If only it were that simple. I don't think you can just say magic is genetic without any sort a basis for such a claim. If it was, then the chance of Elsa being the only one in Arendale to have magic is unlikely, since someone in their family (or some other family) had to have the gene (and be affected) for it to be passed down. If that is the case, then the town should not fear Elsa's power. Knowing Disney, they would give some kind of reason, but instead, it felt more like "This is Disney, they already know there's magic, we don't need an explanation."