First of all, you're not 'an idiot' for having these concerns! That's what these forums are for! Here you are among friends and we speak openly.
I know that many ponies feel embarrassed, ostracized, and misunderstood. But the bottom line is, who cares? Do other people (non Bronies) care what you think of THEIR interests? Should they care if YOU disapprove? They don't because they feel secure in their perfect little world that is tailored to fit the accepted norm and conforms rigidly to the ever-dreaded 'peer pressure'. They are constantly propped up by the not-so-great majority. But they will never enjoy all the world has to offer because they are too cowardly to venture into it for fear that the lowest common denominator of humanity might disapprove. Those who make fun of Bronies might as well wear a sign around their necks that say 'Too afraid to think for myself', 'I'm insecure' or 'I need to have constant approval from others'. Have you ever really thought about what it is to be 'cool'? To me, 'cool' is just a pretty way of saying 'fashionably foolish'.
There are so many different facets of humanity on the earth and individuality is what makes them interesting. When I became a Brony, it never even crossed my mind that someone might not approve, because I don't care. I've never followed the crowd and never will. Do you know what that makes me? It makes ME the one who calls the shots that others must adhere to, because I won't bend to meet THEIR demands. And I'm immune to their insults because I see them for what they are; FEAR. It's the individuals that shape the world. It may not always seem so because the weak ones use easy tactics like teasing, ostracizing, and other hateful means because that's all their capable of and it's easy to manipulate other weak people that way, giving them a large but uninspired army of mindless crowd-followers.
If someone insults you and you want to retort, tell them they have your sympathy. Since they are probably too shallow to understand what you mean, offer to discuss it with them in detail. That's where the enemy loses his footing, because they can't keep up with an intelligent response that might demand the same from them. They might not look good in front of their friends when they're outsmarted and forced to display actual intelligence.