The only way on how to fully understand why "haters" hate bronies so much is to take a look into their mind and see if any of them have had anything that was wrong with them. Say as in bullying or something.
The majority of them as you have said, do in fact say that because they mostly want to show off in front of people and sometimes just want to show you off, or make you feel embarassed in front of a crowd. (which doesn't usually work at all)
Being a brony myself, (of course) I think that they say that because yes, It was originally meant for 5-year old girls, and they have never seen the show before.
But Pokemon was pretty much the same, but yet you see people around their teens watching it, and playing the games, and same as Yu-Gi-Oh!.
That ends my daily rant for today.