I don't think that there is one legit answer. But I think it mainly has to do with how widespread the ponified artwork and things-such-as are. You can search for anything, literally anything, on google images and if you scroll far enough down the page, there will be a pony somewhere.
Go ahead... I'll give you time to try it for yourself.
Having to be constantly bombarded by all things pony can get aggravating to some. Which then leads to them not liking the fans that post pony related stuff in the first place. Beyond that, it's probably just homophobia or some shit.