Funnily enough, today at school I was walking down a crowded corridor and one guy shouted "oh Hey! That's the brony I was telling you about!" He knows because his friend introduced me to the show and I'd often talk about mlp in front of him.
Now the 50 odd people in that corridor know. Lol at this point in my life though I don't care, I wear anime merch all the time now so I don't really care what people know about me anymore.
I'm only in school for another 3 years (including 2 extra years I'm staying in school to do tests to get more qualifications) so I know I'll never see more than 6 people again.
Most people that I play games with know since any clan tag is (mlp) or (cmc) and if possible mlp designs on anything (seriously you should see my Forza garage).
I'd say it's hard to tell though since I only hang around with a few people and don't really talk about it as much anymore, although I did make a 20% cooler phone case 2 years ago.
I always make references and say "eeyup" so I guess it's easy for bronies to tell that I'm also one.
TL;DR: probably a little bit difficult (unless people call you out as being one in corridors.)