Yes! Very much so.
When people ask me if I like to draw, for example, and I say yes, they ask me what I like to draw and I'm stuck. The three things I like to draw are dragons, pokemon and ponies, and none of those are really acceptable for others, since I'm a teenage girl.
I don't carry around my sketchbook to many places, but if one of my friends comes over to my place, I always hide it somewhere in my closet so they won't see my drawings. Sad, I know.