I would say that since you said yourself that you both have agreed to disagree and moved on then I would argue that you shouldn't take it to heart. I had a similar experience with my father who still doesn't believe that 'real men' watch the show. My response to him was that you know me, and I like the show, this should invalidate any theories you have about people that watch the show. If that doesn't get through then I would just say move on and don't worry about it. If it doesn't sour your relationship or your enjoyment of the show just remember that and move on.
Hope that helps! Welcome to the herd my good fellow!