I don't own a dakimakura, and don't see myself owning one in the near future.
If you live your life thinking about other people's opinions and views of you, you're not going to go anywhere - you can't please everyone, and you certainly wouldn't be pleasing yourself. I mean, they're family right - it's not even severe, it's just a pillow with an image on it. I say go for it.