You don't need to feel obligated to feel sad. It's normal that this wouldn't affect you very deeply if you weren't close to him. I'm sure he'd want you to be happy anyway.
I know that feeling, to be honest. My parents didn't invite me to my mama's funeral because they were worried it would be too much for me, but it wasn't for a week or two later, that i actually felt anything