A bit of both. If you always listen to your fans, your passion for your work will fade away because it will no longer feel like it's your work and so the quality of your work will decline.
If you always do everything your way, and never listen to your fans, you will lose them and it may be sad to say it this way, but more fans means more money, usually, lol.