Yeah, a lot of people don't quite know what hacking is, and movies don't help with the matter. Although it is true that hacking would normally be done trough a command line interface.
And another problem is that a lot of people never even see terminals on their computers, so they don't know much about them and as per standard for the unknown, fear them and think they are evil.