I always hated that part. I find it best to take a step into the room where they see you, and run away, that way they'll funnel through the doorway and be less likely to surround you.
Also, sorcery is WAAAY overpowered in that game. There's a spell that catches the attention of most types of enemies nearby (similar to an alluring skull); using that, I can just casually walk past hordes of enemies, and they won't even acknowledge my existence! Oh, and when I got hexes, it was all over.
You're right, though, that game is really hard without using some form of magic, I applaud anyone who can play the game without it.