Well, beating that party there is a bit illusionary, true.
But I don't intend to do that now, I just wanted to get to Lum's machine for the stats increase.
Anyway, I didn't really suffer those fear-problems except for a few times.
If you are in the room where a demon inniciates a conversation with you... there's a Succubus in there who tends to frighten your people. Interrupt her spellcasting and you should be done.. if i remember correctly.