But Terror is a long range spell?! I can cast it at creatures long range away, why shouldn't they hit back at long range?

I think group spells are the only spells irrelevant of distance. If monsters are in the combat area, the spell will hit their group. Or at least that's what I seem to notice anyway.

That's not quite true. You can fight with monsters "out of range", and these will not be targettable by group spells. But as soon as one of them comes into long range they're ready for it.

I'm not sure about groups with everyone being out of line of sight. might be possible to hit them with (say) insanity, but i'm not sure.

No. Only ball spells work around corners. Which is very handy for the Alchemizers, what with both Quicksand and Death Cloud on the list...