I can't get to the chess match

I defeated the four elemental rooms with a my mage and fighter while leaving the other three characters in the hallway. After defeating the slime in the final room nothing happened. Now there are no monsters in any room in the entire tower(except for random encounters) and I cannot get to the chess board(?never been there either). It is getting slightly frustrating. Any help would be appreciated. I really do not want to have to start TotSC over as I have already defeated most of the other quests.

Have you killed all the invisibles, too?

In the last (SE) room there's an air aspect, then some invisible stalkers. You have to walk further into the room before they'll attack you. Once you're through that good luck with the chess match.

Come to think of it, I prefer to 'do' the slime room last, as it only has a fission slime in it that can be taken out with lots of fire (like the stone figure says). Good chance to get to the chess match relatively unharmed.