I had no problems with casting spells in the boogre prison, cause nearly all my warlocks are female
or i would suggest that you first turn your party back to "normal" shape at the altar of (whatever itīs name is, something with a).
I fought the lilies inside the area with long narrow tunnels with 90° turns. I cast two or the meteorstorms and fire storms and then run around the corner and cured the poison like mad