When you find slaves, you are either in a cave or on one of the plantations. I don't know exactly who holds the key to free the slaves on the plantations, but in a cave there's always a guard nearby, so all you have to do is kill him and loot the key from his corpse.
__________________
|