Shinobi

Just a thought, you run into kol again in dragonspire and you could try to pickpocket the key from him then, though i'm not sure if he'd still have it by then, worth a try though.