Welcome to the board, Edward!!!
Go back and check that the drawbridge is still lowered. Sometimes it isn't and if it isn't - you can't be a ninja yet.
This quest sounds harder than it is - not disturbing the spirits - all you need to do is not fight anyone. Run in, lower the drawbridge and split!
Let us know if you've made ninja!
Wyv