And it's done like any other skill check, with the DC being the creatures hide/move silently, score vs the spot/listen scores. One thing I have noticed when I've maxed out spot/listen is that I can see ghost images, for lack of a better term, of creatures that are not yet visible to me, such as in a room with the door still closed, and I'd say that was the game engine's way of letting me "hear" what's in the next room?