I have searched all around the Toad Village... ...is it no where near that?

I assume you have been inside . . . go thru the tunnel with knee-deep water, past a room on your left, ignore the room with ladder on right. Right before this room is a tunnel on the left follow it til you come to another room with ladder on your right go up and advance to meet shinwiki . . .

I did that... ...I have met Shinwiki, and he gave me the quest to rid evil from the sacred Toadem grounds... ...but I can't find the sacred Toadem grounds.

After you talk to Shinwiki, turn around the trap door behind you should be open.

Doh! Didn't notice it.