scatter and zig are correct, loudhy, in as much as you dont need that particular key to get to the castle/get access to ferro and/or becoming a templar. having said that, i always go back and complete everything for the experience.
thinking of ferro's items and experience, i seem to recall that i picked up quite a few levels on the way in with those lower level parties. so i probably managed to make level 14 before reaching ferro the first time. that's probably why i've never experienced the dearth of items. lucky me