Yeah, it's too late because the adventure takes place in the world and you're in hell. There's no way of getting back.

If you wanted to you could load a previously saved game and do it, but you'd have to do the Irenicus fights again.

------------------
a bunch of long hairs.
trux

If I'm not mistaken, once you install ToB you'll get another chance to have your once in a lifetime adventure.

------------------
The last arrow of Legolas kindled in the air as it flew,
and plunged burning into the heart of a great wolf-chieftain.
All the others fled. -J.R.R. Tolkien, The Lord of the Rings

Its not that big a deal, kind of a let down actually...