I can't see a problem with your LE Elf Mage (assuming female), and that recruiting him by the 'neutral'-alignment option shouldn't a.f.a.i.k. interfere with your romance prospects.
From Extremist's Romancing Guide at
Sorcerer's Place, 'Legal game rules on romances' section ...
"10. No romancing in dungeons!!! There are some dialogs that don’t follow this rule and those are ONLY dialogs that are triggered with REST button. See dialogs for details.
11. Besides dungeons, Aerie, Anomen and Viconia won’t romance in the open space around de’Arnise hold (AR1300), only Jaheira will."
... and the first romance dialogue should be ...
"I prithee, my lady...it fills me with no small amount of wonder that you have not asked me of my journeys ere we met? We have traveled a short while, and yet we know next to nothing of each other."
I'm not sure if you've spent the bulk of your time 'in dungeons' or doing the de'Arnise Keep quest, or that you have had this (rather lame) first romance dialogue and hadn't realised that the romance was active.
You can use Shadowkeeper or the
CLUA Console to check and/or amend your romance status.
I wouldn't fret too much about his quest at this point. I'm not sure of the timer, but nine days doesn't seem particularly long in my experience.
Hope something here might be of help.