No it does not permanantely affect your characters.
A restoration spell will remove the drain. Either one of your clerics can cast it or you can go to a temple.
------------------
Never try to teach a pig to sing; it wastes your time and it annoys the pig