I think this weariness thing can only be cured by waiting it out, preferably behind a town gate.
I've gotten weary from:
- Evil Eye Gaze
- Dragon Fire
- Vampire Breath
- Any musical instrument
- Drinking too much at the tavern
I haven't seen anything else cause it, including running nonstop for hours ... (I never walk, old habit from Doom).