If you make a warrior to become a paladin, then decide to switch to a ninja. Do you keep all the traits and abilities you learned as a Paladin?

yes. however you will not learn any new paladin skills ever again. unless some divinity intervenes.

Or unless you reset your adventures - in which case it will be possible to turn the warrior turned paladin turned ninja back to a paladin again.

Yes, this is a great way to get characters to gain unreal power. I had to restart so many times due to glitches that I never got to face a real challenge except for the start and puzzels. The game is about having fun though.