[Azred]

As with many things in life, you may reasonably justify anything you wish. The bottom line is that the character is yours; if you wish to design your character a certain way or justify something, then do so.
The problem with Fallen Paladins is that so many think they must be evil, else they would not be Fallen. Not true. Perhaps a Fallen Paladin has become chaotic good after becoming disillusioned about his corrupt local government or--even better--the Paladin has become Fallen because he believes himself to have some sort of taint that would prevent his remaining a Paladin even though there actually is no taint. That is, the character's irrational belief causes Fallen status, resulting in a self-fulfilling prophecy of failure. Have fun and run with your own ideas.