After doing Anomen's quest properly (I don't want to spoil this for you), his wisdom gains a boost and this will no longer be a problem.
Personally, Anomen is my favorite cleric. The reasons for this are the number of spells he can cast, the high levels he can attain, his turn undead ability (believe me, liches be can killed with it at level 17), his excellent hitpoints, a helping hand in melee combat due to his high strenght and the fact he can attain weapon mastery in cleric weapons. Need to say more?
For me, the fact that he is arrogant, annoying, stuck-up and all that and has a very irritating voice has one purpose: to counterbalance his power. If you can tolerate his behavior long enough you'll not regret it.