You don't bother about charisma? No wonder ypur nickname is 'Skunk'
But seriously... it's all about image. Having a high Charisma can make a big difference in certain elements of the plot - furthering the rewards you can gain. Trust me on this.
Plus - who wants to play an ugly, smelly, insulting hero?