You don't have to worry about people leaving due to alignment or reputation clashes. Sarevok is a very good fighter, and depending on what tactics you use, probably better than at least Valygar. He has an excellent ability called deathbringer assault, which does 200hp of damage (and can affect all enemies, unlike vorpal effects) which occurs on a reasonably frequent basis. Plus from a role-playing perspective it's quite interesting to take him along.
You could always maybe take a set of leather armour along if you want to hide in shadows
