If you ask him, Sarevok tells you he only needs a small part of your soul, that it wouldn't harm you and that he would not be a Bhaalspwan reborn but just a regular mortal instead.
So no, you can easily part with it, without any negative effects. For once, he isn't trying to trick you too much
