That depends on three things: first, if you've been particularly mean or have a reputation the member can't stand, there's a big chance that person will walk away and dissapear for good. Next, there's the place where you dump the person. In certain places the only option is to stay, in others the only option is to go somewhere else.
Finally, and this is true in most cases, after the character walks over to your main character and asks if you want to keep him/her in your party or not, the character will ask wether to return to a certain location or stay where they are.
------------------
The last arrow of Legolas kindled in the air as it flew,
and plunged burning into the heart of a great wolf-chieftain.
All the others fled. -J.R.R. Tolkien, The Lord of the Rings
Legolas, mourning for the USA