Basically which gems should I not dump off on merchants? I know rogue stones, but any others?

Here's the gems you'll need for TOB:
5 star sapphires
2 rogue stones
1 diamond
2 kingstears
1 emerald

I think that's correct but you may want to take an extra rogue stone and diamond just to be on the safe side. Hope that helps.

Not to worry - you'll find plenty of these gems in ToB itself, so you can sell what you have if you need the cash.

er....those gems are all common enough in SoA anyway. I make my living selling gems. It shouldn't be hard to squirrel away that many gems.