[Radek]

I have tried killing Leprechaun at his treasure several times and I've never succeeded. Either he teleported himself away (and got lost from the game - I wasn't able to find him again) or he teleported me away (and vanished from the game simultaneously). He was always able to cast Vanish before he got killed.

Here is the possible explanation: If you attack Leprechaun, you must attack from behind. Otherwise, Leprechaun will Vanish you or himself. You can attack from behind while he is running around on the isle but you cannot attack from behind at his threasure, where he is standing and facing you no matter where you move.