Every now and then I come across a weapon that has 'bound weapon' cast when used on it. How does this work? What's the point? I don't get it. You equip that weapon, and when you use it, you get a bound weapon in your hand, replacing the old one?

Yes. Once you activate the spell, your weapon will be unequipped, placed in your inventory, and the bound weapon will be equipped instead. Once the spell expires, the bound weapon will vanish and your original weapon will be reequipped (not always! A weapon of the same class as the bound weapon was seems to be equipped instead).
You can unequip the bound weapon during activity of the spell and equip another weapon. Nevertheless, you cannot drop the bound weapon on the ground (starting from certain patch level) and you cannot cast another bound weapon spell as long as some bound weapon spell is active.
The bound weapon is always a daedric weapon, it weights nothing, it does not deteriorate, and it is enchanted (it fortifies the corresponding skill or a stat).
Bound armorspells work similarly. Note: if you cast bound cuirass then you cannot cast bound cuirass again until the spell expires but you can cast, for example, bound helm. And so on.

The 'fiend' item must be selected as your active magic, its not a 'cast when strikes' deal.

Oru