If you're an evil cleric, absolutely...after all, they're items that can be put to better uses for your gods. If you're a good cleric, maybe, but only if the items you left could potentially be used for evil. Yes, I know, that can't happen in Morrowind, but that's where the roleplaying aspect comes in.
As far as "decent" weapons are concerned, I found a daedric mace not to long ago that I like quite a bit...though it's still hard for me not to pull out the Umbra Sword anyway
