Thanks!
I just talked to the savant, since he was only about 20 seconds off my path.
I had forgotten the name of where I was supposed to go, and then there was a new icon (or one I hadn't noticed before) on my world map. So I went there.
Daedric ruins are the crumbling-outside ones with the altar-things, and Dwemer ones are the ones that appear fairly well maintiained outside?
I ended up having to kill a flame atronach, but the flame salt or whatever they drop is far more valuable than the books! I'll have to fight them more often, although I'll probably end up getting killed a lot. The guy that appeared when I stole stuff from the altar-thing was more powerful, though. He probably would have killed me if he didn't switch from weapons to magic constantly.
And a question about a different quest entirely. Does the yellow book of riddles have any effect at all on the "courtesy" quest? I know the red book does.