It shouldn't matter that you left and returned to the Temple.
The monkey idols go in the niches located at the ends of the large "X" on the library floor. A long, elevator-like platform should then rise near the far wall.
The serpent idol goes in the central niche and openings should appear on either side leading to the book.
You could also carry chests and crates into the library and build a staircase to the wizard's study. And you can go to this
link to read the contents of the book.
Xignal