I got stuck on that too.
On one of the ledges, there's a fence along the side. At one point, a section of the fence is missing: if you look there, you'll see a steep path you can climb down.
Note: you won't be able to climb back up that path, but you'll find another way to get back eventually.