Fair enough. Although I've read articles about this, my country is fairly advanced in sex change operations (in fact the foremost hospital doing them is connected to my university). And at least where I live, it sometimes happens that a young child already displays a deepseated need to belong to the opposite sex. They get counselling and most of them "grow over it" but some are extremely certain of being in the wrong body and for them it is actually possible to get hormonal treatments that postpone puberty (so their physique, in the case of males, will not be very developed or masculine). Then at an older age when they are considered to be capable of the decision, they have a full sex change. So in those cases at least there is no chance for the young boy's body to develop masculine characteristics before the surgery.
But in any case, sex changes are still quite rare and transgender people capable of competing in the Olypics even more so

so it really isn't such a big issue.