I feel that, for once, it is legitimate for muslims to ask these changes. The games can be moved by one or two weeks.

When a custom concerns a quarter of the participants, the organizer of the party tends to deal with it. If you lived in a muslim country, and that someone organized some activity at work on the evening of christmas, I think you would object as a christian and ask the organizer if it isn't possible to change the date. If you ask him 6 years in advance, I'm sure he will find a way. The same goes with the olympic games.
There is, and should be no legal obligation, but it makes sense to take care of the needs of the athletes.

PS: I'm an atheist.
PS2: I guess we're just two accostumed to the extravagant demands of the religious, and especially of the muslims, and that we now tend to reject them without even considering them

__________________
Client: In six days, do you hear me, six days, God made the world. And you are not bloody well capable of making me a pair of trousers in three months!
Tailor: But my dear Sir, my dear Sir, look at the world, and look at my trousers.
(Beckett)