SARAJEVO, Bosnia-Herzegovina (AP) _ Thousands of Muslim refugees have moved into homes abandoned by Serbs who fled Sarajevo earlier this year, and some of the Serbs who stayed behind have been beaten and harassed.

International relief workers say there is little they can do to stop the Muslim refugees from taking over Serb homes despite provisions in the peace accord that give the homeowners six months to return.

Government officials in Sarajevo argue they have nowhere to put refugees but in the abandoned houses.

About 50,000 Serbs have fled Sarajevo neighborhoods that were handed over to the Muslim-Croat Federation in compliance with the U.S.-brokered peace accord. But some 10,000 have stayed behind.

``We are very concerned about continued harassment of the remaining Serbs,'' said Kris Janowski, spokesman for the U.N. High Commissioner for Refugees.

Janowski, interviewed on Tuesday, said his office had received reports that an elderly Serb man was beaten five times during the previous 10 days in the Grbavica neighborhood.

``When he went to complain to the federal police he (was) ... kicked by a policeman,'' said Janowski.

In another incident in Grbavica, international police found a 95-year-old Serb collapsed on the street, with broken ribs and a multiple jaw fracture. They drove the man, Gojko Pandurevic, to a hospital in adjacent Serb-held territory.

Three days later, a Muslim refugee family moved into Pandurevic's house.

``We will be visiting him, but there isn't much else we can do,'' said Alexander Ivanko, a U.N. spokesman.

International officials monitoring the situation say they think the harassment of Serbs is more likely the work of common criminals trying to take advantage of the chronic housing shortage, rather than an orchestrated campaign.

Sarajevo's leaders maintain they are responding to simple necessity by allowing Muslim refugees into the homes. They argue that the situation is temporary, though there is catastrophic lack of housing in all parts of Bosnia and the little money to rebuild.

``Maybe it is immoral, but it is humane,'' Mirsad Ceman, secretary of the Party of Democratic Action, which governs the Muslim-dominated part of Bosnia.

But Serbs _ and some Croats _ worry that the housing takeovers are part of a campaign to keep them out of the capital.

``It first started with verbal threats _ and at the end open attacks, shooting at the houses, throwing hand grenades into Serb houses,'' said Dusan Sehovac, secretary of the Democratic Initiative of Sarajevo Serbs.

``The people who have left (recently) left because they did not get what has been promised to them, their personal security,'' he told the weekly publication Dani.

Even the Bosnian Croats, the Muslims' federation partners, are complaining. Hrvatska Rijec, a Croat weekly, cited reports of discrimination and outright threats against Croats trying to return to their homes in Grbavica. The publication accused Muslims of ``silent ethnic cleansing.''

Ceman, the Muslim politician, denied there was a campaign to cleanse Sarajevo of Serbs and Croats, either by threats or violence or by resettlement. The immediate concern is to shelter refugees with nowhere else to go, he said.

Janowski, of the U.N. refugee agency, said his organization can understand such arguments.

``There is nothing we can do,'' Janowski said. ``They have to put these people some place.''