WARSAW, Poland (AP) _ Police briefly detained two American diplomats, and the government said Thursday they had joined an illegal May Day march. The U.S. Embassy said they had merely been observing and that officers manhandled them.

The official news agency PAP said the Americans were among 15 people detained in an ''aggressively behaving, leading group of the illegal parade'' Wednesday in the southern city of Krakow.

Leaders of the march were ''chanting anti-state slogans, carrying posters of hostile content and throwing leaflets,'' PAP said.

Supporters of the Solidarity labor movement, and other opponents of the Communist authorities held many illegal May Day rallies that were broken up by riot police.

PAP said charge d'affaires John Davis in Warsaw was summoned to the Foreign Ministry on Thursday to receive a formal note of protest over the incident, the second involving American diplomats in Poland within three months.

The Americans were identified as William Harwood, first secretary of the U.S. Embassy in Warsaw, and David Hopper, consul at the Krakow consulate.

An embassy statement called the charges ''completely erroneous'' and said the United States protested the ''physical mistreatment and detention of these two accredited diplomats.''

It said that when the two Americans showed their diplomatic identity cards to police and one of them was ''pushed, struck, kicked and forced'' into an unmarked vehicle in which both were taken to a police station.

''The two individuals in question were performing normal diplomatic functions as observers repeat observers of events,'' the statement said. ''They were not in any way participating in these events nor were they part of any group.''

The anti-government march in Krakow involved an estimated 2,000 people who left a Roman Catholic Mass in the industrial suburb of Nowa Huta. They had covered about 500 yards when a column of police in full riot gear dispersed them.

PAP's report said: ''The Foreign Ministry in its note stated that the activities of these two employees of the American mission in Poland are in clear violation of their diplomatic status and basic international norms and customs.''

It said police released them as soon as they were identified as American diplomats.

The U.S. Embassy said, however, that the diplomats produced their identification cards when asked by police and ''after a few minutes they were ordered to enter an unmarked vehicle.''

''En route to the police station, the policeman in possession of their documents denied they were diplomats and said they would be accused of throwing stones at police vehicles,'' it said.

Police held the men at the station for 20 minutes, the embassy statement said, then they were ''escorted to the door and released with apologies.''

The U.S. military attache in Warsaw, Col. Frederick Myer, was accused of spying and expelled in February.

The United States retaliated by expelling the Polish military attache in Washington.

U.S.-Polish relations have been poor since the United States imposed sanctions to protest the declaration of martial law and suppression of the Solidarity free trade union in 1981.

Although martial law has been lifted, the two countries do not have ambassadors in each other's capitals and some of the most important U.S. economic sanctions remain in effect.