LAGOS, Nigeria (AP) _ Police battled members of an outlawed Moslem sect in the eastern city of Gombe Friday and 11 people were killed, authorities reported.

The Maitatsine sect was outlawed in 1980 after prolonged riots in which 4,200 people perished, including its founder.

National police headquarters in Lagos said three policemen and eight members of the sect were killed in Friday's fighting, and six policemen were wounded and 11 people were arrested.

''Police have dislodged the fanatics,'' the statement said.

The semi-official News Agency of Nigeria reported initially that hundreds of people, including non-combatants, were feared to have been killed in the clash.

The fighting began early Friday when sect leader Yusufu Adamu resisted an attempt by police to arrest him in Gombe's Pantami district, police said.

They did not say whether Adamu was killed, arrested or had escaped.

Reinforcements were sent into Gombe when Adamu's followers fought with police and the battle spread to other parts of the city, the report said.

Police and troops barred entry to Gombe, a city of one million people in Bauchi state about 150 miles from the border with Cameroon. The news agency said trucks packed with families and their belongings were seen leaving the city.

Residents of the Pantami district started fleeing the area last week when the presence of members of the religious sect was noticed, the agency said.

The Maitatsine sect was outlawed after 10 days of religious rioting in Kano, 200 miles northeast of Gombe, which is the center of the Islamic faith in Nigeria.

Among the 4,200 people killed in 1980 was the sect's founder, Alhaji Marwa Maitatsine, a self-proclaimed prophet of Islam who rejected the teachings of the religion's founder, Mohammed.

He came from Cameroon and preached a form of Islam that rejects all things modern. His main following was among immigrants from other West African countries.

More than 500 people were killed in clashes between police and rioters from the sect in October 1982 at Maiduguri, about 175 miles northest of Gombe, and in the northwestern city of Kaduna and in Kano.

The most recent rioting took 534 lives, by police count, in the northeastern city of Jimeta in March 1984. Nigerian newspapers put the toll at more than 700.

Members of the sect have been seen during riots sprinkling themselves with ''magic dust,'' which they believed made them immune to gunfire.