KARNES CITY, Texas (AP) — An advocacy group says 16 men temporarily transferred out of a Texas family immigration detention center have been returned following what federal authorities called a "disturbance."

During a conference call Friday, the group Raices Texas said 16 of its clients were handcuffed Wednesday at a facility in Karnes City by armed agents with U.S. Immigration and Customs Enforcement. The men were transferred to another lockup for the night.

Some of the men said they weren't told why they were being removed, and that their children didn't know where they were. One man said he was just sitting in his room when he was ordered to put his hands up.

ICE said in a statement that about 40 men "were detained." The agency didn't say what caused the disturbance .