TORONTO (AP) — A lawmaker from Canada's governing Conservative party has apologized for saying women who wish to wear a face veil while swearing the oath of citizenship should stay where they came from.

Parliamentary member Larry Miller made the comments on a radio show Monday. He said he was baffled about a Federal Court ruling overturning a ban on the wearing of niqabs during citizenship ceremonies. He added that women who want to wear the veil should "stay the hell where you came from."

On Tuesday, Miller said in a statement that he stands by his view that anyone taking the oath must uncover their face. But he added, "I apologize for and retract my comments that went beyond this."

A spokeswoman for Canadian Prime Minister Stephen Harper said Miller's remarks were inappropriate and went beyond "our clear position."

"We believe most Canadians, including new Canadians, would find it offensive that someone would cover their face at the very moment they want to join the Canadian family," Anna Tomala said in an email.

The Harper government is appealing the court decision

Harper said in Parliament last week that the niqab is "rooted in a culture that is anti-women." The prime minister's comments come as he faces re-election in October and as Canada's economy struggles with the recent dramatic drop in the price of oil. The government is also in the process of passing new anti-terror legislation.

Opposition Liberal leader Justin Trudeau said the comments prove his point that Harper is stoking fear and prejudice against Muslims.

"This is a prime minister who encourages and condones the politics of division and fear and that is very dangerous in Canada, in a pluralistic, strong, free society like ours," Trudeau said.

Amira Elghawaby, human rights coordinator for The National Council of Canadian Muslims, said that it's unfortunate the Harper government is trying to foster divisions.

"People are uncomfortable with the direction that this discussion is taking. It's really unbecoming of what we expect from our leaders and politicians," she said.

Elghawaby said very few of Canada's 1 million Muslims wear the niqab.