NEW YORK (AP) — The Confederate flags had been in a Manhattan apartment's windows for over a year. And then, in a matter of days last week, they were met with hurled rocks, a punched-out window, a tarp hung over them and legal action.

By this week, the lighted flags were no more to be found in the seventh-floor windows in the East Village neighborhood, where the Confederate symbol had been displayed alongside an Israeli flag and a colonial-era American one.

They'd attracted new attention after an Aug. 12 white nationalist rally to preserve a Confederate statue in Charlottesville, Virginia, spiraled into violence that left a counter-protester dead and stirred debate over where or whether Confederate statues and symbols belong in 21st-century America.

The flags' owner, William Green, said by email Tuesday that the reaction reflects a misunderstanding of their meaning.

"The problem here is media, and schools abusing their power and leaving the population completely ignorant of what the Dixie flag symbolizes," he wrote.

The battle flag represents Confederate fathers who "loved their country the way their God loved the world," said Green, who grew up in New Jersey and is keenly interested in history, including the roots of the design and symbols of the Confederate flag. To him, it stands for "heritage, not hate."

But some of his neighbors don't see it that way.

A man lobbed rocks at the building last week while he and others yelled for the Confederate flags to be taken down, describing them with expletives and calling them "racist," according to video made by a bystander and posted on local news site DNAInfo.

Another man, Darren Keen, put a fist through one of the flag-flyer's windows Friday night after getting to his fire escape from an adjoining roof, according to police, who arrested him on a disorderly conduct charge.

"I just lost my temper when I saw that Confederate flag," Keen said Monday. To the Nebraska-born DJ, the flag symbolizes hatred and intolerance, and it galled him to see the banner displayed across the street from an organization that mentors and educates disadvantaged, largely minority girls from the neighborhood, where funky-chic cafes and pricey condos blend with public housing.

Still, the 34-year-old said he hoped anyone else who feels the same way would "make a sign - don't break out a window."

Green has noted that he'd had the banners up for more than a year, while the controversy unfurled only recently.

By Saturday, his landlord had sued him, asking a court to force the flags' removal. The suit said Green had violated his lease and rent-stabilization laws by creating an "anti-social nuisance" that was causing "fear, civil commotion, violence, protests and unpredictable behavior."

Other tenants had barraged the landlord with complaints about being accosted and accused by people who wanted the flags gone, the suit said. It was withdrawn Monday.

Meanwhile, the area's city councilwoman, Rosie Mendez, cautioned residents in an open letter Thursday that flying a flag isn't illegal in itself and warned them not to break laws in response.

Still, the Democrat asked the flags' owner to take down "a symbol that is so offensive to the surrounding community."

Amid the flap, the flags and their top-floor windows were obscured Friday by a massive tarp hung from the roof, prompting the city to send building inspectors out to make sure the drape didn't endanger the public. The inspectors found the tarp was safely secured, Buildings Department spokesman Andrew Rudansky said.

The tarp, and the flags, then disappeared, though it's not clear how. A message left at the landlord's office wasn't immediately answered.