WASHINGTON (AP) _ The new law that could force an end to the federal government's deficit spending by 1991 swept through Congress with overwhelming majority support.

But in a packed courtroom last week, the measure came under heavy attack on constitutional grounds and even defenders of the Gramm-Rudman Act had trouble mustering enthusiastic support for it.

Michael Davidson, a lawyer for the Senate leadership, said the law is experimental and deserves a chance. It ''should not be terminated so quickly,'' Davidson urged a special three-judge federal panel.

Perhaps the most vigorous backing of the law came from Judge Antonin Scalia of the U.S. Circuit Court of Appeals here who presided over more than three hours of often arcane arguments.

Scalia, who may have been playing devil's advocate, repeatedly challenged the lawyers who were attacking the measure as an abdication of congressional powers and a violation of constitutionally mandated separation of powers.

Leading the charge against the law was Alan Morrison, who heads a pro- consumer organization affiliated with Ralph Nader and who represented 12 members of Congress who filed suit against Gramm-Rudman.

Morrison asserted that a political stalemate caused by Congress' inability to make the tough choices of trimming spending prompted the Gramm-Rudman ''gimmick.''

He said the month-old act unlawfully delagates congressional powers to three agencies headed by appointed bureaucrats.

In effect, Congress has said, ''We hereby abdicate our law-making function, turning it over to others with the fervent hope they will do the job we refuse to do,'' Morrison remarked.

''Congress has refused to pass the very laws to do what it contends must be done - that is, balance the federal budget,'' he said.

But Scalia said Congress delegates authority all the time.

In this case, he continued, Congress decided to ''do whatever it takes to balance the budget.''

A key task under the new law falls to the comptroller general, who heads Congress' General Accounting Office.

The comptroller general is called upon to decide how large mandatory spending cuts must be to meet the law's year-by-year deficit-reduction goals.

Morrison - with the support of the Reagan administration's Justice Department - said the law allows the comptroller general to dictate to the president.

But Scalia said the comptroller general's role was basically that of an accountant.

It's ''something for a guy with a green eyeshade,'' Scalia said. ''It's not power. All he does is to make an economic prediction.''

Scalia also said Congress has delegated great power in the past, to regulatory federal agencies and, for example, when it authorized the Nixon administrtion to set wage and price controls.

Wage and price controls involved more sweeping power than budget cuts, Scalia continued. ''It went to the entire economy. The federal government isn't everything,'' he said.

For some, the future of the economy may also be at stake in Gramm-Rudman.

Hundreds of billions of dollars in spending for defense and domestic programs could be slashed across the board.

President Reagan, who signed the bill with praise but expressed serious reservations about the legislation last week, already is moving to shield American troop strength from the law.

The White House announced plans to exempt military personnel from the law. The administration wants to protect all but $245 million of the Pentagon's $67.9 billion personnel account from the budget ax.

The budget deficit now stands at $212 billion and the new law would pare it down to zero in stages.

The final word on the constitutionality of Gramm-Rudman will be with the Supreme Court. But that could be months away.

The special three-judge court promised to be quick with a decision, clearing the way for the inevitable appeal to the justices. But the three- member panel - comprised of U.S. District Judges Norma Holloway Johnson and Oliver Gasch along with Scalia - gave no timetable.

The first round of budget cuts take effect March 1 and involve the relatively small amount of $11.7 billion.

Until the Supreme Court rules, the Gramm-Rudman act forbids any court from issuing a stay to suspend the law.

But the act does contain a fallback provision that could be its enduring legacy if the courts decide the measure contains unconstitutional flaws.

Under this scenario, the agencies entrusted with setting reduced deficit targets would spell out the needed cuts. But Congress would still have to bite the bullet and actually vote to slash spending.