WASHINGTON (AP) _ Federal regulators on Friday seized California's Columbia Savings and Loan Association, the industry's biggest speculator in junk bonds.

Columbia, of Beverly Hills, worked closely with Michael Milken, Drexel Burnham Lambert Inc.'s junk bond chief, to accumulate a multi-billion portfolio of the risky securities far larger than that of any other thrift institution.

Analysts have estimated the taxpayer's ultimate price tag for protecting Columbia's depositors at around $1 billion, placing it among the dozen or so costliest S&L failures.

Its former chief executive, Thomas Spiegel, was forced to resign in December 1989. Regulators later sought a record $24 million from him for squandering deposits on luxury condominiums, trips to Europe, a gun collection and other luxuries. They are seeking another $29 million in connection with Columbia's headquarters building, which featured a multi-level gym and bathrooms with bullet-proof glass. Spiegel is said to have feared terrorists.

At the end of 1990, Columbia had junk bonds worth approximately $2.1 billion, which it had been trying to sell since new management replaced Spiegel. The bonds were originally purchased for more than $4 billion.

The takeover, ordered by the Treasury Department's Office of Thrift Supervision, brings the government's junk bond holdings to more than $5 billion. That makes the government the largest single owner in the $200 billion-plus junk market.

The Resolution Trust Corp., chartered in August 1989 to conduct the government's thrift cleanup, will manage Columbia until it can be dismantled or sold to new investors with government assistance.

Its 23 offices will remain open during their usual business hours and deposits continue to be federally insured to the $100,000-per-account limit.

News of the takeover preceeded by a few hours disclosure of the first quantitative estimate of next year's expected deficit in the funds of the Federal Deposit Insurance Corp., the insurer of bank deposits.

The New York Times quoted unidentified officials as saying the Office of Management and Budget had said FDIC would be $4 billion in the whole at the end of the 1992 fiscal year, compared with an expected starting position of $4 billion to the good, if nothing is done to boost its available funds.

Columbia's new chief executive, Edward G. Harshfield, who took over in March, had struck a deal last July that he thought had a chance of saving the thrift.

The Toronto-based Gordon Investment Corp. had agreed to pay an unexpectedly high $3 billion for the bonds. But federal regulators rejected the sale because Columbia would have lent Gordon 90 percent of the purchase price and would still be liable for losses if the bonds declined more than 10 percent in value.

The thrift office told Columbia to try again, requiring it to get a larger down payment. But on Friday it rejected the new bids as well.

''These proposals offered little benefit,'' said thrift office director Timothy Ryan. ''If the junk bond market were to improve, the buyers would have enjoyed a disproportionate share of the bond sale proceeds at taxpayer expense. If the market were to decline further, the buyers would have little to lose.''

Some analysts say the government's rejection of the July sale ultimately added to taxpayers' costs, but others say that only a cash deal represents a true sale.

''Unless you get junk totally out of your system, you still have problems,'' said Miami-based analyst Kenneth H. Thomas. ''The best thing to do is sell it and take your hit.''

Last summer, Columbia likely could have found a cash buyer for the bonds willing to pay between $2 billion and $2.5 billion, he said. Now, the price likely will be lower, he said.

Columbia's assets at seizure totaled $6.6 billion. It had $6 billion in 102,114 deposit accounts. It suffered net losses of $591 million in 1989 and $782 million in the first nine months of 1990.

Before the seizure of Columbia, the corporation had inherited about $4.7 billion in junk bonds and had sold $1.7 billion of them, according to agency spokesman Stephen Katsanos.

Columbia was the largest member of a so-called ''daisy chain'' of S&Ls that dealt frequently with Milken, who pleaded guilty to criminal charges last year. The thrifts traded the bonds among themselves at inflated prices, the government charges. Other failed institutions involved included Charles Keating's Lincoln Savings and Loan of Irvine, Calif., and David Paul's CenTrust Savings Bank.

Last week, banking regulators filed a $6 billion civil lawsuit, charging Milken, Spiegel, Keating, Paul and others of defrauding S&Ls in schemes to inflate junk bond prices and hide losses.

''Every thrift that's been a major junk purchaser ... has failed,'' Thomas said. ''The only exceptions are thrifts that are being subsidized by the government'' as a result of previous bailouts.

The S&L bill enacted in August 1989 forces thrifts to sell their junk bonds by July 1994.

Columbia, like many large S&Ls, was a big political contributor to both Democrats and Republicans. According to Common Cause, a group pushing for an overhaul of campaign finance laws, Columbia executives and a political action committee associated with the thrift contributed $285,000 to congressional candidates and political parties between 1981 and April 1990.

Among the top recipients, Common Cause said, were Sen. Donald W. Riegle Jr., D-Mich., chairman of the Senate Banking Committee, $29,000, and Sen. Jake Garn of Utah, the senior Republican on the panel, $25,200. However, Riegle returned all S&L contributions to the Treasury this summer, an aide said.

Columbia was among four S&Ls to fail Friday, bringing the total so far this year to 11. Last year, 211 failed. Others seized Friday were: Trustbank Savings, Tysons Corner, Va.; Center Savings and Loan Association, Clifton, N.J., and Coral Savings and Loan Association, Coral Springs, Fla.