NEW YORK (AP) _ Banks and thrifts expect to lose billions of dollars in tax breaks under the tax overhaul pending in Congress, but any short-term earnings impact likely will be minimal, bankers and analysts say.

In fact, the industry faces more of an impact from the broad economic consequences of the tax changes than from the provisions specifically directed at banks and thrifts.

''If the administration and Congress have guessed wrong, such that we wind up in a recession, that would be far more punitive for the banks in cutting their loan growth,'' said Virginia Adair, a bank analyst for the investment firm Dean Witter Reynolds Inc.

The industry as a whole is not pleased with the bill approved by a House- Senate conference committee and endorsed by President Reagan. According to some industry estimates, banks and thrifts could lose as much as $10 billion in tax breaks over the next four years.

Side effects could include even less lending to depressed business sectors, higher commercial rents and a cutback in bank purchases of municipal bonds.

''It's not a bill that we can support,'' said Kirk Willison, a spokesman for the American Bankers Association. ''The bill hits us pretty hard.''

One of the most criticized changes would end a tax deduction for reserves set aside to cover potential losses from bad loans. It also would require the biggest banks to include the money already in the reserves as part of their regular income over a period of four years at a cost estimated by the bankers association at $4 billion.

Many bankers complain that the government meanwhile is urging the banking system to build reserves against further economic stress in the real estate, energy and farming sectors.

''It's not enough to cause a big tax bite, but it really does a little bit to undermine the safety net the industry had in place, which I think was working very well,'' said Fenwick Garvey, executive vice president of Ryan, Beck & Co. investment bankers.

While banks would lose a tax incentive to make new loans to hard-pressed economic sectors and developing nations, they would retain a writeoff for foreclosures, and Willison predicted some banks forced to seek new tax breaks might foreclose sooner on troubled borrowers.

Other key changes affecting banks and thrifts would:

-Limit the tax credits banks can take based on the taxes they pay to foreign governments at a loss estimated by the bankers association at $1.3 billion in the next four years.

-Sharply reduce the amount of yearly income that thrift institutions may deduct from a tax-free reserve.

-Eliminate the interest deduction on money banks borrow to purchase tax- exempt municipal bonds, possibly forcing municipalities to offer higher yields as an offset.

-Scale back Individual Retirement Accounts, billions of dollars of which are held by banks, thrifts and credit unions.

Cheryl Swaim, a bank analyst for Oppenheimer & Co. Inc. in New York, said a preliminary analysis of the tax bill indicated most banks would see tax- related earnings changes of 10 percent or less, either positive or negative.

She also predicted that over time most of the bill's negative effects would be offset by changes in money management and by lower interest rates expected to result from the legislation.

''The non-deductability of the loan loss (reserve) is a negative in terms of building their reserves. But I don't think this tax bill will be as big a negative to the industry as everyone says it is,'' Ms. Swaim said.