WASHINGTON (AP) _ Louis Rukeyser, host of ''Wall Street Week'' on Public Broadcasting Service, is challenging an Internal Revenue Service claim that he and his wife owe nearly $500,000 in taxes and penalties because the IRS disallowed losses from some of their tax shelters.

Rukeyser and his wife, Alexandra G. Rukeyser, who live in Greenwich, Conn., filed suit in Tax Court disputing the entire deficiency claimed by the IRS. They were billed an additional $63,777 in taxes for 1980 and $339,079 for 1982, plus interest and penalties totaling well over $50,000.

The alleged deficiency for 1980 grew out of the IRS' refusal to allow the Rukeysers an investment tax credit on master recordings they had purchased. The IRS contended the investment was made with a motive of reducing taxes, not making a profit. The agency also refused to allow a deduction for a $36,000 loss claimed as a result of the investment in the recordings.

The IRS rejected the Rukeysers' deduction of $786,428 for losses growing out of their investment in four limited partnerships that were involved in real estate leasing and the sale of commodities and securities.

The suit said the Rukeysers invested $663,773 in cash and notes in the four partnerships. The couple said all the investments were made with an eye toward turning a profit.

The suit relating to the Rukeysers' 1980 taxes was filed June 5. The 1982 taxes suit was filed June 18.