WASHINGTON (AP) _ Sen. Charles Robb, D-Va., said Tuesday that he was dropped from the Senate Budget Committee because his fiscally conservative views upset the panel's chairman, fellow Democrat James Sasser of Tennessee.

But Sasser sought the Virginia lawmaker's ouster from the panel only because the committee's size was being reduced to make it more manageable, and Robb had the least seniority, a Sasser spokesman said. A Republican spot on the panel was also eliminated, as only two of three GOP vacancies created by the congressional turnover this year were filled.

It is unusual - but not unheard of - for a majority party to remove a committee member who wants to remain. Senate Democratic Leader George Mitchell of Maine noted that in his early years in the chamber, he too was stripped of a Budget Committee seat. And ''I wasn't given this much attention,'' he said.

Robb's is often mentioned as a potential presidential contender, and is chairman of the Senate Democrats' election apparatus, the Democratic Senatorial Campaign Committee.

Robb said he was being ousted despite repeated pleas to Sasser and Mitchell, who was responsible for Robb's gaining the chairmanship of the campaign committee.

Robb said Sasser told him recently that ''he would be willing to go to bat for me'' had Robb been more supportive of Sasser's budget proposals.

The dispute - which Robb said he unsuccessfully tried to keep private - already has been seized upon by one Republican leader as a political weapon.

''I'm worried about dealing with a Democrat Party in which any effort to be rational and reasonable on fiscal and defense policy gets punished by the leadership,'' Assistant House Republican Leader Newt Gingrich, R-Ga., told reporters.

While Robb insisted his relationship with Sasser would remain ''collegial and upbeat,'' he challenged Sasser's claim that the Tennessee lawmaker had campaigned for Robb when he was elected Virginia governor in 1981.

''I not only did not request Sasser's participation, but I never met him, I didn't know him, and I don't think there's any public record of his having appeared,'' Robb said.

Larry Stein, Democratic staff director of the Senate Budget Committee, said Tuesday that the panel began considering reducing its size last November or December. Its 23 members made it the second-largest in the Senate.

''It was very, very unwieldy,'' he said.

The final decision to pare the panel's size was made by the 25-member Senate Democratic Steering Committee, which oversees committee assignments.

''The only issue here was reducing the size of the committee,'' said Stein. ''The final vote of the Steering Committee was unanimous. If this was personal, would it have been unanimous? I don't see any way it could have been.''

Mitchell agreed, saying the size of the committee was the only issue. The fact that he appointed Robb to the campaign committee ''in and of itself is the best evidence of the high regard I have for him,'' said the Democratic leader.

Stein said Sasser has ''nothing but the highest regard'' for Robb.

Since joining the budget committee when he arrived in Congress in 1989, Robb has voted for both budgets the panel has approved. He also supported the five-year, $500 billion deficit-reduction deal President Bush and lawmakers agreed to last fall.

But Robb said that at public sessions and at frequent private talks with Sasser, he made clear his desire to press for even deeper cuts in the budget gap.

Last year, when the panel had 13 Democrats and 10 Republicans, the committee's work on a budget was delayed for weeks while Sasser searched for a decisive twelfth vote. Eventually, Robb supplied one of the swing votes.

''It's clear to me from what the chairman said to me, and from what I learned from others, that the chairman felt it would be easier to work his will with the committee if I were not a member of the committee,'' Robb said. ''There's no question in my mind that was the reason.''