WASHINGTON (AP) _ The Supreme Court, studying a dispute between a California real estate broker and a Texas oilman, was urged to limit the financial liability that sellers of unregistered securities face for their investors' losses.

Federal securities law does not force such sellers to pay an investor who lost money if the investor knew the investment was not registered and also persuaded others to invest, the justices were told Wednesday.

The court's decision, expected by July, will be ''important because it bears upon day-to-day regulation of the securities industry,'' Dallas lawyer Braden W. Sparks argued on behalf of oilman Billy J. Pinter.

Pinter and his Black Gold Oil Co. sold interests in Oklahoma oil and gas leases in 1981 to real estate broker Maurice Dahl and 11 friends or family members Dahl contacted about the investment opportunity.

The interests turned out to be worthless, and the investors sued Pinter.

Federal securities law bans the selling or delivering of unregistered securities through the mail. The law also provides for investors to sue the sellers of unregistered securities to recover lost money.

A federal trial judge in Dallas, later upheld by the 5th U.S. Circuit Court of Appeals, ruled that Dahl should recover more than $310,000 from Pinter, and that the other California investors should recover a total of about $82,000.

In the Supreme Court appeal that was granted review last April, Pinter argued that Dahl should not be entitled to recover any money because he was equally at fault.

The 5th Circuit court had ruled that such a defense is not available under that portion of the Securities Act of 1933 on which Pinter's liability was based.

The appeals court also rejected Pinter's contention that Dahl himself should be considered a seller because of his promoting the interests sales among his friends and family members.

Sparks today told the justices that Dahl intended ''not to be merely an investor'' when he initiated his ''solicitor-type behavior.''

''Mr. Dahl's position is 'I'm not a seller.' That seems absurd to me. He's a seller,'' Sparks said.

But John A. Spinuzzi, a Denton, Texas, lawyer representing Dahl, said Dahl's ''only interest was as an investor himself. He cannot be held to be a seller'' under federal securities law.

When Chief Justice William H. Rehnquist asked whether Dahl's motivation in soliciting investments for the Oklahoma wells was ''just to get his friends in on this good deal,'' Spinuzzi answered, ''Yes sir. It really was.''

In responding to questions by Justice Antonin Scalia, Spinuzzi said Dahl could be held legally liable if he had solicited the additional investments for his own financial gain.

If the high court were to rule that sellers ofunregistered securities sometimes may fend off buyers' attempt to recoup investments by contending the buyers were equally at fault, it likely would send the case back to the lower courts to determine whether Pinter could raise that defense successfully.

Representing the Securities and Exchange Commission, government lawyer Richard G. Taranto asked the justices to do just that.

The case is Pinter vs. Dahl, 86-805.