THE HAGUE, Netherlands (AP) _ U.S. and Iranian negotiators in The Hague have made progress toward return of $507.7 million in Iranian funds held at the New York Federal Reserve Bank, an informed Iranian source said today.

Officials from the New York Federal Reserve Bank, the U.S. Treasury Department, the Iranian Central Bank and the Iranian government planned to meet a second time at the Iran-U.S. Claims Tribunal for more talks on the issue.

The two sides met for 3 1/2 hours Monday.

Willem Hamel, deputy secretary-general of the tribunal, said further talks were expected either tonight or Wednesday morning.

''The intention is to continue discussion,,'' said U.S. Embassy spokesman Sam Wunder.

Tehran has linked return to Iran of all its assets blocked in America to Iran's possible intercession on behalf of American hostages in Lebanon.

This is the third series of meetings the United States and Iran have held recently on the issue of Account Number One, the $507.7 million residue of a $3.66 billion account at the New York Federal Reserve. That account was established with Iranian money in 1981 to pay off syndicated bank loans.

The amount was given earlier as $506 million. Today's figure of $507.7 million was given by the New York Federal Reserve and reflected continuing interest.

''They have progressed,'' an Iranian source privy to the negotiations told The Associated Press. ''I hope they reach some outline (for a solution) by today.''

''The tribunal already gave the view that this money is the property of the Islamic Republic of Iran, and they (the United States) should transfer it immediately and promptly,'' said the source, referring to a tribunal decision last August which led to the negotiations.

The tribunal was set up in 1981 to mediate billions of dollars in financial claims between the United States and Iran that resulted from the 1979 Iranian revolution which overthrew the shah.

Account Number One took on added significance when Hashemi Rafsanjan, the Iranian parliament speaker, stressed that Iranian assets in the United States must be returned prior to any Iranian intercession on behalf of American hostages in Lebanon.

''That is the statement, including all Iranian assets and all Iranian property in the United States,'' said the Iranian source, who demanded anonymity.

While both sides in The Hague negotiations have specified that these discussions are only financial in nature, the Iranian source said Account Number One is considered part of those assets.

''It is that for sure,'' he said.

In Washington, State Department spokeswoman Phyllis Oakley said earlier, ''We see no link whatsoever between these funds and the hostages.'' She described the talks as ''technical discusssions.''

Six Americans are missing in Lebanon, and most are believed held by pro- Iranian Shiite Moslems.

The United States has agreed to the return of the bulk of the funds sought by Iran, but details must be worked out.

Leading the U.S. side this week is James Oltman, general counsel of the New York Federal Reserve, aided by Donald Bittker, the bank's deputy general counsel, and a U.S. Treasury official whose identity was not determined.

Legal adviser Assadolah Nouri leads the Iranian delegation, assisted by Mohsen Kakavand, head of the Iranian National Bank's foreign branch.

Issues being discussed include the wording of a release absolving the New York Federal Reserve of any further responsibility for the account it has managed for almost six years; disposition of $10 million to $15 million in interest from the account; the final draft of an agreement, and the mechanism of transferring the funds to Iran.

In addition to Account One, the United States holds millions of dollars worth of military equipment paid for by the late Shah Mohammad Reza Pahlavi, but not delivered after he was overthrown in 1979.

Iran has been at war with neighboring Iraq since September 1980 and needs both money and war material to pursue the conflict.