BALTIMORE (AP) _ A federal judge today vacated all the convictions against former Gov. Marvin Mandel and five co-defendants found guilty more than 10 years ago in a kickback scheme.

A federal jury convicted Mandel and the others on Aug. 23, 1977, of mail fraud and racketeering in a scam to buy Mandel's influence over racetrack legislation with gifts and bribes. Mandel was governor when he was convicted and was forced to resign.

Mandel and the others asked U.S. District Judge Frederic Smalkin this year to have the convictions overturned, based on a U.S. Supreme Court decision in June.

The high court said that federal mail fraud statutes protect only rights in money and property, and cannot be used to prosecute public officials for defrauding citizens of their ''intangible rights'' to honest government.

The ''intangible rights'' argument was used in the case.

The other defendants were Irvin Kovens, W. Dale Hess, Harry W. Rodgers III, William A. Rodgers and Ernest N. Cory.

''Given the recent, retroaction decision (of the U.S. Supreme Court), together with the nonexistence of any federal common law of crimes, it is clear that intangible rights to good and honest government may not be the target of a criminal scheme to defraud under the federal mail fraud statute,'' the decision said.

''However strong the evidence of dishonesty or bribery, the jury was told it could convict for something that did not amount to a federal crime,'' the opinion stated. ''... Even accepting the view that the evidence of dishonesty was strong, it must be remembered that it took the jury a dozen days of deliberation to convict and it took three appellate opinions before these convictions became final.

''In these circumstances and in light of (the Supreme Court decision), the interests of justice favor the grant of the relief sought,'' Smalkin wrote.

Mandel and the others already have served their prison sentences.

U.S. Attorney Breckinridge Willcox said the government will appeal the decision.

Willcox said he was very concerned that the decision could take the federal government out of the business of prosecuting local corruption cases.

Mandel served 19 months of his four-year sentence and the remainder was commuted. He also was disbarred.

Smalkin's order also said the government must repay all fines levied against the defendants within 90 days.