NEW YORK (AP) _ NBC complained when the old A.C. Nielsen Co. rating system showed it with lower Sunday night numbers than it thought it should have had, but things might be worse under the new people meters.

When the first, weekly national people-meter ratings were released on Wednesday, ABC won for the first time since July 1986. CBS News was happy with the new system, too, since it came in firmly in first place.

CBS announced it had signed a new contract with Nielsen for the coming season, joining NBC. ABC has yet to sign.

Meanwhile, there was a glitch in the old Nielsen system that is being phased out this month in favor of people meters, and Nielsen had to recalculate the ratings, delaying program rankings a day.

The people-meter ratings don't become the rule until the new television season officially begins later this month. Meantime, the networks are operating under the old system, but releasing people-meter numbers, too.

Before the recalculation, NBC was in first place under the old system, and presumably will remain so, since the recalculation will bring down ABC's rating slightly.

But under the people-meter numbers, ABC won the week with an average rating of 11.2. NBC was second with a 10.9, and CBS third with 10.0.

Ironically, ABC has not yet signed up with Nielsen for the new season, citing complaints about alleged inaccuracies in the new ratings system.

The ''CBS Evening News,'' meanwhile, which had been struggling in third place under the old Nielsen system most of the summer bounced into firm first place under people meters with a 10.2 and a 22 share. ''NBC Nightly News'' had a 9.3 rating and 20 share and ABC's ''World News Tonight'' had a 9.0 and a 19.

The rating is a percentage of the nation's estimated 88.6 million households with televisions. The share is a percentage of the viewers tuned in during a particular time period.

Coincidentially, CBS announced Wednesday it was signing a new contract with Nielsen, but will continue its contract with a Nielsen competitor, AGB Research. CBS, too, had expressed concern about accuracy of the new Nielsen system but said it was satisfied with guarantees in the new contract.

The old system uses a combination of audiometers attached to sets that automatically record what channel is tuned and a separate, written diary. The people meter records each viewer's sex and age via an identification code that must be punched in each time the viewer watches TV.

Until the new season starts, program rankings are calculated under the old system, and the Nielsen computer apparently failed to accurately calculate ABC's ratings for Sunday when the syndicated Jerry Lewis telethon heavily pre- empted network programs.

The news ratings were not affected, and even under the old-system news ratings, CBS won with a rating of 9.6. NBC and ABC tied with 9.2 each.

The people-meter numbers were welcome news at CBS News, where anchor Dan Rather had been plagued by rumors of co-anchors and drastic changes because of a long, unaccustomed run in third place this summer.

CBS News spokesman Tom Goodman said Rather was pleased by the week's results, but eager to give credit to his production staff.

''It's a real shot in the arm for us,'' Goodman said.