CHATSWORTH, Calif. (AP) _ Computer Memories Inc., which earlier this month lost an IBM contract that accounted for much of its sales, said Friday it laid off more than 36 percent of its workers in the United States.

Computer Memories' chairman, Irwin Rubin, said pink slips were given to 160 of the 440 workers at the company's headquarters here.

At the same time, Rubin said the company is stepping up its shifting of work overseas where costs are lower, indicating that more of the company's Chatsworth workers may be furloughed.

Rubin said company's Singapore plant, which has 840 workers, ''will soon have sufficient capacity to fill all of CMI's production requirements, including the new orders the company is seeking.''

Rubin said the company ''is seeking to reduce costs in virtually every aspect of its business'' because of International Business Machine Corp.'s decision not to renew its contract for disk-drives after this year.

Sales by Computer Memories soared nearly sevenfold during the past two years, but most of that was on the strength of IBM's purchases. For the fiscal year ended March 31, IBM accounted for 67 percent of the company's $108.7 million in sales. In the first quarter of this year, IBM's share of the company's sales rose to 81 percent.

In 1984, Computer Memories had difficulty gearing up for IBM's big demand, and hence it had problems meeting deadlines.