What was the ALPAC report, and why did it stop machine translation research in the US?

In January 1954, a small demonstration by Georgetown University and IBM showed a computer turning a set of carefully chosen Russian sentences into English. Over the following decade, US agencies paid for a string of projects in machine translation (MT), the use of computers to translate from one human language to another. By the mid-1960s, a committee asked a blunt question: what had all that money bought? Its answer, the ALPAC report, is probably the most famous moment in the story of machine translation.

The report's official title is "Language and Machines: Computers in Translation and Linguistics". It was published in Washington, D.C., in November 1966 by the National Academy of Sciences and its National Research Council. The main text is only 34 pages, backed by twenty appendices that bring the whole document to 124 pages.

Who wrote it, and why

In the early 1960s, three US agencies were paying for most MT research: the Defense Department, the CIA and the National Science Foundation. To coordinate their spending, they formed a joint group, and in April 1964 that group set up the Automatic Language Processing Advisory Committee, or ALPAC.

The chair was John R. Pierce of Bell Telephone Laboratories. The other members were the psychologist John B. Carroll of Harvard, the linguists Eric P. Hamp of the University of Chicago and Charles F. Hockett of Cornell (who served only briefly), the former MT researchers David G. Hays of the RAND Corporation and Anthony G. Oettinger of Harvard, and the AI researcher Alan Perlis of the Carnegie Institute of Technology. The committee also heard from people still building MT systems, including Paul Garvin, Gilbert King and Winfred Lehmann.

ALPAC decided early on that MT deserved support only if it promised practical results: cheaper, faster or better translation for government needs. In practice, that meant one job above all: turning Russian scientific and technical documents into English.

What the committee found

The report began not with computers but with translators. Was there a shortage? No. Government posts were filled, hundreds of qualified translators were available in the Washington area, and one translation service was using only a small fraction of the thousands of translators it had under contract. ALPAC even suggested that some material was translated that nobody would read, and that scientists who needed a lot of Russian could learn to read it in 200 hours of study or less.

Then it looked at speed and cost, comparing ways of getting a Russian document into English:

The machine route took longer than sending the work to human contractors, mainly because its output had to be corrected by human "post-editors" and then produced for print. On cost, the committee estimated that once readers' time was included, ordinary human translation was cheaper than raw machine output for any document read by more than about twenty people.

The quality test was designed by John Carroll and reported in Appendix 10. People rated human and machine translations of the same texts. Unedited machine output was mostly understandable, the report said, but sometimes misleading or wrong, and slow and painful to read. The report printed samples from four MT systems and set them beside the 1954 demonstration sentences, noting that the old, hand-picked examples read better than the newer general output.

Its central conclusion was stark: no machine had yet translated general scientific writing usefully, and none was likely to soon. In its own words, quoted widely since, "There is no emergency in the field of translation."

What it recommended, and what happened next

ALPAC did not say language research was worthless. It praised machine aids for human translators, such as automatically produced glossaries used at a German armed forces translation agency and a terminology database built by the European Coal and Steel Community in Luxembourg. It argued that MT research had already enriched linguistics. And Pierce added a note urging the National Science Foundation to fund computational linguistics, the scientific study of language with computers, as distinct from building translation machines.

Its nine formal recommendations, though, were all about human translation: better ways to evaluate translations, speed up translators, cut delays and produce good reference works. Funding agencies read the message clearly. US government support for MT largely ended, and for years many researchers kept quiet about working on the subject at all.

The historian of MT John Hutchins has pointed out that the damage can be overstated. Some projects had already lost funding before 1966, and a few, such as those at Wayne State University and the University of Texas, carried on for years afterward. He also found that the report's famous figure of about $20 million spent on MT included conferences, computer hardware and foreign research, and that the true amount spent on US MT research was likely closer to $12 or 13 million.

Was the ALPAC report right about machine translation?

On the facts of 1966, largely yes. The systems of the day produced clumsy, often wrong English, and they did not save government agencies time or money. The report was also right that tools to help human translators were a better near-term investment.

Its critics, Hutchins among them, see two weaknesses. Comparing working systems on real documents with a tiny demonstration built for a handful of sentences was not a fair test. And by looking only at US government needs for Russian, the committee overlooked other uses, such as business documents in many languages. Meanwhile, research went on in Europe and Japan.

Seen from 2019, ALPAC's lasting gift may be its insistence on measuring translation quality with real readers. Every new translation method since, including the neural network systems behind today's free online translators, has been judged by asking the question ALPAC asked: is the output good enough for the person who has to read it?