What was Google Neural Machine Translation, and how did it change Google Translate?

On 27 September 2016, Google announced that a neural network was now doing the translating in Google Translate, starting with Chinese to English. The system, called Google Neural Machine Translation, or GNMT, read a whole sentence before writing its translation, instead of handling it in separate chunks. Bilingual reviewers judged its output far better than the method it replaced.

It was a big moment for a field with a long history of disappointment. Machine translation had been promised since the 1950s. GNMT was one of the first times a modern deep learning system was put in front of a mass audience for a task this hard, at this scale.

What Google announced, and when

Two things arrived together. On 26 September, the research team posted a technical report to arXiv, the online archive for research papers, titled "Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation." It listed 31 authors from Google, including Yonghui Wu, Mike Schuster, Quoc V. Le, Oriol Vinyals and Jeffrey Dean.

The next day, Le and Schuster, both research scientists on the Google Brain team, explained the work on Google's research blog. They also revealed that GNMT was already live: the Google Translate website and mobile apps were using it for every Chinese-to-English request, about 18 million translations a day. Google said it would extend the system to more of the over 10,000 language pairs Translate supported in the following months.

Sources date the start slightly differently. Google's announcement is dated September 2016, while some reference works give November 2016, when Google switched on neural translation for many more languages.

How it differed from the old method

Google Translate had launched ten years earlier using phrase-based machine translation. That approach splits a sentence into words and short phrases, translates each piece largely on its own using statistics, and then rearranges the results. It works, but it often produces stiff, broken sentences, because no part of the system sees the whole meaning.

GNMT is a neural machine translation system. It treats the whole sentence as one unit. Here is the process, step by step, for a Chinese sentence:

1. Encode. A network reads the Chinese words one by one and turns them into a list of vectors, long lists of numbers that capture the meaning of each word in context.
2. Attend. Before writing each English word, the system looks back over those vectors and decides which Chinese words matter most right now. This is called attention.
3. Decode. A second network produces the English version word by word, using the attended information and the words it has already written.

The building blocks were LSTM networks, a kind of neural network designed to remember information over long sequences. GNMT stacked eight layers of them for reading and eight for writing, with shortcut connections between layers that make such deep stacks easier to train.

The engineering that made it work

Neural translation had existed in research for a few years, but it was too slow and too clumsy with rare words for a real product. GNMT addressed both problems.

For rare words, it split text into wordpieces: common fragments of words. A rare name might be broken into several familiar chunks, translated, and stitched back together. This sits between working with whole words, which misses rare ones, and single letters, which is slow.

For speed, Google ran the system on its own chips, called Tensor Processing Units, using its open-source TensorFlow software, and used lower-precision arithmetic when translating.

Why it mattered

GNMT showed that deep learning could replace a mature, carefully engineered system in a product used by many millions of people. It brought together several research ideas, sequence-to-sequence learning, attention and subword units, and proved they could run fast and reliably enough for daily use.

The rollout moved quickly. Later in 2016 the neural engine was switched on for eight languages to and from English: Chinese, French, German, Japanese, Korean, Portuguese, Spanish and Turkish. In March 2017 Russian, Hindi, Vietnamese, Hebrew and Arabic followed, and nine Indian languages were added at the end of April 2017. The system was also described as able to translate directly between pairs such as Japanese and Korean, which it calls zero-shot translation, instead of passing through English first.

How far has neural translation come since GNMT, and where does it still fail?

Neural translation is now the norm. Google Translate supported 109 languages by October 2020. Outside researchers have tested it in serious settings: a July 2019 study in the journal Annals of Internal Medicine found it a viable, accurate tool for reading medical trials published in other languages, which could help reviewers include studies they would otherwise skip.

The weaknesses Google named in 2016 have not disappeared. Systems still drop words, misread names and technical terms, and translate one sentence at a time without the wider context.

The practical advice is simple. Use machine translation to understand the gist of a text, or as a first draft. For anything that matters, such as a contract, medical instructions or a public notice, have a fluent person check the result.