What is Gemma, Google's family of open models, and who is it for?

Yesterday, February 21, 2024, Google released Gemma, a family of AI language models that anyone can download and run. Google calls them lightweight and says they grow out of the research and technology behind Gemini, the large models that power its chatbot. The name comes from the Latin word gemma, meaning "precious stone."

This is a different kind of release from Gemini. With Gemini, you use the model through Google's services. With Gemma, you receive the model itself and can run it on your own machine, change it and build products on it.

What Google released

Gemma comes in two sizes. The numbers in the names refer to parameters, the adjustable values a model learns during training: Gemma 2B has about 2 billion and Gemma 7B about 7 billion. For comparison, the large models that power popular chatbots are generally much bigger, which is why they run in data centers.

Each size comes in two versions:

Version	What it is	Good for
Pre-trained (base)	The model after its main training on large amounts of text	Researchers, and developers who want to train it further for a specific task
Instruction-tuned	Further trained to follow requests and hold a conversation	Chat, question answering, quick prototypes

Along with the models, Google released a Responsible Generative AI Toolkit (guidance, a method for building safety filters from a small number of examples, and a debugging tool), ready-made notebooks on Colab and Kaggle, and support for the main programming frameworks: JAX, PyTorch and TensorFlow through Keras 3.0. Gemma also works with Hugging Face, the popular platform for sharing models, and with NVIDIA's tools.

Google says Gemma 2B and 7B are the best performers for their size compared with other open models, and that they beat significantly larger models on key benchmarks. TechCrunch notes that Google did not provide a detailed comparison with similar models from Meta and Mistral before the launch, and that results on Hugging Face's public leaderboard would follow. Independent tests will show how they compare in practice.

"Open model" is not quite "open source"

The term matters here. Open source traditionally means anyone may use, change and redistribute software, with very few conditions. An open-weights model, or "open model," makes the trained weights, the numbers that make up the model, available for download, but under its own terms of use.

Gemma is an open model. Google's terms allow responsible commercial use and distribution for organizations of any size. In a briefing reported by TechCrunch, Jeanine Banks of Google explained that the company chose the words "open models" deliberately, because the terms around things like redistribution and ownership of modified versions differ from traditional open-source licenses.

In practice, that means you can run Gemma, fine-tune it, which means training it further on your own examples, and ship it in a product, as long as you follow Google's terms.

How a small model can run on your laptop

Large chatbots need racks of specialized chips. A model with 2 or 7 billion parameters is small enough for a single computer. Google says Gemma runs on laptops, desktops, workstations and Google Cloud, and that it has been optimized for NVIDIA graphics cards as well as Google's own TPU chips.

TechCrunch reports that Gemma uses a dense decoder-only design, the same basic architecture as Gemini. "Decoder-only" means the model predicts text one token at a time, each new piece based on everything before it, like an extremely well-read autocomplete. "Dense" means the whole network works on every token. Tris Warkentin of Google DeepMind told TechCrunch that tasks which once needed very large models are now possible with smaller ones.

Google also describes its safety work: personal information and other sensitive data were filtered out of the training sets with automated tools, and the instruction-tuned models were trained with reinforcement learning from human feedback, in which people's ratings of answers teach the model which responses are preferred. Google reports manual red-teaming and automated attack testing in its model card.

Why it matters

Gemma puts Google alongside Meta and Mistral among large companies offering capable models that developers can download. For builders, small open models bring advantages a cloud chatbot cannot: data never leaves your computer, there is no per-request fee, and you can tailor the model to one narrow job.

For learners, Gemma is a chance to see how a language model works from the inside, on hardware you may already own.

Try it: Write three short tasks you would want a small, private model to handle, such as summarizing your own notes, sorting emails into categories or drafting replies in your style. Keep the list; it is your test set for comparing any open model you install.

How can you try Gemma on your own computer or in the cloud?

The easiest start needs no installation. Google offers free access to Gemma on Kaggle, a data-science platform, and in Colab notebooks, which run in your web browser on Google's machines. Open a ready-made notebook, run the cells in order and type a prompt to the instruction-tuned model.

If you are comfortable with Python, you can get Gemma through Kaggle or Hugging Face and run it with PyTorch, JAX or Keras. Begin with Gemma 2B, which needs the least memory, and move to 7B if your computer can handle it. Businesses can deploy Gemma on Google Cloud through Vertex AI or Google Kubernetes Engine. New Google Cloud users receive $300 in credits, and researchers can apply for additional credits.

Whichever route you take, read Google's terms of use and the model card first, and test the model on your own examples before relying on it.