# What is Google Gemini?

Date: 2023-12-07 · Author: Robert Scales · https://guidetoartificialintelligence.com/en/learn/what-is-google-gemini/

**Answer:** Gemini is Google DeepMind's new family of AI models, announced on December 6, 2023, in three sizes: Ultra, Pro and Nano. It was trained from the start on text, code, images, audio and video together, and a tuned version of Gemini Pro now powers Bard, while the largest model, Ultra, is still in safety testing.

Yesterday Google introduced Gemini, which CEO Sundar Pichai calls the company's most capable and general model so far. It is the first major model from Google DeepMind, the lab Google formed earlier this year. Gemini 1.0 comes in three sizes, each aimed at a different job, and parts of it are already in products you can use today.

The headline idea is that Gemini is **natively multimodal**. "Multimodal" means a model can work with more than one kind of input, such as text and pictures. Many earlier systems did this by training separate parts for each kind of data and joining them afterward. Google says Gemini was instead pre-trained on several types of data at once, then fine-tuned with more multimodal examples, so that it reasons across them from the ground up.

## Three sizes, three jobs

| Size | Designed for | Where it is today |
|---|---|---|
| Gemini Ultra | The most complex tasks | Safety testing; select partners first, wider release planned early next year |
| Gemini Pro | A broad range of tasks at scale | A tuned version runs Bard, in English, across 170-plus countries and territories; developers get access from December 13 |
| Gemini Nano | Running on a device, without the cloud | Pixel 8 Pro, for summaries in the Recorder app and Smart Reply in Gboard |

Google calls the Bard change its biggest upgrade since launch. Developers will reach Gemini Pro through Google AI Studio, a free web tool, and through Google Cloud's Vertex AI. Google also plans a new experience called Bard Advanced early next year, starting with Ultra.

## What multimodal prompting looks like

Alongside the launch, Google published a video called *Hands on with Gemini* and a developer blog post explaining the prompts behind it. The post is useful because it shows the actual inputs: still images paired with written questions, sometimes with a couple of worked examples to show the model what is expected.

One sequence shows three photos of a hand, then asks what game is being played. Another shows frames from a charades performance and asks for the movie. A cup-shuffling game is broken into steps, with two example turns before Gemini tries one on its own. Google adds candidly that the model does not always get the cup game right.

You can borrow the same structure today with any assistant that accepts images:

```prompt
[Image 1] [Image 2] [Image 3]
These three photos were taken a few seconds apart.
Describe what changes from one to the next, then explain
what you think is happening. Say how confident you are.
```

The step that matters most is the last line. Asking a model to state its confidence makes guesses easier to spot.

> **Try it:** Take three photos of a simple process at home, such as making tea, and ask an image-capable assistant to describe the sequence and name the step most likely to go wrong.

## Built-in caution

Google says Gemini has had the most thorough safety evaluations of any Google AI model so far, including tests for bias and toxicity and studies of risks such as cyberattacks and persuasion. It is also holding Ultra back while external red teams (testers paid to find failures) probe it and while it refines the model further, including with human feedback. That is why the most impressive results belong to a model the public cannot use yet.

## How does Gemini compare with GPT-4 so far?

On paper, Gemini Ultra comes out slightly ahead on many tests. Google says Ultra beats the previous best published results on 30 of 32 widely used benchmarks, and its technical report compares it directly with GPT-4. The best-known number is on MMLU, an exam-style test spanning 57 subjects, from law to physics. Ultra scores 90.0%, just above the 89.8% the test's authors estimate for human experts. GPT-4's previous reported result was 86.4%.

Reading the fine print changes the picture a little. The method matters as much as the model:

| MMLU method | Gemini Ultra | GPT-4 |
|---|---|---|
| Chain of thought, 32 samples (CoT@32) | 90.04% | 87.29% |
| Standard 5-shot prompting | 83.7% | 86.4% |

"5-shot" means the model sees five solved questions before answering. "CoT@32" means it writes out its reasoning 32 times and the most consistent answer wins. With the newer method, Ultra leads; with the older standard method, GPT-4 still scores higher. Both readings are in Google's own report. Elsewhere the margins are often narrow; on a set of hard competition math problems, for example, Ultra solves 32% against GPT-4's 30%.

Two more points keep things in proportion. First, the model you can use today is Gemini Pro, not Ultra. Google's report places Pro ahead of OpenAI's GPT-3.5 and comparable to several leading models, but not ahead of GPT-4. Second, benchmarks are run by the company that built the model, under conditions it chooses. They are a useful signal, but your own tasks are the real test.

A fair summary for today: Bard has just received what Google calls its largest upgrade, and Gemini Ultra looks like a serious rival to GPT-4 on paper. Many expect a clearer comparison once Ultra is available early next year and independent testers can try both side by side.

## Sources

- Introducing Gemini: Google’s most capable AI model yet: https://blog.google/technology/ai/google-gemini-ai/
- Gemini: A Family of Highly Capable Multimodal Models: https://storage.googleapis.com/deepmind-media/gemini/gemini_1_report.pdf
- How it's Made: Interacting with Gemini through multimodal prompting: https://developers.googleblog.com/2023/12/how-its-made-gemini-multimodal-prompting.html