Short answer
Think of three nested circles. Artificial intelligence is the widest: any technique that lets a machine do something we would call intelligent. Machine learning sits inside it, covering systems that learn from data instead of following hand-written rules, and deep learning sits inside machine learning, using neural networks with many layers that learn their own features from raw data.
Follow-up question Where do language models fit?
The three terms are often used as if they meant the same thing, and headlines rarely help. They are related, but each one is a subset of the one before it. IBM's explainer compares them to Russian nesting dolls: machine learning is one part of artificial intelligence, deep learning is one part of machine learning, and neural networks are the machinery at the heart of deep learning.
Getting the distinction right is more than vocabulary. It tells you how a system was built, what kind of data it needed, and what sorts of mistakes to expect from it.
Artificial intelligence: the widest circle
Artificial intelligence (AI) is the broad goal of making machines perform tasks we associate with human intelligence, such as playing games, recognizing speech, translating or making decisions. It says nothing about how. A program can be AI even if every one of its decisions comes from rules a person typed in.
For decades, much AI worked that way. A classic chess program searches through possible moves and scores each resulting position with a formula written by people: so many points for a queen, so many for a rook, a bonus for controlling the center. The program can play very well, but nothing in it learns. If its play is weak, a programmer has to change the rules.
Everything built today is still "narrow" AI, meaning it does one kind of task. General AI, a machine as broadly capable as a person, does not exist.
Machine learning: learning rules from data
Machine learning is the part of AI where the system works out its own rules from examples. The idea is old. In 1959 Arthur Samuel of IBM published a paper whose title used the phrase "machine learning"; he reported a checkers program that learned to play better than he did after 8 to 10 hours of playing time, partly by adjusting its own scoring formula as it went.
A spam filter is a good modern example. Instead of writing rules like "block emails containing free money", engineers collect thousands of emails already labeled spam or not spam. The learning algorithm finds which features, such as certain words, links or senders, predict spam, and it can be retrained as spammers change tactics.
In classic machine learning, people still choose the features the algorithm looks at. IBM's explainer stresses this point: human experts decide which characteristics distinguish the categories, and the data is usually tidy and structured.
Deep learning: learning the features too
Deep learning is machine learning done with neural networks that have many layers. A neural network is built from simple units that each take inputs, multiply them by weights (numbers that say how much each input matters), add them up and pass a signal on if the total crosses a threshold. IBM's working definition calls a network "deep" when it has more than three layers, counting input and output.
The important difference is that a deep network learns its own features from raw data such as pixels or text. Nobody tells an image recognizer to look for whiskers or wheels. Given enough labeled photos, its early layers learn to detect edges, later layers combine them into shapes, and the last layers recognize objects. The trade-off is that deep learning usually needs much more data and computing power than classic machine learning.
A worked example: one table, four systems
| System | Circle | Where its "knowledge" comes from |
|---|---|---|
| Rule-based chess program | AI only | Search plus scoring rules written by people |
| Spam filter | Machine learning | Patterns learned from labeled emails, using features people chose |
| Photo recognizer | Deep learning | Many-layered network learning features directly from pixels |
| GPT-3 | Deep learning | A very large network trained to predict the next word in huge amounts of text |
Try it: Pick three apps on your phone, such as a map, a photo album and a keyboard. For each feature you use, guess which circle it belongs to: fixed rules, learning from data with chosen features, or deep learning from raw data. Then look for a clue, such as whether the feature improves the more you use it.
Where do language models fit?
Language models sit in the innermost circle. A language model is a system trained to predict the next word, or piece of a word, given the text so far. Today's leading language models are deep neural networks, so they are a kind of deep learning, which makes them a kind of machine learning and therefore a kind of AI.
GPT-3 is the best-known example. According to OpenAI's paper, it has 175 billion parameters, the adjustable weights inside the network, and was trained on about 300 billion tokens of text drawn largely from a filtered web crawl, plus books and Wikipedia. Nobody wrote rules of grammar or facts into it. It learned patterns of language simply by predicting the next token over and over, and the same paper shows it can then perform new tasks from a few examples in the prompt.
That origin explains both its strengths and its weaknesses. Because it learned from text rather than from rules, it is flexible and fluent. Because it predicts likely words rather than checking facts, it can be confidently wrong. Knowing which circle a system belongs to is a good first step in knowing how far to trust it.
Sources
- AI vs. Machine Learning vs. Deep Learning vs. Neural Networks: What's the Difference? — IBM Cloud Blog (Eda Kavlakoglu), May 27, 2020
- Some Studies in Machine Learning Using the Game of Checkers — Arthur L. Samuel, IBM Journal of Research and Development, vol. 3, no. 3 (reprint hosted at MIT CSAIL), July 1, 1959
- Language Models are Few-Shot Learners — OpenAI (arXiv), May 28, 2020