# What is machine learning, and how is it different from ordinary programming?

Date: 2020-03-16 · Author: Robert Scales · https://guidetoartificialintelligence.com/en/learn/machine-learning-different-ordinary-programming/

**Answer:** In ordinary programming, a person writes the rules and the computer follows them. In machine learning, a person supplies examples with the right answers, and the computer works out its own rules, called a model, that it can then apply to new cases.

Most software is a list of instructions. A programmer decides exactly what should happen in each situation and writes it down: if the cart total is over $50, shipping is free; if the password is wrong three times, lock the account. The computer is fast and tireless, but it only knows what someone told it.

Machine learning flips the order. Instead of writing the rules, you collect examples where the right answer is already known and let a program search for a pattern that connects the inputs to the answers. The pattern it finds is called a model. Once you have it, you can hand the model a new case it has never seen and get a prediction. The best way to feel the difference is to look at one problem solved both ways.

## The same problem, two approaches: a spam filter

Unwanted email is a good test case because everyone knows what spam looks like, yet it is surprisingly hard to describe in rules.

**The ordinary programming way.** You read some spam and write rules for what you see. In his 2002 essay "A Plan for Spam", programmer Paul Graham describes doing exactly this. Rules like "flag messages whose subject is all capital letters" help at first. He reports that one rule, looking for the word "click", caught 79.7% of his spam, but it also flagged 1.2% of his legitimate mail. After about six months of adding rules, he found the last few percent of spam very hard to catch, and every stricter rule wrongly flagged more real messages.

**The machine learning way.** Graham then switched methods. He gathered two piles of mail, about 4,000 spam messages and about 4,000 normal ones, and let a program count how often each word appeared in each pile. From those counts it calculated, for every word, how strongly it pointed toward spam. New mail was scored by combining the scores of its most telling words. He reports that this filter missed fewer than 5 spams in 1,000 while flagging no good mail in his tests. It also found clues he had never thought of, such as "ff0000", the web code for bright red.

Here is the contrast side by side:

| | Ordinary programming | Machine learning |
|---|---|---|
| What a person provides | The rules | Examples with answers |
| What the computer produces | Answers | A model (learned rules) |
| How it improves | Someone writes more rules | You add more or better examples |
| Spam example | "If subject is all caps, mark as spam" | Learns word scores from 8,000 sorted emails |

## The vocabulary, in plain words

Google's free Machine Learning Crash Course uses spam detection as its running example, and its terms are worth learning because you will meet them everywhere.

- **Label:** the answer you want predicted. For email, "spam" or "not spam".
- **Feature:** a piece of input information the model can use. For email, the words in the message, the sender's address, the time it was sent.
- **Labeled example:** one case with both its features and its label, such as an email a user has marked as spam.
- **Model:** the learned relationship between features and label.
- **Inference:** using the finished model on new, unlabeled cases, like the messages arriving in your inbox tomorrow.

A useful way to remember it: in ordinary programming, you write the "if… then…" yourself. In machine learning, the examples write it for you, in the form of numbers.

## When to use which

Machine learning is not always better. If the rules are clear and stable, like calculating sales tax, ordinary code is simpler, easier to check and easier to explain. Machine learning shines when the rules are hard to put into words, keep changing, or depend on many small signals at once. Spam fits all three: spammers adapt, and no single word gives them away.

It also has a cost. A model is only as good as its examples. If your labeled emails come from one type of user, the filter may work badly for others. And the learned rules are usually a long list of numbers rather than sentences a person can read.

> **Try it:** Open your email's spam folder and pick ten messages. Write down three rules you think would catch them all. Then check your inbox: how many real messages would your rules also catch? That trade-off is exactly what pushed Graham toward learning from examples.

## What does it mean to train a model?

Training is the step where the model is built from the examples. Google's course defines it simply: training means finding good values for the model's internal numbers, usually called weights, using labeled examples. A weight says how much one feature should count. In a spam model, the word "click" might end up with a large weight toward spam and the word "invoice" a small one.

The program learns those weights by trial and correction. It starts with rough values, makes a prediction for each example, and measures how wrong it was with a number called the loss. A perfect prediction has a loss of zero; a bad one has a large loss. The program then nudges the weights in the direction that lowers the loss, and repeats across all the examples many times. When the average loss stops falling, training is done.

Graham's filter is a simple form of this: counting words gives each one a score, and those scores act as weights. Larger systems, such as the neural networks behind recent language models, train in the same spirit with millions or billions of weights. The finished model is then tested on examples it never saw during training, because the real goal is not to memorize the old emails but to sort tomorrow's mail correctly.

## Sources

- Framing: Key ML Terminology (Machine Learning Crash Course): https://developers.google.com/machine-learning/crash-course/framing/ml-terminology
- A Plan for Spam: http://www.paulgraham.com/spam.html
- Descending into ML: Training and Loss (Machine Learning Crash Course): https://developers.google.com/machine-learning/crash-course/descending-into-ml/training-and-loss