# What was the McCulloch-Pitts neuron, the first model of a neural network?

Date: 2019-10-23 · Author: Robert Scales · https://guidetoartificialintelligence.com/en/learn/what-was-the-mcculloch-pitts-neuron/

**Answer:** The McCulloch-Pitts neuron is a simplified model of a brain cell, published in 1943 by the neurophysiologist Warren McCulloch and the young logician Walter Pitts: it adds up its incoming signals and "fires" only if they reach a threshold. They showed that networks of these on-off units can carry out logical reasoning, which makes their paper the starting point of artificial neural networks.

Every neural network in use today, from the photo tagger on your phone to large language models, descends from a 1943 paper with a forbidding title: "A Logical Calculus of the Ideas Immanent in Nervous Activity." Its authors, Warren McCulloch and Walter Pitts, took a living nerve cell, stripped it down to its simplest behavior and treated it as a tiny logical switch. Then they showed that networks of such switches could, in principle, compute anything that can be written as logic.

Their model neuron receives signals from other neurons. If enough of them arrive at once to pass a fixed threshold, it fires; if not, it stays silent. That is the whole idea, and it turned out to be enough to start a field.

## Who wrote it, and where

The two authors made an unlikely pair. Warren McCulloch was a neurophysiologist in the psychiatry department of the University of Illinois College of Medicine, based at the Illinois Neuropsychiatric Institute in Chicago. Walter Pitts was a young logician at the University of Chicago, where he had studied with the philosopher Rudolf Carnap. McCulloch brought the questions about the brain; Pitts brought the tools of mathematical logic.

The paper appeared in 1943 in the *Bulletin of Mathematical Biophysics*, volume 5, pages 115 to 133. It is dense and full of symbolic logic. That autumn Pitts moved to the Massachusetts Institute of Technology to work with the mathematician Norbert Wiener, which put the model in front of the small group of engineers and scientists who were about to found cybernetics.

## How the model neuron works

McCulloch and Pitts started from what was known about real neurons. A nerve cell either sends an impulse or it does not; there is no half-firing. This "all-or-none" behavior suggested that a neuron's activity could be treated like a statement in logic, which is either true or false.

They made five simplifying assumptions:

1. A neuron's activity is all or none.
2. A fixed number of input connections (synapses) must be excited at the same moment to make a neuron fire.
3. The only delay that matters is the brief pause at each synapse.
4. If any inhibitory input is active, the neuron cannot fire at all.
5. The wiring of the network never changes.

With those rules, a neuron becomes a small logic gate. Here is a worked example with two inputs, A and B, each either on (1) or off (0):

| A | B | Sum | Fires if threshold is 2 (AND) | Fires if threshold is 1 (OR) |
|---|---|---|---|---|
| 0 | 0 | 0 | no | no |
| 0 | 1 | 1 | no | yes |
| 1 | 0 | 1 | no | yes |
| 1 | 1 | 2 | yes | yes |

Change only the threshold and the same unit switches from "A and B" to "A or B." Add an inhibitory input and you can build "A but not B." Since AND, OR and NOT are enough to build any logical expression, a large enough network of these units can compute any such expression.

> **Try it:** Using the table as a guide, design a three-input neuron that fires when at least two of its three inputs are on (a "majority vote"). What threshold do you need? Then add one inhibitory input that can veto the vote, and check all the cases on paper.

## Memory, loops and the brain

The paper's second half deals with networks that contain loops, which the authors called "circles," where activity can travel around and come back. A loop can keep a signal alive after the original input has gone, so the network holds a trace of the past. That is a simple form of memory. The authors also used their nets to explain a curious sensation: if a cold object touches your skin only briefly, you may feel heat, while a longer touch feels only cold.

They ended with a striking link to Alan Turing's 1936 work. A network of their neurons, given a tape and the means to read and write on it, could compute exactly the same numbers as a Turing machine. In other words, a brain built from such idealized neurons would be, in principle, a universal computer.

## Why it mattered

The McCulloch-Pitts paper gave two young fields a shared language. To brain scientists, it suggested that thinking could be studied as information processing. To engineers and mathematicians, it suggested that machines built from simple switching units could imitate mental activity. Wiener recalled showing Pitts the vacuum tubes used in new electronic equipment and pointing out that they were ideal parts for building his neuron circuits in metal. Wiener's 1948 book *Cybernetics* discusses their work at length.

Its biggest gap was learning. In the 1943 model the connections are fixed; nothing adjusts with experience. Others filled that gap. Donald Hebb proposed that connections between neurons that fire together grow stronger, and Frank Rosenblatt later built the perceptron, a network of threshold units whose connection strengths, or weights, change during training.

## How is the McCulloch-Pitts neuron related to today's deep learning?

A modern artificial neuron is a direct descendant. It still adds up its inputs and passes the result through a function that decides how strongly to respond. The main changes are that each input now has its own adjustable weight, the sharp on-off threshold is usually replaced by a smoother curve, and the weights are learned from data rather than set by hand.

Deep learning, the approach behind the leading image recognition, speech recognition and language systems of 2019, including OpenAI's GPT-2, stacks millions of these units in many layers and tunes their weights using huge amounts of data. The scale would have been unthinkable in 1943, but the building block is recognizably the same: a simple unit that weighs evidence against a threshold.

It is worth remembering, too, how much McCulloch and Pitts left out. Real neurons are far more complex than their model, and today's artificial networks are not faithful copies of brains. The lasting contribution of 1943 is not biological accuracy but a powerful idea: intelligence-like behavior can arise from many simple parts wired together.

## Sources

- A Logical Calculus of the Ideas Immanent in Nervous Activity: https://www.cs.cmu.edu/~./epxing/Class/10715/reading/McCulloch.and.Pitts.pdf
- Artificial neuron: https://en.wikipedia.org/w/index.php?title=Artificial_neuron&oldid=916999097
- Cybernetics: or Control and Communication in the Animal and the Machine: https://archive.org/details/mit_press_book_9780262355902