# What was SNARC, the 1951 machine often called the first neural network?

Date: 2020-01-11 · Author: Robert Scales · https://guidetoartificialintelligence.com/en/learn/what-was-snarc-first-neural-network-machine/

**Answer:** SNARC, the Stochastic Neural Analog Reinforcement Calculator, was a learning machine built in the summer of 1951 at Harvard by two Princeton graduate students, Marvin Minsky and Dean Edmonds. About forty artificial "neurons" made from vacuum tubes, motors and clutches learned, through rewards, to guide a simulated rat through a maze, making it one of the first electronic machines that learned from experience.

In the summer of 1951, two young graduate students spent their vacation in a Harvard laboratory soldering together a machine the size of a grand piano. It had hundreds of vacuum tubes, a tangle of wires and a row of knobs that turned by themselves. Its job was to learn its way through a maze. Its name was SNARC, short for Stochastic Neural Analog Reinforcement Calculator.

The builders were Marvin Minsky and Dean Edmonds. SNARC is often described as the first artificial neural network built as a physical machine: a set of simple, interconnected units, loosely modeled on brain cells, whose connections changed with experience. Minsky, who died in 2016, went on to co-found the field of artificial intelligence, and this machine was his first serious attempt to build something that could learn.

## Who built it, and how it came about

Minsky studied mathematics at Harvard as an undergraduate in the late 1940s, then moved to Princeton for graduate school in 1950. As a student he had been captivated by a 1943 paper by the neurophysiologist Warren McCulloch and the mathematician Walter Pitts, which described brain cells as simple switching units that could, when connected, perform logical operations. Minsky wanted to build such a network for real, and to make it learn.

At Princeton he met Dean Edmonds, a physics graduate student with a talent for electronics. Minsky described the design to George Miller, a psychologist at Harvard, who encouraged him to try it. Miller found a few thousand dollars of funding from the U.S. Office of Naval Research. That summer, 1951, Minsky and Edmonds moved into a Harvard lab to build the machine.

## What the name means

Each word in the name describes part of the design:

| Word | Meaning | In SNARC |
|---|---|---|
| Stochastic | Involving chance | Each connection passed signals only with a certain probability |
| Neural | Like nerve cells | About forty units acted as simple artificial neurons |
| Analog | Using smoothly varying physical quantities | Probabilities were stored as knob positions, not digital numbers |
| Reinforcement | Strengthening by reward | Rewarded behavior became more likely to happen again |
| Calculator | A computing device | It was a special-purpose machine, not a programmable computer |

## How it worked

Minsky's plan called for about forty neurons, each built from six vacuum tubes and a motor. The finished machine had roughly three hundred tubes, many motors and a set of electric clutches the two students machined themselves.

Every connection in the network had a control knob, technically a potentiometer, like the volume knob on a radio. The knob's position set the probability that a signal arriving at that connection would be passed along. This was the machine's long-term memory. Each connection also had a capacitor, a component that holds an electrical charge for a short time. If the connection had just carried a signal, the capacitor "remembered" it for a few seconds. That was short-term memory.

Learning happened like this. When the machine did something right, the operator pressed a reward button, or an automatic signal did it. A motor then turned a long chain linked to all forty knobs. Wherever a capacitor showed that its connection had just been used, a clutch engaged and the chain turned that knob a little, making the connection more likely to fire next time. Connections that had not been used were left alone. The motor that drove the chain was a surplus automatic pilot from a B-24 bomber.

This is essentially the idea Donald Hebb had published two years earlier: connections that take part in successful activity get stronger.

## The rat in the maze

SNARC did not move a physical robot. Instead, a pattern of lights showed the position of an imaginary "rat" moving through a maze inside the network. At first the rat wandered at random. Each time it moved toward the goal, the reward strengthened the connections responsible, and over many runs it made fewer wrong turns.

Then came a surprise. Because of an accident in the wiring, the builders found they could run two or three rats in the maze at the same time, and the rats seemed to influence one another: when one found a good route, the others tended to follow. Minsky recalled in a 1981 *New Yorker* profile, "We sort of quit science for a while to watch the machine."

The random wiring had another benefit. With hundreds of tubes and thousands of soldered joints, something was usually broken, but losing one neuron made little difference to the whole. That tolerance of damage is a property brains share.

> **Try it:** Draw a simple maze with four junctions on paper. At each junction, give the left and right turns 5 "points" each. Run a coin-flip rat through it ten times, adding one point to every turn used on runs that reach the exit. Then bias each coin flip by the points. Does the rat improve?

## Why it mattered

SNARC was one of the first electronic machines that learned from experience, and perhaps the very first. It showed that the abstract ideas of McCulloch, Pitts and Hebb could be turned into working hardware.

Minsky himself drew a sober lesson from it. The machine could learn a habit, he said, but it could never form a plan, because it had no way of predicting what would happen next. For his doctoral thesis he explored networks with a second kind of memory to add prediction, and over time he concluded that brain-like networks would need far more neurons than anyone could afford. He turned toward symbolic approaches to AI, which work with explicit concepts and rules. The machine itself did not survive: Minsky lent it to students at Dartmouth, and it was lost except for a single neuron.

## How is SNARC different from the neural networks we use today?

The core idea is the same: a network of simple units whose connection strengths change with experience, so that knowledge lives in the connections rather than in written rules. SNARC's reward-driven learning is also a direct ancestor of reinforcement learning, the approach behind recent game-playing systems.

The differences are mostly of scale and method. SNARC had about forty adjustable connections; the image and language networks in the news at the start of 2020 have millions or billions, simulated in software on fast chips rather than built from tubes and motors. SNARC strengthened only recently used connections after a reward. Most modern networks instead measure how wrong each output is and adjust every connection by a precise amount to reduce that error.

What has not changed is the question Minsky raised in 1951: is learning habits enough, or does intelligence also need the ability to plan and predict? Researchers are still debating it.

## Sources

- 1951 - SNARC Maze Solver - Minsky / Edmonds (American): https://cyberneticzoo.com/?p=1053
- Learning Machine, 1951: https://digitaleditions.walsworth.com/article/Learning+Machine%2C+1951/3370943/584879/article.html
- Stochastic Neural Analog Reinforcement Calculator: https://en.wikipedia.org/wiki/Stochastic_Neural_Analog_Reinforcement_Calculator