What did Minsky and Papert's book Perceptrons prove, and did it really stall neural networks?

Few books in computer science have a reputation as dramatic as this one. Ask about the history of neural networks and you will often hear a short version: in 1969, two MIT professors proved that neural networks could not even learn a simple logical function, and funding vanished for fifteen years. The real story is more interesting, more technical and more human.

The background: Rosenblatt's perceptron

In 1958 the psychologist Frank Rosenblatt introduced the perceptron, a machine loosely modeled on brain cells. It takes many inputs, such as the light and dark spots of an image, multiplies each by a weight that says how much it matters, adds them up and gives a yes-or-no answer depending on whether the total passes a threshold. Its appeal was that it could learn: when it made a mistake, a simple rule nudged the weights, and Rosenblatt and his colleagues proved that this rule would eventually find the right weights whenever such weights existed.

The idea drew a great deal of attention. Newspaper reports and Rosenblatt's own statements suggested that such networks might soon recognize images and much more. At the same time, a rival approach, symbolic AI, which works with explicit rules and logic, was growing, and the two camps competed for funding and scarce computer time. Minsky and Rosenblatt had known each other since their school days at the Bronx High School of Science, and they argued loudly at conferences while staying on friendly terms.

What the book set out to do

Marvin Minsky and Seymour Papert, both at MIT, published Perceptrons: An Introduction to Computational Geometry in 1969 through MIT Press. They dedicated it to Rosenblatt. Instead of building machines, they asked a mathematician's question: exactly which patterns can a perceptron recognize, and at what cost?

They studied a specific, idealized version: an input "retina," a single layer of simple feature detectors, and one output unit. Crucially, they assumed each detector looks at only a small patch of the input, since the attraction of such networks was that many local pieces of evidence could be combined in parallel. At least one chapter circulated early as an MIT memo: the 1968 reprint of the chapter on learning compares the perceptron with simply storing every example and notes that the perceptron works perfectly only when the examples can be split by a straight dividing line or its higher-dimensional equivalent. Mathematicians call such data "linearly separable."

The worked example: XOR and parity

The clearest way to see the limit is the function called XOR, "exclusive or." It answers yes when exactly one of two inputs is on.

Plot these four points on a square, with the "yes" cases at two opposite corners and the "no" cases at the other two. A single perceptron draws one straight line and says yes on one side. No single straight line can put both yes corners on one side and both no corners on the other, so no choice of weights works.

XOR is the two-input case of parity: is the number of active inputs odd or even? Minsky and Papert proved that a perceptron of their limited kind cannot compute parity unless at least one detector looks at the whole input. They also showed that deciding whether a figure is connected, all in one piece, requires detectors whose size grows impractically large. The book's cover makes the point with tangled figures whose connectedness is hard for a person to judge at a glance.

The XOR myth

The popular version says the book proved neural networks cannot do XOR, and so the field was abandoned. That is misleading. It was already well known, going back to the work of Warren McCulloch and Walter Pitts, that networks of such units with more than one layer can compute any logical function, and the book itself says so. What Minsky and Papert proved was subtler: for certain patterns, even a network with a hidden layer needs at least one unit wired to every input, which defeated the hope of building powerful networks from small local parts.

In their final chapter, however, the authors went further than their proofs. They suggested that extending perceptrons to many layers would probably be a "sterile" direction, and many readers took that guess as a verdict.

What happened next

Early reviews were respectful, and some were enthusiastic: Allen Newell, reviewing it for Science, called it a great book. Neural network researchers objected that the authors had studied a much narrower machine than Rosenblatt's and given it his name. Still, the book is often credited with a long decline in neural network research through the 1970s and early 1980s, a period when symbolic AI flourished and neural network work continued only in small groups outside the mainstream. It came during a broader cooling of support for AI, part of what is now called the first AI winter.

The revival came in the mid-1980s. In a 1986 report on networks with hidden layers, David Rumelhart and his colleagues claimed to have overcome these problems and argued that Minsky and Papert's pessimism about learning in multilayer machines had been misplaced. Minsky and Papert replied in an expanded edition in 1987 (some sources date it 1988), defending their analysis.

Why do neural networks work so well in 2019 if Perceptrons showed their limits?

Because the limits applied to a particular kind of network: one layer of small, local detectors feeding a single output. Today's deep networks stack many layers, every unit can draw on the results of the layer below, and all the weights are trained together, typically with a method called backpropagation. A network with even one hidden layer handles XOR easily, as the two-line exercise above shows.

The book's deeper lesson still holds, though. Minsky and Papert insisted on asking what a given architecture can and cannot represent, and how expensive it is. Researchers in 2019 still ask those questions, now about networks with millions of weights, and the answers still decide which designs are worth building.