# What happened at the first DARPA Grand Challenge, when no robot car finished?

Date: 2020-11-02 · Author: Robert Scales · https://guidetoartificialintelligence.com/en/learn/what-happened-at-the-first-darpa-grand-challenge/

**Answer:** On 13 March 2004, the US Defense Advanced Research Projects Agency (DARPA) offered $1 million to any driverless vehicle that could cross about 142 to 150 miles of the Mojave Desert on its own in under ten hours. Fifteen robots started, none got further than about 7.4 miles, and the prize went unclaimed, but the race exposed exactly what autonomous vehicles still needed to learn.

On the morning of Saturday, 13 March 2004, fifteen vehicles lined up in the desert near Barstow, California, with no one at the wheel. Each had to drive itself along a secret route to Primm, Nevada, near the California–Nevada line. By the end of the day, every one of them had crashed, stalled, wandered off course or been switched off. The best had covered about 7.4 miles.

The event was the first DARPA Grand Challenge. DARPA is the research agency of the US Department of Defense. It wanted to speed up the development of autonomous ground vehicles, meaning vehicles that sense their surroundings and steer themselves, because it saw military uses for them. Instead of awarding a research contract, it announced a public race with a $1 million prize.

## The rules of the race

DARPA launched the challenge in 2003; Wikipedia says it was first announced in 2002. The rules were simple and strict:

- The vehicle had to drive itself the whole way, with no remote control and no human help once it started.
- The course had to be completed in under ten hours.
- The exact route stayed secret until shortly before the start, when teams received it as a file of GPS waypoints, a list of map coordinates the vehicle had to pass through.

Sources give slightly different lengths for the course. The Stanford team's published account of the challenge gives 142 miles; Wikipedia describes a route of about 150 miles following Interstate 15. Either way, it was a long drive over dirt roads, dry riverbeds and mountain passes.

According to Stanford's account, 107 teams registered. Before the race, 21 qualifying teams had to drive a one-mile obstacle course at the California Speedway; seven finished it cleanly and eight more did well enough to convince the judges, giving fifteen finalists.

## What happened on race day

DARPA's own final report, issued at 5 p.m. that day, reads like a list of small disasters. Here is a selection, ordered by how far each vehicle went:

| Vehicle | Distance | What stopped it |
|---|---|---|
| Red Team (Carnegie Mellon) | 7.4 miles | Went off course on mountain switchbacks, caught on a berm; its front tyres caught fire |
| SciAutonics II | 6.7 miles | Drove into an embankment on Daggett Ridge |
| Digital Auto Drive | 6.0 miles | Paused to let a tow truck pass, then got stuck on a rock |
| The Golem Group | 5.2 miles | Could not find enough throttle to climb a steep hill |
| Team Caltech | 1.3 miles | Veered through a fence and could not get back |
| Team ENSCO | 0.2 miles | Flipped over on its first 90-degree turn |
| Axion Racing | Start area | Drove in circles the wrong way |

Several others hit a wall leaving the starting chute, had their brakes lock up or got tangled in wire. Two teams withdrew before the start. Wikipedia notes that within three hours of the ten-hour window only four vehicles were still running.

The winner on distance was Sandstorm, the vehicle entered by Carnegie Mellon University's Red Team. Wikipedia gives its distance as 11.78 km (about 7.3 miles) in one place and 7.4 miles in another; DARPA's report says mile 7.4. Stanford's paper puts it more bluntly: no robot covered more than 5% of the course. DARPA's overview page summed it up in one line: no team completed the route.

> **Try it:** Ask a friend to read you turn-by-turn directions to a place you have never been, but only as GPS coordinates, with no street names and no landmarks. Now imagine your eyes are a few laser beams that can see 20 metres ahead. That is roughly the information a 2004 desert robot had to work with.

## Why a failed race still mattered

The race was widely judged a failure, and on its own terms it was: nobody won. But DARPA did not cancel the contest. It announced a second Grand Challenge for 2005 and doubled the prize to $2 million.

The failure was useful because it was so public and so specific. Every team could see exactly where its vehicle had gone wrong, and so could everyone else. DARPA's deputy program manager Tom Strat observed that some vehicles followed their GPS waypoints accurately but could not sense obstacles, while others sensed obstacles well but struggled to stay on the route, or saw obstacles that were not there. That is a precise diagnosis of the core problem.

Many teams came back. When the second race was held in October 2005, all but one of the 23 finalists drove further than Sandstorm had in 2004, and five vehicles finished the whole course. A field that went from 7.4 miles to more than 130 miles in nineteen months is the clearest sign of how much the first race taught.

## Why was driving through the desert so hard for a computer?

Because driving combines several hard problems that must all work at once. A robot car has to know where it is (localisation), see what is in front of it (perception), decide on a safe path (planning) and turn the wheel and press the pedals accurately (control). GPS readings are never perfectly exact, and on a narrow mountain track a small error is the difference between a dirt road and a ditch.

Perception was the biggest gap. To a laser scanner, a bush, a rock, a dip in the road and a shadow can all look alike, and a robot that brakes for every shadow never gets anywhere. Teams also had to cope with dust, vibration that loosened wires and sensors bouncing as the vehicle bounced. Humans handle all of this without thinking, using years of experience.

The breakthrough that followed came largely from machine learning: letting the vehicle learn from data what drivable ground looks like, rather than relying only on rules written by engineers. That approach is what carried Stanford to victory in the 2005 race.

## Sources

- Final Data from DARPA Grand Challenge (as of 5:00 p.m. PST, March 13, 2004): http://web.archive.org/web/20040807085842/http://www.darpa.mil/grandchallenge04/media/final_data.pdf
- Stanley: The Robot that Won the DARPA Grand Challenge: http://robots.stanford.edu/papers/thrun.stanley05.pdf
- DARPA Grand Challenge (2004): https://en.wikipedia.org/w/index.php?title=DARPA_Grand_Challenge_%282004%29&oldid=970676601