Self-Driving Cars Won’t Go Mainstream Until We Solve This Problem

Autonomous vehicles have come a long way since none of the entrants of the 2004 DARPA Grand Challenge managed to even finish the course. Now there are numerous trials on public roads and Tesla has released a beta of its Full Self Driving into the wild with selected car owners. But there’s a major problem that needs to be addressed before we can let our cars do all the driving for us, and it’s not a technical one. It’s ethical.

The problem goes like this. Imagine an autonomous vehicle discovers a pedestrian in its path and is travelling too fast to stop safely. It has two choices. Either it can try to stop but will probably run over and kill the pedestrian. Or it can swerve away from the pedestrian but will probably hit a barrier that will hurt or kill the occupant. Which does it choose?

This sounds like a simple choice, but the closer you look, the more nuances appear. What if the pedestrian is a mother and child? What if the car is full of children? What if hard braking is likely to mean the car behind has an accident too? In other words, your self-driving car will have to calculate the least harm to human life and that might mean it decides the least harm would be to kill its occupants.

We make these decisions naturally as human drivers. A dog runs into the road. We brake and try to avoid the animal, but most of us would choose to kill the dog rather than harm ourselves and our car occupants. We would try to miss the dog, but not at the expense of our own life. If it’s a child in the road, we’d probably be more likely to allow injury to ourselves to prevent or reduce injury to the child. But we still wouldn’t swerve into the path of an oncoming vehicle to do so.

The question is not just about the least amount of injury, though. In the event of an accident, who is responsible? If the pedestrian has been jaywalking, perhaps it’s their fault, and they deserve to be injured more than the car occupants. But if the autonomous driving system has decided who to hurt and who to harm, is it the fault of the driver, the manufacturer of the car, or the company that programmed the autonomous driving software?

We are already seeing some hints of this argument arising when Tesla
TSLA
cars using autosteer and adaptive cruise control have accidents. It’s not a common occurrence, but when an Autopilot-related crash does happen the question is whether the driver should have done more, or the system failed to do its job properly and therefore Tesla is liable. This is not a clear situation, even if Tesla might argue that Autopilot won’t cause accidents, when used according to instructions. Similarly, Uber halted the trials of its driverless taxis about three years ago when one killed a pedestrian, to assess the implications.

We also can’t just assume that the answers to these questions are the same wherever you live, because they’re not like laws of physics, which are identical anywhere in the world. Research published in Nature a few years ago showed that these ethical decisions aren’t universal. They vary with cultures. The survey, using MIT’s Moral Machine, covered 2.3 million people around the world and made some interesting discoveries. People in more prosperous countries would be less likely to spare a jaywalking pedestrian than those in poorer regions, for example. You can still contribute to the Moral Machine and see the scenarios it proposes on its dedicated website.

Developers of autonomous driving systems are at least taking the ethical dimensions into account. Google
GOOG
is known to hire people for its Waymo developer team that add ethical skills to the mix. However, it’s essential this problem is addressed before autonomous driving goes mainstream. The benefits of doing so would be great because most studies confirm that self-driving cars will be safer than human drivers, due to not getting tired or becoming distracted by their phones. In fact, there’s a handy Rand Corporation tool where you can calculate how many lives will be saved depending on a variety of parameters. Even a basic scenario of autonomous vehicles being just 10% safer than human drivers would save 600,000 lives a year in the USA, and most theorists reckon self-driving cars will be much safer than that.

Overall, we must not assume that the problem with self-driving cars is purely a technical one. Obviously, the cornerstone is the ability to navigate road layouts and markings successfully, with the ability to vary with conditions like traffic speed and weather. But once they have achieved that skill, which now seems to be extremely close, the ethical and moral dimensions will come to the fore. Ethical skills also need to be built into self-driving cars, and until they are, autonomous vehicles will be controversial. But it’s a problem worth solving, because the safety and driving efficiencies will be great as our roads get more and more overcrowded.