The Trolley Problem in Autonomous Driving: Can Morality Be Coded?
In 1967, Philippa Foot (1920–2010) posed an unassuming question about a runaway trolley, five workers on the track, and a lever that could divert the train to kill only one. She meant it as a meditation on the doctrine of double effect, a footnote in the abortion debate. She could not have imagined that six decades later, her thought experiment would haunt the engineering labs of companies racing to put driverless cars on public roads. The trolley is no longer a philosophical abstraction. It is a two-ton machine hurtling through an intersection at forty miles per hour, and the lever has been replaced by a few thousand lines of code that must decide—in milliseconds—which human body absorbs the impact.
The question Foot never resolved is now a design specification. And that transition, from seminar room to software architecture, is where something essential about moral reasoning begins to break apart.
When the Lever Became a Line of Code
Foot's original scenario assumed a moral agent with full knowledge: you see the five workers, you see the one, you know with certainty what pulling the lever will do. Judith Jarvis Thomson (1929–2020) later sharpened the dilemma by introducing the "fat man" variant—would you push a stranger off a bridge to stop the trolley? The brilliance of these thought experiments lay in their radical simplicity. They stripped moral reasoning down to its skeleton: is it permissible to actively cause one death to prevent five?
But a self-driving car operates in nothing like this crystalline certainty. As Heather Roff of the Brookings Institution has argued, autonomous vehicles running on Partially Observed Markov Decision Processes do not confront a single binary choice at a single moment. They make sequential decisions under conditions of radical uncertainty, working from probability distributions about a world their sensors can only partially perceive. The car does not "see" five pedestrians and one cyclist and "choose." It processes fragmentary data—occluded by rain, glare, the unpredictable trajectory of a child chasing a ball—and selects the action most likely to maximize a reward function its engineers defined months or years before the moment of crisis.
This gap between philosophical certainty and computational uncertainty is not a minor technical detail. It is the fault line along which the entire ethical framework cracks open. The trolley problem presupposes precisely what machine perception cannot provide: complete knowledge, a fixed set of outcomes, and a singular moment of decision.
Forty Million Moral Confessions
MIT's Moral Machine experiment, published in Nature in 2018, attempted to crowdsource the answer that philosophy could not settle. The online platform presented users with variations of Foot's dilemma translated into the autonomous vehicle context: should the car swerve to spare an elderly pedestrian at the cost of a young executive? A pregnant woman or a criminal? A group of five or a group of two? Forty million decisions poured in from 233 countries and territories, and the results were revealing—though not in the way the designers may have intended.
Three broad cultural clusters emerged. Respondents from Western nations showed a stronger preference for sparing the young over the old. Southern cluster countries—encompassing much of Latin America and former French colonies—placed greater weight on sparing women and higher-status individuals. Eastern cluster countries, including many in Asia, showed comparatively weaker preferences for sparing the young and a more muted distinction between high and low social status. What the experiment exposed was not a universal morality waiting to be discovered, but the irreducible plurality of moral intuitions shaped by culture, religion, economic development, and the strength of the rule of law.
The data correlated with measurable institutional variables. Countries with stronger rule of law showed greater preference for sparing law-abiding pedestrians over jaywalkers. Countries with higher economic inequality showed a stronger tendency to spare higher-status individuals. The Moral Machine did not solve the trolley problem. It demonstrated, with empirical precision, that any algorithmic "solution" would inevitably embed the moral preferences of whichever culture, class, or engineering team wrote the code.
The Duty of Care That Dissolves the Dilemma
Chris Gerdes, professor emeritus of mechanical engineering at Stanford, has proposed an alternative that sidesteps the entire trolley framework. His argument, developed in collaboration with Ford Motor Company, is disarmingly simple: the social contract already encoded in traffic law provides sufficient ethical guidance. An autonomous vehicle owes a duty of care to all road users. It must follow traffic laws except when violation is necessary to avoid a collision, and it must never resolve a conflict by dragging uninvolved parties into the dilemma.
If a cyclist blows through a red light, the car owes that cyclist a duty of care and must do everything within its physical limits to avoid impact—but it cannot swerve into oncoming traffic and endanger a driver who was obeying every rule. The moral calculus of "kill one to save five" is replaced by a principle of non-contamination: do not transfer risk to those who bear no responsibility for the crisis.
This approach has its own elegance, and it resonates with Immanuel Kant's (1724–1804) insistence that persons must never be treated merely as means to an end. The utilitarian calculation that assigns numerical weight to human lives—the very calculus the trolley problem invites—violates something fundamental about the moral status of individuals. No algorithm should be empowered to decide that your life is worth less than the aggregate of three strangers. Yet Gerdes's framework also reveals something troubling: it works precisely because it refuses to answer the hardest question. When the collision is truly unavoidable and every possible outcome involves harm, the duty-of-care model reaches its own boundary. It tells us whom we cannot sacrifice. It does not tell us what to do when sacrifice is inescapable.
Whose Morality, Whose Machine
Germany's Ethics Commission on Automated and Connected Driving published twenty guidelines in 2017 that attempted to draw exactly these boundaries. Rule number nine stated unequivocally: "In the event of unavoidable accident situations, any distinction based on personal features (age, gender, physical or mental constitution) is strictly prohibited." The principle is admirable. But it exists in direct tension with the empirical findings of the Moral Machine, which showed that actual human moral preferences are saturated with precisely such distinctions. We say we want impartial algorithms. Our intuitions betray a very different appetite.
On March 18, 2018, a self-driving Uber struck and killed Elaine Herzberg as she walked a bicycle across a road in Tempe, Arizona. It was the first recorded pedestrian fatality involving an autonomous vehicle. The car's sensors detected her six seconds before impact. The system classified her successively as an unknown object, then a vehicle, then a bicycle—and each reclassification reset the prediction of her trajectory. The emergency braking function had been disabled to provide a smoother ride. The safety driver was watching a video on her phone. Herzberg did not die because an algorithm solved the trolley problem incorrectly. She died because the system was never designed to treat an ambiguous human figure as worthy of the precautionary principle.
This is the reality the trolley problem obscures. The genuine ethical danger of autonomous vehicles is not the dramatic millisecond decision between two groups of identifiable humans. It is the accumulated weight of thousands of design choices—sensor thresholds, classification confidence levels, the decision to disable emergency braking for passenger comfort—made long before any crisis occurs. The moral architecture of the machine is not inscribed in a single moment of dilemma. It is distributed across months of engineering meetings, cost-benefit analyses, and corporate risk calculations that rarely, if ever, invoke the language of moral philosophy.
Beyond the Lever
The philosopher Barbara Fried, in her 2012 essay on the trolley problem, argued that we should "kill the trolley problem or let it die," because its artificially constrained structure blinds us to the systemic conditions that produce harm in the first place. Her critique applies with even greater force to the autonomous vehicle debate. The urgent question is not what the car should do in the final second before impact. It is who decides the design parameters that make that final second possible or impossible.
What would it mean to democratize the moral architecture of autonomous systems? Not through a global survey of individual preferences—the Moral Machine showed us where that leads. But through transparent, publicly accountable processes that treat the value functions embedded in machine learning systems as matters of collective deliberation rather than proprietary engineering decisions. The German Ethics Commission's guidelines represent one model, however imperfect. Municipal review boards for algorithmic governance represent another. The essential shift is from asking "what should the car decide?" to asking "who has the right to set the terms of what the car can decide?"
When we code morality into a machine, we do not transcend the messiness of human ethics. We freeze a particular set of assumptions—shaped by the culture, class position, and institutional interests of those who write the code—into a system that then applies those assumptions at scale, without hesitation, without remorse, and without the capacity to feel the weight of what it has done. The trolley problem was never meant to be solved. It was meant to make us uncomfortable with the limits of moral reasoning itself. The autonomous vehicle industry's mistake is not that it failed to solve the problem. It is that it believed the problem was solvable at all—and proceeded to build machines on that belief.
Philippa Foot's thought experiment has outlived its author by nearly two decades. Perhaps its true function was never to generate an answer, but to mark the precise point where our confidence in rational ethics falters. Every self-driving car on a public road is a monument to that faltering—a machine that must act where we ourselves cannot agree on what action means. The lever is in the code now. The question of who holds it has barely been asked.
Post a Comment