Safety Engineers as Executioners: Friedman, Cicero, and Montaigne
If you’re interested in safety engineering and AI, I recommend checking out this fascinating Medium post about the failure of an automated truck company - link
Few ethical issues are thornier and more emotional than the question of putting a monetary value on human life. Utilitarians and most economists believe that one ought to determine exactly how much money we should be willing to spend in order to save a life, and yet this calculation conflicts sharply with most people's moral intuitions, which say that the lives of those closest to us are literally priceless. As we come to a new stage in the coronavirus pandemic, where tough decisions have to be made about triaging patients in overflowing hospitals and the order in which people should be vaccinated, these questions are coming to the fore. This essay will deal with the problem of incommensurability by arguing that making these kinds of calculations isn't something most people should have to do - in fact, the decision-makers to whom we entrust responsibility have a special kind of political status, which separates them from the rest of the population, and to which most people do not aspire. The class of people who make these decisions include both safety engineers and executioners.
Some YouTube videos which are popular among the libertarian right because they make simple and often counter-intuitive arguments in effective ways; they own the libs. One of them is Milton's Friedman's argument about The Cost of Safety. The video begins with a nerdy-looking student makes a bold case against a perceived injustice:
Ford produced [the Pinto] knowing that in any rear-end collision the gas tank would blow up because they had failed to install a plastic block costing $13 in front of the gas tank. And Ford estimated in an internal memo that that would cost about 200 lives a year; they estimated further that the cost of each life would be $200,000. They multiplied and they found that the cost of installing those blocks in each of the cars would be more than the cost of saving those two hundred lives. And over the past seven years the car has been produced and over a thousand lives have been lost. It seems to me that Ford did what would be the right thing according to your policy, and yet that seems to me to be very wrong.
Friedman's response is a classic of the genre. He argues that since anyone would concede that an individual life is not worth spending a billion dollars, the debate is not about principle, but about price - Ford was not morally execrable because they made the calculation, but because they made the calculation wrongly. The inexperienced student squirms under the pressure of Friedman's logic - much to YouTube's entertainment. Friedman's ultimate libertarian and utilitarian point is that individuals should have the opportunity and the choice to exchange money for a reduction in risk to life.
Every one of us separately in this room could at a cost reduce his risk of dying tomorrow. You don't have to walk across the street!
And yet here, I feel, the argument breaks down. It is one thing to say that a Ford safety engineer should assign monetary value to human life, in the abstract and the aggregate. She can analyse probability distributions to come up with expected values, basing her decisions on Ford's balance sheet and the competitive landscape in the car industry. It's quite another to suggest that individuals make the same kinds of calculations about their own lives - or more emotively, that mothers should make the same kind of decisions about their children's lives. Friedman's discussion with the student sets up a paradox - how come Ford can make these calculations if an ordinary person wouldn't? Resolving it requires us to understand that utilitarian calculations are a handy model to understand human decision making, but not necessarily how the process actually works. Rather than imposing this structure on individual decision making, we need to employ a conceptual framework that describes reality: safety engineers and politicians place a monetary value on lives, and private individuals do not.
The economic debate, of course, is relatively recent: but analogous debates are far more ancient. Cicero's most influential work, On Duties, is at its core an attempt to answer a basic question: is the honest course of action also the best one? Is it expedient? Not coincidentally, utilitarianism has its etymological roots in Cicero's word for 'expedient' - utile - and so we can explore Friedman's question, of assigning monetary value to human life, through ancient arguments.
Cicero's conclusion was that with a very few exceptions, we should always act in an honest way, because what is honest is almost always expedient too. There shouldn't be a conflict. But further, he argues that the good man shouldn't even make the calculation. Rather, he suggests that everyone can behave honestly by obeying a few rules of thumb (common morality), and that for this reason:
It is immoral either to weigh true morality against conflicting expediency, or common morality, which is cultivated by those who wish to be considered good men, against what is profitable. But we every-day people must observe and live up to that moral right which comes within the range of our comprehension as jealously as the truly wise men have to observe and live up to that which is morally right in the technical and true sense of the word.
De Officiis, III iv 17
Just as Cicero distinguished between morality that only the truly wise can achieve, and the morality which ordinary people should strive for, I would suggest that only a small group of people in positions of authority have to make awful decisions about human lives. The people who make these decisions, by virtue of those decisions, are a class apart from ordinary people. Cicero's framework can help us make sense of the fact that Friedman is accurately describing the job of the safety engineer, but not the decision-making processes of ordinary people.
A millennium and a half later, Michel de Montaigne riffed on the same theme in his essay On the Useful and the Honourable. Montaigne had withdrawn from the traumatic French Wars of Religion to his study, in an attempt to escape the pressures of public life. Living in a post-Machiavellian world of massacre and conspiracy, he understood that it might be necessary to behave in an immoral way for the good of the State or the people - but that didn't mean it was necessary for him to behave in that way.
So, too, in all polities there are duties which are necessary, yet not merely abject but vicious as well... If vicious deeds should become excusable insofar as we have need of them, necessity effacing their true qualities, we must leave that role to be played by citizens who are more vigorous and less timorous, those prepared to sacrifice their honour and their consciences, as men of yore once sacrificed their lives: for the well-being of their country. Men like me are too weak for that: we accept rules which are easier and less dangerous.
While Montaigne was only willing to do what was honourable as well as useful, the State needs "Public Executioners of High Justice - an office as useful as it is shameful". He put these executioners into a class apart, whose members are responsible for making decisions and taking actions which are immoral, and which, according to Cicero, normal honest people shouldn't even consider.
We can use Montaigne's idea to solve the paradox which Friedman elided: we can explain why the engineers at Ford can trade off lives against money, but a mother cannot. For Friedman, the engineer is no different to anyone else, and the decision she makes about car safety is the same sort of decision anyone makes every time they cross the road. But Montaigne suggests that there exists an "office as useful as it is shameful", and "duties which are necessary, yet not merely abject but vicious as well". Holders of these offices are not the same as ordinary people, and they engage in a different kind of decision-making. Because of this distinction, Friedman's analogy isn't as simple as it seems.
Margaret Thatcher, who was of course an admirer of Friedman, enjoyed comparing the Budget of the United Kingdom to a basket of shopping. Although it allowed her to play up her roots as the daughter of a greengrocer, the analogy between the household and the state (so beloved of Aristotle!) doesn't make economic sense for a number of reasons: you can't assume that what is true for an ordinary person is also true for an entire country. Friedman's analogy, I would suggest, did something similar: it failed to acknowledge that the decisions being made by ordinary people are qualitatively different to those being made by those in certain positions of responsibility.
The arguments of Friedman and Thatcher are attractive because they are simple and counterintuitive: something about them feels wrong, but the logic is unassailable: this, I think, is also part of the appeal of effective altruism. In this particular exchange, Friedman and the student are both wrong for the same reason. Friedman understands that safety engineers have to make utilitarian calculations - but he assumes that ordinary people do so too. The student, on the other hand, feels that ordinary people do not make utilitarian calculations about human lives - they regularly act irrationally - but induces from this that engineers should not either. They have both failed to realise that because ordinary people are not executioners, what is true for one group is not necessarily true for the other: if Montaigne's essay has helped us to see this distinction, then it is indeed both useful and honourable.