On predictive justice

Javert: Prisoner 24601, I regret to inform you that your application for parole has been denied.

Valjean: But Inspector, how can this be? I have expressed repeatedly my regret for my actions and profound desire to be an upstanding citizen.

Javert: Nevertheless, Monsieur, based on the available data, the machine has returned a high probability that you will, in the next five years, re-offend; and in the same manner.

Valjean: A machine made the determination? Inspector, this is incredible! You mean to say that you have a machine so sophisticated, so detailed in its understanding of the human mind – and, now I reflect on it, you must also have obtained, I know not how, an accurate model of my own mind: when I was asleep, perhaps? But no matter – so detailed, as I say, as to be able to compute – faster than I myself can think! – the decisions I will make in the future? Whence came these advances in cognitive science? Have I woken in a new world?

Javert: No, monsieur, you misunderstand. I mean to say that we in the judicial system have a great deal of experience with prisoners and their applications for parole. Suspecting the existence of statistical regularities in the relation between past and future behaviour, we have put into the machine this experience, in the form of the numerical and qualitative characteristics of prisoners and their subsequent behaviour when released on license. Our supposition was indeed rewarded—I am informed that the probability of a type 1 error is less than five percent!—and we now use these regularities to inform our assessment of the risk of re-offending.

Valjean: I confess I do not understand what you mean by “statistical regularities,” and why these should be different from regular regularities. But tell me—from which theory of the mind are you working and what characteristics have you used to determine the parameters of that theory in my case?

Javert: Encore, you misunderstand, and I begin to suspect it is deliberate. Monsieur, we have no “theory of the mind”: none is needed and frankly none is wanted! We would not presume to prejudice the workings of our machine with such a “pre-judgement”. No, the machine is a tabula rasa, an open landscape devoid of feature; only the brute facts of the world are taken into account.

Those brute facts are such as are known about you. Your age, to take the simplest of examples; your gender; then of course, your criminal history. We move on to your wealth; your parents' occupations, your school reports from age 5 to 16; your level of education; your height and weight; and so on and so forth. From your départment to your deportment, we enter in everything that is known.

Valjean: Did you put into the machine that my nephew was hungry, that the bread which I stole was to feed him?

Javert: Bien sûr que non! This is not a fact, monsieur, it is a subjectivity. The reasons for the criminal act are manifold, as numerous as the dark cavities of the criminal mind! We could not begin even to codify them, let alone to ascertain them. No, it is on the facts of the matter, and on the facts alone, that we must rest our judgement.

Valjean: You mean, the machine's judgement?

Javert: Quite so.

Valjean: Inspector, I hope you will forgive me if I fail yet again to comprehend your design. Surely, these things do not determine my future course of action?

Javert: None but you can determine your future course of action, monsieur. But these things do, I am afraid to say, dispose us to a belief one way or the other. Were I inclined to gamble—and I am not!—but if I were, I should end up the richer to the extent that I follow the advice of the machine and the poorer if I do not. And it follows, that society will end up the richer, on the balance of probabilities, by returning you to gaol.

Valjean: I suppose that the state must sometimes be forgiven when it acts in ways that would seem inhuman for an individual; just as we forgive our neighbour, but not so the state, when they act from fear.

Yet, I cannot help but marvel that you have managed to find, even in your great experience, another individual who is, in all the characteristics you have described, identical to me. Do you mean to say that you have one such—surely no more than one!—and that this man stole again; and for this reason I am to lose my liberty?

Javert: Not at all! That would of course be improbable in the extreme. But we have many individuals who are, in many of the characteristics, similar to yourself.

Valjean: “Similar”? How do you mean “similar”?

Javert: Near the same age, for example; perhaps from a small town; with the same level of education, albeit from a different school. They are, in the ways that we have proven to matter, rather like you.

Valjean: “Like” me?

Javert: Oui: they are people like you.

Valjean: People “like me” are criminals?

Javert: Monsieur, you say it as if it is somehow a bad thing to say. Yes! You lot—I apologise, you have riled me—I mean, people similar to you tend to be recidivists. I am sorry but that is how it is. The evidence cannot be denied.

Valjean: Inspector, this is not causality but casuistry. It is Kafkaesque!

Javert: It cannot be Kafkaesque: that would be an anachronism.

Valjean: Very well. Then let me ask about, perhaps, the characteristic of age. You say that age is a relevant concern for your machine's evaluation?

Javert: It is.

Valjean: So you allow, in your determination of my status, the consideration of people whose age is “similar” to mine. I must ask again what you mean by similar. Although there are many people born in 1769, my own year of birth; there are fewer in the same month of that year as me. There are yet fewer born on the very same day; fewer still the same hour; perhaps none the same minute. How, then, have you decided what difference in age counts as “similar”?

Javert: I have not, the machine has.

Valjean: But it cannot possibly have allowed for all possibilities. If the world were such that but a minute in age would make a difference to a life, then the machine would have to admit that only those born no more than seconds apart could count as similar. But since, I would imagine, your experience contains only people born at least minutes apart, there can be no similar people in your data. The machine must not even have considered the possibility that a person a few seconds older than me would behave very differently.

Javert: Monsieur, you cannot seriously believe that seconds in age would make a difference?

Valjean: The astrologers do. But it matters not what I or anyone else believes. You claimed that your machine contained no pre-judgement and yet here is an indubitable one: that mere seconds difference in age do not matter. What then does matter? Minutes? Hours? Days? Your machine must have been told.

Javert: It was not told explicitly but of course certain smoothness assumptions are necessary. It would be absurd to expect the outcome to depend arbitrarily on your age. The machine does not consider sudden discontinuities: the relationships it discovers must be reasonable ones.

Valjean: Reasonable or not, that is an assumption. You have not allowed all possible relationships, only certain ones. The possible relationships known to your machine are limited, and I must ask on what grounds? You say it is entirely sensible to make a supposition of continuity of outcomes with inputs; I say that sounds a lot like a theory, albeit one whose foundations have been laid without oversight or plan.

Javert: I don't see how you can disagree with these very light assumptions.

Valjean: Yet I do; and there is worse ahead. For let me cede a great deal: let me allow that indeed that only differences of a year in age matters, nothing smaller. I would guess that perhaps one in twenty of your past cases were 28 years old, the same age as me?

Javert: Exactement! One in twenty is tens of thousands of people: more than enough to see the patterns that emerge.

Valjean: Perhaps. And let us say that broadly the same crime matters. Perhaps one in twenty were convicted for stealing bread?

Javert: Yes: again, tens of thousands of examples.

Valjean: But how many, then, who are both 28 and have stolen bread? That is one in four hundred.

Javert: Well, still: that is hundreds of examples to look at. I am not sure where this is going.

Valjean: But you consider also my parents’ occupation: I am an orphan. You consider my occupation: I was a pruner. How many orphaned, 28-year old pruners have stolen bread? One? Ten? How many of those were born in small towns? With my scholastic record? Having my height? Surely none. If you measure only ten variables, each having only ten possible values, that is more permutations than there are people in the world. Add a few more, and you will have more than have ever lived. Even if we allow the broadest of differences in individual characteristics to count as “similar,” with so many variations and permutations to choose from, I cannot but imagine that, yet, again, there is no one even remotely similar to me.

Javert: Now you are just couper les cheveux en quatre. Of course there is no-one the same in all ways at once; but we take the characteristics one by one. Each, by itself, is taken to influence your own likelihood of offence. You are similar to many people in each specific way and we combine the effects of these similarities.

Valjean: But this, too, is an assumption! You have put into your machine a predisposition not to consider that one variable might affect the reaction to another; that they act, not in ways determined by the other, but separately.

Javert: I have not “put in” that assumption—

Valjean: But you have not allowed for it. It was a priori a possibility, and it was excluded. You have, therefore, made an assumption. Just as a theory, which in your case you say you have not got, is an assumption.

It is not that I blame you. It is simply that it is not possible to model the world without assumptions. And so, we must not say we do not have them; we must instead find out what they are. In the best case, we will have a causal model of the world based on physical principles; we sociologists have it harder but that is not an excuse to give up.

Javert: Prisoner 24601, this is so much sophistry. I do not need to explain the machine to you at all. The fact of the matter—the indisputable fact—is that our machine works. Those whom it scores as high risk have, in the past, reoffended at a greater rate than those whom it scores as low risk. That is all that is needed.

Valjean: It is not all that is needed, and I will therefore tell you the grounds on which I intend to make my appeal.

Neither you nor your machine understands anything about the reasons for my actions. It does not understand why I stole; and it does not have anything remotely like a theory of behaviour that would allow it understand why I might steal in some future circumstance.

Your machine does not know and cannot say what actions you could take to change the outcome; if you suggested an action, it could not tell you the impact of that action. It does not know and cannot say how I myself might reason, nor how that reasoning affects my decisions.

It has based its predictions on a theory. That is not wrong. What is wrong is that we don't know what that theory is. What is wrong is that it is certain to be a bad theory.