If you have ever jumped to the wrong conclusion, made a terrible mistake thanks to your inbuilt biases or been subtly nudged back to your senses, then you are (a) human and (b) already on personal terms with the work of Daniel Kahneman, Olivier Sibony and Cass Sunstein. Thanks to their academic and popular writing, the world is now very familiar with what are collectively called 鈥渃ognitive biases鈥� 鈥� systematic errors in human thinking 鈥� and ways to correct them.
Sunstein co-wrote the highly influential book Nudge: Improving decisions about health, wealth and happiness with Richard Thaler, while Kahneman popularised the work that won him the Nobel prize in economics in 2002 with his book Thinking, Fast and Slow. is the author of You鈥檙e About to Make a Terrible Mistake: How biases distort decision-making and what you can do to fight them.
Advertisement
You may think that, in no small part thanks to their efforts, the swamp of human fallibility has been well and truly drained by now. But that would be yet another mistake. Kahneman, Sibony and Sunstein say there is an even more important source of warped decision-making. The three have banded together in a behavioural science supergroup to draw attention to what they call 鈥渘oise鈥� 鈥� persistent inconsistencies in professional judgements that lead to bad outcomes in all walks of life.
Kahneman and Sibony spoke to 快猫短视频 about the group鈥檚 new book (Little, Brown Spark in the UK; William Collins in the US). Sunstein was due to join the conversation, but was called away at the last minute by his new boss, US president Joe Biden.
Graham Lawton: We are familiar with the idea that human decision-making is bedevilled by cognitive biases. Your book is about a different source of error, noise. Can you explain the difference?
Daniel Kahneman: Noise is the amount of disagreement between people who make professional judgements. Think of an organisation, like a medical system or a justice system, which has individuals who perform judgement tasks. Noise is the variability of their judgements on the same task.
The existence of noise is obvious to everybody: judgement tasks are defined by the fact that reasonable people can disagree, otherwise we don鈥檛 call it judgement. But it turns out there is more noise than people expect. Much more.
Olivier Sibony: One of the mottos of the book is wherever there is judgement, there is noise. And more of it than you think.
You list many areas where noisy judgement is a problem, from criminal justice to medical diagnosis to hiring and firing. But these are just the tip of the iceberg. Can you give some specific examples?
DK: Some years ago, I was consulting in an insurance company, and I proposed to run a study of noise in the judgements of their underwriters. So they constructed cases 鈥� very naturalistic cases from their point of view 鈥� and presented them to about 50 underwriters. They were shown one or more of these cases and made judgements.
I asked a few executives about their expectations. Suppose you take two underwriters at random, and compute the difference between their judgements in percentages. What percentage do you expect to find on average?
The answer was 10 per cent, which turns out to be a very general answer. Olivier has surveyed hundreds of people, and the most common answer is 10 per cent. But what we found for underwriters was 55 per cent. That is a big difference. It is, in fact, so large that it raises questions about the need for people to make that judgement.
This was unknown to the company. It came as news to them that they had a noise problem. So that鈥檚 how it began.
I must admit, when I first read the word 鈥渦nderwriters鈥� in your book, I misread it as 鈥渦ndertakers鈥�, which made me laugh.
OS: We hope there is not too much noise there! Anyway, this process is what we now call a noise audit. And we went through the literature, looking for examples of studies in which people had done noise audits 鈥� without, of course, calling them that. We found a lot.
One striking example is the US judiciary. Years ago, a study was done of the . Again, the difference was in the region of 50 per cent of the mean. So, if you pick two judges at random and you give them a case that has a seven-year average prison sentence, the difference between them is in the region of three-and-a-half years.
That is staggering. The mere fact that the lottery that assigns judges has assigned you Judge X rather than Judge Y means the difference between five years in prison or nine years in prison.
That is serious. Why have we paid such little attention to noise until now?
DK: Noise is difficult to think about. The human mind seems to be specialised for thinking about particular cases, and for thinking causally. It seems to have significant difficulties thinking statistically about ensembles of cases.
What is striking is that when you think about a single case, you can identify bias, but you will never identify noise. No single case appears to be noisy. That is one important reason why noise is so neglected.
鈥淎 lot of professional judgement actually has no connection with reality鈥�
Another is something called na茂ve realism. In general, when we look at the world and make judgements about it, we feel we鈥檈 getting it right. We see the world as it is. And if you鈥檈 sitting next to me looking at the same world, and I respect your judgement, then I fully expect you to see the world exactly as I do. But probably you don鈥檛.
OS: There is a third reason. You have to ask: why don鈥檛 organisations realise this problem? Why doesn鈥檛 the insurance company realise it? Why doesn鈥檛 the judicial system become aware of so much variability? Why don鈥檛 hospitals become aware of the fact that doctors have, quite often, very different diagnoses of the same patient? You would think it鈥檚 in their interest.
We think part of the answer is that organisations are designed to suppress evidence of noise. They鈥檈 designed to sweep the problem under the rug and to create the illusion of consensus. They are not looking for the correct answer. One way to do that is never ask people for their opinions separately. You bring them into a meeting and you ask them to discuss it. Which, of course, gives a strong incentive to the second speaker to agree with the first one, and the third person to agree with the first two, and so on.
One psychologist we interviewed told us that he was working with the admissions department of a university. He advised them that it would be a good thing, when two admissions officers grade an essay, for them to grade it separately and then to discuss their differences. And the answer of the university was 鈥渙h, that鈥檚 how we used to do it, but we disagreed so much that we adopted the current system, in which the second evaluator sees what the first one has done鈥�.
DK: Many professional judgements receive no feedback. You make a judgement and you never know how accurate you are. This is certainly true for judges and underwriters. It鈥檚 true even for radiologists, who make a diagnosis but very rarely hear what the autopsies say.
So, a lot of professional judgement actually has no connection with reality. And this allows different professionals to diverge from each other without knowing that they are diverging.
In the absence of feedback, you have experts. We call them 鈥渞espect experts鈥� because they鈥檈 respected by their colleagues, and what they say has a sense of being true, although there is no objective way of measuring that. That certainly helps obscure the role of noise.
No wonder people are sick of experts. What about science? At least it has peer review.
DK: There are quite a few studies of judgement in science, specifically the judgements scientists pass on each other鈥檚 work.
The amount of noise is embarrassing. There may be agreement among scientists about objective facts, but when it comes to evaluating an article or a grant, it turns out that scientists disagree as much as other professionals do 鈥� again, because there is no obvious objective criterion for the quality of an article. It is a subjective judgement. And where there is subjective judgement, there tends to be a lot of noise.
The book is concerned with professional judgements, but does it also apply to everyday decisions?
OS: To the extent that you want those judgements to be accurate, it does. Now, what judgements in your daily life do you want to be accurate? Do you want to make an accurate judgement when you get married? Personally, I wouldn鈥檛 think about it that way. I think you鈥檈 making judgements of a different type.
On the other hand, if you are trying to decide whether you鈥檈 buying your house at the right price, or whether the job you are going to take is the right job, these are judgements that lend themselves quite well to the type of approach that we talk about, because they鈥檈 quite close to being professional judgements.
What are the psychological sources of noise?
DK: There is a fair amount of psychology to explain it. In Thinking, Fast and Slow, the attempt was to characterise the human mind in general. Here, the main aspect we鈥檈 interested in is individual differences, different people making different judgements. So it鈥檚 a different cut through the psychological cake. Where the psychology becomes interesting, I think, is in the distinction that we draw between different types of noise. We identify three sources of noise.
OS: The easiest way to describe them is to think of the example of facing a judge. You are going to be sentenced, and you are going to be assigned to one judge or another. You鈥檈 assigned to Judge X, and your lawyer says 鈥渙h, thank God, it could have been Judge Y! Judge Y is a real hanging judge. You got lucky. Judge X is much nicer.鈥� That鈥檚 what we call level noise 鈥� different individuals have different average levels in their judgement.
The second thing is that each of us has within-person variability. So Judge X could, in fact, be in a foul mood today. This morning, he happens to be worse than Judge Y. So, there鈥檚 variability within each judge. That鈥檚 what we call occasion noise. The occasion matters. We鈥檈 all aware that there are decisions we are making today that we wouldn鈥檛 have made on another day.
For a long time, we thought that was it, but eventually it dawned on us that there is a third type of noise, which turns out to be the largest. It鈥檚 a little harder to wrap your mind around.
So, if you give two judges 10 different people to sentence, and you ask them to rank these 10 cases from the most punishable to the least, they will not have the same ranking. Their fundamental beliefs are different. Essentially, they have different tastes.
That鈥檚 what we call pattern noise. And this source of difference between individual judges 鈥� in the broader sense, not just the judicial one 鈥� is the most important source. People are different. They have different backgrounds, different histories, different preferences. They鈥檝e learned different things; they have failed to learn other different things. Therefore, we can鈥檛 expect them to be identical.
You mention diagnosis. You would think that two doctors looking at the same case ought to be able to arrive at the same objectively correct diagnosis, but they don鈥檛.
DK: It is actually surprising and depressing the extent to which there is noise in medicine. The worst case is psychiatry. But there is a lot of variability in diagnosis of tuberculosis, for example.
Some of it is occasion noise. If one physician has seen two cases of a particular disease in the past few days, he or she is primed to see more cases of that disease than somebody who has not. In other judgements, it鈥檚 the other way around. If a banker has approved two loans, the banker is less likely to approve the next loan.
Physicians could also have different ways of looking at cases, like different orders in which they consider diagnoses. If a diagnosis comes to mind more easily for one physician than for another, he or she will be more likely to make that diagnosis. And that鈥檚 only one other difference.
While we are on doctors, they have embraced technologies like artificial intelligence to help them make better diagnoses. Is that the answer?
OS: Yes and no. It鈥檚 a way out for tasks in which there is an objective truth. Is this tumour cancerous or not?
Where we have sufficient data to train algorithms, it鈥檚 pretty clear that these tasks of judgement will stop being tasks of judgement. They will become automated tasks. A lot of things that are done by algorithms today used to be matters of judgement.
Deciding whether you got a small consumer loan used to be a matter of judgement. You had an interview with the banker at your local bank. Now there is an algorithm that makes that determination, and we all think that鈥檚 fine.
A lot of simple judgements like that will be automated. Our focus, though, is not on those. Our focus is on the very large number of judgements that 鈥� either because they cannot be automated or because we don鈥檛 want them to be automated 鈥� are going to be matters of judgement for a long time. And we try to give people who make those judgements tools to make them less noisy.
What kind of tools?
OS: One of our first calls to action is for people to do more noise audits to start measuring this problem in their organisations so that they can decide whether it鈥檚 worth addressing. We think, in many cases, their answer will be yes.
Once an organisation sees its noisy decision-making, what can be done?
DK: The first step is to get a good collection of options. On that, we have nothing to say. But if there is a good collection of options, there is a disciplined way of comparing the options, and we recommend that way of proceeding.
OS: We call it decision hygiene 鈥� list all the options, decide how to evaluate them, make those assessments separately and then come to a conclusion.
You admit that decision hygiene is as unglamorous as it sounds. Can you glamorise it a bit?
DK: It goes back to the distinction between bias and noise. If you propose a technique to overcome a bias, that will be the equivalent of medication or vaccination for a known disease.
Decision hygiene is a bit like washing your hands. You have no idea what germs you are killing, and if you are successful, you鈥檒l never know. The procedures are preventive. You want to apply them when you鈥檈 not specifically aiming to prevent any particular bias, just to bring some uniformity and some discipline to judgement.
Is there any risk to eliminating noise? Are there not some benefits to having different outcomes when different people evaluate the same information?
DK: As we define it, there is no good noise, because we define noise as variability in judgements that should be identical.
Variability can be very useful. In fact, it鈥檚 essential when there is feedback and selection. But in the absence of feedback and selection, variability is just a source of error.
OS: There are situations where reducing noise is not worth it. There鈥檚 a cost-benefit analysis to be done where the answer might be that tolerating some noise is just fine because it鈥檚 not very consequential. We think those cases, though, are much less frequent than the cases where noise is large and problematic.
Do the ideas and actions in the book have any relevance to huge societal challenges like, say, pandemic responses?
OS: The pandemic is a really interesting case because, usually, when there is a big crisis like this, you see a singular decision. But here you see all countries responding to what is essentially the same problem with differences in timing and scope and so on.
A striking example was the Oxford/AstraZeneca vaccine side effects, and each health agency in each European country coming to a different conclusion about what to do 鈥� some saying let鈥檚 ban it altogether, some saying let鈥檚 keep it altogether, some saying let鈥檚 ban it for people under a certain age, some saying let鈥檚 ban it for people over a certain age. Looking at the same evidence with supposedly the same set of objectives, very smart and highly qualified people came to very different conclusions. We think if they had more disciplined thinking, we would see less noise.
You have spent years working against bad decision-making, discovering its causes and developing ways to do better. Yet the same types of bad decisions are made again and again. Are you fighting a losing battle against human fallibility?
DK: I wouldn鈥檛 say that things are completely unchanging. I think there is more systematic use of intelligence in decisions than there was before. There has been progress. But it is slow.
OS: The time it takes for ideas to get into the organisations that can actually do something with them is quite long. One of the things that books like this can hope to accomplish is to actually shorten that time.
But the beauty of noise is that you don鈥檛 need to understand the causes of bad judgements in order to fix them. That makes it a lot easier to combat because you don鈥檛 have to point your finger at anybody and say 鈥渉ey, you are biased鈥�. You can say 鈥渙ur system is noisy, let鈥檚 make it more disciplined鈥�. I think that鈥檚 a very liberating thought.
