Lyanna pointed out that I was unclear about the boundaries of when expert judgment is better or worse than an algorithm. Kahneman gets into that in the next chapter. He collaborated with his main opponent to try to figure that out. The takeaway –
At the end of our journey, Gary Klein and I agreed on a general answer to our initial question: When can you trust an experienced professional who claims to have an intuition? Our conclusion was that for the most part it is possible to distinguish intuitions that are likely to be valid from those that are likely to be bogus…If the environment is sufficiently regular and if the judge has had a chance to learn its regularities, the associative machinery will recognize situations and generate quick and accurate predictions and decisions. You can trust someone’s intuitions if these conditions are met. [p 243]
There, that’s more tidy than the way I said it.
If you know whether “the environment is sufficiently regular” you know something many experts do not.
I think this metric is almost worthless; it requires knowledge that can be very hard to find, and in many fields is actually unknown.
How does that make it worthless? I don’t see that at all. (Of course, the quote doesn’t stand on its own. If you read it in the surrounding chapter maybe it wouldn’t strike you that way.)
I’m not llewelly, but maybe it’s the “sufficiently regular” bit? What is a sufficient level of regularity and how do you know if it’s there? That is actually an unknown in many fields.
In any case, thanks for posting more of the excerpt.
I have no problem with the idea of sufficient regularity. Much of my working life and many of my hobbies are strongly based on physical sciences and engineering at the end of the day.
These are very amenable to the sufficient regularity test. Go to biological or social sciences, arts, music. Maybe not so much. There may be aspects where expert intuition work well – for example behaviour of large populations – and aspects where expert intuition is simply an educated guess or even a fabrication – behaviour of an individual.
I see it as an exercise in boundary setting, not at all black and white.
The chapter is mostly about that.
Another helpful bit a few pages earlier –
More regularities for nurses than for stock pickers, I take it he’s saying.
That’s pretty much as I’d expect it. I know a couple of people in the finance industry who have a very poor opinion of stock pickers. Most seem to do worse than the market index with their picks, with a small number doing about the market index and an insignificant number doing consistently better.
Even in medicine the documented error rate suggests that intuition, while useful, is not a basis for practice.
You can see a great example of this dynamic when it comes to things like the scouting of athletes, in particular baseball players. The large number of instances provided by baseball’s long season provide sample size for statisticians to work with (Nate Silver’s predictive models that so accurately pegged the recent election are based on his baseball work.) And they have provided a lot of value in predicting outcomes. At the same time the game provides a very sufficiently regular environment, allowing expert scouts to make decisions based on observation and expertise. This conflict is central to the movie Moneyball (where it is seriously overplayed for laughs at the expense of old timey scouts).
I see more than one concept in one word “intuition”.
On the one hand, to make a given judgment, like a medical diagnosis, someone can test algorithms versus intuitions head-to-head, and I’m fine if survey says algorithms are better than intuitions in that sense. And I’m all in favor of stamping out woo, and I would not support a person who claims to be an “intuitive” as a special kind of person.
On the other hand, I have three degrees from MIT in engineering, and a component of the educational system is building what we call “physical intuition”, and a large part of the value I give in my career is my judgment in terms of “physical intuition”, and helping other people build their “physical intuition”. By this I mean I can go to a whiteboard and tailor mathematical analogies for my colleagues, based on their individual backgrounds, to help them understand, “Are we using the right algorithms?” and, “Are we using the algorithms right?”
You might say my comment is addressing the situation where an environment is not regular yet. But I think my comment is more pointing to a different concept, where algorithms and intuition are not head-to-head, but physical intuition is supervising the definitions and selection of algorithms. And for this meaning of “intuition”, schools teach it by that name. Schools don’t have a course for intuition by that name, but engineers and scientists should recognize what I’m saying.
Personally I treat intuition as “information that didn’t make it to the front of my mind”