I’m reading Daniel Kahneman’s Thinking Fast and Slow. It begins amusingly with his telling us how he and Amos Tversky discovered – during a seminar of Kahneman’s at which Tversky was a guest speaker, their first collaboration – that even statisticians are bad at intuitive statistics.
He tells us about the resemblance heuristic, and starts with a question.
As you consider the next question, please assume that Steve was selected at random from a representative sample:
An individual has been described by a neighbor as follows: “Steve is very shy and withdrawn, invariably helpful but with little interest in people or the world of reality. A meek and tidy soul, he has a need for order and structure, and a passion for detail.” Is Steve more likely to be a librarian or a farmer?
First, of course I knew the obvious answer was the wrong one and I could see that “librarian” was the obvious and therefore wrong one – but I think I would not have chosen librarian even if I hadn’t known the obvious answer was wrong. I can tell you why.
It’s because I frequent libraries a good deal, and I think about things like “what would it be like to be a librarian/farmer/acrobat?” I already know that being a librarian would not be a good fit for someone who is very shy and withdrawn, because librarians spend much of their time interacting with strangers, and besides, colleagues. I also know that farming can be very solitary and even that some people choose it for that very reason.
That’s not actually why librarian is the wrong answer; it’s because there are twenty farmers for every one librarian, and I wouldn’t have considered that at all, so I would still have been wrong, but I would have gotten the right answer for the wrong reason.
I’m a terrible intuitive statistician. I’m confident of that.
Of course the shy, retiring fellow wouldn’t be a libertarian. They’re loud, obnoxious people who delight in telling everyone in sight about their politico-economic fantasies. Another point is the…
…Oh, I see you were talking about librarians. Never mind.
Ha!
They do have little interest in people or the world of reality, though.
The method works well with simple problems for which one has experience. It’s not foolproof (nothing is), and when it goes wrong, the consequences can be somewhat…spectacular.
Spoken as someone who doesn’t know librarians! Many professional librarians operate totally behind the scenes, and don’t have to interact with people. They are the ones who are responsible for all those numbers you see on the books, and the presence of the books in the database you look them up in. They are known as cataloguers.
Many professional librarians are able to go throughout the day without interacting with another living soul, and have never once said “shhh!”.
Just a for fun comment – I’m married to a librarian, and he gets a 0 on the extrovert scale. He spent his life holed up in an office out of sight putting numbers in books and making sure the books were properly catalogued.
But, yeah, it is a shame how people miss the most important things statitically, and focus on something that isn’t relevant. It’s easy to draw someone off on a red herring.
I would like to read this, but on the surface this seems to have a couple of flaws. One is that it assumes the reader knows the relative population of farmers and librarians. I live in a highly urbanised nation, and so I could reasonably guess that librarians were more common. The other is that it is ignoring the additional information, and claims that that is the right thing to do.
Let me pose this to you:
Chris has been drawn at random from the population of primary school teachers. Chris’ doctor tells us that Chris is undergoing treatment for prostate cancer. Is Chris more likely to be a man or a woman?
If you stick with the random selection, as you are apparently supposed to do in the example above, clearly Chris is much more likely to be a woman! Wait, what?
Now in the example above you might argue that “introvert with a passion for order” is not sufficient evidence to outweigh the random selection. That would be entirely reasonable. But it is still evidence, and it does deserve more than a zero weight.
@ 5 – oh. I did wonder about that but decided they meant the ones everybody sees – also you’re right, I didn’t know how big a proportion of working librarians they are.
So I would have answered “correctly” but for the wrong reason twice over.
@6:
So bad at intuiting statistics that even the example is rather poor.
This is really just another example of misdirection in the tradition of “As I was going to St Ives”; if you don’t focus on the first line (I didn’t), you try to answer the wrong question.
So everybody is wrong about everything for wrong reasons? Is that what I should take away from this?
A little Googling indicates that the number of farmers relative to librarians may be fallacious as well. Per the ALA, there are 121,785 libraries in the U.S., while the EPA indicates that there are about 1,920,000 full and part time farmers in the U.S. If the libraries have an average of 15 staff, then there are as many librarians as farmers in the U.S. Also, someone with ‘little concern for the real world’ may have some serious difficulties in a farming profession.
Farmers, besides gthe weather, are virtually always intensely interested in the markets and futures in whatever crop they’re growing or plan to grow. Being “unconcerned with the world” doesn’t work out well for them, as Dalillama, Schmott Guy said. And given that the numbers these guys used for their claim is also likely wrong, it looks like a trick question, a logical fallacy of false dilemma. The correct answer for a “meek and tidy soul ” with “a need for order and structure, and a passion for detai” and “little interest in people or the world of reality” is obviously “statistician”.
Although these particular fellows could use a little more “meek”.
@Daliliama: Thanks for that. I suspected as much, since most farms nowadays have, on average, slightly more than one worker (yes, since that’s on average, it can be just a bit more than one person; it’s statistical, you see?), and libraries, except in very small towns, rarely operate with only one. But I have not been well today, and blew off Googling.
See? We should even think critically about the critical thinking books, because I’ve found errors of this sort in several of them – mistaken assumptions that can be checked out perfectly easily by a simple Google search, or looking in the Almanac.
@Dalillama 11
Kahenman checked, in the book he says that there are more than 20 times as many male farmers as male librarians in the US. (The question assumes Steve is male.)
I love “Thinking, Fast and Slow”.
People should read it before they say that they are certain about anything.
Ophelia @10:
Statistically speaking, there’s a high probability of that.
The important thing about the example is not that we don’t know the true demographic statistics about farmers and librarians or how a particular personality profile fits either job. The important thing is that people will have an opinion without any real data. Even if the opinion happens to be right, it’s usually for the wrong reasons. An important thread in the book is that it is just basic human nature to form opinions, sometimes very strong ones, which are not based on any reasonable evidence.
This book should be skeptic 101.
One would have to be quite naive to believe that the typical self-absorbed person would know enough about her/his neighbour to give even a moderately accurate description of character. I worked with people for years and still knew very little about them. I know less about my neighbours – about as much as they possibly know about me!
@dsmccoy, I completely agree. The deeper I get into this whole skepticism business, the more strongly I’m leaning toward the view that our main focus should be on the psychology of belief and self-deception, such as heuristics and biases, cognitive dissonance and rationalization, patternicity and agenticity, the fallibility of perception and memory, cognitive illusions etc., and the less the “information deficit model” of erroneous beliefs makes sense. The problem isn’t “getting the information out there”. The real problem is that the strongest indicators of truth vs. falsehood – objectively speaking – rarely coincide with what seems most intuitively persuasive to lay people. An important part of becoming a “critical thinker” is learning to ask oneself “Why do I find these kinds of reasons so much more persuasive than those, and am I justified in doing so?”
@Bjarte Foshaug
I agree with everything you say, with the possible exception of the using “lay people” instead of just “people”. One thing which becomes clear in “Thinking, Fast and Slow” is that this is not just a problem with lay people but also experts. Kahneman spent a lot of time working in areas related to economics and is especially good at pointing out where “experts” in such fields do a lousy job of questioning their own assumptions. Being considered an expert is something of a pitfall for believing one’s own opinions, which is possibly one of the reasons many scientific breakthroughs come from cross-disciplinary work. In the book, Kahneman recounts an experience presenting Wall Street “experts” with proof that much of what they do, and reward highly with bonuses, is not skill-based but just gambling. They just ignore it and merely go back to what they think they know.
@dsmccoy, you’re right that “lay people” was a bad choice of words. I stand corrected.
@Bjarte Foshaug, all of us end up with an unfortunate choice of words on occasion. I’m kind of surprised that the worst word choice I can find in my last comment is merely an extra “the” inserted where it makes no sense.
I usually blame my bad English whenever I say something stupid 😉
I’m a terrible intuitive statistician. I’m confident of that.
I’ll lay odds on it…
I don’t understand the point of the question, so for me the whole exercise leaves me with a big FAIL written across my forehead.
The question asks me about Steve, not about statistics. Steve is an individual. I can’t tell a single thing about Steve on the basis of the likelihood of a farmer or a librarian having some particular kind of personality.
It’s like when I used to fight with doctors who would say “you can’t have *that* condition; it’s really rare”. Here’s a tip, buddy (I say to the doctors in my fantasy life); What kind of logic is it that lets you say, looking at me, an individual, that it can’t be X because very few people have X? It may be statistically unlikely, but look at MY symptoms and MY situation”. (But maybe that’s because I already lucked into having something rare, sigh; that might have changed my outlook.)
Same with Steve. ALL librarians except Steve might be extroverts, for all I know, and all farmers except Steve might be untidy and lead unstructured lives — doesn’t change who Steve is or how he operates…
I absolutely HATE, in any situation that matters in real life, having to extrapolate from limited data, but of course I do it all the time. (This brand of canned beans has never given me food poisoning before, therefore it is safe to buy, open and eat the contents of another can of the same brand.) But in anything that characterizes people, especially, I can’t handle that sort of hypothetical.
(…slinks away with big FAIL sign on head…)