Puzzles intrigue me, whatever form they take. This is to explain why I am revisiting what might seem to be a dead issue: the question of how it could be that the Romney campaign could have been taken by surprise by their loss in 2012. How it could be that they seemed so confident right up until election night that they were going to win? Romney later said that the first sign he had that he was in trouble was late on election night when Florida’s result took a long time coming. He had thought he would win that state easily. But how could that be since even casual observers like me realized well in advance that things looked bad for them?

It is possible that they all knew that they were going to lose and were simply lying in order to boost the morale of their supporters and keep the campaign contributions flowing. This was a popular theory but I was not convinced. It is usually the case that staffers leak the truth after the election. Within a couple of weeks after the 2008 election being over, McCain campaign insiders were saying that they had long known that the game was lost. So I waited for a while but now eight months later, there have been no such revelations from the Romney campaign, persuading me that they seemed to have truly believed what they were saying publicly and been genuinely shocked by the outcome. So what could have led even the polling professionals within their campaign to misread the situation?

Although the TV talking heads could yak all they wanted to about what they predicted from the number of lawn signs and the like, it is always the case that the campaigns have statisticians, reality-based numbers people, who are not seduced by vacuous talk of the political pundit class and whose role it is to tell the candidates the unvarnished truth. The poll numbers seemed pretty clear for quite a while before the election that he was going to lose and the statisticians within his campaign surely must have known this.

It turns out that, as is often the case, it was not the raw numbers that are the problem but the models that are used to analyze them. Statisticians use auxiliary data to create various ‘screens’ to analyze the raw polling data to see who are the people who will actually vote on election day, which is what counts. You could use ‘registered voters’ (a straightforward but not a particularly reliable guide since many registered voters do not vote), ‘likely voters’, or ‘very likely voters’ (both based on what respondents say about their intentions and their past voting histories). It appears to have been the case that while the results of applying the first two screens favored Obama, the ‘very likely’ screen favored Romney. And in that analysis, they agreed with internal Democratic analyses

Democrats had argued for months before the election that Republican polling was screening out voters who would ultimately turn up to support Obama. In fact, Obama advisers said, if you applied a tighter likely voter screen to Democratic polling — counting only the very likeliest voters as part of the electorate — you could come up with results similar to what the GOP was looking at.

But the Republicans were wrong in their model that this was an election in which enthusiasm was low and hence the third screen was the best predictor. Why did they think this? They felt that the 2008 election that swept Barack Obama into office was driven by enthusiasm for a young, charismatic candidate that attracted young and minority voters in huge numbers and that this would not be repeated in 2012 because by now Obama would be seen as just another politicians who had not delivered on many of the hope and change promises.

They felt that there was a marked drop in enthusiasm for Obama since 2008 since he had disappointed blacks with not really improving the economy or creating jobs, Hispanics with not pushing for immigration reform, and liberals and progressives and young people with his poor record on civil liberties and human rights and war. Furthermore the significance and novelty of electing the nation’s first black president had worn off, leading to further lack of enthusiasm. Hence they thought that only the very likely voters would turn out on election day.

But what went wrong for them was that minority voter turnout was at historically high levels in 2012, confounding expectations. This seemed to have taken the Romney campaign totally by surprise.

It’s not that the Romney camp failed to meet its targets. They say they actually met their voter outreach goals in Ohio. During the summer, they targeted more than 2 million voters who had not voted in party primaries. Those were the independents they believed would be the key to the race. Since the strategy seemed to be paying off with internal and external polls showing Romney leading among independents, the Romney team felt like they were working their plan. “We did everything we set out to do,” says a top strategist about the Ohio effort. “We just didn’t expect the African-American vote to be so high.” African-American participation in Ohio jumped from 11 percent of the electorate to 15 percent between the 2008 and 2012 elections. “We could never see that coming. We thought they’d gotten a lot last time.” But that wasn’t the only problem. Romney underperformed George Bush’s results from 2004 in the vast majority of Ohio’s counties, not just the ones with big African-American populations.

So what happened to cause them to go so badly astray? This issue of the minority vote and how the Republicans deal with it is going to be the big divisive issue within the Republican party for the next election, as I will discuss in an upcoming post.