P-values are Bullshit, 1942 edition

I keep an eye out for old criticisms of null hypothesis significance testing. There’s just something fascinating about reading this…

In this paper, I wish to examine a dogma of inferential procedure which, for psychologists at least, has attained the status of a religious conviction. The dogma to be scrutinized is the “null-hypothesis significance test” orthodoxy that passing statistical judgment on a scientific hypothesis by means of experimental observation is a decision procedure wherein one rejects or accepts a null hypothesis according to whether or not the value of a sample statistic yielded by an experiment falls within a certain predetermined “rejection region” of its possible values. The thesis to be advanced is that despite the awesome pre-eminence this method has attained in our experimental journals and textbooks of applied statistics, it is based upon a fundamental misunderstanding of the nature of rational inference, and is seldom if ever appropriate to the aims of scientific research. This is not a particularly original view—traditional null-hypothesis procedure has already been superceded in modern statistical theory by a variety of more satisfactory inferential techniques. But the perceptual defenses of psychologists are particularly efficient when dealing with matters of methodology, and so the statistical folkways of a more primitive past continue to dominate the local scene.[1]

… then realising it dates from 1960. So far I’ve spotted five waves of criticism: Jerzy Neyman and Egon Peterson head the first, dating from roughly 1928 to 1945; a number of authors such as the above-quoted Rozeboom formed a second wave between roughly 1960 and 1970; Jacob Cohen kicked off a third wave around 1990, which maybe lasted until his death in 1998; John Ioannidis spearheaded another wave in 2005, though this died out even quicker; and finally the “replication crisis” that kicked off in 2011 and is still ongoing as I type this.

I do like to search for papers outside of those waves, however, just to verify the partition. This one doesn’t qualify, but it’s pretty cool nonetheless.

Berkson, Joseph. “Tests of Significance Considered as Evidence.” Journal of the American Statistical Association 1942;37:325-35. International Journal of Epidemiology, vol. 32, no. 5, 2003, pp. 687.

For instance, they point to a specific example drawn from Ronald Fisher himself. The latter delves into a chart of eye facet frequency in Drosophila melanogaster, at various temperatures, and extracts some means. Conducting an ANOVA test, Fisher states “deviations from linear regression are evidently larger than would be expected, if the regression were really linear, from the variations within the arrays,” then concludes “There can therefore be no question of the statistical significance of the deviations from the straight line.”

Berkson’s response is to graph the dataset.eye facets vs. temperature, Drosophila Melangaster, graphed and fit to a line. From Fisher (1938).

The middle points look like outliers, but it’s pretty obvious we’re dealing with a linear relationship. That Fisher’s tests reject linearity is a blow against using them.

Jacob Cohen made a very strong argument against Fisherian frequentism in 1994, the “permanent illusion,” which he attributes to a paper by Gerd Gigerenzer in 1993.[3][4] I can’t find any evidence Gigerenzer actually named it that, but it doesn’t matter; Berkson scoops both of them by a whopping 51 years, then extends the argument.

Suppose I said, “Albinos are very rare in human populations, only one in fifty thousand. Therefore, if you have taken a random sample of 100 from a population and found in it an albino, the population is not human.” This is a similar argument but if it were given, I believe the rational retort would be, “If the population is not human, what is it?” A question would be asked that demands an affirmative answer. In the hull hypothesis schema we are trying only to nullify something: “The null hypothesis is never proved or established but is possibly disproved in the course of experimentation.” But ordinarily evidence does not take this form. With the corpus delicti in front of you, you do not say, “Here is evidence against the hypothesis that no one is dead.” You say, “Evidently someone has been murdered.”[5]

This hints at Berkson’s way out of the p-value mess: ditch falsification and allow evidence in favour of hypotheses. They point to another example or two to shore up their case, but can’t extend this intuition to a mathematical description of how this would work with p-values. A pity, but it was for the best.

[1] Rozeboom, William W. “The fallacy of the null-hypothesis significance test.” Psychological bulletin 57.5 (1960): 416.

[2] Berkson, Joseph. “Tests of Significance Considered as Evidence.” Journal of the American Statistical Association 1942;37:325-35. International Journal of Epidemiology, vol. 32, no. 5, 2003, pp. 687.

[3] Cohen, Jacob. “The Earth is Round (p < .05).” American Psychologist, vol. 49, no. 12, 1994, pp. 997-1003.

[4] Gigerenzer, Gerd. “The superego, the ego, and the id in statistical reasoning.” A handbook for data analysis in the behavioral sciences: Methodological issues (1993): 311-339.

[5] Berkson (1942), pg. 326.

A Flurry of Science

“Think about being able to do all these experiments on the ground, from airplanes, from balloons, with citizen scientists. It’s pretty breathtaking,” said Lika Guhathakurta, an astrophysicist at NASA. Guhathakurta will shepherd dozens of scientific projects on Aug. 21, including polarized images of the corona, which help scientists measure its temperature; measurements of Earth’s ionosphere — the charged layer of the atmosphere that gives us auroras; spatial disturbances in the atmosphere caused by heat changes; and a whole lot more. (You can get involved, too. With an app called GLOBE Observer and a thermometer, you can collect data during the eclipse and submit it to NASA. And Google and the University of California, Berkeley, are asking for video and images, which they’ll stitch together into an “Eclipse Megamovie.”) Other scientists will be studying animals — creatures as small as grasshoppers and as big as hippos have been documented reacting to eclipses — and us, too. Humans are sure to have a wide range of responses, as we have since time immemorial.

I’ll admit, this comes as a bit of a shock to me. I thought that coronagraphs permitted really good science provided a scientist could grab some funding. Not so, though.

… coronagraphs usually block more of the sun than astronomers would like. Typically, a coronagraph covers an area around 1.4 times the radius of the sun, obscuring arguably the most important region — the one closest to the sun’s surface.

“That is sort of the missing link, the region where space weather is formed, where the corona gets heated, where the solar wind gets accelerated. … So that’s where we want observations to be pristine,” Guhathakurta said.

Which means that my US readers have an excellent opportunity to do some citizen science in exactly three weeks (well, 21 days and 11 minutes from when I post this). In addition to the above links, Space.com has a good rundown of how to safely observe the eclipse and NASA has tonnes more charts and info.

And remember, a solar eclipse was once considered a bad omen for a king or ruler.

William Browder and the Magnitsky Act

While a fair number of people have contributed to Pharyngula’s “Political Madness” thread, one of the top contributors is Salty Current.

I watched Bill Browder’s Senate testimony live the other day. Here’s the link to the recording.

This one is better, actually, but if you follow either link you’ll be treated to a nearly two-hour Congressional hearing featuring William Browder, who learned about corruption at the Kremlin the hard way.

In June of 2007 my Moscow office was raided by 25 officers from the Moscow Interior ministry…. I hired the smartest man I knew, a young man named Sergei Magnitsky… , to investigate who did what and how we could stop them. … he came back with an astounding conclusion… the purpose of stealing our companies was to try and steal our assets, which they didn’t succeed in doing, however they did succeed in stealing 230 million dollars in taxes to the Russian government, from the Russian government.

Magnitsky was arrested by the high-ranking officials, and tortured in jail for much of a year. This included denying or ignoring 20 requests for medical attention to take care of pancreatitis, which left him screaming in pain. Magnitsky was eventually moved to a jail with medical facilities, but (according to Browder) was instead beaten with batons until death. Faced with a government that gave state honours to the people involved, Browder successfully lobbied for the Magnitsky Act in the United States.

That’s just the beginning of the testimony, though. Browder goes on to discuss various murders, Natalia Veselnitskaya‘s role in all this, and the depths of corruption within the Kremlin. It’s riveting stuff, and worth the watch. Can’t watch? Don’t worry, they have a rough transcript below the video. Some of the highlights I plucked from it:

  • “There’s no such thing as a former FSB agent, it’s a lifetime commitment.”
  • Fusion GPS, which hired Christopher Steele to compile that infamous dossier, is claimed to have been indirectly hired by the Kremlin to spread lies about Browder. They deny it, but if Browder is correct then it suggests that dossier was commissioned by the Kremlin so they could discredit or blackmail Trump should they have the need.
  • “Nobody was talking about adoptions, they were talking about the repeal of sanctions so that Russian torturers and murderers could freely travel and keep their money in America.”
  • The Panama Papers contain a treasure trove of information about Russian connections, one of which involves a cellist worth 2 billion dollars.
  • Brower believes that by muscling oligarchs, Vladimir Putin was able to skim off 50% of their transactions for himself and become the richest person in the world.

And there’s a lot more where that came from.

Dead Again

The Senate on Friday rejected a new, scaled-down Republican plan to repeal parts of the Affordable Care Act, seemingly derailing the Republicans’ seven-year campaign to dismantle the health care law.

Senator John McCain, Republican of Arizona, cast the decisive vote to defeat the proposal, joining two other Republicans, Susan Collins of Maine and Lisa Murkowski of Alaska, in opposing it.

The 49-to-51 vote was a huge setback for the majority leader, Mitch McConnell of Kentucky, who has spent the last three months trying to devise a repeal bill that could win support from members of his caucus.

The truncated Republican plan was far less than what Republicans once envisioned. Republican leaders, unable to overcome complaints from both moderate and conservative members of their caucus, said the skeletal plan was just a vehicle to permit negotiations with the House, which passed a much more ambitious repeal bill in early May.

I’ve been following this vote closely, and it looked like a sure pass right up until they were scheduled to vote. McCain may have flipped back thanks to pressure from the Arizona governor. Whatever the case, McConnell has said it is time to move on and signalled he was willing to work with Democrats, to which Schumer agreed.

Breath easy, USians. Your health care system is safe for the moment. Well, safe-ish.

Everything All Right Down There?

Do you need Canada to step in and intervene, USA? Because things are getting weird.

Shock surprise, after repeated denials that the Kremlin influenced your election, Trump is whining that no-one is reporting the Kremlin influenced the election to hurt him. All right, all right, fine, I’ll signal boost the alleged “pee pee tape” that was reported in the New York Times, Washington Post, and a tonne of other mainstream outlets.

If it were just that, I’d declare this an ordinary Thursday. But we also have this:

“Who leaked that to you?” he asked. I said I couldn’t give him that information. He responded by threatening to fire the entire White House communications staff. “What I’m going to do is, I will eliminate everyone in the comms team and we’ll start over,” he said. I laughed, not sure if he really believed that such a threat would convince a journalist to reveal a source. He continued to press me and complain about the staff he’s inherited in his new job. “I ask these guys not to leak anything and they can’t help themselves,” he said. “You’re an American citizen, this is a major catastrophe for the American country. So I’m asking you as an American patriot to give me a sense of who leaked it.” […]

“They’ll all be fired by me,” he said. “I fired one guy the other day. I have three to four people I’ll fire tomorrow. I’ll get to the person who leaked that to you. Reince Priebus—if you want to leak something—he’ll be asked to resign very shortly.” The issue, he said, was that he believed Priebus had been worried about the dinner because he hadn’t been invited. “Reince is a fucking paranoid schizophrenic, a paranoiac,” Scaramucci said. He channelled Priebus as he spoke: “ ‘Oh, Bill Shine is coming in. Let me leak the fucking thing and see if I can cock-block these people the way I cock-blocked Scaramucci for six months.’ ” […]

“The swamp will not defeat him,” he said, breaking into the third person. “They’re trying to resist me, but it’s not going to work. I’ve done nothing wrong on my financial disclosures, so they’re going to have to go fuck themselves.”

Scaramucci also told me that, unlike other senior officials, he had no interest in media attention. “I’m not Steve Bannon, I’m not trying to suck my own cock,” he said, speaking of Trump’s chief strategist. “I’m not trying to build my own brand off the fucking strength of the President. I’m here to serve the country.”

If you haven’t guessed, that was Anthony Scaramucci less than a week into his job. It isn’t often that the most bonkers person in the news isn’t Donald Trump.

Incredibly, though, Scaramucci was sane next to much of your own Senate. Not only did a mainstream publication push a story titled “Senate Republicans hope their own Obamacare repeal won’t become law“, the headline is accurate.

Mitch McConnell is making one last frantic plea to his Senate Republican members to advance the party’s scaled-back Obamacare repeal, assuring them at a private lunch that the vote is merely aimed at getting to conference with the House rather than immediately becoming law.

The Senate majority leader picked up some key votes at lunch, with Sen. Rob Portman (R-Ohio) endorsing the shriveling repeal effort as a bridge to bicameral negotiations. Not everyone was sold, but GOP leaders were emphasizing that the bill, which would slash Obamacare’s coverage mandates and result in millions more uninsured, is not the ultimate goal.

“I believe the leader has been in communication with Speaker [Paul] Ryan on that topic,” said Senate Majority Whip John Cornyn (R-Texas). “The request to go to conference has to come from the House so that would probably be the best people to talk to. But I have every expectation we will.”

To sum up the process so far, the House just barely passed a healthcare bill thanks in part to a promise that they’d fix it in the Senate; today, some key Republicans held a press conference to trash the new Senate bill, with Lindsey Graham in particular saying it was “terrible policy and horrible politics” and a “fraud”… then announced they’d vote for it, provided the House promised not to follow usual procedure and put it to a vote. Instead, they want the House and Senate to hash out their vast differences and come up with a third version of this healthcare bill. If they fail on that task, these Senators want the House to abandon the legislation instead of put it to a snap vote, despite heavy pressure to pass something. All this is happening as no-one has any idea what’s in the Senate bill they’re due to vote on sometime in the next 24 hours, but they do know that critical portions will have to dropped to pass it with 50 votes.

We’re starting to get really worried for you up here. Should we call in France or the UN to help mediate?

Time To Mobilise, America

It’s official.

Mitch McConnell: Tomorrow, I will keep my commitment to vote to move beyond the failures of #Obamacare. I will vote yes on the motion to proceed.

The fight for your health care just kicked up a few notches. Republicans are desperate for a “win,” even though they don’t know what they’re fighting for, to the point that they’re flying in John McCain immediately after his blood clot surgery and brain cancer diagnosis. The vote happens tomorrow, so you’d better get active now. Ben Wikler has some good advice.

The first big tell is Tuesday morning, when Republicans need 50+1 votes to start final debate on… something bad, don’t know what yet. 2/

Nobody knows how Tuesday’s vote will go. I have a sinking feeling. As do others. Need constant, maximal pressure. Call: 202-224-3121 3/

Unlike earlier phases of this battle, the hard right is now fully engaged. Koch organizations, Trump admin, the works. We must be louder. 4/

If McConnell votes FOR the motion to proceed but it FAILS, he can’t bring it up again and you can exhale. 7/

that’s very unlikely. Leaders nearly always switch their votes to “nay” so they’re voting w majority and reserve right to bring back up. 8/

If the Rs win the Motion to Proceed, we enter 20 hours of debate. Surreally, that’s all the debate we’ll get on Trumpcare. 11/

In this case, though, the MTP isn’t necessarily a preview of final vote because we won’t know what the final bill will be. So TURN IT UP 13/

Tue-Wed, we’ll have 10 hours of R speeches, 10 hours of D speeches, furious dealmaking, & (your job) mega public outrage 14/

Then vote-o-rama: the weird Senate thing on budget reconciliation bills where each side proposes unlimited amendments & ALL get voted on 15/

This time, vote-o-rama will be used by Rs to propose radical restructurings of the health care system without time for debate or review 17/

Meanwhile, Dems will, I hope and expect, come ready with 100s or 1000s of amendments to extend vote-o-rama as long as possible 18/

That’s the only way Ds can delay the final vote. But the Parliamentarian could rule them dilatory. Or McConnell can change rules anytime 20/

At the end, McConnell will intro an amendment that wipes away all previous amendments. That’s the final bill. Possibly unseen till then 22/

And then the Senate will vote on Trumpcare, whatever Trumpcare is at that point. A mystery bill that could shape all of our futures. 23/

In a series of moments, each a second or two long, a handful of Republican senators will vote yay or nay—death or life for untold 1000s 25/

If the Senate votes yes on Trumpcare, it’s very likely that the House votes the bill through intact & with mind-blowing speed. 29/

If the GOP passes the Motion to Proceed on Tuesday, I’d suggest basically putting your life on hold until the Trumpcare final vote is over.

This is one of those extremely rare moments in politics when everything is on the line—AND nobody knows which way it’ll go.

His advice for what to do begins around here in the Twitter thread. One thing I’d like to point to directly are virtual call banks, where you repeatedly call your representatives from the comfort of your home, but Wikler details a lot of other ways you can help and dishes some great advice (eg. don’t call people who don’t represent you).

Now get moving! Your health depends on it.

Stat of the Union

Time to do another deep dive on polling in the US. The first item comes via Steven Rosenfeld over at AlterNet. A number of polling companies have examined Trump’s standing in swing states, and compared it to how they voted. Their findings? They like him more than the average American, but less than when they voted for him. As Chuck Todd/Mark Murray/Carrie Dann put it at MSNBC,

In the Trump “Surge Counties” — think places like Carbon, Pa., which Trump won, 65%-31% (versus Mitt Romney’s 53%-45% margin) — 56% of residents approve of the president’s job performance. But in 2016, Trump won these “Surge Counties” by a combined 65%-29%. And in the “Flip Counties” — think places like Luzerne, Pa., which Obama carried 52%-47%, but which Trump won, 58%-39% — Trump’s job rating stands at just 44%. Trump won these “Flip Counties” by a combined 51%-43% margin a year ago.

So the sagging of support I mentioned a few months ago continues to happen. Rosenfeld also links to a few interviews with Trump voters, to get a more qualitative idea of where they’re at. There’s no real change there, they have a pessimistic view of what he’ll accomplish but praise him as a disruptor in fairly irrational terms. Take Ellen Pieper.

Poll respondent Ellen Pieper is among those disapproving of the president’s performance so far. The independent from Waukee voted for Trump and said she still believes in his ideas and qualifications. It’s how he behaves that bothers her. “He’s trying to move the country in the right direction, but his personality is getting in the way,” she said, calling out his use of Twitter in particular. “He’s a bright man, and I believe he has great ideas for getting the country back on track, but his approach needs some polish.”

Still, Pieper says, she’d vote for him again today.

Rosenfeld also makes some interesting comparisons to Nixon, but you’ll have to click through for that.

The second item comes via G. Elliott Morris, who’s boosted some diagrams made by Ian McDonald as well as their own. [Read more…]

Change Of Plans

I’ve had a draft cooking for a while over Laci Green’s view of trans* people. I don’t claim to know why she’s hanging out with MRAs or treating TERFs as if they were feminists, but if she’s going to sit down and attempt to make logical arguments the least I could do is return the favor.

But then this happened. [Read more…]