I’d never heard of Surgisphere before. Apparently, no one had. They just suddenly appeared out of nowhere with vast amounts of data from numerous hospitals, a gigantic database that they’d used to address the question of the utility of hydroxychoroquine in treating COVID-19, and came back with the expected answer: no, it’s not any good. They got quoted all over the place! Great PR! Suddenly, lots of people had heard of Surgisphere.
Unfortunately, Surgisphere is a crock.
The World Health Organization and a number of national governments have changed their Covid-19 policies and treatments on the basis of flawed data from a little-known US healthcare analytics company, also calling into question the integrity of key studies published in some of the world’s most prestigious medical journals.
A Guardian investigation can reveal the US-based company Surgisphere, whose handful of employees appear to include a science fiction writer and an adult-content model, has provided data for multiple studies on Covid-19 co-authored by its chief executive, but has so far failed to adequately explain its data or methodology.
Data it claims to have legitimately obtained from more than a thousand hospitals worldwide formed the basis of scientific articles that have led to changes in Covid-19 treatment policies in Latin American countries. It was also behind a decision by the WHO and research institutes around the world to halt trials of the controversial drug hydroxychloroquine. On Wednesday, the WHO announced those trials would now resume.
Hey! Nothing wrong with citizen input from science fiction writers and adult-content models. There had better be more substance behind the claims, though. It turns out that there is confusion about how many employees the company has (100? 6? 3?) depending on the source, there don’t seem to be any people with the special skills need for the study — this is Big Data stuff, lots of statistics and computer science — and the data has been falling apart. The study claimed to be derived from “96,000 patients with Covid-19, admitted to 671 hospitals from their database of 1,200 hospitals around the world”, but various hospitals have reported that the data doesn’t match what they’ve reported.
And then, the big question: how did this company get access to so much confidential medical information?
One of the questions that has most baffled the scientific community is how Surgisphere, established by Desai in 2008 as a medical education company that published textbooks, became the owner of a powerful international database. That database, despite only being announced by Surgisphere recently, boasts access to data from 96,000 patients in 1,200 hospitals around the world.
When contacted by the Guardian, Desai said his company employed just 11 people [nobody seems to know how many people work there]. The employees listed on LinkedIn were recorded on the site as having joined Surgisphere only two months ago. Several did not appear to have a scientific or statistical background, but mention expertise in strategy, copywriting, leadership and acquisition.
What is clear is that there was a massive falsification of data. It also looks like the chief executive of the company, Sapan Desai, is a con artist with a history of pseudoscientific schemes.
What’s interesting about the story, though, is that it demonstrates how everyone is a bit gullible, and is willing to suspend skepticism a bit when the science, pseudo or otherwise, seems to support prior expectations. Lots of people got fooled by this one. Researchers even suspended ongoing trials because they thought Surgisphere had just provided the definitive answer! At first, it was only the hydroxychloroquine fanatics who were skeptical of the study, and embarrassingly, they were right, in this one case. But the real difference is that the real scientists, like David Gorski, will reassess their conclusions in the light of new information, admit to their error, and move on.
That’s the difference between the cultists and me. I’ll change my mind if they present new information that checks out when I dig into it. It’s also a lesson that a believer’s skepticism when examining something he disagrees with will always be far more rigorous than when looking at a study that goes against what he currently believes. Think of it as a somewhat embarrassing reminder to myself (coupled, perhaps, with a bit of self-flagellation) to remain humble in the future and not to be too fast to dismiss criticisms coming from even the cultists.
Surgisphere’s papers are getting trashed. The legitimate hydroxychloroquine studies have resumed — way too many studies than the treatment deserves, if you ask me. If they come back with positive information about the value of the drug (I don’t think they will, since the claims all originated from sources as quacky as Sapan Desai) then I’ll accept new treatment recommendations. The question is, will the drug’s proponents accept any evidence from any studies that show its efficacy is baseless?
Just for the record – if it can fit on one commercially available hard drive it is not big data.
I agree that what we actually need now is a small number of well-conducted studies, not a whole raft of scientists scrambling for quick answers with a patchwork of differently flawed studies. But that aside, this is another editorial failure at the Lancet — this is the same editor who published the Wakefield paper against reviewer advice and refused to retract it for 12 years.
At least this time they’ve retracted early. The paper has a whopping RETRACTED message across every page of the pdf. But I can’t understand how it passed peer review. It’s not what everyone is talking about now — the key flaw was when Australian scientists noticed that the study had included more deaths from this country than had actually been reported. Obvious when you know what to look for; not obvious to a reviewer who wasn’t intimately familiar with the numbers from each country (which is expecting too much of reviewers). There are other reasons the paper should have been rejected that the review system should have flagged. Here are just a few quotes from the study.
“The registry comprised 671 hospitals located in six continents (appendix p 3).” This is not what a registry is.
“Real-world data are collected through automated data transfers that capture 100% of the data from each healthcare entity at regular, predetermined intervals, thus reducing the impact of selection bias and missing values, and ensuring that the data are current, reliable, and relevant.” So they have a magic system that captures “100%” of the clinical data from hospital records.
“The registry uses a cloud-based health-care data analytics platform that includes specific modules for data acquisition, data warehousing, data analytics, and data reporting.” What? This is just marketing spiel, not medical research. What does it matter to the trial if the data was put in cloud storage versus local or network-attached storage?
“Once this data dictionary is harmonised with electronic health record data, data acquisition is completed using automated interfaces to expedite data transfer and improve data integrity.” Well now we’re into pure buzzword gobbledygook worthy of Theranos.
“Collection of a 100% sample from each health-care entity is validated against financial records and external databases to minimise selection bias.” Now they have access to hospital financial records and undisclosed “external databases” as well!
“The data collection and analyses are deemed exempt from ethics review.” THIS STATEMENT ALONE SHOULD HAVE CANNED THE PAPER.
“…if the patient’s electronic health record did not include information on a clinical characteristic, it was assumed that the characteristic was not present.” Holy crap, that’s bad.
There are also extremely troubling findings in the results as well. For instance, while heart disease was associated with higher risk of death (no surprise), being on a statin was significantly protective — but there is an almost 100% overlap between having heart disease and being on a statin. How is it possible for these two highly-correlated risk factors to have opposite effects? Even more hard to explain: immunosuppressed patients were at no higher risk of dying from COVID!
And I know that this is not in the original paper, but I can’t resist…
When challenged by the Lancet and journalists, the contributor who also founded Surgisphere replied: “We have reviewed our Surgisphere database and discovered that a new hospital that joined the registry on April 1, and self-designated as belonging to the Australasia continental designation…In reviewing the data from each of the hospitals in the registry, we noted that this hospital had a nearly 100% composition of Asian race and a relatively high use of chloroquine compared to non-use in Australia. This hospital should have more appropriately been assigned to the Asian continental designation.”
How could such a thing happen to a 100% capture data extraction system with automatic interfaces that harmonise with electronic health records in the cloud?
Oh, and finally, the Guardian reports “In a statement Surgisphere said it stood by the integrity of its data, saying all information from hospitals ‘is transferred in a deidentified manner’ but could not be made public.”
Not only does it make no sense that the data cannot be made public, but if the data was deidentified, how could they do all the crossmatching with other databases they boasted about in the paper?
Very mysterious what the motive is behind this hoax. You don’t get paid a penny to publish in the lancet, and you wouldn’t be able to get research funding without proving the bona fides of your data including documentation from all of the hospitals that supposedly contribute to it. So this was just a completely crazy thing to do, and obviously the chances of getting caught were pretty much 100%. So why?
How in the hell is this kind of shit allowed to happen? Are there no systems in place by researchers at the WHO, the CDC, and Universities to actually evaluate studies before decisions are made? The scientific establishment ought to be ashamed that a fraud like this on such an important subject could take place. Shit, I’m embarrassed and I don’t even play a role to in this. What a fiasco!
Pretty sure it was basically a marketing move for Surgisphere. Get a lot of attention with an important study in the Lancet, float it past the financial reporters in an easily quoteable press release, watch the investment money roll in. At least, that was the theory.
I mean, fucking hell, shit like this and the idiotic statements that masks were useless early in the pandemic plays into the hands of those who say the experts are not to be trusted. This kind of fucking incompetence feeds the doubt that anyone knows what they’re doing, and I am starting to think they have a point. Heads need to roll.
Note that the big scoop with Surgisphere’s fake data came out only days before the first results of chloroquine randomized-control, blinded studies came out.
One on using it as prophylactic found no difference in outcomes, and ended early because of futility.
Another on using it as treatment of serious cases found no differences in outcomes also.
Neither saw the horrible rate of death that came from the Surgisphere data. The drug was well tolerated, but useless.
@ 1 vytautasjanaauska
if it can fit on one commercially available hard drive it is not big data.
Since Surgisphere is refusing to release the data, for all we know it could be in 44 gallon barrels or Greek amphorae.
@ 4 psychomath
re there no systems in place by researchers at the WHO, the CDC,
Basically no. That is not their function. WHO is an international body with a mandate from the members of the UN and CDC is a US Gov’t agency with a specific mandate.
and Universities to actually evaluate studies before decisions are made?
If the researcher works for the university, yes but this appears to be a private company.
To a considerable extent, its like
the Wild West out there and that is even before we get to crazy alt-medicine and out-and-out fraudsters selling bleach or colloidal silver as a cure for Covid-19.
And even if there is a system in place as there is in France for human clinical trials, an idiot like Didier Raoult seems to have started his first “study” before getting a legally required ethics approval.
@ 6
fucking hell, shit like this and the idiotic statements that masks were useless
You do have a source for that other than said the aforementioned fraudsters and the more rabid anti-vaxers?
Given the desperate pleas for more Personal Protective Equipment (PPE) all around the world I doubt that anyone in WHO , various departments of health or the CDC categorically said that “masks were useless”.
I don’t think that even the Orange Maron or Bolsonaro, the Orange clone, have said that.
psychomath —
This has nothing to do with the WHO or the CDC. They did not fund, approve, or otherwise engage in this study. It is completely on the heads of the authors and the Lancet unless it turns out that Harvard or Brigham and Women’s Hospital had some involvement.
At this time I am inclined to suspect that neither Harvard not B+W had any direct involvement. My reason for saying this is that the paper notes: “Funding: William Harvey Distinguished Chair in Advanced Cardiovascular Medicine at Brigham and Women’s Hospital.”
Except this is not a funding body. It is a job position. Held by the lead author. Here’s an example of what a funding citation should look like, from another COVID paper that includes Brigham and Women’s researchers on the author list:
“Funding KMD and NS-B are supported by the National Institutes of Health Ruth L. Kirschstein Institutional National Research Service Award (T32-AR-007258). HKC is funded by National Institutes of Health (P50-AR-060772). JAS is funded by NIH/NIAMS (grant numbers K23 AR069688, R03 AR075886, L30 AR066953, P30 AR070253 and P30 AR072577), the Rheumatology Research Foundation K Supplement Award, the Brigham Research Institute and the R. Bruce and Joan M. Mickey Research Scholar Fund. ZSW is funded by NIH/NIAMS (K23AR073334 and L30 AR070520).”
What I suspect is that Surgisphere stumped up the money but wanted to make it look like they had been funded through a respected hospital research grant.
@chrislawson @2:
“What does it matter to the trial if the data was put in cloud storage versus local or network-attached storage?” It only matters in that it means that Surgisphere doesn’t need its own data center, so when people went looking for their office it wasn’t unexpected that they didn’t own a data center. Basically, it’s cover, and also the current Standard Operating Procedure for things with lots of data.
I’m occasionally lightly horrified how much medical data is stored on Amazon Web Services (for example).
The WTO made decisions based on these fraudulent studies before validating them. The CDC told the American people not to use masks, presumably because they didn’t want individuals to fuck up the procurement systems for medical professionals. Well, if they are going to be incompetent and/or deceitful in their processes, it is completely reasonable to ignore their statements — which is exactly what none of us should want!
It is hard enough for me to try to convince people to listen to the experts under normal circumstances. If the scientific authorities are going to lie and/or be incompetent, then I can’t be their ally anymore. How is this even a question?
@JustaTech, if you’re horrified by AWS storage of medical data, don’t look into FEDRAMP. It was largely AWS based, now going to Microsoft’s cloud services because the Orangutan in Thief hates Amazon’s CEO.
Military hangs off of cloud services, as does pretty much every federal agency.
I really am upset that people don’t seem to realize how serious of a fuckup this is. This is going to make my job ten times harder, and we are talking about people’s lives here. I am fucking infuriated.
wzrd1: I’m actually OK with that, as long as the people running the cloud know what it is that they’ve got, in a categorical sense. Like, it’s find to store medical data in the cloud, as long as that cloud is designed and maintained with that type of data in mind.
It’s when people put, say, real-time heart-monitoring data into a database that isn’t intended to have the kind of uptime that data needs, then it’s a problem.
From what my friends say, AWS > Azure any day, but maybe the Softies have gotten around to fixing it.
In early March the CDC and local health departments were discouraging mask wearing for the general public, but they meant N95 and surgical masks which were needed for health care workers. The thing was though, by the time they said this you couldn’t buy medical grade masks anywhere, so it was a rather pointless directive. Then when they started advising people to wear cloth masks it played right into the hands of the Covid-hoaxers who said the CDC was flip-flopping and couldn’t be trusted.
@psychomath, I agree, but I also say, that job should be harder, rather than pencil checking peer approval. I wrote some papers that are classified, they were peer reviewed by people who respected me greatly and tried to apologize for scathing reviews. I halted them and apologized for producing a shitty paper.
Later, my training officer defended me against my lead, who complained “he keeps making mistakes”, he said, “Yes, but he doesn’t repeat the same mistake twice, he’s trying and doing well”. I thrive under a steep learning curve, slow curves end up boring me into near defeat and I dig in, bone up and persevere against the stupidity processes in place.
@JustaTech, I couldn’t agree more. Did five years in military information assurance. Security is job #1 always.
Still, don’t get me started on Microsoft’s security, which still remains job #2 in deference to soupy sales.
@horrabin, knew all about it and was phenomenally frustrated that few science explainers explained that boo-boo. Shit, got caught in that initially, had respiratory distress within the COVID-19 window and am phenomenally high risk. Thankfully, it was an odd type of heart failure due to untreated Graves’ disease.
Still, ginned up a mask made of cloth infant diapers, multilayer, check, a tad better than the Chinese derived 1918 cloth mask, check, pleated wrong, check dammit.
Got some military cravats in today, I’ll incorporate my failed experiments into them and become the greeno bandito, who tips well. ;) When life gives you lemons, squeeze them into your lightly sweetened iced tea!