The problem of changing scientific conclusions


People tend to like certainty. One of the things that people get wrong about science is that they think it should provide them with that certainty. But scientific conclusions, while they tend to be the most reliable that we have at any given time, can change in the light of new evidence which is why results are usually phrased conditionally. Unfortunately that nuance is often missing when the media reports science results and this can be disconcerting for some people when new results seem to contradict the old. I deal with this question quite extensively in my book The Great Paradox of Science.

This is particularly the case with research in medicine and health where controlled experimentation is more difficult to do than in the physical sciences and thus inferences are necessarily more tentative.

(Pearls Before Swine)

We see that also clearly with the pandemic advice. When the extent of the pandemic first became apparent, we were given various forms of advice to minimize the chance of contracting covid-19. We were told to avoid close contact with people, use hand sanitizer or in its absence to frequently wash our hands with soap and water, and disinfect surfaces that may be infected. Wearing of masks was not emphasized that much. It now appears that the chance of virus transmission by touching surfaces is small and that it is through the air person-to-person transmission that is most problematic.

Hence mask wearing has become more important and disinfecting less so, though washing of hands frequently is still recommended as being as good as hand sanitizers. It appears that if you are outdoors, you are unlikely to catch the virus simply by passing someone on the street.

Basically it is when you are indoors where the risk is greatest and where wearing masks is most helpful. But instead of welcoming the increased sophistication of the advice, some people are using the changing guidelines to cast doubt on science and the vaccines. We see this also with the grandstanding by some Republican members of Congress who badger Anthony Fauci about the changing guidelines, implying that he is making stuff up just to make life difficult for people.

Perhaps one good thing that emerges from the pandemic is that it has revealed how scientists respond to rapidly changing new evidence. The quote attributed to John Maynard Keynes who supposedly told a critic, “When the facts change, I change my mind – what do you do, sir?” is very apropos even if he never said it. I hope the media becomes better aware of how science really works and thus become able to better able to report on it, and that people are also better able to deal with new results.

Comments

  1. DonDueed says

    Today’s Boston Globe features a headline about fully-vaccinated people who have contracted COVID-19. Fully expected, of course, but you can bet the lunatic right will seize on it and amp up the vaccine hesitation problem.

  2. invivoMark says

    CDC blundered pretty badly in its messaging early in the pandemic, even with the limited information we had.

    We knew that the dominant mechanism of SARS 1 transmission was suspended respiratory droplets and aerosols. We had no reason to suspect that SARS 2 would be different, and yet CDC signs and information on their website emphasized washing hands and surfaces.

    We also knew that close, continuous contact with an infected person was the biggest risk factor, but instead of recommending that people limit the size of their social circles, CDC defined “social distancing” as staying 6 feet away from each other.

    There was massive confusion caused by arguments over the definition of “airborne,” since CDC defines that as droplets smaller than 5um that act as aerosols, and bigger respiratory droplets weren’t technically airborne because they would eventually settle out. But if you tell people it isn’t airborne, they’re going to think it isn’t transmitted through the air.

    So instead of wearing masks in public and maintaining strict social bubbles, we got Purell stations put up at every building entrance and got ready for the Grand Re-Opening of Everything in April.

    Massive stupidity that could have been avoided.

  3. brucegee1962 says

    Regarding the cartoon — I think that nutrition has generally been the worst offender of all the sciences. It’s a poisonous combination — it has the biggest effect on peoples’ daily lives (everyone wants to eat healthy food, but what is healthy?) and it also has the worst track record in terms of outright reversals, as the cartoon suggests. Vaccine science is much more firmly established, but peoples’ experience with the vagaries of nutrition give the anti-vaxxers fertile soil to work with.

  4. Sam N says

    @3, I get that, but part of the problem is simply focusing far too much on any single pathway. The other day I became interested about fructose and how it appears to have very different effects from glucose. Then I got sucked into trying to understand the influence of fats on satiety, saw the fiber modulated it.

    But I de-focused, integrated as much as I could, and I came away with pretty clear conclusions that eating lots of vegetables with complex carbohydrates and balancing that out with about even amounts of fats and proteins is overall healthy, and will do very well at giving you a lengthy satiety for the amount of calories consumed.

    Other than a very clear offender like trans fats, focusing on any other factor seems spurious to me. And that veggie, protein, and fat diet seems to have been a stable conclusion for at least a couple decades now, unlike various fad diets (that inevitably seem to focus in on just one metabolic pathway at the expense of everything else. I throw up my hands when people zone in just eating sugar free, or no fats (terrible idea), or proteins only.

  5. Matt G says

    Someone (Will Rogers?) said “The problem with the world is that the ignorant are so cocksure while the knowledgeable are so full of doubt.” I didn’t get it quite right.

  6. GerrardOfTitanServer says

    Wearing of masks was not emphasized that much. It now appears that the chance of virus transmission by touching surfaces is small and that it is through the air person-to-person transmission that is most problematic.

    I remember arguing with so many otherwise well-intentioned and pro-science people on reddit about this back in the day. It’s a respiratory disease that -- at first glance -- primary infects the lungs and respiratory track. Of course it’s primarily spread by air. There is this old-fashioned dogma in mainstream science that there’s a hard distinction between “airborne” and “dropplets” which doesn’t actually stand up to scrunity, but it still has a lot of sway in the mainstream scientific and medical communities.

    As far as I can tell, this obsession with this hard divide is based in large part on the medical policies in place. For merely “dropplet” transmisson, the previous policy in place was facemasks and not much else. For “airborne”, the medical community would have nurses and doctors in like full body suits with negative pressure patient rooms. This makes some sense. Measles is traditionally considered to be “airborne”, and it can spread from one patient room to another down the hall. By contrast, it seems like COVID-19 can’t do that (or it’s incredibly rare).

    However, this obsession with this hard distinction led to silly conclusions from some in the medical community that only big dropplets can carry COVID-19, and these big dropplets fall out of the air immediately, and therefore there’s no chance that COVID-19 can spread from dropplets that persist in the air for more than a few seconds.

    I was arguing with these people after the one study where they traced infections for some people in a restaurant from the airflow from table to table caused by independent air conditioning units -- people at another table in the airflow from an infected table became infected, but other people at other tables in the same room which didn’t share the same airflow did not get infected. It was at this moment where it seemed pretty clear to me that airborne transmission, in the laymen sense, was pretty important, no matter the minutae of technical terms of the medical community. I think thje refusal to call it “airborne” very badly hurt initial public trust and hurt initial efforts to contain it, and it happened all because the medical community clung to what should have been an outdated and disproven model of transmission (hard divide between dropplet transmission and airborne transmission), which was driven in large part AFAICT by the desire of the medical community to avoid using the magic word “airborne” which would trigger all sorts of advanced and expensive medical precautions like full body suits for nurses and doctors and negative air pressure rooms for individual patients.

    /rant

    Perhaps one good thing that emerges from the pandemic is that it has revealed how scientists respond to rapidly changing new evidence.

    Unfortunately, my experience of this debacle has been the opposite. I saw many proper leading medical experts refuse for the longest time to use words like “airborne”, and continue to insist for the longest time that “(large) dropplet” transmission was the primary mode of exposure and explicitly say that you’re perfectly safe if you’re 2 meters away. It was asinine. This eposide lowered my opinion of the competency of the medical community. At least a little.

    To Sam N
    This is not a problem of the critics of the medical community focusing too much on a single pathway. The problem is that much of the medical community outright claimed that one pathway was completely impossible in its messaging to the public in spite of pretty good evidence to the contrary. The proper thing would have been to say “we don’t know, assume both routes, so wash your hands frequently, don’t touch things, don’t touch your face, and use high quality face-masks like N95 or better whenever you’re near other people, especially when you’re in an enclosed room with other people, and especially when the room has poor ventilation”. Instead, we got bullshit like “don’t worry -- you can’t catch it from the air if you’re more than 6 feet away”.

    Again, I think this was a purposeful decision by many in the community to avoid triggering a panic and a run on supplies and to avoid the legal mandated responses of nurses and doctors in full suits and negative air pressure individual rooms for patients. I think that lying to the public is shameful. I am strongly against lying to the public in this context because of this reason. Instead, strong government action should have been taken to prevent hoarding of supplies and to get supplies to the medical persons who needed it the most, and the medical community should be more open to quickly changing their medical policies re full airtight suits and negative air pressure individual rooms in light of the evidence instead of being dogmatically stuck in their existing response templates.

  7. Sam N says

    @6, Gerrard, I suspect you are quite correct. My mind is quite scattered these days, and in my reply to commenter 3, I was really only talking about nutritional science. No broader implications intended. I recently went down a rabbit hole about it, putting together as much information as possible, and came to unsurprising conclusions: basically, what physicians have been consistently telling people for about 20 years now. (Or at least the physicians I have seen).

    Whenever I see reports that a study suggests this or that specific food is particularly helpful or harmful, I ignore it.

  8. GerrardOfTitanServer says

    To Sam
    Oh yea. I understand. In related news, I consider most nutritional “science” to be no better than a scam. I try for a balanced diet, with lots of different vegetables, and my personal opinion at this moment is sugars, especially refined sugars, are really what should be avoided, and fats aren’t actually that bad. (It’s hard though because chocolate is so tasty.) Oh, and plenty of exercise.

Leave a Reply

Your email address will not be published. Required fields are marked *