(For other posts in this series, see here.)

The philosopher of science Pierre Duhem said in his book The Aim and Structure of Physical Theory (1906, translated by Philip P. Wiener, 1954) that despite the fact that there is no way to isolate any given theory from all other theories, scientists are saved from sterile discussions about which theory is best because the collective ‘good sense’ of the scientific community can arrive at verdicts based on the evidence, and these verdicts are widely accepted. In adjudicating the truth or falsity of theories this way, the community of scientists are like a panel of judges in a court case (or a panel of doctors dealing with a particularly baffling set of symptoms), weighing the evidence for and against before pronouncing a verdict, once again showing the similarities of scientific conclusions to legal verdicts. And like judges, we have to try to leave our personal preferences at the door, which, as Duhem pointed out, is not always easy to do.

Now nothing contributes more to entangle good sense and to disturb its insight than passions and interests. Therefore, nothing will delay the decision which should determine a fortunate reform in a physical theory more than the vanity which makes a physicist too indulgent towards his own system and too severe towards the system of another. We are thus led to the conclusion so clearly expressed by Claude Bernard: The sound experimental criticism of a hypothesis is subordinated to certain moral conditions; in order to estimate correctly the agreement of a physical theory with the facts, it is not enough to be a good mathematician and skillful experimenter; one must also be an impartial and faithful judge. (p. 218)

This is why the collective judgment of the community, in which individual biases get diluted, carries more weight than the judgment of a single member, like the way that major legal decisions are made by a jury or a panel of judges rather than a single person.

Duhem’s idea that we are ultimately dependent on the somewhat vague collective ‘good sense’ of the scientific community to tell us what is true and what is false may be disturbing to some as it seems to demote scientific ‘truth’, reducing it from being objectively determined by data to an act of collective judgment, however competent the community making that judgment is. Surely there must be more to it than that? After all, science has achieved amazing things. Our entire modern lives are made possible because of the power of scientific theories that form the foundation of technology. In short, science works exceedingly well. How can it work so well if the theories we have developed were not true in some objective sense?

Such feelings are so strong that people continue to try and find ways to show that scientific theories, if not absolutely true now, are at least progressing along the road to truth. Popper’s idea of falsification seemed, at least initially, to provide a mechanism to understand how this steady progress might be occurring.

It was Thomas Kuhn who delivered the most devastating critique of Karl Popper’s idea that scientific theories can be falsified if a key prediction of the theory turns out to be contradicted by experiment. In Kuhn’s landmark book The Structure of Scientific Revolutions (1969), he pointed out that falsification fails in two ways. One way is an extension of Duhem’s argument, that it is never the case that a pure theoretical prediction based on a single theory is compared with a piece of empirical data. In the event of disagreement, there are always other linked theories that can be blamed. Secondly, even if we accept the idea of falsification at face value, it would not describe actual scientific practice. Kuhn’s book contains a wealth of examples that show how scientists live and work quite comfortably, for decades and sometimes even for centuries, with a theory that has been contradicted by data in a few instances, until finally discarding the theory or resolving the contradictions. As long as a theory seems to be generally working well, scientists are not too perturbed by the occasional disagreement, seeing them as merely unsolved problems and not as falsifying events. In fact, he points out that new theories almost always have very little evidence in support of them and disagree with a lot of data. If Popper’s model were applied rigorously, every theory would be falsified almost from the get-go.

So how do old theories get rejected and replaced by new ones? Kuhn says that during the period of ‘normal science’, most scientists work within a given scientific ‘paradigm’ (which consists of a basic theory plus the rules of operation), picking problems that promise to elucidate the workings of the paradigm. They are not looking to overthrow the paradigm but to stretch its boundaries. In the process, they sometimes encounter problems that resist solutions. If these discrepancies multiply and if a few key ones turn out to be highly resistant to attack by even the best practitioners in the field, science enters a period of crisis in which people start seriously investigating alternative theories. At some point, individual scientists start switching allegiance to a promising new theory that seems to solve some outstanding and vexing problems that the old one failed to solve and this process can begin to snowball. Kuhn suggests that the switch from seeing the old theory as true to seeing it as false and needing to be replaced by the new one is similar to a gestalt switch, a sudden realization of a new truth that is not driven purely by logic.

Kuhn’s views aroused considerable passions. Some anti-science people (religious and non-religious alike) have seized on his idea that scientific revolutions are not driven purely by objective facts to extend his views well beyond what he envisaged and claim that science is an irrational activity and that scientific knowledge is just another form of opinion and has no claim to privileged status. Kuhn spent a good part of the rest of his life arguing that this was a distortion of his views and that scientific knowledge had justifiable claims to being more reliable because of the ways that science operated.

Next: The rational progress of science