(For other posts in this series, see here.)

Karl Popper’s model of falsification makes the scientific enterprise process seem extremely rational and logical. It also implies that science is progressing along the path to truth by successively eliminating false theories. Hence it should not be surprising that practicing scientists like it and still hold on to it as their model of how science works. In the previous post in this series, I discussed how Thomas Kuhn’s work cast serious doubt on the validity of Karl Popper’s falsification model of scientific progress, replacing it with a seemingly more subjective process in which scientists switched allegiance from an old theory to a new one based on many factors, some of them subjective, and that this transition had some of the elements of a gestalt switch. This conclusion was disturbing to many.

Another historian and philosopher of science Imre Lakatos was one of those concerned that Thomas Kuhn’s model of gestalt switches implied a certain amount of irrationality in the way that scientists choose a new paradigm over the old or the way they pick problems to work on. In his major work The Methodology of Scientific Research Programmes (1978) he argued that scientists are rational in the way they choose paradigms, and he proposed a new model (which he called methodological falsificationism) that he contrasted with Popper’s older model (which he called ‘naïve falsificationism’), that he claimed solved some of its difficulties

In Popper’s naïve falsification model, when there is disagreement between the predictions of a theory and observations or experiment, the theory must be abandoned. Kuhn and Lakatos agree with Duhem that when such a disagreement occurs, it is not obvious where to place the blame for the failure so summarily discarding the theory is unwarranted. In such situations Duhem appealed to the vague ‘good sense’ of the individual scientists and of the collective scientific community to determine what to do. Kuhn refined this by saying that the choice of which direction to proceed is based on whether the scientific community perceives the existing paradigm to be in a crisis or not, and that when there is a crisis, the revolutionary switch to a new paradigm is akin to a gestalt switch, whose precise mechanism is hard to pin down, in which individual scientists suddenly see things in a new way.

Lakatos agrees with Kuhn (and disagrees with Popper) that experimental tests are never simply a contest between theory and experiment. At the very least they are three-cornered fights between an old paradigm, a new emerging rival, and experiment. But he disagrees with Kuhn that a crisis within the old paradigm is necessary for scientists to switch their allegiance to a new one (p. 206). He argues that a new theory is acceptable over its predecessor if it (1) explains all the previous successes of the old theory; (2) predicts novel facts that the old theory would have forbidden or would not even have considered; and (3) some of its novel predictions are confirmed. (p. 227)

Lakatos says that Kuhn places too much reliance on vague psychological processes to explain scientific revolutions and that the process is more rational, that scientists proceed in a systematic way in choosing between competing theories. In Lakatos’ model of methodological falsificationism, he emphasizes that experimental data is never free of theory. An experimental result in its raw form is simply a sensory observation, such as dot on screen, a pointer reading on a meter, a click of a Geiger counter, a track in a bubble chamber, a piece of bone, etc., none of which have any obvious meaning by themselves. In order to give them some meaning, we have to use theories that interpret the raw sensory experience. For example, a fossil bone that is found is useless unless one can determine what animal it belongs to and how old it is, all of which require the use of other theories. In addition, we have to assume that our knowledge about the other elements surrounding the raw data is unproblematic.

Meanwhile, a theoretical prediction is never the product of a single theory but consists of a combination of four components: the basic theory being investigated, the initial conditions, various auxiliary hypotheses that are needed to actually implement the theory, plus the invocation of ceteris paribus (roughly meaning “all other things being equal”) clauses. For example, to understand the origins of the Solar System, we need Newton’s laws but we also need to make assumptions about the initial state of the gas (the initial conditions), that the laws have not changed since the time the Earth was formed (an auxiliary hypothesis), and that no other unknown factors played a role in the formation (the ceteris paribus clauses).

Lakatos said that when there is a disagreement between a theoretical prediction and experimental data (where the two are interpreted in these more complex ways), scientists use both a ‘negative heuristic’ and ‘positive heuristic’ to systematically investigate and isolate the cause of this disagreement and that this process is what makes science rational.

The ‘negative heuristic’ says that one must deflect attention away from the ‘hard core’ theory when there is an inconsistency between predictions and experiment. In other words, scientists look for the culprit in all the factors other than the basic theory. The ‘positive heuristic’ consists of “a partially articulated set of suggestions or hints on how to change, develop the ‘refutable variants’ of the research program, how to modify, sophisticate, the ‘refutable’ protective belt.” (p. 243) So the positive heuristic tells scientists how to systematically investigate the initial conditions, auxiliary hypotheses, ceteris paribus clauses, etc., in short everything other than the basic theory. These two strategies protect the basic theory from being easily overthrown. This is important because good theories are hard to come by and one must not discard them too hastily.

Lakatos claims that this process rationally determines how scientists select problems to work on and how they resolve paradigm conflicts (contrasting it with Kuhn’s suggestion that scientists intuitively know what to do in such situations). In some sense, Lakatos seems to be fleshing out the rules of operation that Kuhn refers to but does not elaborate.

Lakatos argues that as long as a basic theory is fruitful and the negative and positive heuristics provide plenty of avenues for people to investigate and thus steadily produces new facts that both advance knowledge and are useful (a state of affairs that he calls a progressive problemshift), the basic theory will be retained. This is why Newtonian physics, one of the most fruitful theories of all time, is still with us even though it would be considered falsified using Popper’s criterion. It is only when the theory runs of steam, when all these avenues of investigation are more or less exhausted and do not seem to provide much opportunity to discover novel facts that we have what he calls a degenerating problemshift. At that point, scientists start abandoning their allegiance to the old theory and seek a new one, eventually leading to a scientific revolution.

Next: Truth by logical contradiction