I came across an interest research article in PLOS One:

What helps and hinders reproducible research? Researchers’ perspectives from a cross-disciplinary interview study by Magdalena Kozula et al.

Debates and policy initiatives addressing research reproducibility have expanded considerably in recent years. Yet, many of these measures remain generic and risk overlooking the lived realities of research practice. This study aims to explore researchers’ perspectives on the barriers, facilitators, and motivators that shape reproducible research across diverse fields and career stages, using qualitative methods. Semi-structured interviews were conducted with 60 researchers affiliated with universities and research institutions across the European Union and the United Kingdom. Participants were sampled to ensure diversity in discipline, career stage, gender, and geography.

As is set forth already in the abstract, there are five themes for barriers and facilitators influencing reproducibility:

• navigating the research ecosystem – incentives and policies of institutions, journals, and funders
• social and cultural dynamics as drivers and barriers – disciplinary norms, generational differences, competition, and collaboration
• resourcing reproducibility – skills, infrastructure, guidelines and standards, time, funding, and awareness
• inside the research process – field-specific constraints, methodological transparency, research material sharing, and external restrictions
• personal commitment to shared responsibility – reflective motivations, pragmatic drivers, and perceptions of accountability

Looking at these themes it is hardly surprising that we are still facing reproducibility issues. As the abstract goes on to explain

Researchers described reproducibility as less of an individual choice but as a socially and institutionally mediated activity, dependent on enabling conditions such as supportive policies, adequate infrastructure, and equitable resource distribution. Reproducibility reform cannot rely solely on individual researcher commitment or one-size-fits-all policies. Effective interventions must account for disciplinary and methodological diversity, provide targeted resources and training, and realign incentive structures to reward transparency and rigour. These findings highlight reproducibility as a collective responsibility across the research ecosystem, requiring coordinated action by researchers, institutions, funders, and publishers. Promoting reproducible practices in this systemic, context-sensitive manner is essential for fostering a more credible, equitable, and sustainable scientific enterprise.

The scientific process is still the best way to find out things, but unfortunately, our current system rewards new findings over reproducing findings. New discoveries get papers in Science, Nature etc. while attempts at reproduction at best get a foot-note, unless they fail, and show some theory or discovery to be wrong. And even then, most of the time, they won’t get an paper, but at most a retraction of the original paper.

I work in the world of software development, where we have the concept of technical debt. This is stuff that we ought to do, or redo, but which we postpone until the costs and consequences of the “debt” is too high to ignore. It might be some complex piece of code, that really needs to be simplified, but which works – but every time anyone needs to change anything in it, something important breaks, and a lot of time needs to be spent on fixing it.

In my opinion reproducibility is the technical debt of the science fields. It can be ignored, until the costs are too high, and it turns out that the fundament a lot of work is based on, is deeply flawed.

One way of handling technical debt in software development, is to set aside a certain percentage of your time to go back and “pay” the debt. I.e. time where you actually revisit the issues and fix the problems instead of pushing it ahead in an ever increasing pile.

Maybe something similar could be done in research. Make sure that a certain percentage of resources are set aside to replicate findings from other peoples’ research. Not in order to debunk the work of others, but in order to make sure that the fundament the current research is built upon, is firm.

Anyway, please read the article, it is interesting and also shows clearly why we are nowhere near removing the barriers to reproducibility.

—

Something completely different; while reading the PLOS One research article, I couldn’t help think about a 2019 American Economic Review article:

Does Science Advance One Funeral at a Time? by Pierre Azoulay, Christian Fons-Rosen, and Joshua S. Graff Zivin

We examine how the premature death of eminent life scientists alters the vitality of their fields. While the flow of articles by collaborators into affected fields decreases after the death of a star scientist, the flow of articles by non-collaborators increases markedly. This surge in contributions from outsiders draws upon a different scientific corpus and is disproportionately likely to be highly cited. While outsiders appear reluctant to challenge leadership within a field when the star is alive, the loss of a luminary provides an opportunity for fields to evolve in new directions that advance the frontier of knowledge

I can’t help we would something similar in regards to reproducibility. If there is a grand old man (and it is pretty much always a man) in a given field, it is highly unlikely that he and his disciples’ findings would be challenged on the grounds of reproducibility. However when the grand old man leaves the field, this might change and people will start question the validity of the original research, and try to reproduce it.