The Matthew effect

One of the influential metrics used in support of the idea that we are experiencing a climate crisis relates to the level of consensus amongst climate scientists. The assumption is that it is inconceivable for such a level to exist in the absence of a shared knowledge attesting to the veracity of the claim. Not only is this consensus used to justify eye-watering levels of investment in an urgent pursuit of Net Zero, it also appears to vindicate the use of the term ‘denier’ when referring to those who remain sceptical. From that position much more follows on in the form of endless pontification upon how such individuals can possibly think the way they do. Is it a disease? Is it motivated reasoning? Is there something wrong with their values? Are they anti-science conspiracy theorists? Or are they just plain stupid? Whatever it is, scepticism can’t possibly be justified. Can it?

Without wishing to call into question the quality of thinking and the evidence that lies behind the science of climate change, it is sobering to reflect upon the fact that the development of consensus amongst scientists does not actually require an explanation based upon such quality. Faced with a starting point in which there may be a number of competing ideas with none having superior merit, the mere act of referencing each other’s work will cause an emergent narrowing of viewpoint, caused by nothing more than a basic statistical effect. This effect is an example of what has been dubbed the Matthew effect, in an allusion to the parable of talents from the New Testament:

“For to every one who has will more be given, and he will have abundance; but from him who has not, even what he has will be taken away.”

Matthew 25:29, RSV

The more technical term for this is ‘accumulated advantage’, and it works like this:

Assume two papers have been written on a subject and each heavily cites reference material and related research. Let us assume that both bodies of citation have equal merit, i.e. there is no a priori reason to prefer one citation list over the other, or indeed any one citation against the next. However, it is statistically likely that there will be an overlap, i.e. some references will appear in both lists. Then a third person writes a paper and, in order to generate his or her own bibliography, he or she samples from those of the first two. Statistically, those citations that appeared on both of the first two lists will be more likely to appear on the third, further increasing their preponderance. A fourth paper written on the subject will accentuate that bias, and a fifth even further. After only a few generations, the field will appear to be dominated by the views of a small number of scientists whose ascendancy was based upon sheer dumb luck fuelled by a positive feedback. The careers of scientists are very much dependent upon the number of citations received, and so it is tempting to speculate that most of the high profile and successful scientists who dominate the narrative today do so because they are the beneficiaries of the Matthew effect rather than because their work is of superior quality. It is an empirical fact that a minority of academics attract more citations than the rest of the community put together, and this matters because consensus is not a simple numbers game, it coalesces around success.

You might think that this is a fanciful and naïve scenario that bears no relationship to how science actually works in practice. But there are plenty of scientists and sociologists out there who take it very seriously. Take, for example, American sociologists Harriet Zuckerman and Robert K. Merton, who first coined the term ‘Matthew effect’.¹ They point out that this is how society works in general (fame and fortune attracts more fame and fortune) and there is no reason to believe that the society of scientists operates any differently.² Furthermore, it does not require any conspiracy or ulterior motives; it would happen anyway as the inevitable consequence of a simple statistical effect. And when it comes to the society of scientists it is an effect that is in danger of undermining the epistemological value of consensus.

There are actually a number of ways in which the Matthew effect operates within the scientific community. It is there when faculty heads select candidates for tenure who happen to share their views or research interests.³ It happens when funding is made available to those who have a higher profile; higher because they have previously received the most funding. And it happens when editors of journals assume that the author’s canon of previously accepted papers should be taken into account when assessing the merit of a paper under peer review. This last-mentioned example is of particular interest since it can be shown to have a potentially very damaging impact on the epistemological diversity within a field. And, once again, no conscious or unconscious bias is required. It is just a dumb statistical effect resulting from the application of Bayesian logic in the assessment process. As Remco Heesen and Jan-Willem Romeijn, the philosophers of science who have studied this effect, put it:

“This paper concerns biases that are rooted not in the prejudices of editors or reviewers, but rather in the statistical characteristics of editorial decision making…Hence, even if editors manage to purge their decision procedures of unconscious biases, they will be left with biases of a strictly statistical nature. The statistical biases contribute to the already existing tendency towards a mono-culture in science: a purely statistical Matthew effect.”

The full details of how the mathematics works can be found by consulting the paper.⁴ Put crudely, it concerns how Bayesian updating facilitates the assessment of latent quality when the editor has privileged access to knowledge of an author’s previous work. As a result, the work of the familiar author is more readily accepted. It is the opposite of familiarity breeding contempt. The work of the unfamiliar author will be dismissed as being of inadequate quality but the truth is that the latent quality is not as obvious. The system favours the previously favoured without the need for anyone to send intimidating emails to an editor!

To counteract the pernicious effects of editorial assessment, Heesen and Romeijn propose a fundamental solution:

“Indeed, the ultimate resolution of threats to epistemic diversity through biases in editorial decision making might turn out to be a truly radical one: to do away with editor decisions altogether. Depending on the details of such a system, new problems will undoubtedly emerge, but we may hope that statistical Matthew effects are not among them.”

A lot is made of the need to have one’s work published in a peer-reviewed journal to have any credibility, but if the cost is the creation of a mono-culture with little epistemological diversity, then, as Heesen and Romjein put it, “we may be better served by a system like ArXiv rather than one based upon centralised collection and curation”.

And climate change

Nobody seems to be doing a lot of research into the operation of the Matthew effect in climate science and that is pretty much because the Matthew effect operates within climate science. However, one can find glimmers of interest at the interface between science and policy. Take, for example, the Open University’s ON-MERRIT project (Observing and Negating Matthew Effects in Responsible Research & Innovation Transition).⁵ The project’s manifesto states:

“Open Science for the few is just the extension of privilege. Equity is a key aim of open science, but could Open Science policies actually worsen existing inequalities? Open Science needs resources (funding, time, knowledge, skills), and the traditionally advantaged people usually have more of them. Will their privilege mean that they are the ones to benefit most? How can we avoid the dynamic of the rich getting richer, known as the Matthew effect?”

So it seems the project is more focused upon matters of social justice and equality within science/policy than it is regarding the epistemological significance of the Matthew effect. For example, amongst other things, the project will “highlight the role of gender across all investigated questions.”

I’ve no doubt that this is an important issue, but I do not believe it gets to the heart of the matter. The question to be asked should be whether or not the Matthew effect has distorted the science, or the science policy interface, to such an extent that one cannot take the current consensus as a reassurance that the resulting policies will be the correct ones. The project does not seem to be using an epistemological yardstick to answer this question. It’s as if the existing consensus in climate science is still assumed to be sound even though certain groups are deemed to have been excluded from the right to ask the ‘investigated questions’. They can see the equity and privilege issues but have failed to go further and question whether this has any implications for the inherent value of scientific or policy consensus. I think this just illustrates how effective the Matthew effect can be in covering up its own tracks.

I have emphasised more than once that the Matthew effect is a phenomenon of statistics in which the illusion of merit and agreement can emerge from an initial random seeding that is subsequently reinforced through positive feedback. There does not need to be a spark of genius lurking behind the initial flicker of popularity, nor is enduring success a foregone conclusion for its beneficiaries – despite everything I have said, there is plenty in the scientific method that can act to moderate the effect. On the other hand, there is also plenty in the method of scientists that can amplify and worsen its effects. Whilst no conspiracy or wrong-doing is required for the Matthew effect to work, there can be no doubt that such malfeasance can aid and abet the statistical trend. We have Climategate to thank for giving the world an insight into how some scientists surf the wave created by accumulated advantage and how the system can be gamed. It also demonstrated how success not only breeds success but also breeds a contempt for, and fear of, the inquisitive outsider. Unfortunately, we also have Climategate to thank for giving us an insight into how quickly such insights can be extinguished, as a gamed Matthew effect quickly flexes its arm in a flurry of ‘independent’ enquiries seeking to exonerate. This results in the paradox of a wrong-doing that is as self-evident as it is exculpated. The sad fact is that, once established, a consensus may not be allowed to topple if too much has already been invested in it.

I have not written this article to demonstrate the absence of epistemological value in the climate crisis consensus; rather I simply say that it has a significance that should not be taken at face value. The article is, I hope, a further demonstration that scepticism of consensus has a rational basis and is not the province of the cognitively challenged, as John Cook and Stephan Lewandowsky would have us all believe. Perhaps if these gentlemen took more time to reflect upon the socio-dynamics of science and how even very basic statistical effects can play out, they might not be so quick to condemn the sceptical as ‘anti-science’. One may wish for such an outcome, but I am not holding out too much hope. If anyone exemplifies the fruit of the Matthew effect and how success can therefore be divorced from talent, it may be those two gentlemen. They are surfing the wave and I fear they are not for changing their minds any day soon.

Notes and References:

[1] See “The Matthew effect in science”, Science, New Series, Vol 159, No. 3810, (Jan 5, 1968), pp. 56-63.

[2] Merton has described a number of habits and behaviours that characterise the manner in which recognition is bestowed and communicated within the scientific community. Copying bibliographies without actually reading the cited papers is one of them.

[3] It has been suggested that this is a major reason behind the dominance of string theory in fundamental physics. I have previously discussed such matters here.

[4] See, “Epistemic Diversity and Editor Decisions: A Statistical Matthew Effect”, Philosophers’ Imprint 2019, Vol. 19, No. 39, pp. 1-20. For further works by the same authors, see here and here.

[5] The ON-MERRIT project website can be found here. The project is particularly focused upon institutions and individuals working in agriculture, climate and health.