When Professor Shapiro (of Case Western Reserve University) recently derided sceptics for being unable to grasp the basics of reasoning under uncertainty, he certainly wasn’t the first. The evaluation of the scale, nature and significance of uncertainty is central to understanding climate change, so such accusations are tantamount to refusing sceptics a voice in the debate. Furthermore, Professor Shapiro is a psychotherapist, and so it is unsurprising that his criticisms included comparison to his patients, thereby pathologizing the sceptical position. Whilst debating the Shapiro article with a CliScep colleague, I was asked to provide some resources on the subject of uncertainty analysis. I repeat my response below and, as you read it, I invite you to treat it as the clinical testimony I would offer to Professor Shapiro, should I ever be unfortunate enough to be lying on his couch.
My interest in uncertainty analysis began way back in my project leadership days when I questioned the validity of using risk management tools to estimate completion dates for software development projects. These tools employ Monte Carlo Simulation. One has to understand its limitations, so my first reference is:
It transpired that my misgivings were justified, since a tool that was developed to deal with aleatoric uncertainty was being used to model epistemic uncertainty. For a good account of the importance of this distinction when modelling uncertainty, see this paper.
In turning away from Monte Carlo based modelling, I became more interested in using Bayesian techniques to capture and evaluate uncertainties within predominantly epistemic settings. Since, at the time, I was still working in the field of software development, I gained my understanding of Bayesian methods by reading the output of the likes of Professor Norman Fenton. See his website for a number of valuable resources:
You might also be entertained by reading a historical narrative of the development and acceptance of Bayesian methods in the face of fierce opposition from frequentist statisticians.
I have since come to appreciate that Monte Carlo methods retain a relevance in providing a basis for determining the a priori probabilities used in Bayesian inference, but I remain concerned that their application in climate modelling may be resulting in an inaccurate propagation of uncertainty. This suspicion is based upon the tendency to underestimate uncertainty when restricting oneself to purely probabilistic techniques (perhaps I should say ‘restricting to precise probability’). Since my thoughts on this matter started when I was currently involved in transport analysis, this led me to the following paper:
A good example of the application of possibility theory in climate modelling is:
Other non-probabilistic techniques that have found application in climate modelling include Dempster-Schafer Theory. See:
You may also be interested in this application of Info-Gap Decision Theory:
Thus far in this account, I have concentrated upon variability and incertitude as the prime sources of uncertainty. The third source is vagueness. This has led to the development of Fuzzy Logic as a means of capturing uncertainty when modelling systems and evaluating uncertainty in decision frameworks. There are many sources of information on this subject—just steer clear of Bart Kosko. He may know his stuff but he is also a jerk.
For an example of fuzzy logic’s application in climate modelling, see:
On a related subject, I can highly recommend this overview of the philosophical and linguistic implications of vagueness and its importance in the modelling of uncertainty.
Finally, on the subject of uncertainty in climate modelling, I found the following quite informative as a general overview:
You will note in the above that reference is made to ontological uncertainty (the unknown unknown). I like to think of this as a second-order epistemic uncertainty. The impact of the unknown unknown has been popularized by Nassim Taleb in his book, The Black Swan.
Naturally, given his distrust of a purely stochastic appraisal of uncertainty, Taleb is an advocate of applying the precautionary principle for tackling climate change. I think too much has already been written on the subject of the legitimacy of the precautionary principle. I advise that you restrict yourself to the clarification paper provided by UNESCO, and then make up your own mind:
My involvement in project leadership, quality assurance and various aspects of corporate governance required that I gain an understanding of Decision Theory and Decision Analysis (the distinction between the two is not always clear, but it suffices to say there are prescriptive and descriptive elements to the study of decision-making). I found that these subjects provide useful insights into the nature of risk and uncertainty (and the subtle relationship between them).
There are several methodologies that prescribe the logic that should be followed in making a decision. See, for example, Multi-Criteria Decision Analysis (MCDA):
Other strategies for dealing with situations characterized by ‘deep uncertainty’ include Info Gap Decision Theory (see above) and Robust Decision Making (RDM), see:
If you become interested in this area, you will find yourself getting sucked into Game Theory, in which you will encounter strategies such as Minimax (to minimize the maximum loss) and Maximin (to maximize the minimum gain). See:
The descriptive study of decision-making addresses how decisions are actually made (as opposed to how they should be). This invariably involves an understanding of cognitive science. In gaining an understanding of this area, I was influenced by the seminal work undertaken by Tversky and Kahneman. A good introduction to this can be found in Kahneman’s book, Thinking Fast and Slow.
Another important contributor was Daniel Ellsberg, who drew attention to the irrationality that can arise when decision-makers allow their ambiguity aversion to influence their perception of risk. See:
Given my interest in risk, uncertainty and decision-making, I was intrigued to see that the IPCC had devoted a whole section to these subjects in their Fifth Annual Report:
Unfortunately, I found the IPCC document to be surprisingly uninformed on formalisms (it covers expected utility theory, cost benefits analysis, and that’s just about it) and unduly concerned with the benefits of psychological trickery to gain the public’s support for the alarmist agenda.
Finally, although too many within climate science are determined to downplay uncertainty, I found one person who certainly seems to have understood what is going on and is not afraid to point it out:
I appreciate that this very brief synopsis of my educational journey leaves out a number of important topics (no mention of measurement theory, basic statistics or the neuroscience of decision-making, for example). But I still hope I have provided enough information to justify asking the following question:
Professor Shapiro, is there any hope that you might be able to cure me of my dichotomous thinking?