A recent study by Jonathan Bamber et al solicited expert opinion regarding the possibilities for Sea Level Rise (SLR) resulting from Antarctic ice sheet collapse. The study’s methodology included a technique known as ‘Structured Expert Judgement’ to improve the reliability of the conclusions drawn by aggregating the opinions within the expert community. It has concluded that the IPCC AR5 estimates are now understood to be too conservative. According to the paper:
“This study suggests that experts’ judgments of uncertainties in projections of the ice sheet contribution to SLR have grown during the last 6 y and since publication of the AR5. This is likely a consequence of a focused effort by the glaciological community to refine process understanding and improve process representation in numerical ice sheet models.”
The study is quick to defend the apparent paradox that greater understanding can lead to greater uncertainty:
“This negative learning…may appear a counter intuitive conclusion, but is not an uncommon outcome: as understanding of the complexity of a problem improves, so can uncertainty bounds grow.”
But how, I hear you ask, can a greater level of uncertainty be seen as an improvement? Once again, the paper wastes no time in providing the explanation:
“We note that for risk management applications, consideration of the upper tail behavior of our SLR estimates is crucial for robust decision making. Limiting attention to the likely range, as was the case in the Intergovernmental Panel on Climate Change AR5, may be misleading and will likely lead to a poor evaluation of the true risks.”
This is not the first time that I have come across climate scientists asserting that “consideration of the upper tail behavior” leads to a determination of “true risks”. Nor is it the first time that I have encountered the perils of allowing “refined process understanding” to create an increased perception of risk.
I worked for some time in the field of safety risk assessment – long enough for an important principle to become apparent: The more sophisticated and detailed the risk assessment, then the greater will be the estimated risk level. This is as a direct result of the increase in the number of extreme impact possibilities that are identified by the ‘enhanced’ analysis, combined with the greater uncertainty associated with the low probabilities that should be attributed to them. In transport safety analysis (my particular field) this effect was captured by the following apocryphal set of safety assessments, each more sophisticated than the previous:
Q: What is the worst case scenario for a failure of this particular road safety system?
A: A road traffic accident resulting in fatalities.
More sophisticated analysis:
Q: What is the worst case scenario regarding road accident fatalities
A: One that involves a busload of schoolchildren
Even more sophisticated analysis:
Q: What is the worst case scenario regarding the death of schoolchildren
A: That their bus collides with a nuclear fuels convoy
This sequence of analyses was known in the trade as the ‘school bus – nuclear convoy scenario’. The point was this: If one analysed a road safety system long enough, one would always end up with the ‘school bus – nuclear convoy’ scenario, and that is no good when trying to decide the required safety integrity of a given safety-related system. Yes, one can design all systems to be catastrophe-proof, irrespective of likelihood, but that’s not how safety management works. Terms such as ‘reasonably practicable’ apply, and so safety assessments are constrained to only address primary outcomes (e.g. road fatality) without speculating upon the full range of possible, non-zero-probability contexts (e.g. school buses and nuclear convoys). Put another way, if you allow yourself to engage in conjecture within areas of high uncertainty you will simply become a hostage to your imagination.
So when climate scientists declare that a further 6 years of modelling of ice sheet dynamics has led to confident proclamations of an increased 90th percentile SLR impact (courtesy of increased model uncertainties) I have to ask whether they are simply succumbing to the thrall of a ‘school bus – nuclear convoy’ scenario. Their models may be more sophisticated, but if that sophistication is simply adding to the uncertainty (glibly attributed to increased complexity) then they are inferior models for the purpose of risk assessment. Sometimes it is important to refrain from adding complexity and concentrate instead upon the evaluation of a more tractable, albeit simpler risk model.
Of course, this wouldn’t be an issue if it were not for the climate scientists’ habit of defining risk in terms of the worst-case scenario impacts that current levels of uncertainty fail to discount. Such an approach simply allows the increased uncertainty to open the door for more speculation of the ‘school bus – nuclear convoy’ variety. The scientists continue to talk of risk aversion and the identification of the “true risks”, but the reality is that they are just replacing risk aversion with uncertainty aversion. No amount of structured expert judgement can save you from this problem.
In conclusion, Jonathan Bamber et al may believe that they have discovered something new about the reality of the “true risk”, but they should not be so hasty. Even if they can argue that previous levels of uncertainty are no longer scientifically justified, they cannot argue that increased levels of uncertainty provide a better basis for risk assessment. As far as I am concerned, the experts consulted could just be falling into the trap of basing their ‘enhanced’ assessments on the climatological equivalent of ‘school bus – nuclear convoy’ speculations. Give them another 6 years and their uncertainty will flood the whole world.