‘Twas the night before Christmas, when all through the house, not a creature was stirring, not even a mouse…
Well, when I say ‘not a creature’, I mean to exclude those that were stirring over governmental impotence in the face of an international crisis. I think I may also be ignoring those who were stirred by tales of stolen elections, jackboots crushing civil liberties and transgender madness. In fact, come to think of it, ‘twas the night before Christmas and all hell was breaking loose. They say Christmas is a time for all the family to get together – before tearing a lump out off each other. Well, if the Cliscep family is anything to go by, this tradition appears to be going from strength to strength. A bit more stirring is the last thing that Cliscep needs.
For that reason, I now wish to turn to a subject that is so bereft of emotional valence, so soporific to discuss, and of so little interest to the vast majority of you, that there can be no risk of disharmony breaking out. Yes, I’m afraid this is going to be like one of those New Year’s Day lectures you used to sit down to watch on television once the festive fisticuffs had petered out. But instead of an audience of middle-class, privileged, swotty little kids, I choose instead for my audience a group predominantly featuring the disenchanted white elderly male. All I ask is that you press the like button as though I had made my point clearly and as if you appreciated the angst-free distraction it provided.
My Chosen Subject
For this year’s lecture, I choose to cover the subject that I believe to be at the centre of the climate debate and one that has been my main preoccupation on this website for the last three years: Uncertainty.
Specifically, it is my intention to answer a question that was asked by Thomas Fuller over on the ATTP website shortly before Christmas:
Just how do you go about quantifying subjective uncertainty?
Given the professed expertise shared by the denizens of ATTP, one might have expected an orderly queue of volunteers lining up to answer the question. After all, it is a perfectly reasonable question to ask and it does have a perfectly reasonable answer. But, alas, it seems they all had better things to do. Never mind. I’m sure what I have to say below was on the tips of their tongues.
Before I start, I should point out that the term ‘uncertainty’ means a lot of different things to different people, and this applies even more so to the expression ‘subjective uncertainty’. This makes any differences of opinion on this subject difficult to analyse. Consequently, it would be wise of me to begin by clarifying what I mean by uncertainty.
Put simply, uncertainty is the condition that appertains when the probability ascribed to the truth of a statement is neither unity or zero. Such uncertainty may be due to either an inherent variability relating to the factors that determine the probability, or an incertitude arising from lack of information. The former, I shall refer to as objective uncertainty. The latter, I shall refer to as subjective uncertainty. I shall also be making the assumption that Thomas had this distinction in mind when he asked his question.
In practice, it is rarely the case that uncertainty falls neatly into one or other of the above classes. This is a problem because the propagation and management of uncertainty differs according to class, and a number of misconceptions can arise if the two basic sources of uncertainty are not carefully separated. But, for today, let us stick to the question of quantification.
The Best that Probability Can Do
The first clue regarding the correct quantification of subjective uncertainty lies in the fact that probability is its hallmark. For objective uncertainty, probability distributions may be constructed that reflect the likelihood that a variable of undetermined value may lie within a specified range when measured, and it is the objectively determined vagaries of nature that lie behind the shape of that distribution. For subjective uncertainty there may be little objective basis upon which to construct such a distribution, so this leads many to assume that the probabilities cannot be quantified. But this is not the case. Probability has always been an ambiguous concept that applies equally to both objective and subjective uncertainties, and can therefore be used to quantify both classes. The real question is how to interpret the quantification and, in particular, how one can judge the proximity of the subjective quantification to any objective reality.
For example, a fair coin will land on heads with a probability p=0.5. But if I were to tell you only that you are dealing with a two-state system that has adopted one of its two states, what probability would you ascribe? In the absence of any further information you should still ascribe p=0.5, since this is the expression of maximum epistemic uncertainty relating to the nature of the two states. Upon learning that you were dealing with the flip of a coin, you would update your calculation but it would, in this example, still remain at p=0.5. All subjectivity and ambiguity has been removed, but the overall uncertainty appears to be unchanged. In this example, the subjective probability and objective probability happen to be the same.
Given that I defined uncertainty in terms of deviation from p=0 and p=1, it should come as no surprise that p=0.5 represents maximum uncertainty for a two state system. In an n-state system the maximum subjective uncertainty is represented by p=1/n for all states. Of course, as further information is revealed, some states may become more favoured and the probability distribution will become less flat. In the general case, the uncertainty (H) encapsulated by a probability distribution is given by Shannon’s equation for entropy:
H = – Σ p * loge(p)
It makes sense that this entropy reflects uncertainty, since the less information is contained in the signal, the less confidence one can have in what it is telling you.
Shannon’s entropy formula applies irrespective of the degree of subjectivity and so it provides the answer to Thomas’s question. However, it should be immediately apparent that the question itself ducks the issue. The real question isn’t the quantification of subjective uncertainty, or indeed the quantification of the extent to which the uncertainty is subjective. The real question is how far, in a given instance, is the value given by Shannon’s equation from the value one would get if the uncertainty were entirely objective. The problem with this question is that it cannot be answered in advance of the subjectivity being removed. However, a measure of confidence can still be quantified if one abandons probability theory and concentrates instead upon the strength and heterogeneity of evidence available and what this tells you regarding justified belief. For this you need a non-additive mathematical approach such as that provided by possibility theory.
The confidence of which I speak is the confidence to be gained when there is a big difference between the possibility of something being the case and something not being the case. This is analogous to the low uncertainty associated with the probability being close to either zero or unity. However, in possibility theory, the weight of evidence supporting each possibility is used as the basis for determining the calculation. I have covered this subject before on Cliscep, and so I will not repeat myself here. Those wanting to know how such a quantification is performed may follow this link and read what is said under ‘Confidence Measured Objectively’. Suffice it to say, the calculation accounts for the weighting of evidence and reflects what this weighting has to say regarding both the possibilities and what is already necessarily the case. The relevant formula is:
Confidence(x is A) = Possibility(x is A) + Necessity(x is A) -1
This is a confidence that is cognisant of what the evidence is telling you but also takes into account the credence one should place in the evidence. As such, it delineates and accommodates the concepts of likelihood and ignorance in a way that probability cannot. More to the point, it is a quantification of subjective uncertainty that captures the extent to which the objective reality remains hidden.
A Reflective Moment
For those that have made it this far, I commend you for your appreciation of the undramatic yet mysterious world of uncertainty analysis. But I am under no illusions and suspect that this article will attract little interest. Who wants Open University when the real-life soap storylines are so dramatic nowadays? This was not an article to get the heart pumping. There are no villains to boo, no heroes to cheer. The only discordance on show is the probabilistic discordance measured by calculations of entropy. And yet I hope there was still enough to engage the reader. The knowledge that uncertainty (and even the extent to which subjectivity is an issue) is amenable to mathematical calculation should be a source of wonder and comfort to those who seek reassurance that the world is not completely intractable. These mathematical insights are out there, waiting for those who are inquisitive enough to investigate. They are vital for anyone wishing to quantify the uncertainties that bedevil the interface between science and politics. And there was simply no excuse for those who chose to ignore Thomas’s question.