Uncategorized

The Curious Incident of the Test that was Negative in the Night Time

Before I start, I must confess that I am no Sherlock Holmes. What is more, my understanding of virology extends no further than can be gleaned from having caught influenza more than once. Nevertheless, such experience alone should be sufficient to instil a healthy fear of what SARS-CoV-2 may do to an ailing and aging male body – no matter how sceptical that body may be. But when one witnesses and experiences the civic and economic damage that a government is prepared to inflict upon its people in order to manage a pandemic, the fear can become anything but healthy.

Given such mental health challenges, one certainly would not welcome any further distress arising from the simple desire to understand the case statistics upon which governments are basing their decision-making. Unfortunately, that is exactly the position I am in. There are things I think I know for certain, and there are things that have happened that appear to flatly contradict those certainties. This is all very destabilizing. I’ll start, if I may, with the widely understood certainties, after which you are invited to follow me down the rabbit hole.

Firstly, when interpreting a medical diagnostic test result, one has to take into account the possibility of false negatives (i.e. tests that fail to detect the presence of a disease) and false positives (i.e. tests that record the presence of the disease, notwithstanding its absence). These are respectively referred to as the sensitivity and specificity of the test. RT-PCR testing is no exception to this rule. Indeed, Lancet has advised that the specificity of RT-PCR testing is such that between 0.8% and 4% of negative cases are likely to register as false positives. When the a priori probability of the disease is high (for example, when testing those who are presenting symptoms or have been in contact with a confirmed case) the number of false positives will be significantly exceeded by true positives, and so a positive test result is highly significant. However, once testing becomes more random, the a priori probability drops and the false positives start to dominate, to the extent that the test results become pretty meaningless. All of this is all very uncontroversial; it is just standard Bayesian statistics and a reminder of the dangers of base rate neglect. Indeed, the British Medical Journal has produced an online tool that enables anyone to try various a priori probabilities to see how this affects the reliability of RT-PCR test results.

So imagine my surprise when the UK’s Office of National Statistics wrote this about their national COVID-19 Infection Survey:

“We know the specificity of our test must be very close to 100%”

Their logic was impeccable. If, as they claimed, only 159 positive test results were found in a sample of 208,000, then the least that the specificity could be was 99.92% — a full order of magnitude more specific than the most optimistic figure quoted by Lancet. Given the random nature of the ONS testing, and the relatively low prevalence of Covid-19 within the broader community, the specificity suggested by Lancet would have meant encountering far more false positive test results than genuine ones, and it seems more than a little convenient to me that this had not proven to be the case with the ONS survey. Even more puzzling was the apparent lack of curiosity within the scientific and journalistic communities. Rather than question these results, everyone seemed happy to assume that the ONS was using some especially accurate test technology, despite there being nothing on the ONS website to justify such an assumption. On the contrary, the ONS academic partners have confirmed there was nothing out of the ordinary about their testing arrangements:

“The nose and throat swabs are sent to the National Biosample Centre at Milton Keynes. Here, they are tested for SARS-CoV-2 using reverse transcriptase polymerase chain reaction (RT-PCR). This is an accredited test that is part of the national testing programme.”

On the face of it, a team of top-class statisticians were working back from their data to deduce a test specificity that flew in the face of all of the known science regarding RT-PCR testing, and no one seemed the least bit concerned about this.

Normally, in these circumstances, it is safe to assume that one is missing something very significant. It would only require someone to point out my mistake and I would be able to move on, albeit somewhat chastened and embarrassed. I have tried to resolve the mystery myself, but the best I have come up with is the rather outlandish theory that the ONS sample size of 208,000 was completely misleading. If (let’s say, due to quality control problems) the effective number was nearer to 50,000, then the small number of positive results can still be reconciled with the expected Covid-19 prevalence and a more plausible RT-PCR specificity. But other than to point to the fact that survey participants from 12 years old upwards were allowed to self-administer the swabs, I could think of no credible excuse for assuming that such a catastrophic failure in quality control had taken place. I had no alternative but to live with the prima facie contradiction and get on with life. But then I came across the New Zealand Ministry of Health’s Covid-19 statistics.

If New Zealand is to be believed, by early May, only 25 of its 1,138 Covid-19 cases had been asymptomatic. That represents only 2.2% of the cases, and it contrasts sharply with the statistics arising in other countries (e.g. 40% in US nursing homes and 90% in Northumbria University). Just as problematic is the fact that the New Zealand figures were determined as a result of extensive community testing, i.e. circumstances where false positives would be certain to dominate the asymptomatic Covid-19 headcount, and single-handedly account for far more than 25 individuals. Not only does New Zealand owe the world an explanation for its low asymptomatic count, it also needs to explain how, like the UK’s ONS, they were able to achieve near 100% specificity with RT-PCR testing. Furthermore, there is this online statement to be accounted for:

 “When tests were done on samples without the virus, the tests correctly gave a negative result 96% of the time.”

This is a far from impressive specificity, and one which should result in a significant false positive problem for the NZ Ministry of Health to deal with. But only a couple of paragraphs later they say:

“We expect very few (if any) false positive test results…”

And yet, despite this completely illogical expectation, they are proven correct? This is beginning to make the ONS conundrum look perfectly straightforward in comparison.

I trust that you can now see why I should be left so utterly confused. Two organisations that we should presume to be above reproach are making statements that just do not add up. It is no wonder that I am beginning to doubt my own rationality and powers of comprehension. I am hugely sceptical regarding the ONS and New Zealand figures but I feel obliged to be simultaneously sceptical of my own scepticism. Sir Arthur Conan Doyle famously believed in fairies, so I ought to feel in good company. However, I can’t help but suspect that entertaining such cognitive dissonance for any length of time is the sure path to madness. If someone doesn’t rush to my rescue soon and point out where I am going wrong I may end up in an institution listening to the sceptical voices in my head.

Oh yes I will.

71 thoughts on “The Curious Incident of the Test that was Negative in the Night Time

  1. There’s the famous quote (with various wordings attributed to various people such as Artur C. Clarke and Napoleon Bonaparte):

    Never ascribe to malice that which is adequately explained by incompetence.

    Well, if you can rule out incompetence, that does leave open a certain possibility.

    Liked by 1 person

  2. I have just read back my article and noted an error. The 0.8% to 4% Lancet statistic was obviously referring to negative cases that registered as false positives. This has now been corrected. My apologies for the clumsy error and any confusion it has caused.

    Like

  3. John

    It’s worth reading this, as it will give you an idea of what the problems with the PCR test are.

    As I understand it, for the tier 2 & 3 tests, the cycle threshold was not defined by the government or NHS and the labs were left to set their own. I’ve seen references to these thresholds being set at 35-40. This would have given positive results for many people who, not only were not infectious but may never have had the virus.

    If the ONS is certain about its test specificity then it’s likely that they are deliberately setting the threshold low, in the 25-30 range, which would give them very few false positives, but may increase the false negatives.

    Remember that the ONS and NHS are using the test for different purposes, the ONS what to find the spread of the virus in the population while the NHS was using PCR as a diagnostic tool, for which it was never designed.

    Liked by 1 person

  4. Bill,

    Thanks for the link. It was most helpful. However, I am still not entirely happy. My thoughts are as follows:

    If the ONS had opted for very high specificity by setting the cycle threshold very low, you are quite right to suggest that the sensitivity would suffer. Of course the ONS would need high specificity for their purposes (to deal with low a priori probabilities) but I’m not sure how useful an insensitive test would be to them. Is there a sweet spot here that still suited the ONS purposes? Can such high specificities be achieved with anything like a reasonable sensitivity?

    Why would the ONS leave off such important discussion from their website? There is no mention of a policy of favouring high specificity over high sensitivity. Was this reticence just to keep poor folk such as I guessing? Seems like vitally important information to me.

    The cycle threshold explanation doesn’t cover the NZ Ministry of Health statements. They were quite explicit in quoting a very modest specificity, before then going on to predict (correctly!) a low false positive rate.

    There is still the problem of the discrepancy regarding asymptomatic cases to explain.

    Even so, as a result of your assistance, I feel a lot closer to a correct understanding, Bill.

    Like

  5. I don’t know if this helps John, but false positives is a very specific term with a very specific meaning. The rate of false positives is not the percentage of positive tests which are falsely positive, it is not the percentage of total tests which are falsely positive, it is the percentage of people who are uninfected who test positive.

    https://lockdownsceptics.org/might-most-positive-tests-be-wrong/

    “Consider the claim that ‘the test is 95% reliable’ as a gloss on the observation that the test has a false positive rate of 5% (and, for the sake of simplicity, a 0% false negative rate). That gloss betrays an important misunderstanding of what is meant by the ‘false positive rate’. In fact, the false positive rate denotes the percentage of people without the disease whose tests come back positive. Crucially, this is not the same as either

    the percentage of positive tests that are falsely positive; or

    the percentage of total tests that are falsely positive.

    When ordinary intelligent people describe a test as 95% reliable, they mean that 95% of the results that it yields are correct. The reality concerning a false positive rate (“FPR”) as it is in fact defined, however, is quite different: since the FPR is defined as the percentage of individuals who are not infected but who test positive, it simply does not follow that a 5% FPR translates into 95% reliability for the test, as that is ordinarily understood – even if we assume a nil false negative rate. Rather, the reliability (in the layman’s sense) of the test is a function of both of two factors: (i) the tendency of the test to identify as positive people who are not and (ii) the overall prevalence of the disease in society.”

    Like

  6. Jaime, by my reading John’s already on top of all that. Your link highlights that it’s important to have a good handle on “incidence in society”. Some articles from both an orthodox stance and those of skeptical stance, are using ONS figures as the baseline to determine this. E.g. for the latter as already mentioned elsewhere, Yeadon. Usually, because they are considered “more reliable” than other tests / methodology. Hence critical to know how ONS claims were achieved. In the case of your link, if “Government figures” means the ONS yet again, then despite the authors here *don’t* think they’re more reliable, they run with them as the example benchmark. However, by the claims of the ONS themselves, if we are to believe, they *are* more reliable, specifically having a much lower false positive rate. This wouldn’t be a case of the government ‘blithely assuming’, it is a case of the agency telling them said rate is genuinely low, for whatever it is they’re doing (and per Bill above, maybe low cycle threshold is a part of the explanation).

    Like

  7. >”…by my reading John’s already on top of all that.”

    I’d like to think so. Nevertheless, it is still useful to re-emphasise the nature of base rate neglect, since it can lead to counterintuitive results. And it is all too easy to slip up when trying to explain the problem, as indeed I did in my first draft.

    Liked by 1 person

  8. Confusion ramifies rapidly from a two by two confusion matrix…

    Questions, offerings, no answers:

    How does a false postive arise, since the primer is supposedly tuned to a particular RNA sequence? What is it getting a handle on, if not WuFlu? Another virus? Or is it detecting the merest trace of covid e.g. by cross-contamination and amplifying the bejeesus out of it?

    Low false positives: could these be due to some of: re-tests, confirming the first test result; double tests (two simultaneous tests of same subject, only reporting positive result if both tests are positive); quantitative methods (only reporting a positive when some threshold breached, not mere detection of viral genome); statistical jiggery-pokery (a calculation of actual positive cases using known or assumed likelihoods of wrong tests)?

    Something I have wondered for a while is why “cases” are not reported with confidence intervals, which a) ought to be possible for a smartypants to calculate and b) a truer picture of uncertainty but c) possibly confusing to the guys and gals presently banned from the Dog and Duck.

    Like

  9. Jit,

    Specificity and sensitivity are, by definition, qualities that refer to a single test. So when someone claims to have achieved ‘near 100% specificity’, they are saying that few, if any, of their swabs returned a false positive result, i.e. retesting did not substantially alter the numbers. They are not saying that their testing regime was designed to account for false results, thereby rendering their case statistics nearly 100% reliable.

    That said, retesting is obviously the way to deal with any realistic levels of specificity and sensitivity. For example, when using the test to validate a clinical diagnosis, a single negative test is never taken at face value – the impact of letting a case slip through the fingers is deemed too high, especially when dealing with health workers. Similarly, if false positives are deemed to be the main problem, a policy of retesting would deal with them. The ONS survey involves regular retesting of its participants and that may be why they are untroubled by the false positives. But that does not explain how they can claim not to have had them in the first place.

    Liked by 1 person

  10. The government has plans to perform whole-community testing, i.e. test everyone within a given town or city. To do this they will need a test that is a lot cheaper, quicker and easier to perform than RT-PCR. For that reason, they are looking to use RT-LAMP (reverse-transcribed loop-mediated isothermal amplification). This is a lot easier since it only requires a sample of saliva and it is possible to obtain the test results more or less in situ. However, the key question is whether or not it has the required specificity and sensitivity for the purpose. Ideally, of course, one would like that RT-LAMP can be made both highly specific and highly sensitive at the same time. However, I’m not sure this position has been achieved yet. I found the following meta-study that seems to provide all the relevant data, and it seems the jury is still out.

    https://www.medrxiv.org/content/10.1101/2020.07.09.20150235v1

    I do hope they get it right before committing.

    Liked by 1 person

  11. Here we go!

    Announced today: Three towns (Redcar, Accrington and Bishops Stortford) to be used for a pilot in which every citizen is requested to take a Covid-19 test, whether they have symptoms or not. The test is a saliva test that returns a result in less than 30 minutes, i.e. it is RT-LAMP. The purpose is to get a better idea of how many cases, particularly asymptomatic ones, exist within the community.

    As we all know by now, boys and girls, such a survey demands very high test specificity, otherwise a bogus epidemic of asymptomatic disease will be ‘detected’. The meta-study I cited above (the study that looked at all of the published studies that make claims for test accuracy), says this about RT-LAMP:

    “In conclusion, our systematic review and meta-analysis reveals the current state of nucleic acid POCTs for human coronavirus. Overall diagnosis accuracy of these POCTs reported so far is high but the quality of these studies was still in question.”

    That was the situation just 3 months ago. Doesn’t seem to be a problem now!

    Like

  12. For anybody out there who is still interested in some science, an interesting development has been reported by the BBC this morning:

    https://www.bbc.co.uk/news/health-54786130

    The article announces plans to do city-wide Covid-19 testing – piloted in Liverpool. It is an interesting article to me since it contains a few juicy quotes that add to the general confusion regarding test specificity and sensitivity. Sir John Bell of Oxford University is quoted as saying it is a “game changer”. The BBC report states:

    “The pilot will start this week and will include a mix of existing swab tests and new lateral flow tests, which can provide a result within an hour without the need to use a lab.”

    I presume ‘existing swab test’ is an allusion to RT-PCR . Lateral flow is an emerging technology, and I’m surprised that it should be involved. On the other hand, RT-LAMP does not get a mention. This also surprises me since that is the technology being used in the other three pilots (Redcar, Accrington and Bishop’s Stortford).

    Given this confusion, I don’t know quite what to say regarding test accuracy, but that isn’t stopping the BBC, who claim “the quick tests give about one in 1,000 false positives”. They also quote Sir John Bell of Oxford University as saying that they give “quite a good indication of people who are infectious”. I assume this is a reference to test sensitivity, since they go on to quote Sir John warning of the subsequent dangers of trusting a negative result.

    An analysis of the news item is offered by the BBC’s Health and Science Correspondent, but I am not convinced he fully understands the concept of base rate neglect, since he suggests that the specificity problem is one of testing large numbers of people but fails to mention the real issue of targeting asymptomatic people. Even so, I have him to thank for bringing my attention to SAGE’s consensus statement on the topic of mass screening. I won’t provide the link because it is to a pdf document, and I know this causes some of you problems. Search instead for “Multidisciplinary Task and Finish Group on Mass Testing”.

    The SAGE statement cites test specificity and sensitivity for the various test technologies but I see nowhere a claim for 99.9% specificity, as per the BBC article. I might come back to this when I have had time to read the SAGE statement more carefully.

    Like

  13. John: I heard that on the radio, early, and thought of you.

    Does that make me “still interested in some science”?

    Answers in a test tube, whenever you’re ready.

    Like

  14. Richard,

    >’Does that make me “still interested in some science”?’

    Maybe. The important thing, though, is that you were thinking of me. Why else would I post here? It’s all just attention-seeking when all is said and done.

    As for the SAGE statement, this seems to capture the whole issue nicely:

    “ The cheaper, faster tests that will be useful for mass testing are likely to have lower ability to identify true positives (lower sensitivity) and true negatives (lower specificity) than the tests currently used in NHSTT. Problems of low sensitivity would be decreased by very frequent testing. In populations with low prevalence of infection, mass testing that lacks extremely high specificity would result in many individuals receiving false positive results. In such circumstances, rapid follow-up confirmatory testing will be needed to determine whether individuals should continue to self-isolate – it is important to rapidly isolate infectious individuals, but efforts will be needed to quickly release false positives.”

    At last! A statement that makes complete sense. Instead of making implausible claims for specificity, SAGE has acknowledged the problem and explained how it is to be addressed. That’s all I ever wanted.

    Liked by 2 people

  15. Update:

    The government has now abandoned plans to test Redcar, Accrington and Bishop’s Stortford so that they can concentrate on Liverpool. Understandably, there are now three sets of councillors that are a bit pissed off that they were tasked with making preparations, only then to be told that the government has changed its mind. There are also three sets of townsfolk who have been offered a testing service that has now been withdrawn. There is much grumbling.

    As for test technology, everything but the kitchen sink seems to be intended for Liverpool, i.e. RT-PCR, RT-LAMPS and lateral flow tests are to be used.

    The BMJ is now citing 99.3% specificity for lateral flow. It also makes the point that false positives will be virtually eliminated by performing follow-up tests. I suppose that sounds feasible when one is using a test that has a 30 minute turnaround.

    Liked by 2 people

  16. John, do you have the mathematical acumen to tell us what the chances are, using a test that gets say 4 out of every 5 positives wrong (to pluck a figure out of the air), and across a random population with very low infection, what the false positive rate would be after the second application of *the same* test? i.e. when it is then testing a population with a far higher infection rate (1 in five). I presume this will depend on those curves you showed us.

    Like

  17. Andy,

    Since you ask the question, it would be ill-mannered of me not to attempt a response before I sod off for good.

    I think the simplistic way of thinking about false positive rates after retest would be to point out that the odds of a double false positive are much lower than the odds of a double genuine positive. The former is a function of the specificity, irrespective of prevalence. The second is largely a function of prevalence and sensitivity, i.e. with good sensitivity, a genuine positive is almost certain to be followed up by another positive, irrespective of specificity. As a result, in a group of double positives, the double genuine positives can be assumed to greatly dominate the number of double false positives in all but the lowest of prevalences.

    I am conscious that I may not have explained that very well and I have not actually answered your question.

    Anyway, thanks for the debates we have had. That’s a thing I may learn to miss. Cheers.

    Liked by 3 people

  18. Richard,

    Just for the record, I should point out that my decision to no longer contribute to this site has nothing to do with a sense of humour failure. Comments like ‘little gang’ and ‘gang of four’ and unsubstantiated accusations of groupthink are characteristic of a line of arguing that replaces rationality with factionalism. Geoff’s “Drake’s Army” quip, no matter how light-hearted the intention may have been, simply confirms that this is now an accepted line of reasoning on this site. We used to pride ourselves that our scepticism could not be neatly packaged and that individuals should be treated as such. But Jaime seems determined to deal with you and I as if we are some sort of chimera, and there seems to be no limit to how much she can conclude from a single ‘like’. I don’t think this way of thinking should be encouraged. Now that Jaime has started to dominate the airwaves, Cliscep is turning into the very thing that it claims to abhor.

    Moreover, Geoff’s quip brought it home to me just how easy it is to slip into lazy labelling in which agreement on one point starts to be taken as partisan agreement. When that results in someone assuming I am in your ‘army’, Richard, the risk to my reputation is low. However, should anyone start to assume I am in Jaime’s army I would have to ask myself, do I really want to be associated with someone of such stripe? The answer I have come to is, no, I’m more than happy to retire from the forum and leave its destiny in the hands of the self-appointed, one true sceptic.

    That said, tomorrow is another day…

    Liked by 2 people

  19. Oh John, it’s sad indeed that you have decided to vacate the Cliscep airways, but alas your stated reason for doing so – me – should be regarded with some degree of scepticism I feel. I do not ‘dominate’ the conversation here, as you suggest. For some time now, Richard’s (and others) condemnation of my ‘fascism rhetoric’ and one Holocaust analogy have dominated the conversation. You chose to comment on a post which was put up by Richard with the purpose of ridiculing me personally for my opinions and my alleged use of ‘rhetoric’ and hyperbole. If that post had not been put up, none of this would have been an issue. I was baited. I didn’t want to take the bait, but when I did, I chewed down hard and sadly, your argument (plus that of Richard, Mark and Alan) fell apart, an argument which went beyond whether it was wise or appropriate to bring up reference to Nazis, the war, the Holocaust, or fascism, straying into mask diktats and government interventions generally, with me then questioning the quality of scepticism on this site and pointing out the actuality that I was being criticised on the same issue by four people – which I was, often not very constructively.

    Further to skirmishes on other threads, you decided to challenge me further on a post which was written to ridicule me. Did you expect that I would not respond? That I would not defend my position? I did. You just didn’t like the way I responded. You can’t blame me for that I’m afraid. You mentioned my ‘troubling hyperbole’ on the Covid, Climate, and Brexit thread and I responded to that by directly quoting from Lord Sumption’s lecture which was indeed very troubling but which you chose to ignore and instead emphasise that my rhetoric was troubling simply because it troubled Richard. Thus, you subsequently aligned yourself with Richard to criticise me on the Dad’s Army satire post.

    Now, you accuse me of this:

    “But Jaime seems determined to deal with you and I as if we are some sort of chimera, and there seems to be no limit to how much she can conclude from a single ‘like’. I don’t think this way of thinking should be encouraged. Now that Jaime has started to dominate the airwaves, Cliscep is turning into the very thing that it claims to abhor.”

    Like, I have an agenda to ‘deal’ with everyone on this site do I? What of the accusation of me drawing unlimited conclusions from a single like? How on earth did that come about? This is how it cam about. I responded to your comment:

    “You then responded with:

    “Yep. We’ve seen this with smart motorways. We’re seeing it with ‘climate change’ and net zero. We’re seeing it clearly with Covid-19 lockdown policies. Deaths will result/have resulted from the imposition of all three madcap policies. Deaths from net zero will remain effectively hidden for years, but reliance upon scare stories generated by computer models and uncertain science to formulate government policy will hopefully come into sharp focus in the coming months as the death toll from lockdown becomes more obvious.”

    Do you recall who liked that comment?”

    With this:

    “You liked my comment on avoidable deaths caused by madcap government policies, but you also liked a comment from Mark just above which included this absurd, non-evidence based and actually deeply offensive comment:”

    ” In a local supermarket the other day a man with a mask (thus demonstrating he didn’t have a medical reason for not wearing one) took off the mask while he made a tedious and lengthy call on his mobile ‘phone. All he had to do was go outside, or deal with the call a few minutes later, but instead he preferred to break the rules.

    It’s thanks to mindless and selfish morons like these that a weak government is imposing ridiculous restrictions on those of us who do have sufficient concern for our fellow citizens. I despair both for the reasons I expressed earlier, and also for the reasons expressed by Alan. Despair all round, really…”

    What I concluded, far from being limitless, was that you agreed with Mark on that point and I made this clear in the ensuing online debate and you made no effort to counter that assumption. That was it.

    So, whatever you decide, please do not try to shift the blame onto me for having an agenda to ‘deal’ with everyone on the site, to ‘dominate the airwaves’ or promote some line of argument which replaces ‘rationality with factionalism’. I have consistently stood up for rationality and individualism throughout my years spent commenting and writing on this site. That is very plain to see.

    Liked by 1 person

  20. Jaime,

    This is an automated message. Please do not respond.

    John’s army has always stood for scepticism tempered with mutual respect. For further information please visit our website: johnnolongergivesaratsarse.co.uk

    Like

  21. Anyway, back on topic, with or without John to keep us company. The Liverpool moonshot mass testing is not going too well by the looks. I mean, I just can’t understand it. Why aren’t healthy Liverpudlians queueing up to get a swab shoved up their nose by one of these grunts?

    Liked by 1 person

  22. “Why aren’t healthy Liverpudlians queueing up to get a swab shoved up their nose by one of these grunts?”

    Probably because if you test positive, whether falsely or not, and then even accidentally do something that infringes the vague but emotively backed rules, you’ll get a hefty fine 0:

    Like

  23. Jaime:

    For some time now, Richard’s (and others) condemnation of my ‘fascism rhetoric’ and one Holocaust analogy have dominated the conversation.

    Where’s some objective proof of that?

    A number of people have recently joined me in questioning not just your rhetoric but the unreflective certainty about motives that goes with it. But for me that debate has been a minor part of Cliscep, even since your hateful quip about your fellow-countrymen in September. I don’t regret calling you out at that point, nor about your faux certainty about the evil already entrenched in government, in your view, in March. It would have been the easiest thing in the world for you, just once, to admit that you had gone too far, especially in September. Instead, toys were thrown out of the pram (thanks for that phrase just now) and you left Cliscep in high dudgeon, outraged that someone had dared to tell you that you had crossed a truly dreadful line.

    We’ve seen you play the victim like this before.

    When was the last time that you admitted that your critics had a point?

    I made a lighthearted comment to John earlier about the secret emails I’d been sending to control “Drake’s Army” as Geoff foolishly dubbed what he thought was going on yesterday. The real joke is that there are none. I have never talked about you behind your back. Can you and others say the same about me?

    Interested to know the answer there, before we continue.

    Like

  24. It seems from John’s comment at 1.24 pm that I am responsible for his going, for which I’m truly sorry. I thought it was clear that my Drake’s Army was no more than a dig at the title of Richard’s post, which contained a dig at Jaime, who replied with a dig at Richard… All very futile I know, but hardly a resignation matter. I suggest that the rest of us try and raise the level of debate to encourage him back.

    I started to write a post last night which had “Two Chimeras” in the title, and now I see John has put two in his comment! Spooky?

    Like

  25. Richard,

    “Jaime:

    For some time now, Richard’s (and others) condemnation of my ‘fascism rhetoric’ and one Holocaust analogy have dominated the conversation.

    Where’s some objective proof of that?”

    Your very words are objective proof of that. Your ridiculous, insulting Dad’s Army post is objective proof of that. The Covid Climate Brexit thread is proof of that. You are OBSESSED with bringing me to account for my use of language and I will NOT play your game any longer. You are also extremely keen to cast aspersions upon my character. I shall no longer respond to your obsessive demands to account for my alleged sins nor will I respond to your personal attacks. AFAIC you are largely responsible for this monumental bust up, not me, because you just couldn’t let sleeping dogs lie – and you still can’t.

    I’m not going to stomp off in a huff. You’ve lost a valuable contributor to this site because of your behaviour but I will continue to post comments.

    Over and out.

    Like

  26. I repeat this part with emphasis:

    A number of people have recently joined me in questioning not just your rhetoric but the unreflective certainty about motives that goes with it.

    Did you read that? Have you responded to it?

    If you had removed the comment about the cattle-trucks in September we would have talked about the issue far less, I’m sure of that. Perhaps we can agree on that and go from there?

    You now seem to be blaming me for John’s departure. Am I right that you are blaming me for that? He seemed to be blaming you, and a culture he thinks you have come to dominate, triggered by a phrase from Geoff. I assume it’s repairable.

    Like

  27. Guys, guys, calm down. I’m every bit as much an attention seeking prima donna as the best of you and I’m sure I’ll be back as soon as Netflix begins to lose its sparkle – mark my words 🙂

    Liked by 3 people

  28. JOHN
    Oof. Thank for that. The Judaean People’s Liberation Front is reunited again. Wasn’t the genius of Monty Python simply that they reproduce what happens when half a dozen people get together, and making it funny instead of tragic?

    Liked by 1 person

  29. folks – as a long time reader I want all points of view to be heard & argued over.
    it’s like a pissed pub chat, at times it gets heated & people fall out, only to come back the next night & say “I never meant it that way” & the other says “ok – maybe I over reacted”

    Liked by 2 people

  30. Hi everyone,

    How are we all this bright, fine November morning? For the foreseeable future I intend restricting my comments to those germane to this thread. This seems the temperate and moderate thing to do in view of recent controversies.

    In that vein, I would like to point out that the BBC said yesterday (regarding the Liverpool pilot) that lateral flow tests will be employed in the field, with RT-PCR testing being used to confirm any positive test results. This sounds very sensible and will deal with the menace of false positives (who wants to use a testing protocol that is destined to generate asymptomatic cases, when one is looking for asymptomatic cases? It would be a bit like using a PCA algorithm that generates hockey sticks to look for hockey sticks; as if anyone would ever do that!)

    The Liverpool pilot is very important since it will provide the first reliable data on the asymptomatic proclivity of the disease. Wildly different values for this have been reported around the world so it would be nice to finally get some handle on it. Uncertainty regarding the role of asymptomatic covid-19 in driving the pandemic lies behind much of the precautionary strategy that is being adopted. It would be nice to be able to confirm the New Zealand figure (only 2% asymptomatic), but we will have to wait and see. In the meantime, here’s to good old-fashioned data collection.

    Liked by 3 people

  31. Andy,

    You are quite right to point out that the Liverpool pilot will involve only voluntary testing. Clearly, the protection of civil liberties remains imperative. The concern, of course, is that the voluntary nature of the testing may lead to a poor take-up.

    However, of greater concern would be a lack of proper advice given to those who receive the tests. Do you believe that they will be warned that a positive result still means that they are more than likely to be free of covid-19? Do you think that they will be warned that only a swab retest analysed in the laboratory will be able tell them whether they actually have the virus? Do you think that Joe public will understand that this is actually how it is intended to work? Do you think that the numbers of people receiving positive tests that are then overturned will escape the attention of the press? Do you think that the press will understand what is going on, instead of immediately assuming that the testing is being botched? Do you think they will resist the urge to point the finger at the nearest private, outsourced service provider (which usually means Serco) and just whinge on about how the contract was awarded to an incompetent organisation out of pure, political cronyism? And, in the meantime, what about all of those head teachers shutting their schools in reaction to the false positives? Do you really believe that the schools will be immediately re-opened when the result is declared a false alarm? And, even if they are, do you really think the Head teachers will understand that it was really all their own fault for not understanding about base rate neglect?

    As you said earlier, what could possibly go wrong?

    Liked by 1 person

  32. I’m getting more than a little worried about how things are going at the moment. It isn’t that the government is already rolling out the mass testing to 67 regions in addition to Liverpool:

    https://www.bbc.co.uk/news/uk-54885657

    Rather, it’s that no-one in the media or government is making any mention of the fundamental difference that exists between this testing and the NHS track and trace. With the NHS T&T, a positive result was likely to be accurate. With the mass testing, a positive result still means the individual probably hasn’t got Covid-19. A follow-up RT-PCR test is essential before a diagnosis can be made. This is so profoundly significant one would have thought that it would be the first thing mentioned by anyone who covers the subject. But no. Not a dicky-bird. For example, the message from the following BBC article is loud and clear:

    https://www.bbc.co.uk/news/explainers-54872039

    According to the BBC, if you get a positive lateral flow test result, then you have Covid-19 and you have to self-isolate immediately!

    We’re doomed.

    Liked by 1 person

  33. I was horrified by the mass testing in Liverpool and the way it has been rolled out in schools, avoiding parental consent in many cases. But it seems the government’s coercive tactics have back-fired, because the army, using lateral flow testing, after testing 23k people so far, have only come up with a figure of 0.7% positive, which is about the same as the false positive rate. The ONS estimate is 2.2% positive in the NW. So they’ve shot themselves in the foot, proving only that Covid-19 is either not as widespread in Liverpool as suggested by PCR, or in fact it’s not even there at all.

    “The Government will be left with a choice when faced with the gap between the 2.2% figure from PCR testing and the 0.7% figure found by the Army using the new test:

    They could argue that cases fell by two thirds, from 2.2% to 0.7% in a week, and risk being proved wrong with the next round of ONS testing.
    They may claim that these new tests are missing two thirds of cases, and then be forced to abandon the new test as defective. They will then be left with the contradiction of there being no cases being diagnosed in the Liverpool community, but apparently continuing problems in hospitals where everyone is tested.
    They will have to admit that the 0.7% test is actually more accurate and that therefore there are serious problems with false positives from the PCR test results and finally start addressing those problems.”

    https://lockdownsceptics.org/2020/11/11/latest-news-190/#mass-testing-by-the-army-turns-out-to-be-a-good-thing

    Liked by 1 person

  34. Jaime,

    I have the following points to make:

    1) I’m a bit puzzled why Dr Craig is choosing to interpret the bulletins the way she is. Her assumption appears to be that the published data is always the uncorrected data prior to RT-PCR corroboration. Given that the false positive problem was known to SAGE, and that the additional RT-PCR testing was built into the test protocol specifically to deal with it, it seems strange that unreliable data should be knowingly published, even though there has been plenty of time now for the RT-PCR corrected data to have become available. For example, in this morning’s tweet Dr Craig writes as if RT-PCR results are still awaited, but that cannot be the case because it is only through the RT-PCR results that the authorities would be able to confirm the specificity that they have encountered using the lateral flow test (0.3%). I may be wrong, of course, but I am inclined to conclude that the 0.5% figure quoted is the positivity rate after RT-PCR adjustment, rather than before adjustment. If I’m wrong, we can expect an imminent announcement that there is no COVID-19 in the Liverpool community anymore. In fact, that announcement would be overdue since I suspect that fact should already be known to the authorities.

    2) On a positive note, it is indeed good news that the figures appear to be dropping. However, once again, I think this serves to suggest that the figures are the adjusted ones after false positives had been removed. If the positivity rates were to be interpreted as being almost wholly due to specificity problems, then there would be no expectation of any significant fall.

    3) As for the comparison with ONS data, I wouldn’t read too much into that. Remember that the ONS has claimed near 100% specificity for their testing. The implausibility of their claims is the basis for the article I have written here. Also, it is probably worth mentioning that the ONS pilot used random samples (albeit within selected geographical areas and demographics). The Liverpool tests are performed upon volunteers – the very antithesis of a random sample! One would expect the results to be different for this reason alone.

    4) I would prefer it that accusations of civil liberties being violated be kept off this thread, since there are plenty of other Cliscep threads that are better suited for that purpose, and I have chosen to withdraw from that debate. Here, I want to concentrate on technical matters. The only concern I have regarding the concentration on school children is that it is bound to skew the statistics (i.e. saying more about prevalence in schools than prevalence within the broader community). Children being tested because their parents were unable to arrange an exemption letter are of lesser concern to me, at least for the time being.

    Liked by 1 person

  35. John, those 44k tests were carried out yesterday. I’m not sure they would have had time to re-test all those people with PCR. Besides that, people are apparently reporting that they’re being told to isolate immediately after a lateral flow positive, without undergoing the confirmatory PCR.

    Liked by 1 person

  36. Jaime,

    Are you sure that the 44k were carried out the previous day? Dr Craig’s tweet makes it sound like the cumulative total:

    “So far Liverpool has tested 44k…”

    I’m not at all surprised that individuals are being required to act upon receiving the preliminary test results without awaiting RT-PCR confirmation. This is a problem that I anticipated on this thread earlier:

    “According to the BBC, if you get a positive lateral flow test result, then you have Covid-19 and you have to self-isolate immediately! We’re doomed.”

    I wouldn’t even be surprised to hear reports of those, who tested positive on lateral flow, subsequently being advised that they would not be receiving a further test – not everyone on the frontline is going to be on-message. However, I would be a little surprised (but much more so, disappointed) to find out that re-tests were not actually being performed, since this would be an abandonment of the SAGE advice and all common sense. According to Sir John Bell of Oxford University’s calculations, if you were to test 60 million people twice a week this adds up to nearly a quarter of a million people incorrectly told to isolate every week. In his lifetime, the Reverend Bayes was largely ignored. Methinks he shall have his revenge.

    The reality is that I’m struggling here because I am reacting to tweets and attempting an interpretation of assertions that do not cite the original source. I need to get hold of some first-hand information before I can comment sensibly.

    Liked by 1 person

  37. The end-of-week results for the mass testing have been announced by the Liverpool City Council: Around 90,000 tests performed and 336 ‘cases’ detected:

    https://www.bbc.co.uk/news/uk-england-merseyside-54931723

    Of course, it is still as clear as mud as to whether the 336 represents confirmed cases or just lateral flow positives, but the wording used by the article was interesting: “Lateral flow tests have confirmed 336 positive cases”. If the BBC knew anything about the subject, they would know that lateral flow tests can confirm diddly-squat – you need the RT-PCR follow-up tests for confirmation.

    Interestingly, 99.3% has been claimed for lateral flow specificity, and so if everything had gone according to plan, there should have been at least 630 cases from false positives alone. So if the 336 is genuinely the total number of lateral flow positives prior to adjustment for false positives, not only must we believe that Covid-19 is virtually absent in Liverpool, but also that lateral flow testing has turned out to be far more specific than was previously understood. Credibility is being stretch here I think, so I am going to carry on for the time being thinking that 336 is the figure after false positives have already been accounted for (albeit subject to an under-representation due to delays in the availability of the latest RT-PCR confirmation results).

    Even more interesting, though, is the complete absence of anything said about the rate of asymptomatic cases. A major point of the mass testing was to root out the supposed army of asymptomatic Covid spreaders. Prior to the testing, Terry Whalley, Covid-19 testing director for the NHS in Cheshire and Merseyside, had said “We are working on the assumption that we are finding less than half of the people who are walking around with Covid and are therefore inadvertently and unknowingly passing it on to other people.”

    Maybe I have got this wrong, but if that had been the case wouldn’t they expect a dramatic increase in the number of detected cases once the mass testing had got under way? Does the meagre Liverpool 336 validate what had been Terry’s working assumption? I wonder what his working assumption is now.

    Liked by 1 person

  38. Rock God guitarist, Brian May, has now succumbed to the Base Rate Neglect pandemic that is sweeping the nation. On his Instagram page he has posted the following regarding the Rapid AG antigen test:

    “I think this is important! Maybe there is some flaw in my reasoning, but, as I see it, this could be one of the most life-changing innovations in the world right now. There has been very little talk of it in the media – where the news has all been focussed on new hopes of an effective vaccine. But it seems to me this rapid COVID-19 test is the answer to a prayer.

    You can do it yourself, and the result is ready in exactly 15 minutes. This is my first result. The fact that there is no colored lower line in the display means that I have tested negative – in other words, there is at the most, a tiny amount of the virus in my body, and I am not infectious to those around me. If I were anywhere near becoming infectious, as I understand it, there would be a faint line in the lower position. And if the line were solid, it means I am definitely infected and infectious (whether or not I have symptoms)and must quarantine immediately.”

    Sorry to rain on your parade, Brian, but there is indeed a flaw in your reasoning, and you would know this if you had read the following before getting too excited:

    https://www.cdc.gov/coronavirus/2019-ncov/lab/resources/antigen-tests-guidelines.html#table3

    Just to spell it out: Low Pretest Probability = Low Positive Predictive Value

    So I am afraid that ‘definitely infected’ just doesn’t come into it. If a Phd in Astrophysics can’t get this right, then what chance does a humble Health Secretary or BBC journalist have?

    Liked by 1 person

  39. A few days ago I said on this thread that “Children being tested because their parents were unable to arrange an exemption letter are of lesser concern to me, at least for the time being.” I chose my words carefully, saying ‘of lesser concern’ rather than ‘of no concern’. The most important thing is that everyone understands the purpose of the pilot, and is able to interpret the results correctly. In particular, I have expressed the concern that, as a result of such confusion, people will be put in self-isolation unnecessarily. But that is not to say that there aren’t other questions of medical ethics associated with the approach taken by various authorities, particularly when children are involved. I think such concerns are nicely captured by the following letter posted in the BMJ:

    https://www.bmj.com/content/371/bmj.m4268/rr

    In particular, the following is well worth heading:

    “Whilst we would all welcome rigorous research for the new rapid tests and population studies, these must surely be conducted with appropriate ethics committee approval and a sound methodology. Above all, parents must not be placed under psychological pressure to participate, a clear contravention of agreed ethical standards in the Declaration of Helsinki 1964/2013.”

    Liked by 1 person

  40. Here’s a simple calculation to ponder upon:

    Liverpool City Council have reported 99.7% specificity for the lateral flow testing, which is somewhat better than the 99.3% previously understood, but still a lot lower than the phenomenal (and implausible) specificity claimed by the ONS for its RT-PCR testing. This means that, from 90,000 lateral flow tests, there must have been about 270 that were false positives, i.e. were subsequently negative after the confirmatory RT-PCR test. The council also reports 336 positive cases (presumably the lateral flow positives that were corroborated by RT-PCR). That means that the probability that an individual actually had Covid-19 after a positive lateral flow test was only 336/(336+270) = 0.55.

    That’s certainly a lot better than I had anticipated, based upon an expected 99.3% specificity, but it is still only just over a half. If the authorities are doing what they set out to do, the Liverpool 270 would have been freed in good time, but I question whether this is actually happening and why the press has not picked up on this story either way. An even more interesting question is just how many of the asymptomatic cases ended up being amongst the 336. We can expect the majority of the 270 to have been asymptomatic and, in the absence of RT-PCR intervention, we can expect that they would have been put down as being amongst the army of asymptomatic cases. Now that we can discard them, what is the actual figure for asymptomatic Covid-19? This data must already be known. Why are we not being told yet? The authorities seem keen enough to release all the other figures. Remember that the whole purpose of the mass testing pilot was to track down and quantify the asymptomatic Covid-19 hordes. This is a vital figure because it has a significant bearing on little questions like ‘when will it be safe to release the lockdown?’

    Also, given the data emerging from the Liverpool pilot, what are we now to say regarding the hundreds upon hundreds of asymptomatic students placed in isolation as a result of the ‘asymptomatic covid test programmes’ that were unleashed upon them without checks for false positives?

    You want to talk about scandals? I’ll tell you what is scandalous. It is a world that is prepared to draw conclusions without having the first clue about base rate neglect.

    Liked by 1 person

  41. This is the nearest I have been able to get to working out the asymptomatic rate for myself:

    According to Liverpool City Council, as of three days ago:

    https://liverpool.gov.uk/communities-and-safety/emergency-planning/coronavirus/cases-control-and-testing/report-published-13th-november-2020/

    “Of the 1,390 confirmed cases in the last 7 days, 164 (11.8%) were detected using Lateral Flow Testing (LFT) kits. At an assumed Covid population prevalence of 2% in Liverpool, one in five LTF positive results will be false positive. They will require confirmatory Polymerase Chain Reaction (PCR) testing.”

    This statement makes little sense to me. It talks of 164 ‘confirmed cases’ and then it suggests that they await confirmation!

    In fact, a 2% prevalence would mean that 2,000 cases would exist within every 100,000. For 1 in 5 positives to be false, there would therefore need to be 500 false positives per 100,000. So the LCC appears to be assuming here an LFT specificity of 99.5%. With such a specificity, there should already be significantly more than 164 false positives, given the volume of LFT tests already undertaken (90,000), and so the figure of 164 must be the already confirmed cases.

    The naïve assumption is that these 164 are all asymptomatic cases because the LFT kits were designated purely for the mass testing of people who had declared themselves to be asymptomatic. Equally, the remainder of confirmed cases could be assumed to be symptomatic because they were the ones detected at the standard mobile testing units that (in Liverpool at least) are now reserved for those declaring themselves to be symptomatic. On these dodgy assumptions, I would say that 11.8% is the asymptomatic rate.

    Of course, if people are not playing the game and are turning up to the wrong type of station (and not being turned away) then this result will be inaccurate. Nevertheless, we are going to have to be a long way off before we end up confirming Terry Whalley’s working assumption of over 50% asymptomatic cases roaming the streets of Liverpool.

    Incidentally, on the same day that the LCC were issuing the above-referenced report, claiming 164 confirmed LFT results, the BBC were saying:

    “…lateral flow tests have confirmed 336 positive cases, Liverpool City Council said.”

    If 164, rather than 336 is the correct figure for true LFT positives, then the odds of incorrect isolation being inflicted upon an asymptomatic subject rises from 0.45 to 0.62.

    Like

  42. Given the data that is coming out of the Liverpool mass testing, the lesson to be learnt ought to be that the numbers of asymptomatic cases out there do not justify further investment in mass testing.

    But that cannot be! The government was so confident in the pilot’s results that it has already rolled out the mass testing to 67 other areas. This is its flagship Moonshot programme that is going to save the nation. So what is to be done?

    I know, get the Public Health Director, Dr Susan Hopkins to declare that 700 asymptomatic cases have been detected already. It’s a big number that, taken out of context, seems to justify the investment. Just remember not to say ‘only 700’.

    https://www.bbc.co.uk/news/uk-england-merseyside-54966607

    Like

  43. There is an error in the statement I quoted earlier from the LCC, which I should have pointed out before now.

    The 2% figure they quoted for the prevalence of Covid-19 in Liverpool is irrelevant because the population they are testing is not randomly selected. Instead, they should be using the expected prevalence of Covid-19 within the asymptomatic subset of Liverpool. This should be a lot lower (figures suggest perhaps as low as 0.2%), which, for a specificity of 99.7% (as reported) would mean that about half of the positives would be false, not the one in five that the LCC seems to think. This basic error (which, admittedly, I should have seen straight away) does not instill confidence in the reporting.

    That said, whichever figure for specificity is taken (my deduced 99.5% or the reported 99.7%) the case figures that are being reported are below the number of expected false positives, suggesting that they must be the figures after RT-PCR confirmation (as one should expect).

    Like

  44. Just a reminder of what the Liverpool City Council said on their website at the end of the 7th day of mass testing:

    “Of the 1,390 confirmed cases in the last 7 days, 164 (11.8%) were detected using Lateral Flow Testing (LFT) kits.”

    Nevertheless, on the same day, the BBC were quoting the LCC as saying the number of ‘confirmed cases’ resulting from LFT was 336. Now, only 3 days later, the Director of Public Health England is claiming that the number of asymptomatic cases is 700. On the assumption that LFT kits are used exclusively for the detection of asymptomatic cases as part of the mass testing, the discrepancy between 164 on day 7, and 700 on day 10 looks rather odd. What am I missing?

    Given such inconsistencies and anomalies, I now have zero confidence in the ability or willingness of the authorities to report accurately and reliably on this subject. I think I should probably pack in now and find something more productive to do. Trying to make sense of the reports is proving to be a fool’s errand.

    Liked by 1 person

  45. @ John, thanks for keeping track of all this.

    I doubt any sensible numbers will emerge. I also doubt (high certainty) that the media will ever make anything of it, because the maths is too hard. I am constantly amazed by how difficult it is to interpret testing when you have at root a very simple binary test. There is no way the average hack is going to know how to interpret a press release, nor be able to turn it into a contingency table.

    I have given up trying to understand the reported figures. Do the asymptomatic bunch include false positives? Why do the reported figures vary wildly for the same testing in the same area? It would be rational for the authorities to publish the actual data in the form of a contingency table, but that isn’t going to happen. People just will not understand the concept that if they tested positive there is say only a 50% chance of being infected.

    Liked by 1 person

  46. I’ve been trying to think of a suitable nickname for Operation Moonshot. I first thought of ‘Operation Moonshit’, and then I thought ‘Operation Moonpig’. In the end, I decided to compromise with ‘Operation Pigshit’.

    Here is what the biostatisticians were saying about Operation Pigshit in the Guardian yesterday:

    “The government’s £100bn Operation Moonshot mass Covid testing scheme is like ‘building a Channel tunnel without asking civil engineers to look at the plans’, experts have warned.

    They say there is no evidence the plan will offer any benefit, the effectiveness of the tests it uses is weak, and the programme itself has been structured without input from the body responsible for advising ministers on screening strategy.”

    So far on this thread, I have concentrated on the menace of false positives, but experts are just as worried about the lack of sensitivity of lateral flow antigen testing. Either way, the rolling out of the Liverpool pilot will do more harm than good and the infliction will cost us all an eye-watering fortune. This government appears to be so adept at pulling off this trick that one is inclined to say they have turned it into an art form.

    https://www.theguardian.com/world/2020/nov/16/operation-moonshot-like-building-channel-tunnel-without-civil-engineers-covid-testing

    Like

  47. The BMJ is highly critical of the Liverpool testing and especially of the fact that it’s being rolled out across the country:

    “Despite claims by the city council that the Innova test is “very accurate with high sensitivity and specificity,” it has not been evaluated in these conditions. The test’s instructions for use state that it should not be used on asymptomatic people. A preliminary evaluation from Porton Down and Oxford University9 throws little light on its performance in asymptomatic people or in the field. It suggests the test misses between one in two and one in four cases. The false positive rate of 0.6% means that at the current prevalence in Liverpool, for every person found truly positive, at least one other may be wrongly required to self-isolate. As prevalence drops, this will become much worse.

    Spending the equivalent of 77% of the NHS annual revenue budget on an unevaluated underdesigned national programme leading to a regressive, insufficiently supported intervention—in many cases for the wrong people—cannot be defended. The experience of the National Screening Committee and National Institute for Health Research (NIHR) tells us that allowing testing programmes to drift into use without the right system in place leads to a mess, and the more resources invested the bigger the mess. This system should be designed with up to 10 clear objectives to deliver the aim of reducing the impact of covid—for example, to identify cases more quickly or to mitigate the effects of deprivation on risk of infection and poor outcomes. Progress in each objective (or lack of it) should be measured against explicit criteria. Screening programmes based on experience and on the literature relating to complex adaptive systems14 offer a model for rapid progress.

    At a minimum, there should be an immediate pause, until the fundamental building blocks of this mass testing programme have been externally and independently scrutinised by the National Screening Committee and NIHR. In the meantime, nobody’s freedom or behaviour should be made contingent on having had a novel rapid test. It is premature to offer testing as the route to individuals’ release from restrictions. Instead we must heed the advice of the World Health Organization and the government’s Scientific Advisory Group for Emergencies (SAGE), radically improve the woeful performance of the “find, test, trace, and isolate” system, and renew the focus on identifying symptomatic people, especially among those sections of society most at risk. The current approach will open Pandora’s box.”

    Johnson’s mad ‘Moonshot’ program is the equivalent of spending £100bn on a state of the art launch pad plus monitoring and control systems, then popping down to the shops to buy some rockets and launching them in the direction of the moon on several evenings.

    https://www.bmj.com/content/371/bmj.m4436

    Liked by 1 person

  48. Jaime,

    Indeed, the BMJ has been critical of the proposals from the outset. For example:

    https://www.bmj.com/content/370/bmj.m3699

    and:

    https://www.bmj.com/content/370/bmj.m3585

    I could see the value of a pilot to explore questions of viability and to collect data to see if the problem that Moonshot is designed to address even exists. However, everything I have seen so far seems to confirm that even the correct conduct of a pilot is beyond the government’s capabilities. It’s one thing to come up with a stupid idea and it is quite another to be so stubborn and/or incompetent as to be unable to undertake the basic steps required to recognise it.

    I repeat, Bayes will have his revenge.

    Like

  49. This is just nuts. According to this, up to 15th Nov, 42270 PCR tests on symptomless people were carried out. 1407 tested positive (2.98%). 65792 residents were given LFTs and 402 tested positive (0.61%). So, a very large difference in rates of positives between the two tests.

    If you go to the local government website, they provide these figures up to Nov 16th:

    71,684 Liverpool residents tested using lateral flow
    51,855 Liverpool residents tested using PCR
    Total of 119,054 Liverpool residents tested – some people have had both lateral flow and PCR tests
    In addition, 13,775 people from neighbouring areas have been tested using lateral flow
    There have been 588 positive lateral flow tests – 439 of which have been Liverpool residents.

    Add the PCR and LFT figures and you get 123539. As only 119054 residents got tested, this means that only 4485 residents received BOTH tests! So the idea that LFT positives would be confirmed with a ‘gold standard’ PCR test is definitely out of the window. It was EITHER LFT OR PCR.

    Just what was the point of this exercise?

    The site tells us:

    “The Liverpool Mass Asymptomatic Serial Testing pilot aims to inform a blueprint for how mass testing can be achieved, and how fast and reliable Covid-19 rapid testing can be delivered at scale.

    Whole-city testing aims to protect those at highest risk and find asymptomatic cases.”

    If this is indeed the blueprint for nationwide mass testing then I think we can safely say that Operation Pigshit will become known as Operation Pig’s Breakfast.

    https://liverpool.gov.uk/communities-and-safety/emergency-planning/coronavirus/how-to-get-tested/mass-testing-faqs/

    Like

  50. Jaime,

    Welcome to the madhouse reserved for those trying to make sense of the Liverpool pilot pronouncements, or indeed any pronouncements involving specificity and sensitivity.

    I think the first thing you have to do to stand any chance is to hold the following simple (perhaps simplistic) model in your head:

    As well as the new trial of asymptomatic testing there is still the ongoing testing of ‘symptomatic’ citizens. The former is conducted at a number of newly established, dedicated test sites, and that is where the lateral flow antigen technology is used. The symptomatic tests are held at the pre-existing mobile sites and for these PCR technology is still being used. The reason why the PCR positivity rates are so much higher than the LFT rates, therefore, is because the PCR tests are used upon predominantly symptomatic individuals and any others likely to have Covid. In comparison, LFT, being reserved for the asymptomatic screening, will be associated with a much smaller positivity rate.

    In addition to the above, there are a small number of PCR tests performed at the asymptomatic screening stations to validate the small number of positives that are picked up by LFT. Given this small number, I am actually puzzled that the number of people that appear to have received both types of test is as high as it is. There was talk at one time of RT-LAMP being used. I wonder if that explains the discrepancy.

    Meanwhile, what happened to the 700 asymptomatic cases reported by the BBC? Does that higher number include asymptomatic cases still being picked up at the mobile stations, even though they are not supposed to be? I don’t know. I’ve given up. I don’t see why the authorities feel the need to make everything so unclear in their reporting.

    You ask what is the purpose of this programme. Damned if I know anymore, and I suspect that goes for the majority involved. I think the BMJ has it right. It’s all things to all people but will end up being universal pig shit if they are not careful.

    Liked by 1 person

  51. John, I’m not sure that the PCR testing in this mass testing pilot was actually for those with symptoms or just to validate positive LFTs. If you go to the website, it says that those with symptoms should book an appointment for testing but that those without symptoms can just turn up and get tested at any of the testing sites, where they are using both PCR and LFT tests. If this is the case, then the very high rate of PCR compared to LFT is something which the government needs to explain and justify urgently. A court in Portugal has apparently just ruled that detention of travellers is illegal on the basis of unreliable PCR testing, which I think may be a first.

    “The centres listed on the map below are all open from 7am-7pm for people with no Covid-19 symptoms to drop in for a test without an appointment.”

    https://liverpool.gov.uk/communities-and-safety/emergency-planning/coronavirus/how-to-get-tested/symptom-free-mass-testing/

    “All residents and workers who consent will be tested using a combination of existing PCR swab tests, as well as new ‘lateral flow’ tests which can rapidly turn results around within an hour, without the need for processing the swabs in a lab.”

    https://liverpool.gov.uk/communities-and-safety/emergency-planning/coronavirus/how-to-get-tested/mass-testing-faqs/

    The latest figures up to the 17th Nov confirm that the PCR positive test rate is about 3% and that of the LFT about 0.6%.

    [https://liverpool.gov.uk/media/1359852/17112020_external_pm.pdf]

    Like

  52. Jaime,

    I’m not sure of anything. My technique is to come up with a mental model that can explain the content of the pronouncements and then ask myself if the model is plausible. The problem is that the pronouncements are so often nonsensical in themselves, downright contradictory or hopelessly lacking context that the game reduces to the question of which is the least unbelievable thing to accept. It is obvious to me that many of those responsible do not themselves understand what is going on and so I see no prospect of someone like me getting to the bottom of it. Jit summed it up perfectly. There is no consistency and no coherence.

    Liked by 1 person

  53. I should add to the above that I do not rule out that some non-confirmatory PCR testing may be happening at the mass screening test sites, but I still don’t understand why they would do this when LFT is so much cheaper, quicker and better suited to mass screening (unless they ran out of LFT kits). Also, the reasonable assumption that LFT is NOT being used on self-declared symptomatic individuals turning up at the original mobile sites would be reason enough to explain the different positivity rates associated with the two test technologies.

    Throughout all of this, I have been guided by the following logic: PCR testing has been used extensively to assist in the diagnosis of individuals presenting with symptoms, or who live or work in an environment that significantly increases the risk of them having Covid-19. In these circumstances, where the pre-test probability is relatively high, the specificity of PCR is adequate for the job and false positives are of less concern than false negatives. The sensitivity of the test may be a problem, but negatives can be (and were) corroborated with routine re-testing. You can afford to do this when dealing with relatively low numbers. However, the costs and delays in receiving test results renders PCR ill-suited to the large-scale screening of asymptomatic individuals. A cheaper, quicker testing technology was required. LFT provides that technology but it has two drawbacks.

    Firstly, the specificity is no better than PCR and so, when it is used in very low prevalence scenarios (such as screening the asymptomatic) the false positives become a problem. This problem, however, can be solved by using PCR to double-check the positives (a second LFT test would achieve the same purpose but it is better to double-check using a different technology such as PCR, particularly since the cycle threshold for the PCR confirmation could be customised to favour good specificity over high sensitivity). Once again, one can afford to do this given the relatively low numbers.

    Secondly, LFT is not as sensitive as PCR. In low prevalence scenarios, low sensitivity is not usually such a concern, unless you had actually set out specifically to capture all asymptomatic cases. The problem is that individuals incorrectly testing negative, although small in number in comparison to the legions of true negatives, will be emboldened to behave as if they are Covid-free. In that sense, the screening may do more harm than good. I’m afraid this is the price to pay for having to resort to a cheap and slick testing technology. It would be nice to immediately double-check the negatives but this would mean just about everyone being double-checked — this would not be cost-effective. The compromise is to advocate regular re-testing.

    Anyway, the point I’m making is this: I choose to believe my mental model because it aligns with what SAGE and other experts had said should happen and it makes sense both economically and operationally. That is not to say, however, that my model is accurate. There has been enough written and reported to suggest that the planned intention and what has actually happened may differ. It is difficult to say when the information is so ham-fistedly presented by sources with a track record of saying stupid things.

    Liked by 1 person

  54. John:

    I’m not sure of anything. My technique is to come up with a mental model that can explain the content of the pronouncements and then ask myself if the model is plausible. The problem is that the pronouncements are so often nonsensical in themselves, downright contradictory or hopelessly lacking context that the game reduces to the question of which is the least unbelievable thing to accept. It is obvious to me that many of those responsible do not themselves understand what is going on and so I see no prospect of someone like me getting to the bottom of it.

    So we’re back on the subject of climate.

    Liked by 1 person

  55. “So we’re back on the subject of climate.”

    I don’t know how you can say that. Operation Pigshit is a cripplingly expensive programme designed to address a problem that may very well not exist (large-scale asymptomatic transmission) requiring technology that does not yet exist. It has been pushed through without a semblance of basic programme governance (audit scrutiny, risk management, cost benefits analysis, etc.) and, although the subject is mired in uncertainty, those who express doubt are being branded ‘deniers’. The media could save us from this madness by asking the right questions but they seem to have gone AWOL.

    Okay, I see your point.

    Liked by 3 people

  56. Operation Pig’s Ear sounds apt. Part of the science informing Pig’s Ear is that we should urgently lock down to save Christmas, which has dutifully been achieved. Thus Sage informs us that we may be able to have a family Christmas, but that we must ‘pay’ for every day spent together at Christmas by a further 5 days in isolation during January. So, we ‘saved’ Christmas by locking down beforehand but then we must save ‘save’ ourselves from the dreaded after effects of having enjoyed Christmas by locking down throughout January. In climate-speak, I guess this may translate to something like: you can EITHER drive your diesel car for one day OR enjoy a burger, but for two weeks afterwards you MUST go vegan, forgo use of the car AND walk to work every day.

    Liked by 1 person

  57. They are letting the students out for Christmas and, unsurprisingly, this will entail mass testing to screen out the infectious ones before departure. I have a fag packet in my hand (which I suspect is a metaphorical one since no one in the house smokes) so let’s do some rough and ready calculations to see how this may pan out. For example, let us examine the TV article, presented on my local news last night, regarding the proposal to mass test up to 10,000 students in a local gymnasium.

    If one assumes a 2% prevalence of covid-19 one might expect there to be 200 infected students in that group. However, that would be assuming that the 10,000 represents a random sample. In fact, the testing is voluntary and the sample is self-selected, with the main selection criterion being ‘I think I’m fit and I want to go home’. The last time I looked, feeling fit and wanting to go home was not on the list of acknowledged covid-19 symptoms. So the more realistic assumption would be that only asymptomatic/pre-symptomatic cases are going to be found. Liverpool mass testing figures are suggesting that the covid-19’s asymptomatic proclivity runs at 20%, at most, and so I think a better estimate would be for 40 cases present in the group. Let’s assume perfect test sensitivity so they are all detected.

    Now for the false positives. Test specificity is always subject to uncertainty but 99.5% is a commonplace assumption. That would give 50 false positives in a group of 10,000.

    These are very rough calculations but they are going to be in the right ballpark. And given that the calculation suggests that false positives are likely to outnumber the true positives, one would expect that a second, confirmatory test would be a no-brainer. After all, what’s another 90 tests after you have forked out for 10,000?

    Except, there was no mention of this on the news article. Instead, it was the usual mantra: If you’re negative then you can go home but if your test is positive then you have to stay put and self-isolate for 10 days. I looked on the government’s dfemedia blog for some reassurance but found none – it just said the same thing. All I found in my searches was a letter sent to all universities by the Universities Minister, Michelle Donnelly, saying , ‘The tests we are deploying have a high specificity which means the risks of false positive test results is low’. Way to miss the point there!

    The worry is, of course, that in our group of 10,000 students, 50 of them would be incarcerated in their digs unnecessarily, and their contacts (even if they tested negative) will be told they can go home but they still have to (unnecessarily) incarcerate for 14 days when they get there. Add to that, as a result of the exercise, the prevalence of Covid-19 in the student community will have been grossly over-estimated and the myth that students with covid-19 are predominantly asymptomatic will have been reinforced.

    I did some casting about on the internet for any articles that covered the subject, and mentioned second tests being performed, but I found only one in the Metro online. This does not fill me with confidence.

    Liked by 1 person

  58. I’m feeling a bit more relaxed today after reading these two articles:

    https://www.bbc.co.uk/news/uk-scotland-edinburgh-east-fife-55006105

    https://www.theguardian.com/education/2020/nov/11/mass-covid-testing-will-allow-students-to-return-in-january-says-minister

    Both articles are at pains to point out that both negative and positive test results are to be corroborated. In recognition of LFT’s modest sensitivity, two tests are to be taken 3-5 days apart and both have to be negative for the student to be declared covid-free. In recognition of LFT’s specificity and the low pre-test probability, any positive result is to be immediately corroborated by administering a PCR test.

    The only niggle I have is that the reports stress the ‘accuracy’ and ‘sensitivity’ of the PCR test, as if that is the reason for the re-test. It is true that PCR is sufficiently specific and sensitive for the purpose of confirmation, but the purpose of the second test is not to use a more specific and sensitive test. It would have been nice if this had been made clearer by the reports.

    Liked by 1 person

  59. Yesterday I was pleased to report how St Andrews and Edinburgh universities seem to have got their act together. I wish I could be as confident regarding Durham University.

    Durham has always portrayed itself as being ahead of the game. Under the leadership of Professor Jacqui Ramagge, Professor of Mathematics and Statistics, they have been boasting that they have led the way in providing their students with self-administered LFT kits to enable mass testing. Only this morning, the aforementioned professor was being interviewed on local television explaining to a witless reporter how they will be using LFT to screen their students prior to the great Christmas jailbreak. As has become customary when describing the test regime, the professor failed to acknowledge the perils of false positives and made no mention of confirmatory PCR testing. Even more worrying was the absence of any mention of the 3-5 day retesting to guard against false negatives. Instead, the good professor simply expressed a naïve confidence in the accuracy of LFT.

    Nowadays I would put this down to a simple desire to keep things simple for the benefit of a too easily confused public. Normally, I would grant the benefit of doubt and trust that things are actually being done properly behind the scenes, despite the professor’s simplistic account. After all, if one cannot trust a professor of mathematics and statistics to get it right, then who can we trust? However, in this instance I’m not so sure. This is the same Durham University that was implementing mass campus lockdowns around about the time that Northumbria University was reporting a 90% asymptomatic epidemic of Covid-19 within its students and staff. Indeed, according to a Palatinate interview of Professor Ramagge, “50% of [Durham] students who have contracted Covid-19 have been asymptomatic. The national average of asymptomatic Covid cases is 20%”.

    https://www.palatinate.org.uk/durham-pushing-to-be-leading-university-in-improved-mass-testing-for-covid-19/

    I have previously reported on this thread that the figures emerging from the Liverpool mass testing seem to be indicating a 20% asymptomatic proclivity for Covid-19 and I have provided rough calculations that suggest that a 50% proclivity might be concluded by anyone who is not doing the testing properly. Coincidence? I bloody hope so for Ramagge’s sake. She may be a professor of statistics but that is not going to protect her from a naïve belief in 100% specificity.

    As I’ve said before, all it would take to avoid all of this doubt and confusion is for one — just one — journalist to ask the right bloody question.

    Liked by 1 person

  60. Geoff,

    What is particularly bizarre regarding the censorship of the Portuguese ruling is that there isn’t anything remotely controversial or ground-breaking about the ruling. It does nothing more than represent the avoidance of the Prosecutor’s Fallacy.

    https://en.wikipedia.org/wiki/Prosecutor%27s_fallacy

    The importance of the Prosecutor’s Fallacy has been long-since established within the legal profession and I suspect it is now taught to all students of law. The Prosecutor’s Fallacy is just base rate neglect within the context of expert testimony provided in the courtroom. It gained notoriety in the famous case of Sally Clark, who was wrongfully found guilty of double infanticide, based upon the incorrect statistical reasoning of the prosecutor’s expert witness, Sir Roy Meadow.

    It should be noted that the Portuguese court did not rule against the use of PCR, only that an incarceration based upon the results of a single test would be unlawful. Outside of the legal system, there seems to be only a patchy understanding, at best, of the issues involved. Significantly, once one involves a profession that is well-briefed on such matters, the idea of single-test incarceration gets immediately kicked into the dustbin where it belongs. It would only need a similar case to be brought to the UK courts and the same ruling would be inevitable (unless, of course, all statistical logic is to be overruled using dubious health and safety legislation).

    Like

  61. Just to spell out what has happened with this Portuguese ruling, it goes like this:

    Step 1: Someone who hasn’t a clue how the statistics work incarcerates someone on the basis of a single PCR positive.

    Step 2: The party affected says ‘this can’t be right’ and takes it to court.

    Step 3: The legal system, which does have a clue how the statistics work, rules in favour of the plaintiff.

    Step 4: The ruling is reported on-line but someone who hasn’t a clue how the statistics work thinks ‘this can’t be right’ and censors it as fake news.

    This is what you get when the fact checkers have doo-doo for brains.

    Like

  62. Matt Hancock was on Good Morning Britain today crowing about the incredible success of the Liverpool mass testing (it seems to be one of the reasons why he appears to think he deserves a big pay rise this year). According to him, it is responsible for the Covid-19 prevalence in Liverpool dropping to one third of its former level.

    Wow! How could I have ever doubted the wisdom of project Moonshot? But how, you might ask, can testing have achieved such a remarkable result? Of course, the answer is that it didn’t.

    The theory behind Moonshot was that, at any given time, more than 50% of Covid-19 cases are asymptomatic and are infecting the community undetected. Moonshot was designed to identify those people and isolate them, thereby denying the virus its most important vector. However, initial figures from Liverpool indicated that only 11% of positive results were from asymptomatic individuals. This figure has since crept up to nearly 24% but it is still less than half of the figure upon which Moonshot was premised. Nevertheless, the relatively small number of asymptomatic cases identified by Moonshot is deemed to have made all of the difference. Not only is this implausible, it also ignores the fact that the Liverpool figures were on the decline well before Moonshot began and that much of the decline had already happened.

    So was Moonshot such a huge success? I prefer the way Mayor of Liverpool, Joe Anderson, has put it:

    “Until we get a vaccine, we need to look at all the tools available to us – [lateral flow testing] can play a role.”

    Tell me again how much this tool is costing us (Moonshot, I mean, not Hancock).

    https://www.theguardian.com/uk-news/2020/nov/21/liverpool-mass-testing-finds-hundreds-with-covid-19-but-no-symptoms

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.