More GHCN errors – thumb on the scale?

This post follows on from the recent remarks by John Bates about Tom Karl having his “thumb on the scale” and the apparent instability of the GHCN adjustment algorithm, raised by Bates and discussed in a series of blog posts by Peter O’Neill and a blog post here.

As well as errors in the adjustment algorithm, there are also errors in the raw input data. One example of an error of omission is for Iceland, where for Teigarhorn and other sites there are chunks of data missing, even though the data is available from the Iceland Met Office.

Peter O’Neill has a new blog post showing an example, Cork Airport, where the input data used by GHCN is simply wrong. It disagrees with the data from the Irish Met Office, and it’s “obviously wrong” in the sense that the same number, 10.4, or 1040 in hundredths of a degree, keeps coming up in the GHCN unadjusted file:

cork

An average temperature of 10.4C seems suspiciously warm for January 2015, quite apart from the unlikeliness of January 2013 and 2012 and three Decembers having the same value, and Peter’s post shows that according to the Irish Met Office it should be 5.4C.

Is this just an isolated problem for one site? No, the same problem occurs for another site, Shannon airport, where the mysteriously repeating temperature value is 11.8C.

shannon

Note that as well as the dodgy repeating numbers, the files also include the number -9999 in some places. This indicates that the number is “missing”. But at his blog, Peter has cunningly located the numbers that GHCN seem to be unable to find, by looking them up at the site of the Ireland Meteorological Service.

Notice that the GHCN files include some letters alongside the numbers. These are explained here. The W after a number indicates that the computer has noticed that this number is the same as the previous one. The M and the C indicate the source of the dodgy data.

So where does the dubious data come from? The M and C indicate that it’s from “Monthly Climatic Data of the World (MCDW)”, which is another branch of NOAA. The MCDW file for January 2015 is located at https://www1.ncdc.noaa.gov/pub/data/mcdw/mcdw1501.pdf, and you can find the data for other months by changing the file name in the obvious way. You can see the incorrect data in there. In fact, all of the numbers for Ireland for January 2015 look suspiciously high.

Who is responsible for this incorrect data? Whose thumb is on the scale? Well, at the front page of each of these MCDW files, there’s a signature:

karl

 

19 thoughts on “More GHCN errors – thumb on the scale?

  1. Good post Paul. It seems to me NOAA needs to do the grunt work of cleaning up their data before using automated adjustment methods.

    Like

  2. The output was politically desirable, and time was running out, so who had time for such details? They can be fixed later as and when they’re noticed, and who knows, that might even improve upon the results. The notion of a pause, of temperatures no longer rising, was, in my opinion, becoming prevalent. A tiny gradient change would do for the politicians and the kiddies in the mass media to help them hang on to their precious sense of crisis unfolding.

    Karl put in an heroic effort on the front line, and now that the flipperty-floppity, procedures be damned, shonky inputs, nature of the computations has been revealed, he may progress to martyr status. Or, in other words, another useful idiot bites the moral and intellectual dust. But perhaps it is still too early to say that for sure. Gleick, for example, fell and rose again, and Mann never even noticed his descent, while Pachauri got done not so much for his junk talk about science and the IPCC, but for his treatment of his staff. Hansen’s multiple arrests have not held back his intemperance. Flannery is seemingly unphased by being dramatically, and extremely expensively for Australians, wrong in his advice to government. Nor has the inconvenient behaviour of hurricanes nobbled Trenberth, and Pope was not sacked from the Met Office for going over the top on temperature rises. Gore never batted an eyelid about his breakthrough presentation in which he showed the key effect preceding its notional cause. I know this list is not exhaustive – it is but what I found off the top of my head – but it is at least illustrative. So, maybe Karl will soldier on after all.

    Like

  3. Paul Matthews finds another dastardly deception by wicked climate scientists. The horror, they are trying to fool us all by missing out Ireland! Strange thing is that June 2015 data for Ireland appears in the October file under “Surface Later Reports”. I wonder where the other Ireland data has been cunningly hidden.

    “…there’s a reason that I’m not a professor”

    I think I just found it.

    Like

  4. Bad luck, Lenny, you’ve just screwed over climate science by mistake. You might have to lose your enforcer badge. Paul refers to July and August and you think you have solved the problem by locating…… JUNE. Climate enforcers are even funnier than climate pscientists

    Liked by 1 person

  5. The same is true in other records. I have a particular interest in Rutherglen (ag research station, pristine well maintained rural site). Data available from 1913. BEST has nothing before 1949. Another example is USHCN 415429, Luling Texas. 10 of 12 months in 2013 are marked estimated (as data was missing), and all 12 months for 1934. Yet the Texas State Climate report contains that information; it isn’t missing.

    Like

  6. The subject report cannot legally be referenced by any government dept. as it does not meet the minimum requirements for such documents. No part of this report may be used as reference, the entire report is completely meaningless. The whistleblower is the person who wrote the regulations for NOAA. There is no more to said, the report has no validity and its authors are guilty of attempting to politically influence the elected officials by committing scientific fraud.

    Like

  7. Just to be clear and for the record.

    Thumb on the scale refers to cheating.

    Are you claiming that Karl is guilty of fraud or misconduct?

    For the record.

    Like

  8. Mr Mosher, given that you are so proud of your reading skills, I find it hard to understand how you missed the question mark. So it is surely redundant to say to someone of your intellectual eminence that it was a question rather than a claim. And now you are the person suggesting that it can only resolve to fraud or misconduct. Why not come down off the fence and tell us what someone of your unimaginable importance believes?

    Like

  9. On further reflection, the claim that Karl has to be guilty of either fraud or misconduct is a little simplistic for someone with the intellectual sophistication of Mr Mosher. For example, it could be inattention, inadvertence, carelessness, inattention, negligence or maybe just stupidity. Given that computing a global temperature is a foolish thing anyway, any of those ideas might be a better explanation than either fraud or misconduct. When you are computing something worthless, why bother to put too much effort into the task?

    Like

  10. It is amazing how many blog electrons have been spent in defense of Karl et al. This will be investigated and we will find out more. Why however have the defenses of Karl been so breathless and voluminous? The result will be better science. Sounds like a plan to me.

    Like

  11. From the JC blog: with Mark Goldfinch | February 11, 2017 at 5:05 pm

    (“So what did you learn about “NOAA methodology and results”?-)
    “We learned that NOAA is a political organization, not a scientific one. We can now discount everything they say as being based on their political beliefs, not on any science that may accidentally be involved. Note that I say “may be”, because when your approach is to support your pet theory by cooling the past and warming the present, you have to say that there is no actual science here.”

    Where is anyone to defend the scientific method, of hard science? All seems to be “I are self appointed Skyintist you must believe me”!

    Like

  12. For all that did not ‘get’ the 12 Feb 17 at 9:26 pm message! The popular ‘belief” of CAGWis now called ‘science’! The same as when political\religious folk what ridiculed Galliano’s proclaimation dat Da Sun is near da center locus of dis solar system, never da Earth! Go away religious kooks!

    Like

  13. Steven Mosher says: 10 Feb 17 at 9:22 pm

    “Just to be clear and for the record.. Thumb on the scale refers to cheating.”

    Not at all for actual folk!! “Thumb on the scale” refers to the butcher, carefully looking at da price of some commodity, den honestly adjusting de price to fit da needs of da buyer! Is dat ever ‘cheating’?

    “Are you claiming that Karl is guilty of fraud or misconduct? ”
    I will gleefully and publicly accuse retired Thomas Karl, of NOAA , of intentional fraud and governmental misconduct! Will someone please sue or applaud?

    Liked by 1 person

  14. “Are you claiming that Karl is guilty of fraud or misconduct? ” Tomas Karl may be found to be human by some court with reference to some physical birth certificate. Does Mommy agree with that? Why?
    I will gleefully and publicly accuse retired Thomas Karl, of NOAA , of intentional fraud and governmental misconduct! I refuse to acknowledge that Thomas Karl has any Human attributes! Will someone please sue or applaud?

    Like

  15. I was a management accountant in various businesses for around 25 years. I am used to proper audits. I have had experience of auditors looking at stock counts. They make random samples checking from the count sheet to the actual inventory and from the actual to the stock. If they find an error they then widen the checks, until they have confirmed it is an isolated instance. If not, the whole exercise is repeated until it is certain that the count is accurate. Otherwise it will be not signed off.
    It looks like the GHCN data should be audited. But first you need laid down standards of data collection and also of adjustment. Have looked at adjustments in a various areas a couple of years ago, what I believe that will be found it that data has a lot of genuine noise or random local variation, not just measurement biases, such as re-siting or the UHI effect. Over time the “adjustments” will likely become necessarily larger and larger, including replacing real data with phoney data and deleting other data to make calculation of regional and global averages possible. My guess is that the deletions and replacements will be done by computer algorithms, and nobody will check the outputs, beyond to see if that is “sensible”. As a result, you will not get a more objective data set as a result of auditing, but an admission that average temperature trends cannot be computed with any reliability from such incomplete data sets.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s