Bishop Hill Don't forget GISS
Nov 30, 2009 Climate While we're all banging on about CRU, it's important not to forget that there are other people producing temperature series and temperature reconstructions.
And other people looking at what they do.
One of these is EM Smith who runs a blog called Musings from the Chiefio. Like Climate Audit, it's pretty hard going sometimes but as the first outsider to actually get NASA's GISSTEMP global temperature index running, it's very important. Fortunately, the Chiefio has written a layman's introduction to what he has found.
It's all disturbing, but his comments on "The Great Thermometer Dying" are simply astonishing.
Since about 1990, there has been a reduction in thermometer counts globally. In the USA, the number has dropped from 1850 at peak (in the year 1968) to 136 now (in the year 2009). As you might guess, this has presented some “issues” for our thermal quilt. But do not fear, GIStemp will fill in what it needs, guessing as needed, stretching and fabricating until it has a result.
Reader Comments (25)
Bishop
You're quite right to point to this interesting but difficult website. He has analysed hundreds, even thousands of data points. He has demonstrated how a temperature recorded on an airfield can spread over thousands of kilometers of ocean, how the thermometers have an inexorable migration to the equator and towards sea level (no cooling bias in either of those). Yes, the language is hard to follow sometimes, indeed frequently off-putting, but the content is really serious. Deserves greater consideration. Needs to be picked up by a competent summariser (do we know of one?)
My apologies for OT comment Bishop -
According to the radio Australian Opposition Leader, Malcolm Turnbull (a strong supporter of an Emissions Trading Scheme), has lost a party leadership vote to Tony Abbott (who is not a strong supporter of ETS).
Using that Norway/Nordic raw temp data on the previous post and overlaying it on the Norway graph in that CRU file I have been researching I think we can safely assume that 2008 -2009 file does contain pre-'corrected' temp data - and still shows no significant warming
http://strata-sphere.com/blog/index.php/archives/11643
If that file really is the raw monthly temps from around the world before being smeared with proxy data, that would be a very interesting development CRU and IPCC would have trouble knocking down. It is their data after all.
BTW - if you don't want me posting here please let me know via email. I hope you find these of modest interest.
Cheers, AJStrata
Funny video about ClimateGate
YouTube
E.M Smith's posting is really an eye opener for the depth of the corruption of temps data collection and subsequent manipulations. Just imagine - California has only (4) four stations and (3) three are located south of LA. These must sure give a good reading of temps in SF. Sheeesh
I wish I knew more science and I curse my lazy teenage high school brain for my current state of ignorance. I've been plodding through this stuff for over a week now and am only just beginning to get past the acronyms and the terms you all use with such ease.
One thing I do know is people and the emails told me enough about the made-men of climate science to start digging. People confident in their expertise do not hide their light, or their data/code, so to speak, under a bushel. They willingly share it, almost to the point of obnoxiousness, as a shining beacon for others to follow.
In that spirit I thank you and others for shedding some light for me on what is a difficult subject. I am, indeed, learning.
Another jaw-hits-floor moment - how can the warming community expect us to take their conclusions seriously when there are these gaping holes not just in the treatment of their data, but in the actual collection of them?
Thank you for the kind words about my summary. To the extent that GIStemp is incredibly hard to work out, the language reflects that complexity. I'm sorry to hear that you may have found some of it "off putting". I try to swap between more technical postings for "geeks and programmers" and more human oriented posts for "everyman". If there is a page that has some part that is unclear, feel free to ask for a clarification on that page. I can usually translate "geek speak" to regular English if someone wants it. Unfortunately, the "turf" I've taken on in this AGW "issue" is the technical part of it. It wasn't being done (probably because it is hard and technical, not sexy) but I figured someone had to take it on, might as well be me.
If there is interest in a glossier general public top layer to a topic, I can likely put one together. Just need to know what folks want to see. I do think it is important to have the deep detail geek layer too, though. That way folks can tell I'm not just making stuff up (as some, er, um, "scientists" seem to have done...). But that can be a "backing layer" under a pleasant face. At least, after I've worked out just what the issue is in a given area... So I usually plough through some technical dreck, posting the rough detail as I find it, then figure it out, and put a top layer posting on it to tie it together. But if the top layer isn't readable enough, I need to polish a bit more...
E.M.Smith
Hi EM Smith
I don't think you're doing anything wrong. You have to do the geek-level stuff - that's where the real work gets done, but the layman's summaries will get you the attention. Attention then gets you comments and then commenters can start explaining things to each other.
Lies, damned lies, and statistics.
The falsification of data and the conspiracy to commit same etc, constitutes serious criminal activity. Further, the granting of public funds for research warrants a federal investigation. I’m hoping the perpetrators, including possibly Professor Michael Mann, director of Pennsylvania State University’s Earth System Science Centre and a regular contributor to the popular climate science blog Real Climate, and their facilitators will be tracked down and prosecuted to the fullest extent the law allows. -- Michael Santomauro
Bishop,
"While we're all banging on about CRU, it's important not to forget that there are other people producing temperature series and temperature reconstructions"
All in pretty good agreement - even UAH which is run by 'sceptics'
"And other people looking at what they do.
One of these is EM Smith who runs a blog "
Aside from having a blog, is there any reason to suppose EM Smith has the first clue about what s/he is doing? Is this just another ignorant bodge like the NIWA smear referenced on 'fiddling in the antipodes' below?
Argumentum ad verecundiam
Why don't you check out his site Frank.
You can check to see if his work has any validity.
Or to slightly rephrase your comment-
Aside from having a comment, is there any reason to suppose Frank O'Dwyer has the first clue about what s/he is saying?
Tony, Bishop,
Do you choose your doctors the same way?
If you've no evidence that Smith is competent just say so.
"You can check to see if his work has any validity."
I might, but generally I haven't got the time to check every wild claim made by some 'sceptic' blogger. Have either of you checked his work? Has anyone?
The reason I ask is that this site has recently referenced a 'study' as evidence of 'fiddling'. It turned out to be the most abject nonsense. Given that GISS agrees pretty well with other independent constructions (inc CRU and UAH), it seems likely that this will turn out to be a more convoluted version of the same thing.
Incidentally I notice that Smith seems a bit surprised by enhanced winter warming. This doesn't inspire confidence in what he's doing, does it?
Maybe some kind of quality filter is a good idea after all. I dunno, perhaps it could be called 'peer review'.
Choosing a doctor is not the same as a scientific argument.
Would you ignore Einstein because he was a patent clerk?
"Would you ignore Einstein because he was a patent clerk?"
If you did you would be right more often than not.
Most, almost all, patent clerks are not Einstein.
Besides, Einstein's work was reviewed by Max Planck and replicated by others. GISS results have also been replicated by others - so I ask again, has anyone checked this Smith guy's work? Who is he? Have you badgered him for his data and code - that's what you guys do, isn't it? Don't tell me you simply took his word for it?
Who will audit the auditors?
"Given that GISS agrees pretty well with other independent constructions (inc CRU ...": it's surely rather difficult to take at face value the use of "independent" when it applies to (i) workers who seem to collude together, and (ii) databases (or datamiddens) that are probably based on the much the same observation sets.
"Incidentally I notice that Smith seems a bit surprised by enhanced winter warming. This doesn't inspire confidence in what he's doing, does it?" It does for me - it means that though there was a feature of global warming that was unfamiliar to him, he was frank about its discovery and his surprise. It's the fundamental lack of frankness that is one of my two main complaints about the Global Warmmongers. It's good to know that Smith is frank, but otherwise your point doesn't matter much since his work is about the internal shennanigans within the GISS code and database.
Who will audit the auditors?
You guys. That way we progress.
And don't forget NASA / GISS ModelE:
A GISS ModelE code fragment
Coding Guidelines and Inline Documentation: GISS ModelE
Another NASA/GISS ModelE Code Fragment
Pattern Matching in GISS/NASA ModelE Coding
Yet Even More NASA/GISS ModelE Coding
All internal URL links over there are broken at this time.
dearieme,
"Given that GISS agrees pretty well with other independent constructions (inc CRU ...": it's surely rather difficult to take at face value the use of "independent" when it applies to (i) workers who seem to collude together"
A decade of emails shows no collusion to make GISS and CRU match - I think someone would have mentioned it by now if they did.
And good luck trying to find evidence that UAH and CRU colluded to back each other up.
"and (ii) databases (or datamiddens) that are probably based on the much the same observation sets."
Probably? GISS and CRU are - which is why they are independent corrobations of each other, different code, much the same data, much the same results. If you doubt this you could also do your own reconstruction with 'much the same data', as I suggested before.
But then the satellite observations also corroborate CRU and are not based off the same observation sets. There are also multiple other independent proxies and analyses and lines of evidence which show a warming trend.
No, it appears that the reason for the pretty good match is the simple one: it really is warming.
Last but not least if they were fiddling the numbers why would they allow the 'sceptics' to get away with claiming it was cooling for so long? Did they forget they were part of a conspiracy? They also seem to have forgotten to mention it in their private correspondence for that decade.
Frank you are trying hard butnot convincing. They are all singing from the same hymn-sheet, the song is the same, it has the same unhappy ending. They make their hockey-sticks look like each others - is that proof of them being independently "right"? They are all manipulating the data to tell the same story . They read each others papers. The fact they are not whispering to each other doesn't mean they are not all up to the same game.
As I said earlier, perhaps on a different thread, it seems likely that there has been some warming if only because, as Frank points out, the satellites suggest that there has too. This is not to say that we should accept the surface records - the problems with them are clearly overwhelming - but I think there has to be a real question mark over the magnitude of the trend and the error bars.
You might want to have a look at the blog currently up on DotEarth
http://dotearth.blogs.nytimes.com/2009/12/01/a-climate-science-forecast-in-the-wake-of-climate-files/
For those who wondered at my skill, background, or code / methods: It is all published and public on my site. The data I use is the same NCDC produced data set that is downloaded by GIStemp (that is, GHCN). The code I use is put in various web pages.
I've taken on the "computer code" side of it predominantly, and I've got about 1/4 century of work in the field as a computer professional. There is nothing particularly hidden in any of what I'm doing, unlike others. So I would hope that having a long resume of professional work in computing and programming would serve as qualifications for examining computing and programming... I've also done computer security work and forensics. I think that is also a needed skill here. To find where the "tricks" are located.
But like I said, everything is upfront and public. Nothing hidden. All of it can be reproduced by anyone (and in several cases I asked folks to test; and got confirmation from different sources- one guy reproduced results using a MS Windows system and some database - and that code was posted too.).
The Team has been to visit, so I'm pretty sure if there were something bogus in what I've done it would have been on the front page of RC or somewhere related. Instead it's just dead air from them... (That, btw, is how you can use them to guage a point. The AGW Team goes silent when you have a good point. If they attack, you have a weak point to polish. IF you are studiously ignored by them, then you have gold...)
BTW, my surprise was not so much at winters warming as it was at the complete LACK of summers warming. Yes, kind of a 'half full vs half empy' thing. But if we are having "global warming" with a "tipping point" as it gets hotter and postive feedbacks, well, one would expect to see some evidence of that when it gets hotter... and it just isn't there. Then when you look at winters at long lived sites, you find them not warming much either. The only "winter warming" comes in the arrival of new thermometers at places with warmer winters... It is that lack of positive feedback (and a pattern that is in line with a negative feedback) that is the "surprise".
And one final point: It isn't about me. It is all about the data. Science never ought to be about the person, it always ought to be about the data and what they say. Given the tendency to character assasination practiced in the UEA as evidenced in the emails, I'm even more comfortable now with an observation that has served me well: When someone attacks the messenger instead of the message, they are waving a flag of The Team. It is a signature behaviour of their 'style'. Not quite diagnostic, but certainly part of a pattern. So watch for who tosses rocks at messangers, and look at who reads the message. There is truth about motivation in behaviour that can not be hidden.
Frank you are trying hard butnot convincing. They are all singing from the same hymn-sheet, the song is the same, it has the same unhappy ending. They make their hockey-sticks look like each others - is that proof of them being independently "right"? They are all manipulating the data to tell the same story . They read each others papers. The fact they are not whispering to each other doesn't mean they are not all up to the same game.
It is worse than that! The three major players (NCDC, GISS, UEA CRU) point to each other in a circle each saying "I'm like him, so ignore my problems, I must be right and you must be mistaken". Unsaid in all this is that they all start from the same highly biased input data.
The emails finally shed light on where CRU got their data. In complaining about the folks wailing over lost data, one of them said they could recreate it from GHCN, since their data were substantialy identical.
GHCN is the data product made by NCDC.
GIStemp uses GHCN as it's dominant input. It added "USHCN for the USA only" up until 2007 when a file format change broke that input. Just a few weeks ago I posted how to use USHCN.v2 including code. A couple of weeks later NASA GISS put in a patch to use USHCN.v2 ... I'm sure it's only a coincidence ;-) but as of now, version 2 of USHCN is back in GIStemp and the deleted US thermometers have partly returned (USHCN.v2 has about 400 deletions still...) That's 2% of the earth surface... only 98% to go...
But at the end of the day, NCDC, HadCRUt, and GIStemp all agree because they all use substantially the same "cooked" input data: GHCN. I've published several pages on that, which is what I've dubbed The Great Dying of Thermometers. I've also done "by latitude" and "by altitude" studies on the changes along with a new series just started of "by longitude". The pattern is clearly one of deletion of cold thermometers.
Don't believe me? Download GHCN and stick it in Excel. Do you own graphs and charts. The only 'hard bit' is you need to match the "inventory" record to the "temperature" record by station ID. Not particularly hard. File layout is documented under the "GIStemp" tab on my site along with ftp location for download. (And at least one person has done just that - I've got a link in a comment to a fellow who has duplicated the "Great Dying" chart in Excel and added an interesting study of GHCN "raw" vs GHCN "adjusted" and found the NCDC "adjustments" also make the past colder..l)
Like I said before: Everything I'm doing is public, code is published (or available for new bits that I've not posted yet, like the new "by longitude" code), data is freely available, and methods are simple enough for anyone to replicate.
So next time the Evil Triplets say they must be OK after all because they are all in the same club together, ask them what input data they use and have there been changes in thermometer location in that data they all share...
Finally, on satellites and ageement: The GIStemp code holds the present stable and rewrites the past. It has a pivot point hard coded as just prior to the satellite era. The present overlap with the satellites can match just fine, and GIStemp will still find dramatic recent warming, by the simple expedient of making prior to 1970 colder... So all you need to catch this is that satellite record from before 1970... But I'm sure it was all just an innocent programming decision... at least until the email is leaked ...
And no, you need not beleive me on this at all. In fact, I encourage you to assume the worst possible: Just go download the code and read it. It is in there. I've documented the code on my site, but you can get your own copy and see for yourself. STEP1 does some of it, but it is mosly the STEP2 program PApars.f that you ought to look at. And no amount of questioning me, my background, or my motivations will change that bit of code: So read it and weep.