Tuesday
Jul172012
by 
Bishop Hill
Bishop Hill Homogenisation is the root of all evil
Jul 17, 2012 Climate: Statistics Climate: Surface Well, something like that anyway. Anthony Watts is reporting a presentation by Demetris Koutsoyiannis, which finds that many of the homegenisation adjustments applied to surface temperature data are ill-founded.
Homogenization practices used until today are mainly statistical, not well justified by experiments and are rarely supported by metadata. It can be argued that they often lead to false results: natural features of hydroclimatic time series are regarded [as] errors and are adjusted.
Reader Comments (56)
"Homogenization practices used until today are mainly statistical, not well justified by experiments and are rarely supported by metadata."
Isn't that what led Matthew Menne to conclude in his "Workshop on Creating Surface Temperature datasets to meet 21st Century Challenges" in his final section on "Takeaways from the the Lessons Learned" that
"Lesson 1 Metadata records are helpful, but we must be prepared to have less than comprehensive station histories"
Wonder why!!!
http://wattsupwiththat.files.wordpress.com/2010/09/7_1wed_exeter-menne.pdf
And it's not only surface temperatures that our climate scientists feel need to be homogenised -
they feel our historical radiosonde records need adjusting too -
http://journals.ametsoc.org/doi/abs/10.1175/2010JCLI3816.1
Let me be the first;
"I'm shocked I tell you! Shocked!"
From WUWT: "...... after taking homogenization errors into account find global warming over the past century was only about one-half [0.42°C] of that claimed by the IPCC [0.7-0.8°C]."
Looks like the Met Office are going to have to "retune" their models.
expect some lukewarmers to show up on this thread
Oh no, its worse than we thought. If all this weather weirding is happening with just half the temperature rise we thought, what does that now mean for the 2, 3, 4 degC rise projected?!
Is it good news or bad news? I don't know any more!
With apologies to Tom Lehrer -
"Homogenize,
Don't let the data deceive your eyes,
Remember why the good Lord made your eyes,
So don't shade your eyes,
But homogenize, homogenize, homogenize -
Only be sure always to call it please 'research'."
Missing heat mystery solved.
They've also systematically fiddled raw data: http://stevengoddard.wordpress.com/2012/07/16/how-ushcn-hides-the-decline-in-us-temperatures/
Realclimate are not going to take this LYING down.
I wonder whether this abstract
Steirou, E., and D. Koutsoyiannis, Investigation of methods for hydroclimatic data homogenization, European Geosciences Union General Assembly 2012, Geophysical Research Abstracts, Vol. 14, Vienna, 956-1, European Geosciences Union, 2012.
will be published in a peer-reviewed journal and if so will it make it to AR5?
But this 0.4 degrees of warming does not agree with the models!
Ergo the data is clearly flawed and needs "adjustment"
So... the data doesn't fit the theory so the data is wrong!!
And worth quoting this from the report -
"In 2/3 of the stations examined the homogenization procedure increased positive trends, decreased negative trends or changed negative trends to positive"
Why am I not surprised!!
http://wattsupwiththat.files.wordpress.com/2012/07/steirou-and-koutsoyiannis-results.png
US July data systematically falsified: http://stevengoddard.wordpress.com/2012/07/14/all-july-warming-in-the-us-is-due-to-data-manipulation/
'USHCN has adjusted recent US July temperatures upwards by 1.5 degrees F, relative to the 1890s.'
Well I said shake baby shake
I said shake baby shake
I said shake it baby shake it ...
There's a whole lotta shakin' systemic homogenization goin' on....
That's what you get when you shake....
Interesting post from Steve McIntyre on homogenisation -
"I expressed a particular concern that Menne’s algorithm might be spreading UHI [Urban Heat Island] warming at low-quality stations to better-quality rural stations through biased detection of changepoints. In a comment on the Berkeley study,which used a similar method, I noted their caveat that the methodology had not been demonstrated against systemic biases (such as widespread UHI)"
http://climateaudit.org/2012/07/17/station-homogenization-as-a-statistical-procedure/#more-16449
>Let me be the first;
"I'm shocked I tell you! Shocked!"
"What are these? Your Grants sir."
From the Ecclesiastical Uncle, an old retired bureaucrat in a field only remotely related to climate with minimal qualifications and only half a mind.
Is homogenization another name for one of the manipulations used in MBH 98 and described in the HSI? Or is it a replacement or addition?
Of course Anthony doesn't refer to the presentation as a "presentation" but as a "paper". He had originally referred to it as being "peer reviewed" until pulled up on it by one of his readers.
http://uknowispeaksense.wordpress.com/2012/07/18/a-conference-abstract-is-not-a-paper-anthony/
What is interesting to note though is the way everyone is just taking his word for it that the temperature rise is half what is reported. Regardless of whether the authors are accurate or not, that isn't what they are actually saying. They are essentially suggesting that if they are correct about the homogenisation techniques, and they do couch it in those terms, then the likely warming is somewhere between 0.4 and the reported 0.7-0.8. Naturally, those in the denial camp latch on to the lower end of that scale and run with it. What remains to be seen if this non peer reviewed conference abstract morphs into a paper atsome point and if that paper then passes peer review and is published in a reputable journal. I won't hold my breath. Finally, even if they are correct, that still doesn't account for the ocean warming or the satellite data or the land temperature data from the 70% of the world that wasn't included in their assessment.
uknowispeaksense
That's a nasty abusive blog you have. With headlines such as "Denier comment of the day" - I hope your not looking to start trolling here.
"That's a nasty abusive blog you have. With headlines such as "Denier comment of the day" - I hope your not looking to start trolling here." - Jul 18, 2012 at 8:18 AM | Jack Cowper
Jack, are you trolling me? How about instead of trying to act like the blog police and start a food fight, you act like a grownup and make an informed response to my informed comment? As for my "denier comment of the day", you should respond to that over at my blog rather than attacking me here? That's good form.
uknowispeaksense
When you yourself can act like a grown up and engage in polite discourse without the use of the lowest form of ad-homs etc then perhaps I will find the time to read your articles and give you some time. In the meantime for the same reasons why I do not read the likes of Delingpole I will not be bothering to engage with you. It's people like you who reduce the debate to a food fight.
I am more than happy to engage in polite discourse with you Jack. You however, rather than respond to the content of my comment, impolitely insinuated that I might be trolling and attacked the content of my blog. Now whether you like it or not, that kind of behaviour is immature and your apparent indignance at being told so is symptomatic of that immaturity. Now, if you'd like to actually discuss the content of my original message in a polite and civil manner I am more than happy to accommodate you. If however, you are going to continue with your temper tantrum I will continue to respond to you as if you are a petulant child.
uknowispeaksense is the latest troll doing the rounds of CA and WUWT. He's been spanked roundly everywhere he appeared and now this blog is the latest place he has started infesting. A visit to his blog will show you what kind of a person this is.
uk.. As long as you're going to gob off about "those in the denial camp" I couldn't give two squirts of guinea pigs piss what your views on anything are.
Or bother to answer them if I was.
Having just read the comments by uknowispeaksense (aka 'Mike') on his own blog - http://uknowispeaksense.wordpress.com/2012/07/15/geography-youre-doing-it-wrong/ I have to agree with Jack that he runs a nasty and abusive blog. Does that make me a petulant child?
Mike, it saddens me greatly that you teach at a UK university. Aside from ecology, do you have any other areas of expertise?
Venter
Why won't you address me directly? As for being spanked, well I guess if deniers refusing to answer direct questions and resorting to personal insults is being spanked then yes, I've been spanked. I'll add you to that list as well.
GrantB
Such an intellectually stimulating comment. If you are only prepared to discuss things with people who agree with you, well, you're never going to get past that closed-mindedness. It's your choice of course. If you are offended by the word "denier" well, how about I refer you to as a "contrarian"? Would that make you feel better?
You are a troll and I have no hesitation in naming you for what you are, uknowispeaksense. Unlike your name, you speak nonsense and run a hateful blog.
lapogus
not at all, but if you have a comment about my blog, perhaps the thingto do isgo there and comment? I made a perfectly legitimate comment in here about Anthony Watts' misrepresentation of a conference abstract and all his followers misunderstanding what it actually said and as yet, noone in here wants to comment about that. Instead, all I get is comments about my blog. I can't help thinking you all agree with me. That's great.
Venter
I'm feeling the love. Clearly you need an excuse not to discuss my original comment in here. So be it.
uknowispeaksense
How about you take your issue up with Anthony Watts?
Lapogus, uknowispeaksense does not come across as a learned gentleman IMHO I would not put much value on anything he/she writes about themself on their blog. After all he/she has not got the courage to use their real name. Enough said and time wasted on this troll, I do wonder if it this is ZDB using another name.
What comment? The crap you are peddling? Anthony referred to it as a peer reviewed paper, then acknowledged his mistake and made the necessary corrections stating it as a presentation. You have been going around at CA and here still pretending that Anthony is cheating. As to the substance of the presentation I have not seen a single analysis or comment from you as to what's wrong with it. So you're basically trolling dishonestly, which is your modus operandi.
Jack - Indeed.
uknowispeaksense/Mike - I had a look at your blog and was not impressed by the level of discussion. I think it was the comment where you said you hoped that in future our grandchildren would dig up our bodies so they could cut our heads off. And you think of yourself as a scientist? As Jack rightly said at the outset, you run a nasty and abusive blog. No more to be said. Please don't come back here.
Venter
That's one massive mistake! No, I very much doubt it was mistake. He corrected it only when he was called on it and then he wrote some nonsense about thinking it was "in press". How he could mistake a conference abstract for an "in press" paper is beyond me. He is also STILL referring to it as a paper, which it isn't.
As for what's wrong with it.. Anthony takes two erroneous statements, one from the unreviewed abstract and one from the even less reviewed slides. The first came from the slides. It said, "... of 67% of the weather stations examined, questionable adjustments were made to raw data that resulted in: "Increased positive trends, decreased negative trends, or changed negative trends to positive,” whereas “the expected proportions would be 1/2 (50%)."
Nowhere do they explain why you would expect a 50/50 proportion. When the majority of the changes to the network invoved changing from warmer settings to cooler settings e.g. cities to airports or north wall settings to fully screened Stevenson boxes, you would expect to not get a 50/50 split. For examples and further explanations see Begert et al (2005), Menne et al (2010), Brunetti et al (2006).
Begert, M., Schlegel, T., and Kirchhofer, W.: Homogeneous temperature and precipitation series of Switzerland from 1864 to 2000. Int. J. Climatol., 25, 65–80, doi: 10.1002/joc.1118, 2005.
Menne, M. J., Williams, C. N. jr., and Palecki M. A.: On the reliability of the U.S. surface temperature record. J. Geophys. Res. Atmos., 115, no. D11108, doi: 10.1029/2009JD013094, 2010.
Brunetti M., Maugeri, M., Monti, F., and Nanni, T.: Temperature and precipitation variability in Italy in the last two centuries from homogenized instrumental time series. International Journal of Climatology, 26, 345–381, doi: 10.1002/joc.1251, 2006.
Next he quoted partly abstract and partly slide with “homogenization practices used until today are mainly statistical, not well justified by experiments, and are rarely supported by metadata. It can be argued that they often lead to false results: natural features of hydroclimatic times series are regarded as errors and are adjusted.”
Rather than get into how homogenisation actually improves data quality I'll just direct you to
Venema, V., O. Mestre, E. Aguilar, I. Auer, J.A. Guijarro, P. Domonkos, G. Vertacnik, T. Szentimrey, P. Stepanek, P. Zahradnicek, J. Viarre, G. Müller-Westermeier, M. Lakatos, C.N. Williams, M.J. Menne, R. Lindau, D. Rasol, E. Rustemeier, K. Kolokythas, T. Marinova, L. Andresen, F. Acquaotta, S. Fratianni, S. Cheval, M. Klancar, M. Brunetti, Ch. Gruber, M. Prohom Duran, T. Likso, P. Esteban, Th. Brandsma. Benchmarking homogenization algorithms for monthly data. Climate of the Past, 8, pp. 89-115, doi: 10.5194/cp-8-89-2012, 2012.
lapagos
Please don't attribute quotes to me that I didn't make. Also, please don't misrepresent the comments of other people. The comment you are referring to was made by someone called Martin and he said,
"However, what really saddens me is that when these people finally wake up to the reality of who it is that has been lying to them for so long, they are really going to hate themselves.. I just hope their grandchildren will be more forgiving (and not decide to exhume their bodies and cut off their heads when the reason they cannot go and play outside is explained to them)."
I said, "I hope they do. These moronic anti-establishment sycophants blindly following the words of any charlatan who comes along feeding them drivel that they readily accept because they lack the intellect to understand the more complex aspects of science, deserve to be made examples of. They need to be held up as the greatest examples of stupidity. As for the greedy perpetrators of the lies, well, I can only hope they are still alive when their own grandchildren disown them."
Now for context. When I said "I hope they do", that was in reference to the comment that they "are really going to hate themselves."
I am going to assume you have made an honest mistake in falsely attributing a comment to me and misrepresenting the comments of another. If it was deliberate, well.......
Sock puppetry anyone?
I don't know why you're all coming down so hard on ukwnwsee or whatever his name is. He seems to be talking good sense.
Well said! Where do we start? Gore? Hansen? Gleick? Most politicians? The rent-seekers who are in favour of carbon taxes? You?I mean, look at this
Iknowyouspeaknonsense
A certain mathematician named Doug Keenan was so insensed by what Prof Jones did with Chinese weather station data that he showed no hesitation in accusing Jones of Fraud. Prof Jones showed no hesitation in er um hesitating to respond.
Anthony is not exactly breaking new ground here, perhaps you should accuse Steirou, E., and D. Koutsoyiannis of Fraud, you might get lucky.
Mr Jackson
Nice one sir ^.^
Ah yes I remember now
I was going to point out that whether an article has or has not been published and peer reviewd means absolutely diddly squat these days. What always mattered and even more so today, is whether an article or paper stands up to close scrutiny I always believed that peer review did that but now I know different.
Venema, V., O. Mestre, E. Aguilar, I. Auer, J.A. Guijarro, P. Domonkos, G. Vertacnik, T. Szentimrey, P. Stepanek, P. Zahradnicek, J. Viarre, G. Müller-Westermeier, M. Lakatos, C.N. Williams, M.J. Menne, R. Lindau, D. Rasol, E. Rustemeier, K. Kolokythas, T. Marinova, L. Andresen, F. Acquaotta, S. Fratianni, S. Cheval, M. Klancar, M. Brunetti, Ch. Gruber, M. Prohom Duran, T. Likso, P. Esteban, Th. Brandsma. Benchmarking homogenization algorithms for monthly data. Climate of the Past, 8, pp. 89-115, doi: 10.5194/cp-8-89-2012, 2012.
31 authors. A veritable grant mine!
Mike Jackson
Well done. Very clever. Would you care to address any of the points I made about Anthony Watts' misrepresentation of the paper and its contents, and also the flaws in the paper itself or are you just going to behave like the rest of the people in here who lack critical thinking skills? You at least showed some creativity in quotemining me. Perhaps you could astound me with some logic.
hmmm... there is a nasty abusive troll in the house who has already established a pathetic track record at CA and WUWT..... I will say DNFTT but others will make their own calls of course, if it's around for long.... I wouldn't recommend anyone else wasting a click on the troll's blog but that is a nasty and pathetic piece of work for sure.
I'm still trying to work out why you bothered to post here in the first place. This is Bishop Hill not Watts Up WithThat. If you have a problem with one of Anthony's postings, take it up with him.
Andrew's lead-in quite clearly saysso I'm not sure what you're bitching about. Inter alia Koutsoyiannis is sayingSo? Quite a few people here and on other blogs and even in parts of the climate science community itself have been casting doubt on the accuracy or reliability of some of the "adjustments" for years. You appear to be a newcomer (or, as Jack Cowper suggests, a sock puppet — or perhaps even just a puppet. Whose, I wonder?)
If you disagree with Koutsoyiannis' conclusions, fine. Just don't come on here and accuse everybody else of lacking critical thinking skills just because they don't agree with you.
The quote I threw back at you applies equally well — probably better — to the climate science "establishment" than to the extremely large (certainly larger than you would like) and intellectually diverse and in some cases highly qualified body of observers, bloggers, and commenters that is sceptical of many of the more extreme pronouncements that come to us from theWe also tend to be better-mannered.
If you want a civilised discussion on climate and the science thereof you can have one.
If not, crawl back under your stone. (And before you think that is needlessly rude, I have read your blog and I really don't think it is rude enough but I have to draw the line somewhere.)
Mike
The post at WUWT also then goes on to explain his concerns and how they can be resolved. The trolls post to me seems to want to make a storm in a teacup. Watts made a mistake and then corrected that mistake - I see no problem here. I also think the troll should be applauding the fact that he then quotes McIntyre who gives a well reasoned solution to any problems. The troll then goes on to post a number of references in a "Got you with that one" - "Look how clever I am" style - this is not debate, if he could have said has anybody considered reading these papers perhaps there could have then been some discussion and learning instead he reduced the thread in to a food fight.
Mike
"We also tend to be better-mannered."
Really? My first post in here was polite and well-mannered and all I copped right off the bat was abuse. Sorry buddy but if you think you are all better-mannered you are kidding yourself...and to think I paid you a compliment. Anyway, whatever. If you wish to comment on my blog, perhaps you should take it up with me over there....see how that works?
uknowispeaksense
Quote from your 1st post:
"Naturally, those in the denial camp latch on to the lower end of that scale and run with it"
That's your version of polite is it.
Jack
We're used to that sort of "polite", aren't we?
RealClimate, Cook at SKS, ZDB, BBD, and now uknow on here — in fact, almost any one of the paranoiac tendency who appears to be scared shitless if anyone opposes their increasingly dubious world view.
In my experience only a deep sense of insecurity (or Tourette's) calls up that level of anger.
Jack, if you're going to quotemine that one line which really isn't that offensive, to justify ignoring everything else in that post then you probably need a thicker skin. My challenge to you is to find where some contrarians commenting on that "paper" have actually acknowledged that the authors have suggested a range of 0.4 to 7-8 degrees rather than just running with the lowest number? If you can't then I am completely justified in making that statement. If you won't well......now there's a moot suggestion hey?
uknowispeaksense
You obviously don't speak sense - if you cannot see why that is rude. You also link to your ugly abusive blog and you take great delight in the fact that your site stats appear to give another abusive sign at what you call deniers - you really need to grow up and start acting like a man and not like a bully in a playground. I frankly cannot be bothered with the ugliness you bring to the debate.
Make a storm in a teacup if you like; my advice would be to go somewhere else your attention seeking is tiresome.
You also have not answered someone else’s query about the work of Doug Keenan and his allegations of fraud - http://www.informath.org/pubs/EnE07a.pdf.
Although I am resigned to the fact that you still don't feel it necessary to respond to my statements, I will respond to the fraud allegations raised by another poster and now yourself..
The 1990 paper was one of many that the IPCC used for that purpose.
Wang was investigated and cleared by his university
Jones revisited the temperature data from China and came up with the same result. This was peer reviewed and published by GRL in 2008
Li, independently examined China's meteorological record and the results were identicial to Jones'.
Doug Keenen's fraud allegation is full of conjecture and what if's and relied heavily on assumptions being made about China during the Mao uprising. His claims that the uprising would compromise the availability of weather data are unfounded.
The correct course of action for Keenan to take would have been to obtain the data, analyse it himself and publish it. If he wanted to do it as a contradiction he should have published in the same journal as the original Wang paper. Instead, he went on the attack with baseless allegations of fraud which have been dismissed. To continue to bandy this about is like flogging a dead horse.
Jones, P.D., Lister, D.H. and Li, Q., 2008: Urbanization effects in large-scale temperature records, with an emphasis on China. J. Geophys. Res. 113, D16122, doi:10.1029/2008/JD009916.
Li Q. and W. Li, 2007: Development of the gridded historic temperature dataset over China during recent half century, Acta Meteroloigca Sinica, 65, 293-299