Bishop Hill Another Hockey Stick broken
Jun 7, 2012 Climate: MWP There are some important findings at Climate Audit today. Once again I have tried to set out a layman's version of the discussion there.
One of the perennial problems with temperature reconstructions has been a lack of data covering the southern hemisphere - the Hockey Stick itself was a northern hemisphere reconstruction, although the IPCC billed it as global in extent. However, a recent paper by Gergis et al sought to partially remedy this by presenting an Australasian temperature reconstruction for the last millennium, based on 27 proxy records, primarily from tree rings and corals. The headline was, perhaps not unexpectedly, that late twentieth century warming was unprecedented:
The average reconstructed temperature anomaly in Australasia during A.D. 1238–1267, the warmest 30-year pre-instrumental period, is 0.09°C (±0.19°C) below 1961–1990 levels.
;)
However, when the paper was examined in more detail, alarm bells began to be sounded. Concern centred around the proxy data sets used in the study.
Gergis et al reported that they had used 27 proxy series, but that they had selected these from a larger body of data. This was a critical step in the process and one that could well have led to a bias in the results. Many of the studies in the field of temperature reconstructions rely on "sorting" or filtering the data in some way, either choosing only proxy series that correlate well with their local temperature or alternatively weighting them according to how well they correlate. On the face of it, this is a reasonable approach, as the argument might well be made that if there is no correlation then the series is clearly not a proxy for temperature. However, the problem with this approach is that it amounts to a circular argument; it could be that the correlation between temperature records and proxy data is coincidental. This flaw has been demonstrated by studies in which the proxy series are replaced with dummy data series that wiggle up and down at random. In these studies, it has been shown that most of the time the resultant "reconstruction" is a hockey stick.
However, the good news was that Gergis et al were apparently aware of these issues and they declared that they had found a way around them. According to the paper:
For predictor selection, both proxy climate and instrumental data were linearly detrended over the 1921–1990 period to avoid inflating the correlation coefficient due to the presence of the global warming signal present in the observed temperature record.
In other words, when doing the correlation calculations to determine which proxies they would use, they first removed the twentieth century trend (actually, the 1921-1990 trend) from both the proxies and the temperature records, so that instead of filtering for series with a twentieth century uptick, they were picking records that appeared to match in their year-to-year wiggles.
So far so good, but this would have been a calculation that had to be done with immense care and a replication of it could therefore have been illuminating. However, in order to do this, it would be necessary to obtain all the Gergis data including the series that had been filtered out and not used, and unfortunately Gergis was not playing ball:
The compilation of this database represents years of our research effort based on the development of our professional networks. We risk damaging our work relationships by releasing other people’s records against their wishes. Clearly this is something that we are not prepared to do.
We have, however, provided an extensive contact list of all data contributors in the supplementary section of our recent study ‘Southern Hemisphere high-resolution palaeoclimate records of the last 2000 years’ published in The Holocene (Table S3):
http://hol.sagepub.com/content/early/2011/12/16/0959683611427335
This list allows any researcher who wants to access non publically available records to follow the appropriate protocol of contacting the original authors to obtain the necessary permission to use the record, take the time needed to process the data into a format suitable for data analysis etc, just as we have done. This is commonly referred to as ‘research’.
The refusal to release data was troubling, as it prevented anyone ever being able to replicate Gergis's work. And when Gergis's blog was unearthed and it was revealed that she was a committed environmentalist, the alarm bells became louder still.
Despite the flat refusal to allow replication of the paper, it was still possible to verify certain aspects of the filtering process. In particular, the data for the 27 proxies that had been used was available and so it would be possible at least to replicate the calculation that showed that these had significant correlations to their local temperature once the 1921-1990 trend had been removed. This task was taken up by statistician "Jean Sibelius" but rather remarkably he found himself unequal to the task:
Steve, Roman, or somebody, what am I doing wrong here? I tried to check the screening correlations of Gergis et al, and I’m getting such low values for a few proxies that there is no way that those can pass any test. I understood from the text that they used correlation on period 1921-1990 after detrending (both the instrumental and proxies), and that the instrumental was the actual target series (and not the against individual grid series).
Sibelius's difficulties were confirmed by others, including Steve McIntyre, but perhaps most significantly, by CSIRO's Nick Stokes, who is no sort of a sceptic. Stokes agreed with Sibelius that, when detrended, the correlations for the 27 proxies used in the Gergis reconstruction were insignificant, completely contradicting Gergis's paper. However, extraordinarily, Stokes also ran the calculations without detrending and found correlations that were significant.
I’ve run Steve’s code with and without detrending, and with and without the Quenouille correction. Without detrending (but with zero mean) or AR1 correction all (exc maybe Madang) proxies do seem significant
This seems to suggest that Gergis's declaration that the correlations were based on detrended data was false and that she and her co-authors had indeed fallen foul of the circular argument noted above. The finding of unprecedented warmth reported in the Gergis paper appears as though it is a function of the methodology used rather than of the underlying data.
Bishop Hill
Paul Matthews points out another serious problem with the Gergis et al paper:
Worse still, proxies were selected with positive or negative correlation. In other words, some were used 'upside-down'.
Reader Comments (121)
Anyone surprised by this , for after all the idea was to support an already established ‘conclusion ‘ not to see if that conclusion was 'actual valid' in the first place .
In that Gergis is merely following standard climate science methodology .
Can anyone help me out here? May be a dumb question but I wonder if anyone could expand on how the detrending works at a layman level? I couldn't see a reference in the Gergis paper for their method but it seems to be accepted by all sides as straightforward step to implement. Is it just subtracting the linear slope that is the agreed trend for 20C warming?
They say they detrend on both the proxy and instrumental records so I had assumed it meant some sort of high pass filter applied to both - hence the explanation why low frequency upside down 20C trends could make it through.
That looks like a fair old bump around 1300...
If proxies are selected using detrended comparisons against 20C temperatures, then won't this step alone provide adequate proof of the validity of the treemometers?
a) Get a detrended temp record for 20C
b) Get a detrended thing record from the proxy
c) Select proxies from (b) that match (a).
d) Plot set (c) against absolute temps and check correlation - if not a great correlation, then step (c) was random.
Upside-down proxies are not by definition a problem - why shouldn't trees dislike hotter weather? Selecting both ways up within a similar temp range is a problem.
When one "cherry picks" the data, one can "cherry pick" the conclusion. This ain't science.
I will shortly be forwarding you an email with an interesting development on this from an Australian perspective.
Like JamesP I seem to have spotted a re-appearance of the MWP?
If you take the same trend off the proxy as the instrumental record, and they match, so what? That just means they match before and after, surely?
Same old stuff.
Start with an answer (20C temps unprecedented), and work backwards until you can get the data -- or selected bits of it -- to agree.
Post-modern science, expertly carried out.
In a few years from now the data sets will be graph paper and a box of crayons.
Sorry I can't release the data sets as they are pinned to my fridge.
Look, Bish, I'm busy ok. My first peek at BH today and there's
1. Worstall again (on the heart of the matter this time)
2. Gleick "cleared"
3. Another Hockey Stick broken.
Sigh. Brilliant though. Talk about a Gordon Brown "bad news sandwich".
"There are some important findings at Climate Audit today"
What makes them "important" compared to almost any other example of the same behaviour in pursuit of the same graph? This is something that might merit a speculative sentence or two.
McSteve has been on fine and twinkly form carrying out his, aha, "research" and it's educational, but what makes the failure of this paper so significant?
I suppose the anticipation that it may be cited in AR5, and / or the transparently non-transparent behaviour of the authors and lack of critical review?
Oh, that reads poorly; to clarify, I meant a speculative sentence of two from the Bish in his post article, not my thinking out loud.
Upside-down proxies are not by definition a problem - why shouldn't trees dislike hotter weather? Selecting both ways up within a similar temp range is a problem.
Which comes the root cause of these problems, some species of tree will react to temps different from others, but why don't they involve Biologists to inform them how a species will react, same reason they do not include Statisticians, it will kill the idea of trees giving historic records of temps.
mrsean2k:
You suppose right, in my book. It's significant and exciting to see this happening as the weighed-down AR5 train tries to lug itself through to a fatuous extension of the hockey stick, albeit on life support.
It is with a sense of dark satisfaction that one notes the half-life on these paleo-dendrophrenological offerings rapidly decaying into the weeks, if not days, on one hand.
As Marge Simpson once said: Get ready, skanks. Here comes the Truth Train.
BoFA - "some species of tree will react to temps different from others"
Plant growth is affected by a number of factors - temperature, light, disease, predation, light, inter-plant competition, rain, soil, altitude - to name but a few.
It is dangerous to ascribe growth to any one factor
mrsean2K
Richard Drake
It's not speculation about AR5, apparently the Gergis, et al (2012) is already cited in the first draft:
Gergis, et al (2012) cited in AR5, 1st draft
The statement quoted by Steve McIntyre might, of course, be expanded in detail into the 20th century for a SH hemisphere "hockey stick".... in any case, getting Gergis, et al (2012) into AR5 would seem to be a key element of having it affirmed as part of the "settled science" we hear so much about.
It does not surprise me that Gergis' hockeystick is as good as a conjuring trick. As I pointed out in the previous thread Joelle Gergis talks up her results, close inspection of the instrumental data shows that there has been no 20th Century rise in temperatures in Australasia:
John Daly's summary of what the Australian stations say - http://www.john-daly.com/stations/stations.htm#Australia
An adjustment like Alice - http://notalotofpeopleknowthat.wordpress.com/2012/03/15/an-adjustment-like-alice/
Hansen tampering down under too - http://www.real-science.com/hansen-tampering-down-under-too
The Chefio looks at New Zealand data - http://chiefio.wordpress.com/2011/02/18/comparison-temperatures/
NIWA has abandoned the official national temperature record
Hence it would be have truly extraordinary for her to have produced a hockeystick regardless of her data selection methods and procedure.
The point for the Team seems to be both to get this Gergis et al (2012) paper into AR5 and to pretend that it is new and significant evidence adding to the understanding of SH climate. Steve McIntyre on the background to IPCC's AR5 (First Draft) claim that Gergis et al (2012) is providing analysis of "new" paleo proxy records from SH (these are just excerpts, please read that entire post for details and context):
======================================================================
[from Climate Audit post linked below]:
The “New” Paleo Records
What are the “new paleo records from Australasia” that shed this new light on the medieval period?
Gergis et al has only two proxies in the period 1250-1318: tree ring series from Tasmania and Oroko Swamp, NZ, both from Ed Cook. Neither of them are “new’. Nor is a third long proxy, Law Dome O18 (screened out by Gergis.)
The “New” Gergis Proxies in IPCC AR4
Not only are the “new” Gergis proxies not new; they were prominently displayed in the AR4 section on SH proxies. Indeed they are the only two long proxies so displayed....
....The two long series illustrated in AR4 come from the same two sites as the two long Gergis series.....
Thus, the statement in AR5 that the Gergis results for the medieval period come from “new paleo records” is untrue. I wonder if reviewers picked this up.
Mann and Jones 2003
IPCC AR4 referred to Mann and Jones 2003, which included a SH reconstruction, as a primary SH reference, noting that they used “only three series”....
....Remarkably, and I hadn’t noticed this until just now, IPCC AR4 didn’t show the Mann and Jones SH reconstruction, for reasons that will become clear below....
=======================================================================
Gergis, et al (2012) cited in AR5, 1st draft
Why do these papers get away with using tree rings? Tree rings are not good proxies for temperature doh!
The author herself said it best:
...climate style.
Skiphil: absolutely, thanks.
Bish
You have put words into Nick Stokes's mouth - and he is a master of the evasive statement. As far as I can tell, he makes no comment on the correlation of the detrended figures. You might assume that that implies that he agrees with Jean S but that is dangerous given that he is a noteworthy dissembler and nit-picker.
This is worth noting, from Jean S:
(2011 paper, not this one)
From my reading that means the same tree species correlated in opposing fashion at different locations. For example, at one location a wide tree ring is interpreted as warming, but at another location where the same species is examined a wide tree ring interprets as cooling.
... another zebra down ...
just another day on the Serengeti
[I'm not a lion merely a tourist.... but it is fascinating to see the lions in action, in a morbid kind of way]
@diogenes
As he says: "I just observe the numerical results"
Am I right in thinking that the consensus is that a temperature of 1-2C above current levels is optimum?
So even if we take the data at face value, shouldn't the papers be reporting ...
'Earth saved from catastrophic cooling.
Man-made CO2 emissions in the last century may be helping to cancel out the gradual cooling of the previous 600 years, saving us from environmental disaster.
"Another 400 years like the last 100 and things will be perfect", said a delighted environmentalist. "I have just ordered a Hummer."'
So let's recap.
1) We have the refusal to release data- so that the work cannot be replicated and verified
2) Nick Stokes found that, when detrended, the correlations for the 27 proxies said to be significant and used in the Gergis reconstruction were insignificant.
3) Claimed "new" paleo records are in fact "old".
I think that this amounts to academic fraud and should be called as such.
The reason this travesty of a paper is so significant is that the Australian Government has endorsed it as input to AR5, and also that there is not a lot of work on the Southern Hemisphere climate history that covers this ground, so they have been able to write their own ticket.
I am very grateful to Steve M, Jean and others at CA who have applied their expertise and effort to analysing it. It is not so long ago that this kind of tendentious junk would have been a poster child for the Cause without attracting much (if any) critical analysis. Thanks also to the Bish for bringing it to a wider audience.
Go find it yourself. This is commonly referred to as re-search.
When a proxy is de-trended is it against a local, regional or global temperatures?
Is it de-trended for against the slope over the entire record or the slope year over year?
"When a proxy is de-trended is it against a local, regional or global temperatures?"
None of the above. The proxy is detrended against itself - i.e. any trend in the proxy data is removed.
@Don Keller
re your point 2, I think Nick Stokes found that the non-detrended figures were significant-ish but said nothing about the detrended figures. He said nothing about the latter.
@Diogenes
"Stokes agreed with Sibelius that, when detrended, the correlations for the 27 proxies used in the Gergis reconstruction were insignificant, completely contradicting Gergis's paper. However, extraordinarily, Stokes also ran the calculations without detrending and found correlations that were significant."
The point being that Gergis specifically said that
"For predictor selection, both proxy climate and instrumental data were linearly detrended over the 1921–1990 period to avoid inflating the correlation coefficient due to the presence of the global warming signal present in the observed temperature record."
So what exactly is Gergis doing?
Playing fast and loose I believe.
I'm getting seriously worried now about the quality of research published in journals. Say I wanted to start from scratch to get up to speed with climate science: I could study the basics and intermediate level stuff, as that's old science, but when I try to get to grip with more modern research in this field, how on earth do I know what is 'true' science, and what is just utter garbage? You could waste a lifetime reading complete and utter nonesense like that written by Gergis et al and never know if it wasn't for the excellent forensic work done by intelligent people in their spare time.
Science is really in a terrible terrible state.
Slightly off topic but relevant.
How is this for a bit of detrending?
http://wattsupwiththat.com/2012/06/06/noaas-national-climatic-data-center-caught-cooling-the-past-modern-processed-records-dont-match-paper-records/#more-65130
Over to you Rob (Wilson) :-)
Your Grace! Tsk,tsk.
You mean (of course) "'a committed environmentalist' and far leftist".
I spent 10 minutes on her (recovered) blog. (I needed a shower thereafter.)
@Leopard in the Basement,
If you have experience of scientific or engineering terminology, you will find climate science uses common terms in bizarro ways. It is usually possible to perform a running mental translation - if you are broadly familiar with the methods of climate science. The first lesson comprises How to Make Hockey Sticks. A clear and brief exposition is here:
http://noconsensus.wordpress.com/hockey-stick-posts/
I said this before.
Detrending, even if 'done right', does not resolve the circular inference issue. It is merely a sandwich between two entities that ought not be compared with one another to derive a standard.
I have had a good day. I have just drunk a 12 year old bottle of St Emillion. I live 15 kilometers from St Emillion in France on the river L'isle. When I ponder the lengths these so called climate scientists go to, to produce a tome in line with the faith, I do actually feel sorrow for them, for it is a wasted endeavour to corrupt the truth but there is enlightenment in revealing it. So the likes of M&M and all those who point out the falseness of this kind of work will have a sense of achievement in putting their critical analysis to the world. Enlightenment is the Eureka moment the other is the Piltdown syndrome. This will be the state of climate science until the money for getting the required preordained result is taken away. Then the faithful will have a hard time at looking back on the life's wasted work and time.
Jun 7, 2012 at 7:50 PM | Luther Bl't
Thanks for that link, I had read that ages ago but it was worth a reminder. However that refers to the Mann method which I believe didn't relay on detrending before screening.
Just noticed on the Climate Audit thread someone called Matt Skaggs answered a similar question to mine with a link to an interesting looking paper that seems to cover the kind of thing I was looking for. Looks detailed and will keep me occupied getting up to speed with the more complex stuff. Detrending seems a hell of a lot less straightforward an issue than I thought!
I notice Steve McIntyre made a rather pointed comment about detrending that kind of gives a flavour of the issues going on here, re the different approaches:
As their are causal variables other than temperature that affect the response parameter measured in proxies, to remove any significant number of observations simply because of correlation issues dilutes the value of any subsequent analysis. Unfortunately, the publish or perish environment in today's academia has led to much less rigor on this point (a dirty little secret that few will ever bring up). This situation allows the shorter research time required for the published output demanded but quality is lessened at the expense of quantity. Given that for any work to be seriously considered it must be built upon prior published research, to properly consider the value of today's work it should be viewed with a skeptic's lens. Anyone who believes their work has a global consequence, such as global warming, should expedite the ultimate validation of their work with high transparency, especially in regards to the foundational data.
I understand why it is prudent to keep confidential research-in-process but once published, any honest necessity to maintain confidentiality escapes me. If someone uses any such data in later publications, won't an accompanying citation only add value to the original research from whence it came? For those warmers who refuse transparency here, given the dire consequences to the world if nothing is done, it is logical to conclude they are either 1) hiding something that would otherwise invalidate their results or 2) putting their careers ahead of their duty to their fellow man.
All claims to validity based on the peer review process should be ignored as the reviewers likely took little time to properly review the foundational data or they are birds-of-a-feather members of the same "exclusive" club. If they were passengers on the Titanic, I am sure they would have put together some very good advice on how to arrange the deckchairs.
Worse still, proxies were selected with positive or negative correlation. In other words, some were used 'upside-down'.
Climate change research 'from the land 'down-under' style.
Seriously, what does Dr Robert Wilson think of the garbage that is published in his cherished journals ? Methinks that they should be published on small squares of soft 2-ply tissue neatly arranged with perforated end-to-end mechanical joints and rolled over a cardboard tube.
As I nothed over at Climate Audit, in the case of one of the only two proxies used by Gregis to cover the period 1250-1318, tree ring series from Oroko Swamp, NZ, suffers from the problem that the official NZ temp used to calibrate the proxies has been revised since the study took place, and in the course of that revision the agency responsible for the official series (NIWA) has stated “Because of lower confidence in early temperature measurements the revised temperature series is not constructed prior to 1900″ although the study used the 1866 -1880 period to verify its models.
At the very least Gregis should have reviewed the impact of these changes on the Oroko proxies before using them. It has a direct impact on the post 1957 component of the series, would have reduced the confidence in the proxies as less data was available to make the comparison with the instrumental record, including using 20th century data for model verification.
DJ at 7:32 pm,
It isn't just climate science. The whole formal, academic scientific enterprise is collapsing in a mess from bad statistics, careless attitudes toward quality, and political domination.
Joelle Gergis gets money to play in the big leagues:
The above is from her now defunct blog.
http://web.archive.org/web/20100405120650/http://joellegergis.com/?p=240#comments
I also noticed a link to this webpage on Gergis' blog which seems to be the public page of the 'scientific research' she is carrying out.
http://climatehistory.com.au/
What will she do once the project is over? Work for CSIRO perhaps?
Skiphil:
... another zebra down ...
just another day on the Serengeti
You took it out of my mouth. ;-)
sHx
I think you get all credit... didn't you say something like that on another thread recently? I was trying to think of where I saw that previously.... anyway I know it's not my original mockery of Dr. Mann's bloviating, so maybe you get the credit?
btw, I wonder if the serious scientists at Real Climate are in a hurry to correct their recent confident praise of the Gergis et al (2012) paper, or will they do a decade long rear-guard action, stalling to acknowledge any problems?? From Real Climate (passage is hyperlinked):
"Fresh hockey sticks from the Southern Hemisphere...." "First, a study by Gergis et al., in the Journal of Climate uses a proxy network from the Australasian region to reconstruct temperature over the last millennium, and finds what can only be described as an Australian hockey stick. They use an ensemble of 3000 different reconstructions, using different methods and different subsets of the proxy network. Worth noting is that while some tree rings are used (which can’t be avoided, as there simply aren’t any other data for some time periods), the reconstruction relies equally on coral records, which are not subject to the same potential (though often-overstated) issues at low frequencies. The conclusion reached is that summer temperatures in the post-1950 period were warmer than anything else in the last 1000 years at high confidence, and in the last ~400 years at very high confidence."
Skiphill
I think it is Mann who gets credit for the Serengeti analogy. I am just glad it is catching on. :-)