Click images for more details



Recent comments
Recent posts
Currently discussing

A few sites I've stumbled across recently....

Powered by Squarespace
« Greens in sight - Josh 303 | Main | Their lordships bestir themselves »

Significance doing the rounds

I'd like to commend to readers a couple of postings on the subject of statistical significance in the temperature records.

Last week a little visited website called Real Climate had an article by climatologist Stefan Rahmstorf, which addressed many of the issues discussed here in recent months. What I found interesting was that there was a measure of agreement:

...the confidence intervals (and claims of statistical significance) do not tell us whether a real warming has taken place, rather they tell us whether the warming that has taken place is outside of what might have happened by chance.

You will hear no dispute of that sentiment from this quarter. However, as readers here know, it is hard to demonstate that the post-industrial warming is outside the range of natural variability, a subject that the Rahmstorf article rather glosses over, instead preferring to focus on the pause:

...the question the media love to debate is not: can we find a warming trend since 1998 which is outside what might be explained by natural variability? The question being debated is: is the warming since 1998 significantly less than the long-term warming trend? Significant again in the sense that the difference might not just be due to chance, to random variability? And the answer is clear: the 0.116 since 1998 is not significantly different from those 0.175 °C per decade since 1979 in this sense.

It goes on to look at a changepoint analysis of the data performed by Niamh Cahill, a postgraduate student from Dublin.

The optimal solution found for the global temperature data is 3 change points, approximately in the years 1912, 1940 and 1970. There is no way you can get the model to produce 4 change points, even if you ask it to – the solution does not converge then, says Cahill. There simply is no further significant change in global warming trend, not in 1998 nor anywhere else.

And so if she is to be believed we are just seeing long-term warming with natural variability superimposed. However, Ms Cahill's conclusions have been challenged at WUWT, where reader Jeff Paterson seems less than impressed:

[Cahill's] basic thesis is that since a CPA analysis detects no significant recent change in the slope of the GISS dataset there is no pause. Unfortunately, the analysis is of no value because, as is commonly known, the CPA cannot be used on auto-regressive time series.

I am insufficiently expert to be able to say who is right here. Ms Cahill lists time series analysis as one of her areas of expertise so I am dumbfounded if she really has made the error Paterson says she has.

Do any readers here know the answer?

PrintView Printer Friendly Version

Reader Comments (26)

Googled 'change point analysis autoregressive time series' and got as first reference

'Change-Point Analysis: A Powerful New Tool For Detecting Changes' by Dr. Wayne A. Taylor

About halfway down that page:

'Data not appropriate for a change-point analysis and control charting include autoregressive time series data such as stock prices.'

Seems Jeff Paterson may have a point!

Dec 10, 2014 at 11:52 AM | Unregistered CommenterChGr53

I know nothing, but the first Google hit for 'cpa analysis time series' is this:

and it immediately states in its Assumptions:

CPA assumes that the process (time series) must be DISTRIBUTED IDENTICALLY, and the observations must be INDEPENDENT (at least there is no strong autocorrelation)

so there do seem to be limitations to the method. I would have thought that historical temperatures aren't independent, but as I said, I know nothing.

Dec 10, 2014 at 12:04 PM | Unregistered CommenterChris Long

Worryingly, that page also says

'There are numerous approaches to performing a change-point analysis.'

Another hit from that google is:

is entitled 'Fourier Methods for Sequential Change Point Analysis in Autoregressive Models', so one would need to know more about the methodology used by Ms Cahill, I suppose.

Dec 10, 2014 at 12:07 PM | Unregistered CommenterChGr53

"The optimal solution found for the global temperature data is 3 change points, approximately in the years 1912, 1940 and 1970"
But according to the IPCC there was little anthropogenic CO2 prior to the 1950s, so tow of those change points (ie., 1912 and 1940) would not appear to be the result of manmade CO2 emissions.

If 2 of the 3 changepoints were not the result of manmade CO2 emissions, why would the third be the result of manmade Co2 emissions? This is particuolarly important since one probable explanation for the 1970 change point is that that is when we see the effects/corruption/contamination of data caused by UHI, station drop outs and/or inappropriate homogenisation/adjustments.

When considering that point, one should not forget that the land based thermometer record shows warming from tyhe mid/late 1970s through to say 1997, whereas the satellite data shows no warming between 1979 (the date it first streamed back data) and the run up to the Super El Nino of 1998 such that the satellite data suggests that there is some warming bias in the land based thermometer record at any rate as from 1979, and it would not surprise me that the warming bias took place earlier and explains the apparent change point of 1970.

Dec 10, 2014 at 12:13 PM | Unregistered Commenterrichard verney

I am with Ernest Rutherford, 1st Baron Rutherford of Nelson, OM FRS on this:

"If your experiment needs statistics, you ought to have done a better experiment.

Dec 10, 2014 at 12:56 PM | Unregistered Commentergraphcconception

I remain convinced that we actually know beggorah all about what is happening, or has happened, in the climate - which is probably why we always seem to know beggorah all about what might happen in the future.

Dec 10, 2014 at 12:59 PM | Unregistered CommenterMorph

The point is moot. The projections of the earlier IPCC ARs have diverged from reality. That is more significant than the (lack of) warming trend since 1998.

Dec 10, 2014 at 1:00 PM | Unregistered Commenteronion

This is particuolarly important since one probable explanation for the 1970 change point is that that is when we see the effects/corruption/contamination of data caused by UHI, station drop outs and/or inappropriate homogenisation/adjustments
... and also the point when the politico-environmental activists got into top gear which may be totally irrelevant (or then again, may not).
Since they spent the next five years worrying about global cooling it is quite obvious that "change points! can only be identified well after the event. Perhaps it would be as well if they bore that in mind when drawing assumptions.
On the basis of a warming (~1912 to 1940) and a cooling (1940 to 1970) perhaps it is time they started looking for a change point around 2000-2005.
I presume that honest researchers are doing that. Anyone of them care to comment?

Dec 10, 2014 at 1:02 PM | Registered CommenterMike Jackson

So being "insufficiently expert", and instead of asking Ms Cahill herself for any supporting information, or maybe contacting your local university to see if any statisticians are able to explain things in more depth; you read/link to an article on one of the most inept, anti-science websites on the internet, where a dodgy bloke performs some dodgy "analysis" that completely ignores any underlying physics; then tops it all off with some dodgy curve fitting baloney, and tells us that ONE "cycle" of unknown physical origin amounts to long term cyclical behaviour!

Dec 10, 2014 at 1:08 PM | Unregistered CommenterQuentin Wallace

If they were asked to peg their salaries to either the "pause" data or the "pre-pause" data, then we would certainly see a mass outbreak of statistically significant honesty.

Dec 10, 2014 at 1:23 PM | Unregistered Commentermichael hart

Time and again I hear nonsense from people saying climate signals are, or are not, "significant".

Unfortunately, this is a meaningless concept unless you have an understanding of the type of variation present.

And it is particularly meaningless applying the standards tests, intended for when the variation is independent from measurement to measurement. We know we have long periods of warm (medieval warming period) and long periods of cold (little ice-age) so right away we can categorically rule out the standard statistical tests. You just cannot use them.

Or to put it in simple terms: unless you understand what is "normal" you cannot know what is "abnormal".

I've got a few articles on the subject: Lies, damned lies, and statistical significance of climate trends

I've got a demo you can play with to start understanding the type of noise and why it doesn't fit standard statistics at and some info on the generator at: About

I've got some results showing how variance increases: Statistics of 1/f noise – implications for climate forecast

And I've even collected together a few examples of how 1/f noise causes errors in interpreting data even by sceptics: Natural habitats of 1/f noise errors.

Dec 10, 2014 at 1:28 PM | Registered CommenterMikeHaseler

I am with Quentin. Why any of you take information from Real Climate beats me too.

Dec 10, 2014 at 1:31 PM | Unregistered CommenterBobthetilde

RC won't and can't improve as long as they run after more and more ex-post-facto excuses and data manipulations in order to keep up an appearance of being always right.

Had Rahmstorf made any of his arguments before the "pause", or explained years ago why change point analysis was going to be thing to do, or or or, his opining now would carry some weight. As it stands it's like open fishing season on the literature looking for something, anything to be the Deus Ex Machina that resolves every problem with the touch of a single article.

Dec 10, 2014 at 1:51 PM | Registered Commenteromnologos

Just to clarify something which is important. Above I've explained why we cannot make any assertions about statistical significance of the real climate. So how can we say there is a "pause"? The answer is that the pause isn't really a pause in warming, but a discrepancy between the projected warming and actual data. So, the test for the pause is very very simple: is the global temperature lower than the range the IPCC gave for it. The IPCC didn't provide a statistical model but instead gave a very simple range which was 0.14 to 0.58C/decade.

So, the only test that matters is whether the current temperature is within the range given by the IPCC.

No other test is meaningful.

Dec 10, 2014 at 2:26 PM | Registered CommenterMikeHaseler

You don't suppose that those highly paid and experienced people at Real Climate are losing their blind faith in the all powerful CO2 molecule?

Just because none of their predictions became true, is no reason why they should not continue to enrich themselves at everyone else's expense.

Dec 10, 2014 at 2:38 PM | Unregistered CommenterGolf Charlie

As I am looking at the sea surface temp data from the 1900s to 1950s (bucket bias stuff - on a discussion thread) it might be worth noting that the Met Office noted a discontinuity around 1940 due to what is believed to be a switch to engine inlet temp measurements rather than bucket hauling. There was a war on after all. Some corrections were made to the temp anomaly record but it may still have a residual discontinuity. I wonder has this been taken into account ?

Dec 10, 2014 at 3:55 PM | Unregistered CommenterMicky H Corbett

Micky: "As I am looking at the sea surface temp data from the 1900s to 1950s (bucket bias stuff - on a discussion thread)"

Steve Mc did a few posts on the subject a few years back which are probably worth checking on.

Dec 10, 2014 at 5:07 PM | Unregistered CommenterDaveJR

I couldn't find anything I'd written which was an introduction to 1/f noise and the climate to go along with the links, so I've quickly written one: Introduction to 1/f climate noise

The conclusion is also relevant: "Therefore, given that the 20th century changes are very typical within the temperature record, this means that if we were only looking at these temperature records, then we would be forced to conclude that the whole 20th century change is most likely due to natural variation."

Dec 10, 2014 at 5:34 PM | Registered CommenterMikeHaseler

Rahmstorf appears to be someone who is easily entranced by new shiny math thingys that he has no understanding of,

Dec 10, 2014 at 6:51 PM | Unregistered Commenterstan

""Unfortunately, the analysis is of no value because, as is commonly known, the CPA cannot be used on auto-regressive time series." I am insufficiently expert to be able to say who is right here.[...] Do any readers here know the answer?"

It depends on what statistical model it uses. The standard basic change point analysis found in undergraduate text books assumes a non-autocorrelated model, and is therefore invalid if the data is autocorrelated.

However, it is entirely possible to construct a change point analysis test assuming an autoregressive model - indeed, Ross McKittrick did this in his paper discussed here.

Ross said: "Fitting a linear trend through a series with a positive step-change in the middle will bias the slope coefficient upwards. When I asked Tim if the VF method could be used in an application allowing for a suspected mean shift, he said no, it would require derivation of a new asymptotic distribution and critical values, taking into account the possibility of known or unknown break points. He agreed to take on the theoretical work and we began collaborating on the paper. Much of the paper is taken up with deriving the methodology and establishing its validity. For readers who skip that part and wonder why it is even necessary, the answer is that in serious empirical disciplines, that’s what you are expected to do to establish the validity of novel statistical tools before applying them and drawing inferences. Our paper provides a trend estimator and test statistic based on standard errors that are valid in the presence of serial correlation of any form up to but not including unit roots, that do not require the user to choose tuning parameters such as bandwidths and lag lengths, and that are robust to the possible presence of a shift term at a known or unknown break point."

I gather from reading WUWT that the RealClimate analysis used the standard algorithm with the non-autocorrelated model, rather than anything particularly clever. But I haven't checked, so I recommend that anyone interested (who understands what the hell I'm talking about!) should do so. I'm not sufficiently curious to look. However, it's not generally true that CPA cannot do autocorrelated series, it can be modified to do so.

By the way, I'm also not endorsing Ross's approach as unarguably the right method either. All statistical significance testing relies on assuming a statistical model, for which you need externally-sourced, independent reasons for thinking is a sufficiently good approximation to the physics. You can't do it just by looking at the data, and while Ross's approach is a lot more plausible, I'm not convinced that his model can be validated from the physics either. This is the same issue that Doug Keenan raised.

To say anything at all about the significance or non-significance of anything to do with the global temperature record, you need to have a validated statistical model of the normal background variation, preferably founded in the physics, and we simply don't have one. For that reason, the RealClimate post is just yet more hot air.

Dec 10, 2014 at 8:18 PM | Unregistered CommenterNullius in Verba

"a little visited website called Real Climate"


Dec 10, 2014 at 10:59 PM | Registered Commenterjamesp

Micky / DaveJR - I came across Steve's page on this a while ago, here's the link:

sea temperature adjustments -

Dec 10, 2014 at 11:39 PM | Registered Commenterlapogus


I've read these in the past and at the time. Shows how long I've been reading about this stuff.
My take on the bucket adjustments especially in Folland 1995 is a bit more fundamental. Or at least in the sense that I'm a bit confused on the approach taken once a model of buckets has been validated to some extent. I'll update my discussion thread later.

Dec 11, 2014 at 8:21 AM | Unregistered CommenterMicky H Corbett

I thought the magic number for changepoint years would have been "17" ... Or else is that now disavowed as the key line for understanding a trend that moves beyond the bounds of the expected?

I bet if that same trendline graph is run with 17-year lines instead of 30, there may well be a recent changepoint. I do see it reasonable that 30 can be chosen, but also 17, based on climate scientists' own words.

Dec 11, 2014 at 11:06 AM | Unregistered CommenterSalamano

I really don't care how close statistical correlations track reality. I care that climate scientists understand causality well enough to predict future results and try to do so while continually trying to falsify their own predictions. Alarmist climate science has given itself a free pass to ignore this type of rigor. Rather than trying to disprove their hypotheses, they try to prove them. This malpractice is written right into the charter of the IPCC. And with so much data available, with a little careful cherry picking, or tendentious modeling, one can find "evidence" for any kind of hypothesis.

Dec 12, 2014 at 4:58 PM | Unregistered CommenterMickey Reno

I'm a little late to this discussion, but I saw Jeff Patterson's post when it first went up, and it is terrible. Patterson has no idea what he's talking about. I commented on his post to point this out when it first went up. I won't quote the whole comment, but my conclusion was:

Pretty much nothing this post says about changepoint analysis is right. I’m not sure about the other things it says. I haven’t been presented any justification for them, so even if they are right, I have no reason to believe it.

To make things worse, Patterson responded to me by saying things which were completely untrue, and, quite frankly, were ridiculous. It's sad such mind-numbingly bad criticisms can get posts on WUWT.

For those who are just interested in this quote:

[Cahill's] basic thesis is that since a CPA analysis detects no significant recent change in the slope of the GISS dataset there is no pause. Unfortunately, the analysis is of no value because, as is commonly known, the CPA cannot be used on auto-regressive time series.

Know it is completely wrong. There are many different ways to try to detect change points. Some cannot handle autocorrelation. Some can. Patterson's claim it "is commonly known" none can is a sign he has absolutely no idea what he's talking about.

Dec 13, 2014 at 8:01 AM | Registered CommenterBrandon Shollenberger

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>