Bishop Hill Statistical sierra
Jun 9, 2014 Climate: Statistics Climate: Surface Sierra Rayne, writing at the American Thinker blog over the weekend, took a gentle pop at AP's Seth Borenstein for making alarmist claims regional temperature trends in the USA while barely paying lipservice to standard statistical techniques.
The AP used "the least squares regression method" to calculate the annual temperature trend for all these regions, but then proceeded to ignore entirely whether the regression method indicated if the trend was statistically significant (the typical criteria would be a p-value<0.05).
This is first-year statistics level stuff. Quite simply, if your statistical test ("least squares regression method") tells you the trend isn't significant, you cannot claim there is a trend, since the null hypothesis (i.e., no trend) cannot be rejected with any reasonable degree of confidence.
In an area like climate, you would have thought an experienced journalist like Borenstein would take some statistical advice before writing.
Reader Comments (20)
Advice?.......He don need no stinkin advice........
Unfortunately, as Robert Brown, Nic Lewis and Steve McIntyre regularly show, most climate scientists can't do statistics. And these are people who should have done at least undergraduate maths. If this is representative of the academic population, what hope is there of getting someone who was probably only an Arts graduate at most understand significance, or even just probability?
"What complete and utter scientific rubbish and exceedingly poor science journalism. A War on Science indeed, except it is from the climate alarmists"
If that article can be described as "taking a pop" I look forward to a frontal assault.
Great article.
Ah, the sarcasm there, Mr Hill, is literally dripping down my monitor like water overflowing an EA managed river in Somerset.
Clearly the geezer had a stance that needed propping up and he wasn't about to let Real Life (tm) get in the way.
Mailman
autocorrelation?
Perhaps he did consult a statistical expert, but the expert just so happen to work in dendrochronology. :)
Borenstein is not practicing journalism when he writes this sort of tripe.
He is evangelizing his religious beliefs.
Facts, figures, ethics, truth are all behind his fanatical imperative to justify his faith and get others to believe in his climate obsession.
Moving on to second year statistics we can make the following points:
- Sierra Rayne's criticism is weak because in fact the probability that Borenstein would have got the results he did by chance is there were no warming trends in 29 states is way below 0.05. If there was no real trend in the 30 states which had no statistically significant warming then the chance of measuring a positive trend by chance would be 0.5 in each such state. The chance of measuring 29 out of 30 as positive would be 30 x 2^-30 which is about 1 in 30 million.
- With enough data a low p value is often beside the point. A low p value will let you establish that the best least squares fit with gaussian noise does not have gradient exactly zero to an infinite number of decimal places. But we usually already know that. The more interesting questions then become what is the magnitude of the slope? and what is the magnitude of the slope in relation to other sources of variation.
- My conclusion is that, assuming the data is not too systematically biased or noisy, there is a widespread warming trend, but in many places the magnitude is small in relation to other sources of variation (or else more measurements would be individually statistically significant). That indicates that for the moment, regardless of statistical significance, the trends are not practically significant. But I think there are some grounds for saying there is a trend there.
you can't calculate the actual significance (as opposed to the nominal significance which simple OLS packages produce) unless you know the sampling distribution of he statistic being considered. #####the simple packages assume that that distribution is normal, but we know this is wrong. This was the starting point of the famous VS thread over at Bart's .
Borenstein is a regurgitator.
We know that.
It seems that there is a general lack of mathematical ability amongst the alamists
Remember the Bob Ward/Phil Jones communication (as mentioned on WUWT, Climate Audit, etc.) where Jones admitted
"I’m not adept enough (totally inept) with excel to do this now as no-one who knows how to is here."
And I seem to recall (in one of the climategate emails) that he also couldn't work out his own age
Jones made a lazy attempt to work out his publication score and admitted it was wrong. Michael Mann decided to use an even more inaccurate and known wrong result for Jones' nomination to a learned society because it was a bigger number which supported his argument better.
This little cameo from the Climategate files tells you much about the people, their morals and abilities and the methods involved in the CAGW scam.
If this is time-series date, linear regression will probably give spurious results anyway.
- Sierra Rayne's criticism is weak because in fact the probability that Borenstein would have got the results he did by chance is there were no warming trends in 29 states is way below 0.05. If there was no real trend in the 30 states which had no statistically significant warming then the chance of measuring a positive trend by chance would be 0.5 in each such state. The chance of measuring 29 out of 30 as positive would be 30 x 2^-30 which is about 1 in 30 million.
My statistics show that there are cooling trends in 29 states. By your logic the chance of this being incorrect is 30 x 2^30. So no matter what statistical trend that we find in 29 states, no matter what the sample size, accuracy or method, there is a 30 x 2^30 chance that the trend observed will be correct? That is truly amazing!
Redbone writes:
I'm not sure I follow you, but if 29 of 30 states had shown a statistically insignificant negative trend then we could indeed conclude that the chance of seeing such an extreme result if the chance of each individual state warming is 0.5 would be 30 x 2^-30. The difference is that we would then conclude there may be a slight cooling trend rather than a slight warming trend.
If, by contrast 15 of the 30 states showed a statistically insignificant warming trend and 15 showed a statistically significant cooling trend then we could not reject the null. That is, we could conclude that if the results were random warming / cooling with 50% chance we might easily have seen the measured data by chance.
If my math holds up then seeing 20 or more states out of 30 going in one direction would reject the null hypothesis that each state has equal chance of warming / cooling at p=0.05. The more rigorous standard of p=0.001 suggested by Sierra Rayne would require seeing at least 24 out of 30 states going in one direction.
But as I said in my first comment, getting hung up on these p values is missing important points. The way that p values matter and the extent to which they matter really depends on an overall understanding of the data you are looking at and it's analysis. Just grinding out p values and declaring victory / defeat doesn't move us closer to understanding.
I wrote:
Apologies. That should be:
If, by contrast 15 of the 30 states showed a statistically insignificant warming trend and 15 showed a statistically INSIGNIFICANT cooling trend then we could not reject the null.
The ritual of fitting least squares regression to climate data should be challenged far more, preferably by statistics professors, although they seem curiously silent. There is nothing gained if the line gives a poor fit to the data. P-values offer one measure of this, although this requires a (null) distribution for the data. Easier, just look at a plot of residuals = data - (fitted line). Often this shows very obvious periods of poor fit. In that case, the regression line is of no immediate worth. Obvious example: regression lines fitted to global average temperatures for the past 35 years. These typically show a clear pattern of negative residuals for the past 10-15 years, so what is their point?
For the data being considered, why not just compare the average of the last five years with the average of the first five years?
statistics is the engineering discipline of "condensing facts"
it uses a toolbox for that, like all engineering.
Ironically its product output mostly results in more froth than you would think to be possible, invariably done with the worst possible chosen tool..(can you build a cathedral with a plastic spoon? well yes, theoretically you can)
JK - are you using the familiar statistical nonsense of assuming that I either will or will not be struck by lighting tomorrow - therefore there is a 50% chance that I will?
FACT: There is nothing gained , for the statistics professors, in fact, to be loud on this. Quite to the contrary in the present "climate" set by sceantists in the BBC and MET and the like.
If we had an INDEPENDENT (as in, free from career politicians' meddling) funding institute querying for cold facts and insights, with the hand on the purse, we would see statistics professors stampede through the door, by the dozen.