Buy

Books
Click images for more details

Support

 

Twitter
Recent comments
Recent posts
Currently discussing
Links

A few sites I've stumbled across recently....

Powered by Squarespace
« Yours truly in the Telegraph | Main | Was Briffa the mole? »
Sunday
Dec132009

Blast and counterblast

Giorgio Gilestro takes issue with Willis Eschenbach's attack on the credibility of the adjustments at Darwin. One should, he says, look at the big picture, and proceeds to an analysis of the adjustments showing that they are normally distributed and average to zero (i.e. there are as many upward adjustments as downwards ones).

Hooray! says RealClimate author Eric Steig in the comments:

This is a very nice analysis, and is really the last word on this entire fabricated scandal.

The last word? Not so, says Climate Audit regular, retired statistician Roman M, who produces this rather amazing graph of how the adjustments pan out over time.

 

 

PrintView Printer Friendly Version

Reader Comments (35)

I see "SOD" who posts around was first in the cheerleader's list in comments. Care to change your mind SOD, GG did a nice bit of work, but unfortunately his analysis, by excluding any time dimension, is worthless in deciding if there was any affect on the trend in the adjustments.

Dec 13, 2009 at 9:28 AM | Unregistered CommenterEd Snack

A further question on this. Why is the Giorgio Gilestro bell curve centred on zero? I thought these adjustments were meant to be to take account of particular local circumstances at a weather station. It doesn't strike me as self-evident that those adjustments should be randomly distributed in such a way that the average to zero over this long period. To take just one example, the urban heat island effect should presumably resulted in a predominance of negative adjustments, so will have to have been balanced by a precisely equivalent amount of negative adjustment for all other purposes - odd that everything evens out. Maybe there's a good statistical reason for this. Can anyone explain it?

Dec 13, 2009 at 10:43 AM | Unregistered CommenterNigel

V for Victory!

Useless internet analogy time. Stealing money then paying it back at a later date doesn't absolve you of your initial crime. It is common sense to consider that an average adjustment of bugger all can still produce an upward trend - adjust the first half of the data downwards and the second half upwards by the same amount.

On Eureferendum Manchester Airport's temp. record came up for discussion. I reckon the adjustments remove the UHI trend but not the effect - the adjustments get smaller as you move forward in time. Yet time is a crucial metric in the discussion of climate change. I am not a statistician, are any of these 'climate' 'scientists'?

Dec 13, 2009 at 10:57 AM | Unregistered CommenterGareth

"V for Victory!"

Or V for Vendetta! Blow up the New Bailey. (BBC)

Dec 13, 2009 at 12:56 PM | Unregistered CommenterJohn Silver

Nigel, great observation (which I made as well on the Statpad blog: http://statpad.wordpress.com/2009/12/12/). Further, why would adjustments form an almost perfect, unskewed, although leptokurtic, distribution? As you say, if nothing else, adjustments for UHI should skew the distribution negatively. GG's distribution suggests something "unnatural" about the adjustments, which, of course, Roman M makes brilliantly clear.

Dec 13, 2009 at 2:43 PM | Unregistered CommenterJPM

I thought there had to be something wrong with that GG analysis. It appears he thinks "trend" means "change in average for the period", not "change in annual averages over time".

I think the scariest thing is why the number of met stations should plummet just as temperature data (so they say) suddenly becomes vital for the future of the world. It's almost as if they don't actually care about the real temperatures out there...

Dec 13, 2009 at 3:01 PM | Unregistered CommenterPhil A

Might the plummeting of the Met stations be related to a switch-over to satellite data, suitably homogenised and adjusted a la Darwin?

Dec 13, 2009 at 3:11 PM | Unregistered CommenterViv Evans

I think the decline in stations is a result of the collapse of the Soviet Union. If I recall correctly, a large number of Siberian stations were closed when the USSR fell apart.

Of course, the alarmists have assured me over and over that the loss of all these colder-than-average stations was properly "compensated" for in the data.

Dec 13, 2009 at 3:45 PM | Unregistered CommenterPPP

It wasn't just in Siberia, there was a great die off of stations in the US used by GHCN and has been documented by Chiefio on his site. Right now GHCN uses something like 136 stations in the US down from a high of 6,000.

Dec 13, 2009 at 5:32 PM | Unregistered Commenterboballab

Why don't we just dispense with the BS and throw out the intstrumental records altogether and replace them with data provided by a consensus of IPCC climate scientists? Let's just admit that there is an over-arching global temperature signal that can be statistically derived at every point on earth. And this signal is a far better metric than what any mercury thermomter.

Dec 13, 2009 at 6:04 PM | Unregistered CommenterJP

One should actually expect a steep downwards slope for adjustments as one approaches now from UHI.

Your adjustment is inherently trying to remove the effect of cities - which are generally insignificant slices of their gridcell, and thus bad representatives of the average gridcell temperature. So the adjustment should be a subtraction.

And yet, we see a steep upwards adjustment instead. Well done.

Dec 13, 2009 at 6:41 PM | Unregistered CommenterAl

Doesn't matter. It's been answered. Time to move on.

Dec 13, 2009 at 7:54 PM | Unregistered Commenteranon

This is all based on the GHCN version 2 that just came out. As I understand it:

1. The number of stations has fallen dramatically.
2. The sites that have been eliminated have been, by a significant degree, tilted toward the "cold" stations.

Shouldn't the adjustments be biased toward the negative side to make up for this?

PS I don't believe the UHI effect matters here, because (again, as I understand it) the GHCN doesn't include any adjustments for that.

Dec 13, 2009 at 8:37 PM | Unregistered CommenterJ Thomason

I looked at GG's code - like written by a ten year old. I hope he doesn't do this kind of thing for a living.

Dec 13, 2009 at 10:25 PM | Unregistered CommenterMike

Ed Snack, Phil A etc. You've got this totally wrong. Giorgio has patienly explained it over and over. I have reproduced his calculation using R.

What he does is compute trends. This of course incorporates time. When looking at Darwin, the figure Willis calculated was that the adjustment changed a -0.7 C/century trend to a +1.2 C/century trend. A change of +1.9 C/century, which is indeed large.

What GG did was to do just this same calc for the other 6753 stations in the same dataset. The resulting set of trend changes is his histogram. And it shows that the 1.9 figure is atypical. It's in the top 10%. The others are spread more or less evenly on either side of zero.The mean trend change due to adjustment is 0.175 C/century.

That's still noticeable. But it doesn't sustain Willis' underlined charge, which was:
Those, dear friends, are the clumsy fingerprints of someone messing with the data Egyptian style … they are indisputable evidence that the “homogenized” data has been changed to fit someone’s preconceptions about whether the earth is warming.

Dec 14, 2009 at 1:13 AM | Unregistered CommenterNick Stokes

Nigel #2
The reason the distribution centres on zero is simple. A large number of stations had no adjustment at all.

Dec 14, 2009 at 1:16 AM | Unregistered CommenterNick Stokes

Giorgio's analysis was a pretty good indication that issues like Darwin station may not be causing major problems overall. However, the USHCN (USA) adjustments are much greater than the GHCN (global) ones. Here is the graph above overlaid with the adjustments found for the USHCN.

Dec 14, 2009 at 1:37 AM | Unregistered CommenterNikFromNYC

I was wondering if Eric Stieg had ever considered the possibility that some people might think that his personal/scientific credibility may be affected by his eager support of anything that appears to support his own position

Dec 14, 2009 at 2:32 AM | Unregistered CommenterTony Hansen

Nick,

I'm glad you think 1.9 deg/century is large.

Giorgio didn't seem to blink an eye at it:

(0 is the median adjustment and 0.017 C/decade is the average adjustment – the planet warming trend in the last century has been of about 0.2 C/decade)

Dec 14, 2009 at 3:28 AM | Unregistered CommenterJohn M

Phil A:

I think the scariest thing is why the number of met stations should plummet just as temperature data (so they say) suddenly becomes vital for the future of the world. It's almost as if they don't actually care about the real temperatures out there...

A late friend who was a meteorologist who'd done a lot of modeling of the atmosphere told me about this reduction back in the late 1990s, saying, "WTF? Why would anyone want less data points as we go forward in time?" He thought it was insane.


Viv Evans:

Might the plummeting of the Met stations be related to a switch-over to satellite data, suitably homogenised and adjusted a la Darwin?

Didn't John Christy's work on the satellite data actually show much less warming than the met stations?

Dec 14, 2009 at 5:19 AM | Unregistered CommenterSteveGinIL

John M
No, I don't think GG mentioned Darwin there. What people don't seem to be noticing is the difference in scale here. The average, 0.017 C/dec, is about a tenth of the Darwin figure, and it isn't large. Certainly not something you could base Willis' skullduggery claims on.

The figure at the head of this post seems to impress people, but the numbers are small there too. The gradient of that seemingly impressive uptrend is about 0.023 C/decade, very comparable with GG's average.

Dec 14, 2009 at 8:58 AM | Unregistered CommenterNick Stokes

The shape is surprising though isn't it?

Dec 14, 2009 at 9:48 AM | Registered CommenterBishop Hill

GG is a wally. It is not the distribution corrects that matter, it is their serial correlation; i.e.: negative early and positive later.

As for Steig! The more I read aout these climate scientists is that they don't seem to have a very good grasp of basic scientific technique.

Dec 14, 2009 at 10:29 AM | Unregistered Commenterrcsz

"The shape is surprising though isn't it?"
No, not really. Remember, it is not a plot of the size of each adjustment, but of the cumulative effect of adjustments. If you thought of GG's 0.0175C/decade average trend as being caused by a more or less uniform likelihood of adjustments happening in any one year, then a downtrend in this plot of 0.0175 C/dec is what you'd expect, and close to what we see. The deviation comes back in about 1900, when the probability seems to shift to having aadjustments of a nett cooling effect. However, the number of stations also diminishes there.

The spread of adjustments is much greater than the relatively small difference of the mean from zero. At a guess, I'd say that going back through the C20, about 45% of adjustments are up, and 55% down. In C19, that switches to about 60%up, 40% down (these numbers just for illustration).

Dec 14, 2009 at 11:05 AM | Unregistered CommenterNick Stokes

The time-dependent adjustment is exactly the problem. Most people agree that the Earth has been warming; longer term since the last ice-age and shorter term since the end of the "Little Ice Age" in the 19 century. Adding the V-shaped adjustment to a steady upward trend will flatten the earlier part and steepen the later part. The result is the "signature" of CO2 emissions-- proof of AGW! The fact that the average trend of the adjustments equals zero is irrelevant.

Dec 15, 2009 at 1:55 AM | Unregistered Commenterrwnj

The average trend isn't zero. It is 0.0175 C/decade, as GG said. The point is that all those nett figures are small, including those in Roman's graph. And there actually is a reason for them.

Dec 15, 2009 at 5:10 AM | Unregistered CommenterNick Stokes

I have two problems with Giorgio Gilestro's/Nick Stoke's bell curve:

First, how come the adjustments have a mean effect of zero? I'd have thought, given the well-known urban heat island effect, the net effect would be negative. Maybe there's something I've misunderstood there. Or maybe there's some other factor I'm unaware of that's as big as the UHI, so it cancels it out. Is there?

Second, what if fraudulent adjustments have been made, not to increase the total warming, but to create a false impression of accelerating warming? Suppose, for instance, the adjustments were designed to turn a straight line into an upwards-turning parabola, with the same end-points? Would I be right in thinking that your analyses would be unable to detect this, given that there would be no effect on the overall trend (measured as a simple difference between temperature at start and temperature at end)?

Dec 16, 2009 at 1:26 AM | Unregistered CommenterBruce

Bruce,
First on point 1, the average effect was not zero but 0.0175 C/decade warming. The GHCN homogeneity is a break detection algorithm - it tries to detect discrete changes, such as equipmeny change. I don't think it claims to adjust for UHI.
I might add that it's unclear what is done with this adjusted file. It isn't used by GISS or Hadcrut. It might be used for NOAA graphs. Maybe people apply further adjustment for UHI.
On your second point, Romanm's graph is actually fairly straight during C20, with a downturn at the end. You should ignore the anomalous endpoints - they are based on very few stations, so they won't affect any aggregates. The total rise is small anyway, but the only acceleration that could possibly show is around 1900. I doubt that advances anyone's argument.

.

Dec 16, 2009 at 3:40 AM | Unregistered CommenterNick Stokes

Bruce

Yes, the postings in question are only about the homogeneity adjustment. UHI and TOBS are separate. TOBS is the big one as far as I know.

Nick

I may be missing something. Roman is showing the average adjustment in any given year I think. If average adjustments in C19 were gradually becoming more and more negative this implies more and more shifts to warmer sites. Then with the arrival of the new century, this trend is reversed and they start moving stations to cooler sites. Am I getting this correct? What happened in 1900?

I can't help but be reminded of the Hockey Stick itself where the behaviour of the graph suddenly changed in 1902 (neatly coinciding with the start of the calibration period).

Dec 16, 2009 at 7:15 AM | Registered CommenterBishop Hill

The main thing that happened around 1900 was the introduction of Stevenson screens.

Dec 16, 2009 at 8:17 PM | Unregistered CommenterNick Stokes

OK, that's plausible. What was causing the shifts to warmer sites before 1900? And isn't the linearity still very surprising?

Dec 16, 2009 at 9:10 PM | Registered CommenterBishop Hill

The graph represents the cumulative effect of adjustments. The linearity says that during C20, there was a constant bias, small relative to the spread, towards negative adjustments.Going back before about 1900, that bias changed to positive.
Incidentally, Wiki tells me that the engineer who designed Stevenson screens was the father of Robert Loius.

Dec 17, 2009 at 7:00 AM | Unregistered CommenterNick Stokes

My reading of Roman's post was that this was the average adjustment for any given year. Where are you seeing that this is the cumulative effect?

Dec 17, 2009 at 10:13 AM | Registered CommenterBishop Hill

The figure for 1950, say, is the average difference between 1950 adjusted values and unadjusted. But for each station that difference is the sum of adjustments that happened between 1950 and 2009. For the population, you can consider it as the cumulative sum of adjustments.

A starting point model for the population of stations is a sort of random walk (back in time), where in any year there is a finite probability of an adjustment up or down, with a slight bias toward down. As you pass 1900 (going back) that bias shifts to up.

Dec 17, 2009 at 10:54 AM | Unregistered CommenterNick Stokes

I took my first loans when I was not very old and it supported my relatives a lot. However, I need the commercial loan once more time.

Apr 2, 2010 at 2:53 AM | Unregistered CommenterLEANNPacheco

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>