Wednesday
Mar142012
by 
Bishop Hill
Bishop Hill Climate Hawkins
Mar 14, 2012 Climate: Models Ed Hawkins, a climatologist at Reading University, has written a short blog post reporting a comparison he has done of climate model output to data. He specifically addressing the question of the Arctic temperatures, which I mentioned in yesterday's post, masking the model output so as to exclude the Arctic and thus giving an apples-to-apples basis for the comparison.
The models used are the CMIP5 ensemble, which I think I'm right in saying is very recent - within the last year or two. I would be interested in seeing some AR4-era model runs and checking how these panned out against the same temperature data.
Reader Comments (45)
Yet another "climate model." Why don't they move on to something worthwhile like modeling How Many Angels Can Dance on the Head of a Pin? That was of great theological importance at one time and surely should appeal to the theological bend most of these Climate Scientists™ have.
I can see them arguing over the size of the pin, the size of an angel, whether there are forcings and of what nature. And if they choose to ignore natural law, nobody but they would care.
Given that the essence of the angel argument was (in modern terms) whether the answer was finite or infinite, the size of the pin is not a relevant issue.
Bish
Yes, CMIP5 is very recent, in fact the data from different modelling centres are still coming in to the archive (several terabytes) so this really is "hot off the press".
I think the AR4 models would look fairly similar for the last decade. Although this is not exactly what you are after, you can get a reasonable idea by looking at Figure 1 in Ed's BAMS paper.
Cheers
Richard
@ RB
"I think the AR4 models would look fairly similar"
Does that mean you don't know
Jonathan Jones
But what are the confidence limits, and can you use PCA to discover the essence of angels? Those are critical issues. Perhaps Richard Betts has some insight?
Mar 14, 2012 at 5:53 PM | Anoneumouse
It means I don't know off the top of my head - however I will have a look around to see if there is anything relevant in the literature (unless Ed fancies re-doing his analysis for the CMIP3 (AR4) models.... :-)
Mar 14, 2012 at 6:02 PM | Don Pablo de la Sierra
Don't know about angels, but being a Hitchhiker fan, I am far more likely to believe in the Great Green Arkleseizure, from whose nose the entire universe was sneezed...! :-)
Not to intrude but the claim that medieval theologian debated how many angels could dance on the point of a needle is nowadays regarded as a modern invention. ie. it is nonsense.
@RB
Literally that's a don’t know then? Scientifically it’s Betts hedging ;-)
yep pun intended
Hi Anoneumouse
No, it's a "this is what I expect from past experience"
Tell you what, if you prefer I could just assert that the AR4 models would look fairly similar, without bothering to check. I'm pretty confident I'd be right.
:-)
Very similar to Lucia's analyses. The next 5 years will be nail biting.
How are we to read the graphs? The grayed areas are from the models, but what inputs did the models have? Did the inputs include all the forcings up to 2010?
"I think the AR4 models would look fairly similar for the last decade"
Or more accurately
I think the suitably adjusted and modified AR4 models with hansen and mannian manipulation, would look fairly similar for the last decade
Can anyone explain where the '17 years' concept in falsifying GCMs came from?
I'm also wondering why someone doesn't test the GCMs by reversing the sign of time in their inputs and showing that they correctly predict the past given today's conditions? Presumably this must work as the GCMs are being used to determine future trends which affect public policy(?)
There is too much of a coincidence in the linguistic and social styles of the language. I believe Ed, Tamsin and Richard are special CAGW agents who've been trained in "reasonableness" (if there is such a word) to infiltrate the skeptical blogs with the goal of persuading the curmudgeons who haunt them that all climate scientists aren't like those we've seen in the Climategate emails.
I am disinclined to believe it's possible to model the climate, not only are there unknown unknowns, there a myriads of knowns which aren't understood. Just discussing models gives them a credence they don't deserve.
RE: geronimo | Mar 14, 2012 at 8:28 PM
Have you ever seen all three of us in the same place at the same time? ;-)
Ed.
RE: Mar 14, 2012 at 7:19 PM | Sara Chan
Hi Sara - yes, these are the "all forcing" simulations up to 2005, and the RCP4.5 emission scenario after 2005, as per the standard CMIP5 protocol.
Ed.
Mar 14, 2012 at 8:28 PM | geronimo
Uh oh, rumbled... :-)
But it's worse than you thought! The Greenpeace/WWF-sponsored modules in Basic, Intermediate and Advanced Reasonableness for Climate Modellers took place in month-long workshops at luxury holiday complexes in Bali, Hawaii and the Bahamas respectively, each at enormous cost to the taxpayer! :-)
Got to go, I need to delete all the emails about it :-)
Cheers
Richard
PS. Hope Doug McNeall isn't offended by you missing him out. Maybe he flunked the course? :-)
Re Geronimo
Models still have a lot of value. Programme them with what we think we know and if the results come out unexpectedly, then we know there's an unknown or an error. Biggest downside for the climate models is having to wait and see if they agree with reality or not. More funding is needed for either reality accelerators, or time machines.
From Ed's blog
"Below is (I hope) a consistent comparison for two observational datasets (HadCRUT3 and GISS)"
But aren't HadCRUT3 and GISS models too. Surely it is best to match GCM's to actual observations than altered observations. ie There is no need to artificially estimate a warming factor for Denver due to it being a mile high city as you can directly compare the thermometer there to the actual mile high area in the GCM itself.
What we see here is a comparison of anomalies, not actual values.
This is an important distinction. Climate models do a very poor job at simulating the actual values, though they behave a bit better in simulating the change in value. It is not the same, if your model is 2 degrees of the mark it will rain instead of snowing, albedo will be different, evapotranspiration another dimension, convective activity will be shut off or not, etc, etc. And the whole energy balance (which in essence is what those models are) will be wrong.
Besides, why don't wee see any regional validations? We now the regional bias is huge. This kind of exercise is like congratulating yourself on getting the weight of an elefant right by assembling parts from an hypopotamus.
Nice try Dr. Betts, but the superabundance of punctuation leads my textual analyser to suspect that you're trying to mislead us. The most damning evidence however is the presence of 'Greenpeace', 'WWF' and 'reasonableness' in the same sentence.
Off the mark
We know
...
It's hard to type on this little smart thingies.
Re: Mar 14, 2012 at 9:04 PM | Ed Hawkins
Ed, thank you!!
I'm not sure why these GCM things are called models at all. Models leads to a test-able predictions. All I see is gale force hand waving. E.g. Where does the 17 years come from? Well it could be 15-20 years, that was the length of time before the two random lines crossed last time, etc. Can you predict the past? Not really tried - sorry.
Not science - just silliness. (Sorry Ed, Richard, et al).
And please don't give us the official extreme gale force hand waving:
'Scenarios are not specific predictions or forecasts of future climate. Rather, scenarios are plausible alternative futures. Each scenario is an example of what can happen under particular assumptions on use of fossil fuel and other human activities. Scenarios assist in climate modeling, help to examine potential climate change and explore vulnerabilities of humans and ecosystems under a changed climate.'
It is annoying. Not science, just annoying.
@ZT
"Can anyone explain where the '17 years' concept in falsifying GCMs came from? "
Yes, it's Santer et al (2011). I always imagined that this paper was produced 'on demand' to rebut politicians "it hasn't warmed over the last 10yrs" statements.
They did various models runs and some had no warming over 10yrs; So zero warming over 10yrs is 'consistent' with models, or so they would argue. Santer claims you need at least 17yrs without warming before you can say the models are wrong.
I say "on demand" because the validity of this study is highly suspect - I don't have a lot of confidence that the models do a good job of modelling climate anyway, so extracting some 'detail' characteristic of what a model does (no warming period) and infering that the earth will also behave with the same 'detail' characteristic is kind of a long shot. The paper was produced so polticians could answer to the statement "there has been no warming for 10yrs" with "Yeah I know, that's what the models say can happen - whats the problem?".
ZT - that my take anyway, Pielke's comments are here,
http://pielkeclimatesci.wordpress.com/2011/09/07/comments-on-the-new-paper-separating-signal-and-noise-in-atmospheric-temperature-changes-the-importance-of-timescale-by-santer-et-al-2011/
@ZT
"Can anyone explain where the '17 years' concept in falsifying GCMs came from? "
Yes, it's Santer et al (2011). I always imagined that this paper was produced 'on demand' to rebut politicians "it hasn't warmed over the last 10yrs" statements.
They did various models runs and some had no warming over 10yrs; So zero warming over 10yrs is 'consistent' with models, or so they would argue. Santer claims you need at least 17yrs without warming before you can say the models are wrong.
I say "on demand" because the validity of this study is highly suspect - I don't have a lot of confidence that the models do a good job of modelling climate anyway, so extracting some 'detail' characteristic of what a model does (no warming period) and infering that the earth will also behave with the same 'detail' characteristic is kind of a long shot. The paper was produced so polticians could answer to the statement "there has been no warming for 10yrs" with "Yeah I know, that's what the models say can happen - whats the problem?".
ZT - that's my take anyway.
[ZT the original post has gone into moderation because of a link to Pielkes comments. the Santer paper was "Separating Signal And Noise In Atmospheric Temperature Changes: The Importance Of Timescale"]
So, Betts is making jokes about the involvement of the WWF in IPCC activities, as though it were a conspiratorial meme rather than reality?
A lot of people are in a fix Betts, Tasmin and Judith Curry find themselves in: having to reconcile the vastness of their community's enterprise, with the scant regard it is held in sceptics' eyes (which they must feel holds a grain of truth)
"Ed Hawkins says:
March 14, 2012 at 4:25 pm
Hi Bishop,
Yes, these are the latest model runs being prepared for assessment in the IPCC AR5, and not all have been released yet. And more years will aid the comparison of course."
How can such a comparison of a series of "hot off the press" model runs with HadCrut or GISS have any meaning at all? Let's have a comparison of HadCrut 3 with the models used to induce most developed countries to sign the Kyoto Protocol in 1997. That would be a real test to find out if models have any predictive value at all. If they fail we can wait to 2014 and Santer's 17 years will be up. Two years to a defining moment.
John PEter Mar 14, 2012 at 10:49 PM
"Let's have a comparison of HadCrut 3 with the models used to induce most developed countries to sign the Kyoto Protocol in 1997."
Good luck with that I do hope you get somewhere! I have been asking Richard Betts of the MO for the actual performance of the subsequent individual MO decadal forecasts (started in 1985) for quite sometime. Just keep getting pointed to hindcasts. Meaningless if you are trying to assess whether a forecasting ability is improving or not. I am begining to suspect that the earlier models showed far better ability.
Mar 14, 2012 at 10:49 PM | John PEter
A comparison of models from the IPCC's first and second (and third) assessment reports with HadCRUT3 is here.
Cheers
Richard
Mar 14, 2012 at 11:14 PM | Green Sand
Hi Green Sand,
I'm very confused here. Why do you think we started doing decadal forecasts in 1985? We didn't start them until 2005 - the first publication on this is here.
Are we talking at cross-purposes and referring to different things when we say "decadal forecast"? By this term, I mean the use of initialised forecasts in which the models assimilate observational data in order to begin the simulation at the correct state for the current moment in time (as distinct from starting the simulation way back at pre-industrial conditions and running forwards through the 19th and 20th Centuries before getting to the present-day, which is what we done before initialised forecasting was developed).
The paper by Smith et al that I linked to above was the first example of decadal forecasting as I define it above. This was the first time that there was any hope of beginning to capture internal variability year-to-year and not just the long-term trend.
If you could be more specific about the kind of forecasts you mean that started in 1985, and preferably post a link, that would be extremely helpful. Maybe you mean the kind of thing shown in the figure I linked to in my reply above to John Peter? If so, then that's not the initialised forecasting that I mean when I say "decadal forecasting". However, if not then it would be great if you could say what else you are referring to!
All the best,
Richard
"A comparison of models from the IPCC's first and second (and third) assessment reports with HadCRUT3 is here."
Up to 2005. Anything with up to 2011?
@GSW Thanks for the Santer reference. I had a quick look at this bafflegab-fest - ugh - what is wrong with these people?
@ZT
"what is wrong with these people?"
I'll hazard, "never fall in love with a theory" - Carl Sagan(?)
;)
Cumbrian Lad:
http://img252.imageshack.us/img252/1351/ar42011.jpg
I am influenced by Ed because he is the one that gave me the idea, and encouraged me, to go into climate modelling :)
As for Richard, it's already been documented that I'm his sidekick (Robin to his Batman?).
Though my favourite new comparison is to Servalan...
Here is a graphical representation of the latest climate model, now it has had the 'bitter winter = global warming" and Australia's overflowing-dams-never-ending-drought modifications added.
http://i148.photobucket.com/albums/s17/brock_blither/Global%20Warming/potato.jpg
Having finally met Richard Betts I can confirm that he is almost implausibly nice in person. I have never met Tamsin, but anyone who forgives me for comparing her to Servalan (entirely on the basis of her twitter picture I'm afraid) must have something going for her.
I think it would be good if for these initialised forecasts, the criterion for success in prediction was tougher. The yearly global temperature anomaly is just one number per year - it is too easy to get it right by accident, especially if the statistical criterion for skill is not too demanding (e.g. how well would quasi-random series with the right degree of persistence do when benchmarked against this sort of test?). For predictions over small numbers of years, as here, I think you'd also want to test things like regional or at least hemispheric anomalies, and perhaps other things like cloud cover, sea-surface temperatures, and so on.
Also, it does seem to me that it would be good if the testing of predictions was done in a more transparent way, in which the judges of success were more obviously different people than those doing the predictions. Predicting the three-dimensional structure of proteins based on knowing only their amino-acid sequence is a hard problem (though probably not as hard as climate prediction) which is approached - like climate - through various types of computer modelling. In that community, tests of model prediction have now been done in this way for some years: crystallography is used to determine three-dimensional structures of some new proteins, but these are not made public immediately. Instead, a competition is run to see who can predict the structures of these target systems most accurately. By removing any possibility of knowing the structures in advance, you remove all possibility that those doing the predictions can fool themselves into thinking they have done a prediction when in fact they - even unwittingly - had their thumb on the scale. These Critical Assessment of protein Structure Prediction (CASP) experiments really helped people work out how well prediction was working.
Thanks Jonathan. I did laugh out loud when I saw the photo of her on Wikipedia (sorry, never watched Blake) - very funny! Better inspired by the haircut than a feeling that I am "a cold, calculating, ruthless sociopath" ;)
Hi Richard, many thanks for coming back.
http://www.metoffice.gov.uk/research/climate/seasonal-to-decadal/long-range/decadal-fc
Figure 1:
“Previous predictions starting from June 1985, 1995 and 2005 are shown as white curves, with red shading representing their probable range, such that the observations are expected to lie within the shading 90% of the time.”
Though I seem to remember that you pointed out that these are in actual fact hindcasts and referred me to the Verification section at the bottom of the page? So they are not “Previous predictions” as stated? Possibly that could be made clearer?
Re MO forecasts, I have no insight into how they are defined within the MO so I will ask the question in different way. When did the MO start making long term forecasts, predictions, projections (you define the length of forecast) of global temperature? I cannot believe and I am sure that you are not saying that they only started in 2005?
Is there a record of how those forecasts in their original state performed against actual observed data?
@Tamsin
No to mention Mele-on Grayza. Another cold, calculating, ruthless sociopath with a short hair doing horrible things to people who don't deserve. That's why I am so totally against haircut. ;-)
Hi Green Sand
Aha, I see. Yes, you're right, that figure caption is indeed confusing. I'll mention that to the guys who look after that page.
On long-range forecasts in general - the exact date(s) will require some digging. I'll get back to you....
Best
Richard
Don't you think you are all taking GCMs far too seriously? I wish they deserved to be so taken, but I fear that is at best premature, and at worst impossible because of the nature of the system. This chap's view of them seems closer to the mark:
'To say that modeling the climate for long-term predictions is difficult given the current state of climate science is like saying that it would be difficult for your five-year-old son to build a 400 horsepower car from re-purposed Toys R’ Us purchases. Imagine that he comes to you with pages and pages of plans he’s sketched out in crayon. The “car” will cost $22,827.35 worth of toys.
Why wouldn’t you reach for your credit card? Is that because you’re against teaching kids engineering? Is it because his sworn enemy, your daughter, is paying you off? Or perhaps it’s because this project is obviously beyond the capability of a five-year-old, and that his crayon schematics don’t offer convincing evidence that he is in fact the kind of once-in-a-generation prodigy who could somehow pull it off.'
Source: http://www.masterresource.org/2012/03/what-the-skeptics-are-skeptical-of/
It seems to me that research into modelling geophysical systems is worthwhile. But I think a more productive focus might well be into sub-systems such as ocean circulations or the effect of mountain ranges on flows and so on. Trying to do the entire climate system looks to me to be absurd.
As an interested outsider, I do occasionally worry that I may be a bit like a passer-by calling out to Michelangelo and telling him he can't possibly paint on ceilings - the stuff won't stick, and so on. But so far, I have not seen any signs of the like of a Sistine Chapel coming out of GCMs (mind you, as an aside while on this arty tack, the Hockey Stick seems to have been produced like a work of art - hey,let's try funny PCA, and be sure to include that simply wonderful pine data, and let's add these temperatures on to the end, just look at those effects!)
How can any of the climate models predict temperatures when the cloud physics supposed to predict how the major global warming from CO2-AGW is exactly hidden by the cooling from polluted clouds is plain wrong, as can be seen from rain clouds getting dark underneath when droplets coarsen?
And yes, there is an explanation; CO2-(A)GW is much smaller than claimed and the real warming is the 2nd AIE! Are these people so dumb they can't look out of the window at reality, or will, if they do so, lose their careers?