Bishop Hill Protomodels
Feb 1, 2012 Climate: Models To the layman, the word "model" implies a scaled down version of something, correct in all its salient details. We all know the kind of thing.
A climate model is not like that. As everyone knows, climate models are not validated out of sample and we don't know if any of their salient details are correct. Some features of the real world are reproduced on a "hindcast" basis, but the ability of models to make meaningful temperature predictions is more in the realms of hope than established fact.
I therefore wonder if the IPCC should adopt the word "protomodel" to distinguish unvalidated "models" from all-singing all-dancing all-validated models. This would be a snappy way to keep the uncertainty in the public's mind - something that needs to be done to allow policymakers to make the correct decisions.
Reader Comments (64)
Ensuring the public is aware of its models uncertainty is the last thing on their minds.
This is about control, and just luge a religion, everything the IPCC does it does to further that control.
Mailman
One problem is that the public may see computer models as being as realistic as the virtual worlds seen in their computer games.
What we have to do is keep pointing out to the public that the climate models are the same as those used to predict the weather and which are used to give us the weather forecasts. All the forecasts I have seen this week have said that they do not know what the weather will be doing by the weekend (ie beyond about 5 days) and tell us to keep tuning in for more updates. These very same models cannot predict the climate more than 5 weeks, 5 months or 5 years out.
With a chaotic non-linear system, too many uncertainties, assumptions and unknowns, it is just crystal-ball gazing. No engineer would design and build a bridge using a computer model with as little (ie none) validity as a climate model.
Hi BH
Please can you explain exactly what you mean by "not validated out of sample". I think that's a bit simplistic and risks giving the impression that there is no evaluation of the models' performance, which is not the case - there is a huge amount of evaluation!
I think it would help people comment from a more informed perspective if you could expand on what you mean and what your specific point is.
Cheers,
Richard
Richard
I've added the word "temperature" to "predictions". I think we agree that there are no temperature predictions out of sample, i.e. with data that wasn't there to be snooped when the model was put together. Obviously this is the salient detail as regards policy, as we need to get a handle on the "unknown unknowns" problem.
When I was writing the post, I wondered about the other aspects of climate that emerge from models and wondered about the validation of these. For example, I know that ENSO has emerged from some climate models (all climate models?) But can we make predictions of the intensity of ENSO intensity from a climate model? I'm guessing the answer is no.
Richard Betts,
It would help people to provide more informed comment if the evaluation of the model(s) were known. Where could we find the results of this extensive evaluation?
I think "model" IS the right word, after all they are going for an Airfix, they tend to Revell in their results, they Monogram their initials all over the science (MBH98) and if your skeptical they try to Tamiya.
Sorry, "you're" not "your".
Dr Betts,
You might recall the long spiel I wrote about circularity of model vetting and prediction activity being carried out within the same timeframe, which becomes the basis for IPCC's attribution argument. The Bish's point is obviously related.
I thought about this for a while. Emergence of real world phenomena inside models is certainly encouraging. But it cannot be a basis (or the basis) for acceptance of quantitative output from the same models, because they are two different things.
I find it interesting that your examples of Spitfire aircraft exactly show that models are not representative of the real world. Why to I state that? Let us analyse it.
An armed aircraft with a four bladed airscrew and pointed rudder means that it must be either a Mk VIII, a Mk XI or a Mk XVI. Mk XIs and Mk XVIs (which are virtually the same aircraft except for the manufacturer of the engine) never had a cut down rear fuselage. Your models must therefore be Mk VIIIs. But, only one Mk VIII ever had a cut down fuselage and it was only experimental. So out of the reputed 17,671 Merlin engined Spitfires only 1 (one) of your model was ever made.
It seems to be really representative of reality :-)
According to the Shorter Oxford English Dictionary (2007), “proto-” has the following definition.
Hence I think that “protomodel” is not accurate.
What about using “simulator”? The Earth Simulator shows that such usage is already established for this.
I think it would help people comment from a more informed perspective if you could expand on what you mean and what your specific point is.
Richard Betts
Help me out Richard.
As an electrical engineer of over 30 years experience I have used many computer generated designs using models (starting with slide rules!). Every input is a know and proven rule/theory. There are so many unknowns in climate theory so, to me, none of the models can be trusted.
I in no way mean to be insulting to you but where do you get your blind faith and trust in the models, especially when people like Mann use spurious statistical inputs like RE?
I ask in all good faith.
Mann's attempts, at such, are 99% glue =)
Richard of NZ - what's a "cut down fuselage" please? Presumably not the same thing as having a bubble canopy.
I think Mr Bratby makes an important point here.
Prior to taking a more detailed interest in all of this, I naively assumed that "climate models" were some form of clever computer simulation of the relevant aspects of our world. I was astonished when I discovered that, as far as I can tell, they seem to be merely (and I use that word cautiously) statistical models that try to attribute cause to correlation.
I think a lot of people just hear models and, as you say, think of Airfix Spitfires.
I vote for simulator.
There are other ways that you can test a climate model out-of-sample (I hesitate to use the word 'validate').
There are new observation streams coming online all of the time, and some that are needed. In a recent paper, my colleagues and I showed that a deep ocean observation network might be useful to track energy on a decadal timescale. There are observations that have been hitherto ignored when building the models, and turn out to be useful. There is also a lot of effort put into model-data comparison in paleoclimate. The uncertainties are bigger there, of course, and the configuration of the system means that the data won't always be easy to use. But people are trying.
Throg, when people say 'climate models', they generally mean physical process models. You might still need statistical models *on top* of the physical process models, in order to help interpret them.
What's wrong with the term "unvalidated model"?
The Met Office's web site (last time I looked) explained that their climate models had been validated by their ability to reproduce the past climate record.
If a model could not even reproduce the data used to construct it, it would obviously have fallen even before the first hurdle. The ability of a computer model to reproduce the data used to construct it does not confirm the correctness of the physical models used to construct it, nor the validity of the numerous assumptions necessarily made to account for incompletely understood factors.
It seems a common failing in "climate science" for researchers to confuse their unvalidated computer models with reality and even to talk of "conducting experiments" when what they are actually doing is run some computer model with varying input parameters. I think Professor Michael Kelly (Muir Russell enquiry) took exception to this.
Modelling, eh? I recall the mindbogglingly complex equations of metal casting technology, and then the surge in capability that computerised FEM - finite element modelling - brought to the design of injection moulding tooling. Provided that the relevant laws of physics are included in the programming, modelling of fluid flows makes a very useful approximation to the real world. The next step - validation - is essential. In the case of manufacturing, a model which resulted in an expensive but useless moulding tool (one which was so far from viable that it couldn't be 'tuned', and must thus be scrapped) would be discarded.
I see a need for Quality Assurance in climatology. By comparing models with real data, in forecasting and hindcasting, a "Useful Range" rating could be assigned. Model A might be accurate within a certain tolerance for, say, 30 days. A more successful Model B might achieve 60 days. And if no model can hindcast a decade, it must be considered a QA failure. Century-scale certification? No chance.
The climatology industry might just be able to salvage some public credibility by limiting its pronouncements to those which can be verified. There would be a jobs cull. The headcount of the remaining reputable rump would, of course, be tiny compared to today's vast gravy train.
What are the chances of climatology accepting QA? With families to feed, this army of jobsworth parasites is as likely to submit to QA as turkeys voting for Christmas or medieval churchmen for an Enlightenment.
The climate models are fantasy physics: 4 major scientific mistakes justified because they were peer reviewed so are sacrosanct to the new religion.
They predict 3-5 times real warming assuming it's mostly GHG-AGW and are corrected by using double real cloud optical depth and a variable net AIE which has no experimental proof [the algorithms used to process satellite data are wrong].
As for the other errors, they are elementary and should never have been made by a professional scientist. No-one who looks at the theory can accept 'back radiation'. The IR physics, the assumption of 100% direct thermalisation, can be rejected just by looking at the advanced text books and Will Happer warned of this in 1993. As for Hansen's claim of 33 k present GHG warming, he apparently concatenated real GHG warming and lapse rate to get maximum traction from his claims of apocalypse.
Now the precedent has been set with Fred Goodwin, will the politicians who got on this bandwagon turn on scientists who unwisely ran with this ball and were given honours?
That's interesting and prompted a Google search that threw up a load of new reading material - for me at least. Thanks. :-)
For a computer model to be fully validated, each of the component models has to be fully validated. For example, where is the validation of the cloud model that has a positive feedback?
That is a mk XVI, (16) and after April 1945 they had the cut-down rear fuslage and bubble canopy. Glad at last to have something to comment on that I actually know about.
Something I've always wanted to know about climate models, but was afraid to ask: Do they keep all runs? Or do they chuck away any result which is 'obvioulsy wrong'?
"I think we agree that there are no temperature predictions out of sample"
Ha. Yet another fantasy ginned up so that you can convince yourself that you can just dismiss the science. And as Richard points out, it is based on nothing but your own feelings. But back in the real world there are plenty of examples where a) GCMs made predictions about future temperature changes that were validated, and b) made temperature hindcasts that initially did not match the observations (showing clearly that there was no tuning to fit), and where the observations were subsequently found to have errors, which when corrected, showed the models were right all along.
Examples of a): Hansen et al 1992 prediction of cooling following Pinatubo, Hansen et al, 1988 scenarios, Hargreaves (2010)
Examples of b): Model hindcasts of LGM temperatures showing that CLIMAP SSTs were almost certainly wrong (Rind and Peteet, 1985), hindcasts of MSU2 showing that the UAH record showing 'cooling' was wrong,
Now you can make up some reasons why none of these are 'real' predictions, or tie yourself in pretzel knots to explain how this isn't what you meant, and I'm sure that will convince the chorus.
But, no, we can't all agree.
Doug McNeall: "I vote for simulator."
Sorry, can't agree. The word "simulator" suggests a fairly close and obvious similarity between the model and the real thing.
Doug
"At an early stage of development" seems about right to me. That I think is what the public and policymakers need to understand about the models.
In response to the comments from people who imply that the models are validated by 'hind-casting', that's nothing of the sort. What they do is to use fudge factors claimed to be derived from the aerosol optical physics of clouds, But that physics is fundamentally wrong AND they falsify LWP etc. to double the optical depth of low level clouds to offset the imaginarily high heating.
This is why the 1988 A,B and C scenarios of Hansen et. al. are coming back to haunt GISS. The key question is when this started to be deliberate fraud. Was it 2004, when NASA substituted Twomey's physics to keep imaginarily high feedback in AR4? Was it 1997 when the direct link between CO2 and the end of ice ages was broken, and we had Mann's arguably fraudulent hockey stick in association with CRU?
Or was it earlier still, perhaps 1975 after the Endangered Atmosphere Conference?
Further thoughts on the requirements for a model to be validated (in addition to containing physically correct models and being able to reproduce data), it must be demonstrated that the model is convergent and invariant to timestep length and cell size, conserves mass, momentum and energy and is insensitive to small changes in initial conditions. Presumably Richard Betts can point us to the documentation where this is all demonstrated.
wishcaster: no-one doubts that the prediction of cooling after Pinatubo were correct - that part of the physics is sound. However, all to do with CO2-AGW is wrong from the ground up, also the aerosol optical physics of clouds, even the satellite processing algorithms.
Thus, the assumption that most recent warming has been from GHG-AGW is nonsense. Natural processes are much more important and there may have been a substantial AGW from Chinese aerosols reducing cloud albedo which stabilised about 2000 when the 'Asian Brown Cloud appeared.
The most competent guy is Ramanathan who is a good experimentalist, but his IR work was amateur. Even so, his use of area averaging IR sensors has limited his work. What's needed in climate science is a management team who can challenge the existing people; just not good enough and have too much to lose by admitting they were wrong.
models: elephant, trunk, wiggle.
MDGNN
Please give references to one or two textbooks that explain the notion of <100% thermalisation - I am eager to read up on it. (And costing less than a good night out for two, if possible)
Another reason why "model" remains a perfectly adequate name is that while a 1/72nd scale model of a Spitfire may look like the real thing and be dressed up in historical colours, there is no way that it will ever behave like the real thing. Same thing with fashion models.
Its a simple model how much heat does it take to melt 1 kilogram of ice and how many litres of water does it produce
Then times that by the amount of minus the amount caused by expansion ice both floating (taking into account the displacement in the ocean) or on land at the poles and that gives you the extra water then add that to the volume of the oceans minus the total outline length of the land continents and that gives you the rise in sea level
The oceans are so vast the sea rise will be less than a few millimeters
And thats only if it actually manages to get that hot
The maths genius brother of FBI agent Epps in Numbers and Sheldon Leonard Raj and Howard from Big Bang Theory can all properly work it out with a calculator
This is a judgement call of course, which we can get good data on. Crucially, *it depends what you are looking at, and your information needs*. I would agree that models are in an early stage of development for some uses, but are quite well developed for others. I'm sure we'll have useful discussions on which uses those are ;)
Throg, I would recommend "A Climate Modelling Primer" by McGuffie and Henderson-Sellers as a good place to start. It's clear, not too crazy-technical, and actually has some code that describes a rudimentary model.
Doug
Can you expand a bit on the areas you think models are in early/advanced stages of development.
I wondered about the other aspects of climate that emerge from models and wondered about the validation of these.
The price of energy in 100 years' time implied by climate models is of course wholly unverified and unverifiable.
O/T (@ Richard of NZ)
The Spitfires depicted are in fact these:
http://agapemodels.com/?p=185
Mark 16e, apparently.
artin A: a starting point is here: http://arxiv.org/abs/0707.1161v4
Using Fourier, Tyndall and Arrhenius as justification, a whole series of assumptions have been made in climate science which have no physics' basis. These have then gone by 'peer review' to secondary sources and in turn this false physics is apparently taught in some courses.
However, as far as I can tell, no physics textbook teaches the GHG 'back radiation' idea. You get the CO2 physics from advanced works on IR spectroscopy.
Tyndall measured his rise in temperature using apparatus with constant volume: as CO2 has a higher CTE than other gases, much of that temperature rise was from the rise in pressure, a false result. The same goes for the BBC PET bottle experiment. Loosen the cap and the temperature rise with CO2 is much lower and may all be from the interaction of scattered IR with the PET walls.
There is no physical mechanism for transfer of quantised vibrational energy from an asymmetrical to a symmetrical molecule. Will Happer warned of this problem in 1993 when he resigned as Director of Research for the US DoE: he refused to lie about this physics for Gore saying 'I don't need the money'!
Don't like "simulator" first thing comes to mind is flight simulators, which are apparently useful.
Regarding "climate computer model" I think a hyphen is missing, should be Mode - L, i.e. a computer in "Lie Mode"
Its ironic that spitfires were designed and built to protect our freedom and democracy in the 1930s and 40s
Yet 70 years later when WE are trying to exercise our right to free speech questioning Global warming and the motives of those behind it
Some unproven myth that is going to cause untold unnecessary hardship for everyone trying to prevent something that dosent actually need preventing
We get ridiculed and harassed and called deniers (as if we are no different from Nazis sympathizers like Nick Griffin and David Irvine)like those who try to deny the Jewish holocaust in WW2
I hope the smug environmentalist and self serving politicians celebrities and scientists
Who are basically just scrounging around for a bit of funding and a few extra votes and an Eco ego boost
I hope they are proud of them self's when they are wearing there poppies on remembrance day
God knows all our old relatives and current who have and are serving in our forces in both the wars and in Iraq Afghanistan Bosnia the Falklands etc etc have given more than enough so we can have free speech websites like this one
Bishop Hill Feb 1, 2012 at 10:56 AM
I disagree completely.
"At an early stage of development" implies, misleadingly, that the models already have some validity, they are continually improving and have potential for significant further development.
Did any climate models predict, in 2002, the current decade-long absence of global warming? If they did, it must have been a fluke because, as Trenberth pointed out:
Date: Mon, 12 Oct 2009 08:57:37
Hi all
Well I have my own article on where the heck is global warming? (blah blah)
The fact is that we can't account for the lack of warming at the moment and it is a travesty that we can't.
They don't have a clue.
It is a travesty that climate models are assigned any credibility whatever.
I think we should not use phrases that imply they already have some validity and that this will improve as work continues (and as Sligo demands) more powerful supercomputers are provided. They are rubbish. Expensive rubbish, but still rubbish.
Completely agree Martin A. They may as well have taken a ruler and drawn a straight line in the direction they hoped temperatures would go, for all the accuracy they've shown.
Martin A: I have a longer post in moderation.
Read this: http://arxiv.org/abs/0707.1161v4
Maybe the problem is the use of the word 'model' rather than an attached descriptive. Climate models (?) are used to make a series of "IF" scenario-projections, not single predictions.
A good example is what happens if a passenger plane loses one engine, two engine, three engines - can the pilot land safely in time. That is why we have flight simulators that allows pilots to go thru each and every likely scenario that may impact on flying, landing and taking off.
So instead of 'climate model', I would argue that 'climate simulation' or 'climate simulator' is a better description of what climate scientists are trying to achieve.
Feb 1, 2012 at 7:48 AM | Phillip Bratby
Indeed, thank you for making that point - and indeed your remarks that followed are fair enough, it has been a very difficult forecasting and risk communication situation this week. On Saturday I was trying to explain to my daughter that the forecast was only that it might snow on Sunday night, and not that it would. As it turned out, it snowed a few miles away but not where we live.
However, the fact that these same models do behave credibly and are validated out-of-sample on a daily basis is one piece of evidence that they are representing atmospheric processes reasonably well. So in the case of last weekend, the model was correct that it would be cold (around zero celcius) and wet where I live, but the critical issue of snow depended on whether it was above or below zero. A difficulty with precision around a critical threshold, but one which should not distract from the more general behaviour of the model being correct.
But the point is, for climate studies we are not aiming to get the day-to-day or even month-by-month or year-by-year changes correct date-for-date. As you correctly point out, this is simply impossible due to the chaotic nature of the system. However the evidence suggests that a reasonable representation of the long-term statistics is achievable (eg: the long-term mean and, to a lesser extent, the statistics of the variability).
You and others are also right that long-term climate simulations driven by scenarios of future GHG emissions / concentrations (amongst other factors) should not be regarded as predictions - more as scenarios or projections. At the risk of inviting disparaging comparisons between climate scientists and fortune tellers (please try to hold back!!!) I found this distinction between projections and predictions (and prophecies) quite interesting.
So the models give us a range of future possibilities consistent with current understanding of atmospheric and oceanic science (and biology and atmospheric chemistry to some extent) as represented in numerical versions of mathematical equations.
Feb 1, 2012 at 1:59 PM | Mac
Thanks, I agree.
@Mac 1.59 PM - but what would happen if the flight simulators in a lost engine scenario increased the power available to the trainee pilot?
MDGNN
artin A: a starting point is here: http://arxiv.org/abs/0707.1161v4
Thank you.
I was really hoping for a standard advanced text on (say) radiative transfer that explains the point that you have made many times, rather than one specifically debunking greenhouse theory.
I think this statement lies at the heart of the point you keep making about "thermalisation".
If I understand your point, it is that a photon gets absorbed by (say) a water molecule but its energy remains stuck there because "there is no physical mechanism for transfer of [its] quantised vibrational energy" to the oxygen or nitrogen molecules it meets up with. Have I understood correctly (at last)?
Martin A
Thank you Richard.
"Possibilities". Why is the Government betting the country's economic future on a vague possibility based on nothing more than "current understanding" (or lack of understanding).
Why is the Met Office (and advisers like Watson) not saying this to the Government in the strongest terms?
Why are politicians being misled about dangerous climate change?
Justice4Rinka
Mark 16e, apparently
Certainly appears to be of the Mk XVI family -- it was designed specifically for low-altitude high-speed flying to be used to shoot down V1 buzz bombs. It had clipped wing tips as the models clearly show.
Richard Betts
I guess I am a little surprised at your apparent lack of knowledge of what a model is all about. The only true test of a model is prediction, although it is not sufficient. For example, Phillip Bratby @ Feb 1, 2012 at 11:22 AM lists a number of other necessary criteria. Merely modeling existent data sets with a "model" is meaningless beyond being the FIRST requirement, which is "does it match the known data?" Beyond that all the other requirements need to be matched. And the more specific the better.
For example, I can match just about any time-series data you might have with a polynomial regression ( i.e. y= ax +b(x*x)+c(x*x*x) ....
And given the clearly cyclic nature of temperature data over many years, such an equation would probably have a very high correlation -- far higher than any of the "models" touted by the climate scientists.
What the climate scientists need is a model that makes a prediction that is testable and verifiable, and do so with high precision.
Perhaps the best example of this is Einstein's Theory of Relativity, in which he predicted a gravitational lens effect around the Sun. That prediction was extremely precise, and in 1922 the Lick Observatory proved beyond all question that the effect was observable and precisely what Einstein predicted.
I go outside and look for all those terrible things "predicted" by these "Models". Somehow, I fail to see them.
Feb 1, 2012 at 2:03 PM | Richard Betts
Your post is suggesting that because the regional forecasts for a couple of days are fairly accurate that this is validation for multi decadal global means?
Perhaps clarification is necessary.