Bishop Hill Protomodels
Feb 1, 2012 Climate: Models To the layman, the word "model" implies a scaled down version of something, correct in all its salient details. We all know the kind of thing.
A climate model is not like that. As everyone knows, climate models are not validated out of sample and we don't know if any of their salient details are correct. Some features of the real world are reproduced on a "hindcast" basis, but the ability of models to make meaningful temperature predictions is more in the realms of hope than established fact.
I therefore wonder if the IPCC should adopt the word "protomodel" to distinguish unvalidated "models" from all-singing all-dancing all-validated models. This would be a snappy way to keep the uncertainty in the public's mind - something that needs to be done to allow policymakers to make the correct decisions.
Reader Comments (64)
Richard - re: "However the evidence suggests that a reasonable representation of the long-term statistics is achievable (eg: the long-term mean and, to a lesser extent, the statistics of the variability)."
What is your "evidence" here? From blogs that have caught my eye I have the impression models are overestimating the long term temperature trend:
http://rankexploits.com/musings/2009/models-over-predict-using-another-version-of-a-long-trend/
http://judithcurry.com/2011/09/18/climate-models-as-ink-blots-2/
And the conclusions of Koutsoyiannis et al are that the models are useless for long term prediction of temperature and precipitation:
http://itia.ntua.gr/getfile/850/2/documents/2008EGU_ClimatePredictionPr_.pdf
Of course one has to remember that only one plane in 5 million crashes.
Perhaps the models are constructed for artistic purposes, much like the hockey-stick chart could have been created to convey a dramatic sense of temperatures shooting up towards the end of the 20th century. (Very effective it was too - I suspect that is the mental image that most folks with one on this now possess. Mind you, the original is no longer so fashionable in the gallery which first carried it to prominence. In fact it seems to have brrn completely removedfrom their latest catalogues).
By 'artistic' I mean intended to share with the viewer the artist's feelings and emotional state. Now those who feel that additional airborne CO2 must have a strong effect on climate, and who are also very upset by that, can select from the array of model outputs/scenarios those which will best share this with others. Since model runs could no doubt be contrived to span the range from snowball earth to year-round saunas in the mid-latitudes, this is a genuine artistic contribution to modern life, raising as it does a vivid sense of living in a nature of which we have control on a global scale. Films to be made, funds to be raised, grants to be won, honours to be shared around like Oscars. Not to mention, uncounted numbers of devoted admirers, keen to be taxed, to be levied, and to be scared witless.
Richard Betts, if you want to get people to understand models and their forecasts/ projections/ predictions may I respectfully request that you get somebody at the Met Office to validate previous and/or existing model output.
For this particular frustrated onlooker a good place to start would be:-
http://www.metoffice.gov.uk/research/climate/seasonal-to-decadal/decadal-prediction
Why is this chart not updated? Please get somebody to update it. Let’s see if the actual observed data meets with the yellow and blue forecast lines. If it does, what a powerful way to promote comprehension, if it doesn’t then the discussion moves to uncertainties with a view of the magnitude involved? At present I am left guessing and sadly rightly or wrongly I find I am guessing about motives rather than data.
Richard Betts Feb 1, 2012 at 2:03 PM
"However, the fact that these same models do behave credibly and are validated out-of-sample on a daily basis is one piece of evidence that they are representing atmospheric processes reasonably well. So in the case of last weekend, the model was correct that it would be cold..."
Richard with regard to last weekend you may find the following of interest:-
"East versus West: Let the battle commence!"
"the outlook remains a major headache for forecasters, as it has for some time. In fact I can't remember the last time there has been so much disagreement between the major forecasting models.
As I've explained before, each computer model runs many times, and each time the initial starting conditions of the atmosphere are changed by a very small amount, in order to see what happens.
An example of this can be seen below, from the American GFS model, from midnight.
It clearly shows that after a few days, when all the model runs are in broad agreement, each separate run of the model thereafter yields different results - with some solutions showing a 15 Celsius difference in temperature at 5000ft."
http://www.bbc.co.uk/blogs/paulhudson/manchester.jpg
Paul Hudson
http://www.bbc.co.uk/blogs/paulhudson/
@Mac - a lot more would crash if the pilots thought they were going to get more power when they lost an engine...
Martin A: the absorbed IR photon doesn't get stuck. What happens is that because at constant temperature the density of IR excited states has increased above that determined by the Law of Equipartition of Energy, another GHG molecule will emit, effectively simultaneously, the same energy photon in a random direction thus restoring the energy distribution,so there can be no temperature change of the gas mixture,
At room temperature, about 5% of CO2 molecules are in the excited state. The problem that climate scientists have, because they haven't been educated in physics or statistical thermodynamics, is they imagine you can follow an excited photon for a 1000 collusions, about a microsecond, and it drips energy to increase the kinetic energy of the gas mixture.
I've seen this argument many rimes, so it appears to be taught, but it's fundamentally wrong. In effect the duration of the extra IR excited state is zero so there can be no temperature rise. Read about the 'Gibbs Paradox', which is the mess you get into if you imagine you can label individual molecules [except at very low temperatures].
Climate science needs an infusion of top level physicists to re-write the textbooks and the courses to get this back into being science rather than pseudo-science.
@Richard Betts: Are there no roles in weather forecasting for senior forecasters these days?
It's been over 20yrs since I've seen a forecast put together in real time so I appreciate a lot may have changed.
In those days the senior forecaster would print out pressure maps, e.g. "wind flag" type points on a map, then join the dots, to complete an isobar type map, compare to previous pressure maps, draw in cold/warm fronts, add in some rain radar info into the picture etc. and use his/her experience to do the forecast for a particular period.
From the weather forecasts on the BBC for example, it appears as if this long hand process has been computerised.
Would that be called a "weather model" in the context of this thread?
Or are the "climate models" in the quote "climate models are the same as those used to predict the weather" something completely different?
Do they now include all the additional data such as albedo, "simple physics" "Co2 feedbacks" and assorted IPCC lingo we're so used to hearing these days regarding "climate models"?
Is it still possible to arrange informal public visits to regional weather centres, such as I used to enjoy at Stockport on an almost fortnightly basis 20 yrs ago? If so this could be a good way to communicate modern weather/climate forecasting techniques to "the unwashed" ;¬)
I'm just a clown, but the July/August 2006 issue of the International Journal of Clownatology was a special issue on Clownatological Simulacrums. [Simulacrum: 1. an image. 2. a mere pretense or semblance, vague representation; counterfeit; travesty; sham.] Sounds good to me.
MDGNN
Thank you. I need to think/read about it.
I'd always assumed that an excited CO2 molecule (excited at a quantised energy permitted by the mode of excitation) would eventually bump into some other molecule and its quantum of excitation energy would get shared as (unquantised) kinetic energy (aka heat) in some proportion between the two molecules. Either that, or it would eventually emit a photon of the same energy as the one it had absorbed.
As I said, I need to think/do some revision. Thanks again.
I have to say that I find all the discussion of models intensely frustrating. I'm with Mac that the term is losing it's usefulness but I don't think that just replacing it with another term like 'simulation' will resolve the issue. The attempt made in the original post to distinguish different sorts of model is a useful direction, but I fear there is much, much further to go in becoming specific and precise.
I think that one source of my frustration is that I am approaching the question of models not in an all or nothing sense, but rather by asking the question how good or how bad is a given model? What is it good for and what is it not?
Many critics of models here seem to approach the model question as all or nothing, as in 'is the model validated'?
I suspect that this difference in approach may reflect a difference between science on the one hand versus technology and policy on the other. I think these are trying to answer different questions.
For scientists the purpose of models is to gain understanding. There is never an end to this process. More research (and more funding!) is always required. 'Validation' doesn't really make sense in this context because there is not a set threshold of performance which will be satisfactory and bring the investigation to a conclusion.
For engineers and policy making, however, the question of 'what is good enough' is usually much clearer. Can a machine be made to run reliably within set parameters? How much should we spend on flood defense? Should we spend money on moving away from fossil fuels? Here it makes much more sense to ask the question, 'is the model good enough to help answer the question at hand?' These sorts of questions also provide some criteria for validation. If a model passes the validation tests it can be considered 'good enough' for practical purposes for its results to be taken into account in some way. If it does not pass the test it is 'useless' in answering the question at hand.
This blog, and most of the climate blogosphere has arisen more out of questions of policy than out of questions of science. It is quite understandable that science is approached from a policy mindset.
But I suspect many scientists who are trying to understand the climate system will react badly to being told that models are 'useless', because a model that is useless for policy may still be useful for understanding.
Let me give an example to try to clarify.
Earlier in the thread (Feb , 2012 at 11:22 AM) Philip Bratby wrote that one requirement 'for a model to be validated' is that it must 'conserve mass, momentum and energy'. The basic point is important, and of course this has been a big area of work in climate modeling. But what does it mean for a model to 'conserve mass, momentum and energy'? The model may be based on equations that perfectly conserve mass, momentum and energy, but no numerical model will perfectly conserve any quantity.
(At least I don't think so. At first glance that would require infinite precision arithmetic. If I am wrong the counterexample would be interesting to find out about.)
The question, then, that I would propose is how well does a model conserve mass, momentum and energy? When that is answered we can ask 'Is that good enough for the task in hand'?
When the approximations used in a model result in too greater failure of conservation 'flux adjustments' have sometimes been used to enforce conservation. Now, as the approximations used to for transport of momentum, mass and energy improve (for example, a finer grid in space and time) flux adjustments are less necessary (as I understand it). They still do not conserve quantities perfectly - no numerical calculation can - but they do not suffer from runaways.
I would describe that situation by saying that models which do not require flux adjustments are better than models without flux adjustments. Does that mean that models with flux adjustments are 'useless' and those without flux adjustments are 'valid'. Perhaps we can say that the newer models can be validated to a higher standard, but I think that misses the point.
For some purposes models with and without flux adjustments might both be useful (for example understanding the basic pattern of trade winds). For other purposes the newer models may be more useful than the old models (for example forecasting of El Nino). For other purposes neither may be good enough (for example climate prediction 100 years out to good enough precision for policy purposes). (My examples in the paragraph may not hold up, I'm just trying to give examples of the sort of problem to which these considerations may apply).
What I am trying to get at - and I am sorry if other commentators feel I am just repeating myself - is that model or generally not 'valid' or 'invalid'. The question of 'validity' depends on the question asked. Attacking approximate calculations (which is all the important models are) as inadequate without specifying the question, and without attempting a better calculation, comes across to me as just attacking attempts at understanding.
Martin A: a key aspect of statistical thermodynamics is that you cannot follow the behaviour of a single molecule, except at very low temperatures and small numbers,. If you try to do otherwise you get the 'Gibbs Paradox'.
Indeed, that is the question I'd like answered too.
You'd expect El Nino/La Nina whether pattern (or is it 'climate' considering its longer duration and considering the fact that 'climate models' are used to predict them) is so much better understood than the global warming we are fretting about.
Yet, the last time I checked the best 'climate models' could do is to predict whether the trend is up or down, NOT what the trend is going to do once it reaches its peak or hits the bottom. From then on it is usually a spaghetti graph.
For example, if you check the ENSO reference page on WUWT, you'll see the second graph on the page shows the current NINO 3.4 Ensemble Forecast.
There is a solid forecast for the next few weeks that La Nina will hit the rock bottom and there is a general agreement that in the next few months it will return to ENSO-neutral conditions. And that is where it ends. From June onwards the forecast from the ensemble of models looks like a jumbled mess.
So if I were a farmer or a dam operator I'd be completely without a clue as to whether this time next year we'd be in El Nino or La Nina conditions, because the climate models themselves haven't got a clue.
Are the climate models that are used to 'forecast' global climate 50 years from now any better tested and validated than the models used to forecast ENSO? I'd like to know because we'll be risking the world economy on it, not just daddy's farm.
Indeed, that is the question I'd like answered too.
You'd expect El Nino/La Nina whether pattern (or is it 'climate' considering its longer duration and considering the fact that 'climate models' are used to predict them) is so much better understood than the global warming we are fretting about.
Yet, the last time I checked the best 'climate models' could do is to predict whether the trend is up or down, NOT what the trend is going to do once it reaches its peak or hits the bottom. From then on it is usually a spaghetti graph.
For example, if you check the ENSO reference page on WUWT, you'll see the second graph on the page shows the current NINO 3.4 Ensemble Forecast.
There is a solid forecast for the next few weeks that La Nina will hit the rock bottom and there is a general agreement that in the next few months it will return to ENSO-neutral conditions. And that is where it ends. From June onwards the forecast from the ensemble of models looks like a jumbled mess.
So if I were a farmer or a dam operator I'd be completely without a clue as to whether this time next year we'd be in El Nino or La Nina conditions, because the climate models themselves haven't got a clue.
Are the climate models that are used to 'forecast' global climate 50 years from now any better tested and validated than the models used to forecast ENSO? I'd like to know because we'll be risking the world economy on it, not just daddy's farm.