Bishop Hill Defining reality
Dec 16, 2013 Climate: Models Science writer Jon Turney has been looking at computer simulations and their role in the decision-making process, considering in particular climate models and economic models. There's a much of interest, like this for example:
Of course, uncertainties remain, and can be hard to reduce, but Reto Knutti, from the Institute for Atmospheric and Climate Science in Zurich, says that does not mean the models are not telling us anything: ‘For some variable and scales, model projections are remarkably robust and unlikely to be entirely wrong.’ There aren’t any models, for example, that indicate that increasing levels of atmospheric greenhouse gases will lead to a sudden fall in temperature. And the size of the increase they do project does not vary over that wide a range, either.
Hmm. Climate models are unlikely to be wrong because they give similar results to each other? Aren't they mean to be tested against - you know - reality? And when you do this, doesn't it show that the models are running too hot?
;)
This isn't the only problem either. Turney cites a suggestion that multiple runs of the same model can help you assess uncertainty:
Repetition helps. Paul Bates, professor of hydrology at the University of Bristol, runs flood models. He told me: ‘The problem is that most modellers run their codes deterministically: one data set and one optimum set of parameters, and they produce one prediction. But you need to run the models many times. For some recent analyses, we’ve run nearly 50,000 simulations.’ The result is not a firmer prediction, but a well-defined range of probabilities overlaid on the landscape. ‘You’d run the model many times and you’d do a composite map,’ said Bates. The upshot, he added, is that ‘instead of saying here will be wet, here will be dry, each cell would have a probability attached’.
But as far as decisionmaking is concerned this misses the point. If there are structural model errors, like the ones we have been discussing in relation to the UKCP09 climate predictions, then multiple runs of the same model are not going to help you.
Turney's article is nicely written and a good introduction to some of the issues, but you might come away with the idea that climate models are a reliable basis for decision making. Nothing could be further from the truth.
Reader Comments (73)
What, if anything does the Met Office say about validation? Perhaps RB could comment.
Dec 16, 2013 at 7:07 PM | Unregistered CommenterSchrodinger's Cat
Here is what the Met Office says about validation...
Are computer models reliable?
Yes. Computer models are an essential
tool in understanding how the climate will
respond to changes in greenhouse gas
concentrations, and other external effects,
such as solar output and volcanoes.
Computer models are the only reliable
way to predict changes in climate. Their
reliability is tested by seeing if they are able
to reproduce the past climate, which gives
scientists confidence that they can also
predict the future.
(Met Office publication. "Warming. A Guide to Climate Change")
Richard Betts has some time back given a reference to a paper with Met Office authors that shows:
1. Their model agrees with past climate if CO2 is included in the simulation.
2. It does not agree with past climate if the effects of CO2 is switched off in the simulation.
Tesco have a trial of infra red cameras at the doors of some of their stores, the hope is that they can run a model that will tell them when they need to open more tills for queuing customers, up to present, customers are still queuing, or not queuing, Tesco say they haven`t worked out the maths yet.
How chaotic are customers coming into a tesco store collecting groceries and paying for them compared with the climate system.
Dec 16, 2013 at 8:08 PM Rob
Surprising Tescos have not worked out the maths yet. It sounds like a standard queueing network problem which has probably already been analysed in detail and its accuracy verified. There is a huge literature on the subject.
Customer arrivals can probably be modelled as a nonstationary Poisson process, equivalent to assuming that they arrive independently, which means there will be an exponential distribution of time between the arrival of successive customers - a Poisson process. It's nonstationary because the mean time between arrivals will itself be a function of time. For example, the time between arriving customers will on average be shorter at 5pm Friday than at 10am Tuesday.
Although modelling queueing networks is nontrivial, and the maths can get quite complicated as soon as you stop assuming that all distributions are exponential, it's an immensely more tractable problem than modelling the climate system.
-------------------------------------------------------------------------
As well as capable of being mathematically analysed, such queueing systems can be modelled with great precision by discrete event simulation. There are (were?) computer languages [eg Simscript, Simula] specifically adapted for programming such simulations.
Martin...Tescos have the maths worked out...when the footfall rises beyond a certain point, then they mobilise extra staff....for all the stuff that needs to be done when people drop goods in the aisles, need to go to the bathroom, want a coffee etc.....all those fractal things that climate scientists do not know about
and, it would seem, all those situations that happen, unbeknownst to folks like Martin A who never go shopping. Has Martin A ever done shopping? My guess is that he is seen supermarkets from a distance and never gone inside. Other people do shopping.
I think it is worth re-posting Geronimo's recent comment with Trenberth's 2007 summary of how crap models are:
I just checked and Richard Betts did not come back to give any update.
Dec 16, 2013 at 6:00 PM | Manniac
Thanks for the laugh.
"None of the models used by IPCC are initialized to the observed state and none of the climate states in the models correspond even remotely to the current observed climate."
So UNFCCC, the main and basically only basis for the IPCC models, is then crap?
Has Martin A ever done shopping? My guess is that he is seen supermarkets from a distance and never gone inside. Other people do shopping.
Dec 16, 2013 at 9:38 PM diogenes
Huh? Of course Martin A does supermarket shopping. But he does not get the point that diogenes obviously feels quite strongly about.
Maybe diogenes thinks that supermarket queue waiting times can't be modelled statistically unless you take into account all the details of every purchase, whether the customer in front has an item that has lost its bar code, whether they have a product that the cashier notices is leaking and calls for a replacement, whether the customer in front finds that their debit card does not work, etc etc.
It was some other commenter who said that Tescos had said they had not yet worked out the maths of their queueing model, not Martin A. (see Dec 16, 2013 at 8:08 PM | Unregistered CommenterRob)
Richard Drake
Mosher has many bankrupt arguments. This one:
"that all so-called observations or observational results are a product of models of some form. We never in our interactions deal with 'raw data', everything is mediated by what we think is going on." -
is a simple confusion of the many disparate uses of the word 'model'. But when the rest of us talk about 'models' it is only a shorter term for GCM, which is specifically a numerical climate simulation. We are not talking about any other type of model, some of which are indeed very good and some of which are rank bad. Moreover Mosher even seemed to confuse Steve McI with this argumentation by insisting that satellite measurements were based on a model too, which McI then later chided me with on his blog. I realised later that Mosher had somehow interpreted mere data post-processing as being a 'model'. This was either accidental or deliberate semantic obfuscation. Post-processing can indeed also be good or bad (checked by testing against solid benchmarks) but it is clearly broken logic to argue that because some models are good then all models are valid. Every model must be tested on its own merits and a bad model remains a bad model. I have tried to put him straight on this directly just as with all his odd arguments on climate etc. but likely to no avail. The man runs with the hare and hunts with the hound and I strongly doubt he has provided anything other than peurile derailment to any debate.
Martin A
With regards to your previous post I've just realised the similarity between the Met Office thinking and Pascal's Wager.
The implicit assumption that more CO2 WILL cause more warming and not doing something about it is like a God who punishes for disbelief.
Nevermind the CO2 (or God) that has no effect / couldn't give a monkeys what you believe.
"but it is clearly broken logic to argue that because some models are good then all models are valid."
Since all models are different, then only one or none can be correct. The average of mostly wrong models, that IPCC use in their reports, is strange and difficult to understand.
The subject of climate models came up several times in a recent press conference at the American Geophysical Union, "IPCC: The Future of the Assessment", and there is now a partial transcript here:
https://sites.google.com/site/mytranscriptbox/home/20131210_ag
"Seems to me the problem with these 'scientists' is they never learned English"
No. The problem with these scientists is that they never learned Science.
"The three major messages by now should be well-known to everybody - I just want to recall them. Warming of the climate system is unequivocal, human influence on the climate system is clear and limiting climate change will require substantial and sustained reductions of greenhouse gas emissions. These are very simple, quotable sentences but their strength lies in the fact that these sentences now are not only agreed by the scientists who have worked on this report but also adopted by the governments of the world participating in the IPCC. "
These claims are basically the same as one finds in the UNFCCC. UNFCCC was made up by politians in 1992 and it's basis is more policybased dogma and less politics based on science.
Reality will bite back at them.
The bizarre spin from owners and operators of models with more adjustable parameters than you could shake a stick at, that running them with and without CO2 shows that only CO2 can 'explain late 20thC warming (actually, none of them model CO2 - instead of modelling it, they add a presumed reduction in heat loss at the outer edge of the model atmosphere), gets dealt another blow with this study reported at WUWT today: "Claim: Solar, AMO, & PDO cycles combined reproduce the global climate of the past"
The authors claim:
Tesco may have infrared cameras to predict queuing at checkouts, whether or not they have done the maths.
However - like the variables which have not been taken account of in climate models, their procedures do not take account of the following:
Checkout lady is serving her neighbour and wants to know in great detail how her daughter's getting on at school
Customer has vouchers which don't seem to be recognised by the laser reader
One item requires the checkout lady to summon a manager from deep in back-of-house for a summit conference
Customer's debit card won't work despite them having ample funds in the bank (allegedly)
One customer is convinced that the End Of The World Is Nigh and has stocked up on enough to fill two trolleys
Two-for-one offer has not registered at the till, requiring a supervisor to be summoned, who then has to go to the very
furthest point of the store to check what it says on the shelf, who then comes back with that 'look' which says 'you
stupid customer - that offer was for 250ml not the 500ml items which were at that precise position on the shelf'...
Apart from that (and a few dozen other variables which I can't think of) - the system should work well...
Yes, Sherlock1 we have all experienced those sort of things.
But it's not at all like climate models that neglect important effects such as transport of heat by convection, evaporation and condensation. Not sure how to say it politely, but if you really think that, you have just no idea what queueing theory is about.
Queueing models make no pretense at analysing the details of the physical situation of what's going on - unlike climate models.
I assume that what Tescos are aiming to do is to get more checkouts open as the number of arriving customers increase at busy times, in advance (or concurrent with) those customers arriving at the checkouts.
Good for them. That will be better than waiting until each checkout has a queue of several trolleys before belatedly calling "all checkout trained staff please".
@ Dec 16, 2013 at 9:03 PM | Martin A
Models do not agree with past climates neither with CO2 or without it.
You would have notice that model results always are given as anomaly of temperature, never the actual temperature. But this is not a correct metric. If you are planning to go skiing in Siberia, do you want to know the temperature anomaly, or rather that the actual temperature may drop to -63 C?
One of the reasons actual temperature is not given is because the models are not very good at simulating it. Departures of more than two degrees are common and some of the less performing models may be in excess of 4 degrees off the mark. Other variables are worse, like specific humidity, but we hardly hear of them.
You may think that it does not matter, a measure of anomaly still will tell us if the future is going to be 2 or 4 degrees warmer, and that is the key issue.
It might be for your skiing holidays in warner climates but not for the models themselves.
A climate model is basically a energy and mass balance of the atmosphere - ocean coupled system. If the resulting actual temperature is wrong the energy balance is wrong, and this have very serious implications.
For starters you will get the wrong phase of water over many regions of the earth. A couple of degrees up or down is the difference between snow or rain, between an albedo higher than 80% or lower than 20%, which will reflect an average of 260 or 65 W/m^2 of solar radiation (admittedly less in polar regions, but still way larger than the 3.5 Watts of CO2 forcing). You will get the amount and time of precipitation and condensation wrong, which will make one of the largest heat pumps in your simulated system to malfunction, etc. etc.
At present, climate models are not mature enough to say anything credible about the past or the future of climate. They are useful, and could be much more, but not in the present form, and they are presented in a very dishonest way by people that should know better.
JamesG: Thanks for your opinion. You obviously didn't read mine. And I'm sure that Mosher doesn't hold the simplistic ones you attribute to him - on models generally and GCMs particularly, which I would disagree with in my turn. But what a mighty opponent he must be, able to influence even Mr McIntyre, while snipers here can only have a go at him from the safe bunker of their pseudonyms - yet still obviously deeply wounded by his disregard of theirs.
Martin A: Thank you for multiple expressions of what I think is the correct opinion! Especially this:
That is the most fundamental point throughout the CAGW disaster: the inability to say (perhaps even to realise) that we don't know. Thank you.
When assessing hydrology models compare with data from rainfall , rivers and boreholes: many are of dubious use. In 2000 VP of CMh2 Hill said computer models were no good at designing remediation of contaminated groundwater because the ground was too complex. If we cannot accurately predict groundwater flow for cleaning up contamination we predict the future climate?
Part of the problem is that many scientists are no longer that keen on fieldwork,: it is too expensive, time consuming , uncomfortable and produces confusing data. it is much easier to construct a virtual world in a computer and produce the results one wants. This is combined with the problem that very few environmental scientists have the statistical skills see McIntyre,McKitrick N lewis and Wegman Report. I would suggest the only areas where failure of stats produces death or loss of mining are drug research and mining.
If one looks at the Royal Society,scientists such as Nurse , May, Beddington and Oxburgh have never been involved in analysing data where a mistake could result in death or loss of vast amounts of money.
The reality is that for many academic and theoretical scientists the only result in admitting a mistake is loss of face. Even if they make a mistake , provided they do not admit it , there is no adverse result.
The massive increase in environmental science outside of engineering, physical , chemical and medical sciences in minor universities has resulted has resulted in large numbers of academics trying to justify their existence. In fact Eisenhower predicted the threat from the massive expansion of the scientific academic community.
If the taxpayer demanded that the environmental science community had the same statistical skills of the mining and pharmaceutical industries ,not many would be employed. I doubt many environmental scientists actually understand the statistical points made by McIntyre,McKitrick N lewis and Wegman.
I think that those who want a rigorous scientific approach actually question the statistical skills of many of those who support AGW.
That is the most fundamental point throughout the CAGW disaster: the inability to say (perhaps even to realise) that we don't know. Thank you.
Dec 18, 2013 at 12:35 PM | Registered CommenterRichard Drake
Yes, thank you Richard.
Years back, a former colleague and I started an investigation into the nature of bullshit in the organisation where we worked, as there seemed to be so much of it around. (Our studies were overtaken: http://en.wikipedia.org/wiki/On_Bullshit )
One of the key findings on the subject is that the purest and most dangerous bullshit comes from those who sincerely believe the rubbish they are talking. I think that the great majority of climate scientists probably fall into this category.
Climate models, on the record, are a reliable basis for making mistakes. Very, very expensive mistakes.
<I>One of the key findings on the subject is that the purest and most dangerous bullshit comes from those who sincerely believe the rubbish they are talking.
Dec 19, 2013 at 9:16 AM | Registered CommenterMartin A
Is self-hypnosis involved?