Science writer Jon Turney has been looking at computer simulations and their role in the decision-making process, considering in particular climate models and economic models. There's a much of interest, like this for example:
Of course, uncertainties remain, and can be hard to reduce, but Reto Knutti, from the Institute for Atmospheric and Climate Science in Zurich, says that does not mean the models are not telling us anything: ‘For some variable and scales, model projections are remarkably robust and unlikely to be entirely wrong.’ There aren’t any models, for example, that indicate that increasing levels of atmospheric greenhouse gases will lead to a sudden fall in temperature. And the size of the increase they do project does not vary over that wide a range, either.
Hmm. Climate models are unlikely to be wrong because they give similar results to each other? Aren't they mean to be tested against - you know - reality? And when you do this, doesn't it show that the models are running too hot?
;)
This isn't the only problem either. Turney cites a suggestion that multiple runs of the same model can help you assess uncertainty:
Repetition helps. Paul Bates, professor of hydrology at the University of Bristol, runs flood models. He told me: ‘The problem is that most modellers run their codes deterministically: one data set and one optimum set of parameters, and they produce one prediction. But you need to run the models many times. For some recent analyses, we’ve run nearly 50,000 simulations.’ The result is not a firmer prediction, but a well-defined range of probabilities overlaid on the landscape. ‘You’d run the model many times and you’d do a composite map,’ said Bates. The upshot, he added, is that ‘instead of saying here will be wet, here will be dry, each cell would have a probability attached’.
But as far as decisionmaking is concerned this misses the point. If there are structural model errors, like the ones we have been discussing in relation to the UKCP09 climate predictions, then multiple runs of the same model are not going to help you.
Turney's article is nicely written and a good introduction to some of the issues, but you might come away with the idea that climate models are a reliable basis for decision making. Nothing could be further from the truth.