Bishop Hill Garden shed tinkerers
Sep 6, 2013 Climate: Models There is a fascinating layman's intoduction to climate models over at Ars Technica. Author Scott Johnson starts out with the standard potshot at global warming dissenters, takes a look at how a GCM is put together and talks to lots of climate modellers about their work and all the testing they do; it has something of the air of a puff piece about it, but that's not to say that it's not interesting.
Here's how it opens:
Talk to someone who rejects the conclusions of climate science and you’ll likely hear some variation of the following: “That’s all based on models, and you can make a model say anything you want.” Often, they'll suggest the models don't even have a solid foundation of data to work with—garbage in, garbage out, as the old programming adage goes. But how many of us (anywhere on the opinion spectrum) really know enough about what goes into a climate model to judge what comes out?
Climate models are used to generate projections showing the consequences of various courses of action, so they are relevant to discussions about public policy. Of course, being relevant to public policy also makes a thing vulnerable to the indiscriminate cannons on the foul battlefield of politics.
Skepticism is certainly not an unreasonable response when first exposed to the concept of a climate model. But skepticism means examining the evidence before making up one’s mind. If anyone has scrutinized the workings of climate models, it’s climate scientists—and they are confident that, just as in other fields, their models are useful scientific tools.
"Useful scientific tools"? Well yes, I think I would agree with that. The article describes how a divergence of model and real-world behaviour can help uncover gaps in our knowledge. This is great - this is what a GCM should be for. What it isn't is a prediction of the future - something we can pin policy measures on. But while the article is entitled "Why trust climate models?", in fact to its credit, the article doesn't push this more expansive claim about the usefulness of climate models very much. The case seems to be that because modellers do a lot of testing against historic data we should trust the models. Not convincing at all, in my opinion.
The flimsiness of the case also becomes clear when Steve Easterbrook makes his entrance:
Easterbrook has argued against the idea that an independent verification and validation protocol could usefully be applied to climate models. One problem he sees is that climate models are living scientific tools that are constantly evolving rather than pieces of software built to achieve a certain goal. There is, for the most part, no final product to ship out the door. There's no absolute standard to compare it against either.
To give one example, adding more realistic physics or chemistry to some component of a model sometimes makes simulations fit some observations less well. Whether you add it or not then depends on what you're trying to achieve. Is the primary test of the model to match certain observations or to provide the most realistic possible representation of the processes that drive the climate system? And which observations are the most important to match? Patterns of cloud cover? Sea surface temperature?
Here, Easterbrook seems to be making a pretty strong case that climate models have no part to play in the policy process. How can you have a model that can't be built and tested to engineering standards informing policy? How can the public trust the moving feast that he describes? And if models really are being built without a specific goal in mind then the funding councils surely have some fairly pointed questions to answer.
The public is being asked to fork out lots of money on the basis of climate model output. Climate modellers have to decide if they are going to be garden-shed tinkerers or engineers whose findings are robust enough to inform the policy process.
Reader Comments (105)
I thought the sum was 10 Guineas and that would be what, £10 pounds plus 10 shillings?
ATheok: always happy to get paid more. I'll take your £10.50
"The Chief Scientist of the Met Office, in testimony to Parliament (if I remember correctly the context in which she said it) said that the Met Office's climate models are tested several times a week because the same models are used to forecast the weather.
Sep 8, 2013 at 8:38 PM | Martin A "
I'm just going to put my face in my palms and shake my head a few times. ::sigh::
I can't say I'm surprised someone actually made that claim.
@TheBigYinJames Software engineering practices like version control have been standard in climate modelling for many years. Model releases are indeed measured against reality in a planned and controlled way.
Doug
How is that compatible with the assertion that the models extrapolate scenarios (to elucidate possible interconnections, etc.) rather than produce predictions? Only (falsification) predictions can be "measured against reality" in a policy relevant way. If that is now what is being done (since when?), then the models are comprehensively falsified.
Great topic and post, much information to consider