Buy

Books
Click images for more details

Twitter
Support

 

Recent comments
Recent posts
Currently discussing
Links

A few sites I've stumbled across recently....

Powered by Squarespace
« Lew deconstruction | Main | Energy Bill second reading »
Tuesday
Jun182013

Some model thoughts

There have been a few interesting bits and pieces floating round on the subject of climate models in recent days.

At WUWT, Bob Tisdale has reviewed the CMIP5 model predictions of Antarctic sea ice and found that they have performed no better than their predecessors.

Judith Curry has focused on a paper by Stevens and Bony that looks at one of the fundamental deficiencies of climate models - their inability to represent clouds - and considers the futility of trying to add complexity in other areas until this basic failing has been overcome.

And then Doug McNeall tweeted the following remarks in defence of the CMIP5 ensemble:

The CMIP5 ensemble is not set up a a calibrated probabilistic prediction system: it doesn't pretend to be one either.

It is just a set of plausible trajectories that the climate might take, given a certain set of forcings.

They're not predictions - just plausible outcomes, and those outcomes have not been borne out in practice. The models have fundamental deficiencies too.

Do the policymakers discussing the Energy Bill in Parliament today understand these issues?

PrintView Printer Friendly Version

Reader Comments (63)

Jun 19, 2013 at 8:40 AM | Registered CommenterMartin A

"You mentioned models in your last answer, and people have asked whether we can really rely on models to tell us about the future of our climate?

It’s a very good question, but of course we have to remember they are the only thing we have to tell us about the future."

But if they can't predict the future, then they are worse than useless. The rest is verbiage.

Jun 19, 2013 at 9:26 AM | Unregistered CommenterAllan M

Re my: "We can't think of anything else"="We haven't really tried to think of anything else". yesterday afternoon.
Lo and behold! When we do try to think of something else, up pops the AMO — about which we had never heard before. Excuse me? Where have you been the last few years? Stuck in your own private echo chamber, that's where

Jun 19, 2013 at 10:24 AM | Registered CommenterMike Jackson

So they are not in a sense tuned to give the right answer, what they are representing is how weather, winds blow, rain forms and so forth, absolutely freely based on the fundamental laws of physics.

Martin A, do you have the source of that quote, please? It might be handy in the future.

Another point in which we are misled by models is in the use of anomalies instead of actual values. Even for temperature the spread among models is larger than 3C at any given time. If these models are a good energy balance representation of the surface of the earth, why such a disparity of results?

3C is already beyond any apocalyptic tipping point. Lucia made a plot of models temperature cacophony some time ago here.

Pielke Sr. had a good summary of where models fail here

And Paltridge has shown that the main positive feedback (water vapour) refuses to cooperate, here.

Jun 19, 2013 at 11:10 AM | Registered CommenterPatagon

Patagon,

The quote comes from an questions and answer session Julia Slingo did. It's on youtube in 3 parts called "Ask the Expert - Prof Julia Slingo, Climate Change". A transcript of it is available on the Met Office website here. The relevant bit is at the start of Part 2 which begins on page three of the transcript.

Jun 19, 2013 at 2:18 PM | Unregistered CommenterGareth

From Slingo's statement:

"...How do we know that they’re good? Well we continually test them against observations of the current climate in lots and lots of ways. At the Met Office we use the same model to make weather forecasts as we do to make our climate predictions, so every day we are testing the model and saying, ‘how well did we do with the weather forecast?’ We know that on many occasions our weather forecasts are incredibly skilful and that’s increasingly giving us confidence that the science in our models is fit to do this ‘crystal ball gazing’ into the future.."

Thetrefore, there is clearly an ongoing audit, and I would like to see the results. If Richard Betts is out there please can he post a reference to it? If not, FOIA anyone?

Jun 19, 2013 at 3:03 PM | Unregistered CommenterRoger Longstaff

But what weather forecasts does she mean? 24 hr, 48 hr, longer? Because they rarely agree, and anyway lack precision at any level. Their accuracy cannot be verified to any reasonable degree because the monitoring network required to do so does not exist.

I suspect a degree of circularity - the model creates the forecast (which is fairly imprecise), the forecast is declared to be correct (within a certain margin of error) which verifies the model (within a margin of error).

Jun 19, 2013 at 4:02 PM | Unregistered CommenterNW

I have sent the following email to the Met Office:

"Dear Sir / Madam,

On the Met Office website the Chief Scientist, Prof. Julia Slingo, states that climate model forecasts have been audited, in comparison to actual conditions that occurred, and talks of "incredibly skilful results":

http://www.metoffice.gov.uk/media/pdf/q/e/julia-slingo.pdf

Please could you provide a reference to this audit, or, if it is not yet publicly available, please send it to me?

Yours sincerely, Roger Longstaff"

I will give the MO the benefit of the doubt, and assume they will reply in good faith. But if they obfuscate I will definitely FOIA it.

Jun 19, 2013 at 4:11 PM | Unregistered CommenterRoger Longstaff

I've read details of how they verify/validate their predictions on the Met Office website. It's there all right. But don't make me find it for you or explain what the words mean. There's a self-justifying pedagogical feel about that site which makes me not want to go there again.

Jun 19, 2013 at 4:56 PM | Unregistered Commenterrhoda

I have seen something on their website, but it describes a complicated method of deriving some sort of accuracy index which of course shows that their forecasts are accurate, but nothing I could see about how "accurate" is defined in terms of temperature, wind force etc.

I just looked up the forecast for Glasgow tomorrow. They give temperature, wind speed and direction at 3 hourly intervals. Where in Glasgow would one measure these parameters, and what would count as accurate? Are any records of forecast vs. actual available?

Jun 19, 2013 at 6:00 PM | Unregistered CommenterNW

"Are any records of forecast vs. actual available?"

This is exactly the point. I want to see the audits of forecast vs. actual - not anything to do with retrospective forecasts/predictions/projections (in other words hindcasts), which is all that I have ever seen for Met Office verification/validation of the models.

Jun 19, 2013 at 7:07 PM | Unregistered CommenterRoger Longstaff

There is also a question of which forecast. It seems obvious that the more distant the forecast the more it must rely on the models rather than observations. As the further out you go the less accurate the forecast this must be an issue.

Jun 19, 2013 at 7:48 PM | Unregistered CommenterNW

"Are any records of forecast vs. actual available?"

Is there any archive of past forecasts at all? If forecasters are making competitive tenders you would think each company would turn up with some easy to understand numbers in hand.

Jun 20, 2013 at 9:53 AM | Unregistered CommenterRob Burton

"On the Met Office website the Chief Scientist, Prof. Julia Slingo, states that climate model forecasts have been audited, in comparison to actual conditions that occurred, and talks of "incredibly skilful results":

http://www.metoffice.gov.uk/media/pdf/q/e/julia-slingo.pdf

Please could you provide a reference to this audit, or, if it is not yet publicly available, please send it to me?"

The Met Office has replied that they will respond to my question within 28 working days.

Jun 20, 2013 at 11:32 AM | Unregistered CommenterRoger Longstaff

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>