Readers may remember Nic Lewis's paper demonstrating a major flaw in the UKCP09 climate predictions. In brief summary, the predictions are a weighted average of a series of virtual climates produced by the HadCM3 climate model, the weight each one gets being determined by how well it matches the observations. Nic discovered that the HadCM3 model was incapable of producing virtual climates that match the real-world climate as regards two key parameters - the climate sensitivity and the aerosol forcing. This obviously meant that the average produced is meaningless.
Nic's paper had a response from Julia Slingo which acknowledged that HadCM3 could not produce low-sensitivity/low aerosol forcing climates, explaining that this was an emergent property of the model. Nic noted that she was therefore implicitly accepting his core argument and I mentioned this in a blog post about the related UK Climate Change Risk Assessment.
Shortly afterwards I had an email from a press officer at the Met Office:
I noticed in your recent blog a line about UKCP09, as follows: “Unfortunately the UKCCRA is based on the UKCP09 climate predictions, which the Met Office has acknowledged contain a major flaw.”
The highlighted section is not accurate – no such acknowledgement has been made by the Met Office (unless you can point me to any evidence to the contrary which you think supports your statement). Could you please remove this reference from your blog?
This was a bit of a surprise to tell the truth. It was pretty hard to imagine how the Met Office could argue that the predictions were sound while acknowledging that the HadCM3 model behind it couldn't generate realistic climates. I therefore replied, trying to get some clarification of precisely what their position was:
As I understand it, the Met Office does not dispute that low CS/low aerosol forcing combinations are not possible with HADCM3. This is most likely the way the real climate system is, at least according to the current observational evidence. As the UKCP09 predictions are supposed to be a probabilistic combination of all possible CS/aerosol combinations, the fact that the model cannot sample the most likely combination is indisputably a major flaw. Prof Slingo has observed that this feature is an emergent property of the model, but this is only to recognise how the problem arose, not to dispute that it is an error. She has also stated that this feature does not create a bias, but I fail to see how she can conclude this without actually correcting the error and finding out what effect it has on the results.
I am happy to add a note to either of both of the BH postings clarifying the Met Office's position, but I would need to be clear what this is. Are you saying that it is possible to create a valid probabilistic forecast without sampling the full CS/aerosol parameter space and, in particular, without sampling the combinations of parameters that is most likely according to observations? I assume not. If the case is simply that the flaw doesn't create a bias (and is therefore not "major") then I would ask how you have reached this conclusion.
However, the reply the following day was evasive on these questions:
From your reply below it’s clear you make a few leaps of judgement to make your conclusion, and these aren’t leaps that the Met Office, or for that matter the IPCC, would necessarily agree with. Certainly they don’t justify the concept that we have acknowledged a flaw, let alone a major one – this is very much your own language and, as such, this should not be attributed to us in the way that you have.
I understand your argument here is based on Nic Lewis’s comment piece and so I note here that Nic himself has made it clear he did no claim there was an error in HadCM3 - in a response to an article on our blog, he said: “I didn’t claim that the HadCM3 model contained an error.” You can see this here (first comment).
Again, this reinforces the point that your statement is very much your own judgement call – not ours and, it seems, not Nic Lewis’s.
As a final point, HadCM3 was not the only model used for UKCP09.
With all this in mind, I’d appreciate if you removed the comment in question or reword it to something which more accurately reflects the situation.
Readers will note the complete absence of any attempt to answer my questions. Moreover, while the criticism had been carefully aimed at the UKCP09 predictions, here the Met Office was insinuating that I had implied a flaw in the underlying HadCM3 model (the argument is more that HadCM3 is unsuitable rather than that it is flawed per se). So this was an distraction by the Met Office. Readers might also like to read the rest of the Nic Lewis comment cited by the Met Office, which reveals a naked falsehood in the report they issued on climate sensitivity.
Similarly, the point about other models being used in UKCP09 was not the kind of thing you would have found in an honest response, as my reply made clear.
Thanks for your email, which arrived too late in the day for me to deal with before I went away on holiday. As I have indicated before, I am happy to make clear the Met Office’s position if you will make clear to me what that position is.
You note Nic Lewis’s comments about HadCM3, but I have been careful to say only that the UKCP09 predictions are flawed. Regarding HadCM3 we therefore appear to be in agreement and my blog posts are correct.
I note also your comments with regard to the use of other computer models in the UKCP09 project. I fear that you may have misinterpreted what your colleagues are telling you. As I understand it, the use of other computer models in UKCP09 is peripheral to the predictions, since they are only used in error estimates. Their use therefore does not in any way mitigate the problems caused by the structural rigidities in HadCM3.
I asked for clarification of the Met Office’s position in my previous email, but you have not responded to the question I asked. I therefore ask again: Are you saying that it is possible to create a valid probabilistic forecast without sampling the full CS/aerosol forcing space and, in particular, without sampling the combinations of those factors that are most likely according to recent observationally-based estimates? (For the avoidance of doubt, I am referring to values in (say) the ranges 1.25–2.25°C for climate sensitivity and -0.4 to -1.1 W/m2 for total effective aerosol forcing.) It appears to me that such a position is untenable and the UKCP09 predictions are therefore flawed. Your silence on the issue seems to represent a tacit admission that I am correct. You cannot say that I am wrong while simultaneously refusing to say why.
In a similar vein, I might ask why the Met Office has failed to withdraw its July paper when the vast majority of the problems Lewis identified in it have gone unanswered.
The Met Office cannot simply try to brush these problems under the carpet by refusing to acknowledge them. Should it do so it will undoubtedly be severely criticised and I therefore urge you and your colleagues to withdraw the UKCP09 predictions and the July briefing paper on your own initiative.
That was sent at the end of October but despite my sending reminders no reply was forthcoming. At the start of December I therefore wrote once more, indicating that I would publish the correspondence and let people draw their own conclusions. This finally elicited a response:
Sorry it has taken a while to respond, we do tend to get very busy in the winter months and I have to prioritise operational work, as I’m sure you’ll understand. With regards to our email exchanges, we appear to be going round in circles, so I thought I’d recap on how we got here.
As you’ll recall, this started when I asked you to change the following line in one of your blogs:
“Unfortunately the UKCCRA is based on the UKCP09 climate predictions, which the Met Office has acknowledged contain a major flaw.”
As I’ve said before, you are entitled to your own opinions but it’s inaccurate and misleading to ascribe your own opinions to someone else – which is what you have done in the highlighted line above. The Met Office has not said anything along the lines above, and that alone is enough to say that it is factually inaccurate.
In response, you’ve asked me numerous questions which don’t have any relevance to this straightforward request. Besides that, I understand Met Office scientists have spent a considerable amount of time over recent years dealing with similar inquiries from your correspondents, Doug Keenan and Nic Lewis in particular. In fact, Nic Lewis was here for an afternoon a few weeks ago and, as I understand, there has already been a robust, open and constructive conversation about many of the issues you raise. I think that is the appropriate forum for those discussions.
Having made my points as clear as I can, I’d once again ask you to remove or change the highlighted statement above and any other similar statements on the grounds that they are inaccurate and misleading. I think we’ve probably exhausted our respective positions, so I’d suggest that unless you have anything new to add, we bring a close to this particular line of correspondence.
If anyone is any the clearer for this response, they are cleverer than I am! Slingo has certainly admitted that the HadCM3 model can't do what it needed to do for the UKCP09 process. This is undoubtedly a major flaw. Ergo she has acknowledged the flaw.
Tiring of the word games, I decided that I would publish the correspondence anyway, but asked one last question:
Can you just give me one more detail please. You say that you think that my questions re whether the inability of HadCM3 to simulate realistic climates is irrelevant. Is this your personal opinion or the opinion of Met Office scientists (or perhaps just a corporate position)?
But unfortunately, this elicited an equally obscure reply:
We need to clarify here, as I wouldn’t want you to misunderstand or use my words out of context.
I said (as you can see in the email below): “...you’ve asked me numerous questions which don’t have any relevance to this straightforward request” [bolded for emphasis]. So to make it absolutely clear, I’m saying that your questions are not relevant to my specific request – which is on a point of fact, namely that you assert the Met Office has said something which it has not. I’ve gone into the details of why that is the case and why it’s misleading to ascribe your own opinions to someone else.
I don’t doubt that your questions do have relevant contexts and, as I understand, these issues have already been discussed at length between people more directly involved.
As I’ve said, my point is specific and on a point of fact. I am acting in my capacity as a press officer to point out factual inaccuracies and reasonably request that those inaccuracies are corrected.
To this I signed off as follows, copying Julia Slingo:
I have asserted that the Met Office has acknowledged that there is a flaw in UKCP09. I say this because Prof Slingo has acknowledged that HadCM3 cannot simulate low climate sensitivity, low aerosol forcing climates, which are the ones that observations suggest are most likely. She says that this is an emergent property of the model.
You have not disputed that she has made this acknowledgement. I therefore assume that you are disputing that HadCM3's inability to simulate these kinds of climates means that the UKCP09 projections are flawed. This is, as your scientific colleagues no doubt realise, is a completely absurd and entirely indefensible position. It is hard to believe that anyone within the Met Office would seriously suggest that a weighted average of climates excluding those most like the real climate has any usefulness as a predictive tool. As if to confirm this conclusion, you have signally failed to answer my questions on the matter.
I assume then that your real objection is that your colleagues are embarrassed by this public airing of their errors. That really is too bad. I reiterate what I said in my earlier email: the duty of the Met Office is to withdraw the UKCP09 predictions of your own volition as soon as possible. To allow both public and private sectors to continue to spend on the basis of erroneous predictions would represent a serious breach of the Met Office's duties towards the public.
I will add a note to my earlier post as follows:
The Met Office have acknowledged Nic's points about the flaw in the UKCP09 predictions but ask me to make it clear that they deny that they have acknowledged a flaw. I don't know what they mean either.
Bishop Hill
;)