Bishop Hill Met Office admits claims of significant temperature rise untenable
May 27, 2013 Climate: Statistics Climate: Surface This is a guest post by Doug Keenan.
It has been widely claimed that the increase in global temperatures since the late 1800s is too large to be reasonably attributed to natural random variation. Moreover, that claim is arguably the biggest reason for concern about global warming. The basis for the claim has recently been discussed in the UK Parliament. It turns out that the claim has no basis, and scientists at the Met Office have been trying to cover that up.
The Parliamentary Question that started this was put by Lord Donoughue on 8 November 2012. The Question is as follows.
To ask Her Majesty’s Government … whether they consider a rise in global temperature of 0.8 degrees Celsius since 1880 to be significant. [HL3050]
The Answer claimed that “the temperature rise since about 1880 is statistically significant”. This means that the temperature rise could not be reasonably attributed to natural random variation — i.e. global warming is real.
In statistics, significance can only be determined via a statistical model. As a simple example, suppose that we toss a coin 10 times and get heads each time. Here are two possible explanations.
- Explanation 1: the coin is a trick coin, with a head on each side.
- Explanation 2: the coin is a fair coin, and it came up heads every time just by chance.
(Other explanations are possible, of course.)
Intuitively, getting heads 10 out of 10 times is very implausible. If we have only those two explanations to consider, and have no other information, then we would conclude that Explanation 1 is far more likely than Explanation 2.
A statistician would call each explanation a “statistical model” (roughly). Using statistics, it could then be shown that Explanation 1 is about a thousand times more likely than Explanation 2; that is, statistical analysis allows us to quantify how much more likely one explanation (model) is than the other. In strict statistical terminology, the conclusion would be stated like this: “the relative likelihood of Model 2 with respect to Model 1 is 0.001”.
A proper Answer to the above Parliamentary Question must not only state Yes or No, it must also specify what statistical model was used to determine significance. The Answer does indeed specify a statistical model, at least to some extent. It states that they used a “linear trend” and that the “statistical model used allows for persistence in departures using an autoregressive process”.
If you are unfamiliar with trending autoregressive processes, that does not matter here. What is important is that HM Government recognized, in its Answer, that some statistical model must be specified. There is, however, still something missing: is their choice of statistical model reasonable? Might there be other, more likely, statistical models?
(There is also a minor ambiguity in the Answer, because there many types of autoregressive processes. The ambiguity is effectively resolved in a related Question, from 3 December 2012, which discussed “autoregressive (AR1) processes” [HL3706]; other Answers, discussed below, confirmed that the process was of the first order.)
I found out about the Question (HL3050) put by Lord Donoughue via the Bishop Hill post “Parliamentarians do statistical significance”. I then discussed the choice of statistical model with Lord Donoughue. I pointed out that there were other models that had a far greater likelihood than the trending autoregressive model used by the Answer. In other words, the basis for the Answer to the Question was untenable.
Moreover, I had published an op-ed piece discussing this, and related issues, in the Wall Street Journal, on 5 April 2011. The op-ed piece includes a technical supplement, which describes one other statistical model in particular: a driftless ARIMA(3,1,0) model (again, unfamiliarity with the model does not matter here). The supplement demonstrates that the likelihood of the driftless model is about 1000 times that of the trending autoregressive model. Thus the model used by HM Government should be rejected, in favor of the driftless model. With the driftless model, however, the rise in temperatures since 1880 is not significant. In other words, the correct Answer to the Question (HL3050) might be No.
Lord Donoughue then tabled a Parliamentary Question asking HM Government for their assessment of the likelihood of the trending autoregressive model relative to the driftless model. HM Government did not answer. Lord Donoughue asked a second time. They did not answer. He asked a third time. Again they did not answer. He then asked a fourth time.
A Parliamentary Question that has been tabled in the House of Lords is formally answered by HM Government as a whole. In practice, HM Government assigns the Question to a relevant ministry or department. In our case, the Questions have been assigned to the Department of Energy and Climate Change; the designated minister is the Parliamentary Under Secretary of State, Baroness Verma. Verma obtains answers from the Met Office. The person at the Met Office with final authority is the Chief Executive Officer, John Hirst. In practice, Hirst delegates authority to the Chief Scientist at the Met Office, Julia Slingo. Thus, it is actually Slingo who was refusing to answer the Parliamentary Questions, with Hirst and Verma backing her (perhaps without thinking).
I have had a few e-mail exchanges with Slingo in the past. Slingo has never really addressed the issues that I raised. Instead, she has replied largely with rhetoric and a display of gross ignorance about undergraduate-level statistics; for an example, see the Bishop Hill post “Climate correspondents”. Thus, I decided that trying to talk directly with Slingo about the Parliamentary Questions would be a waste of time. Hence, I tried talking with Hirst. My message to Hirst included the following.
Last week, Lord Donoughue tabled Parliamentary Question HL6132, about statistical models of global temperature data. HL6132 is essentially the same as HL5359, which the Met Office refused to answer. The Met Office Chief Scientist does not have the statistical skills required to answer the Question; there is, however, at least one scientist at the Met Office who does have the skills—Doug McNeall. I ask you to ensure that the Question is answered.
Doug McNeall is a statistician. He and I have had cordial e-mail discussions in the past. In particular, after my op-ed piece in WSJ appeared, on 12 August 2011, McNeall sent me an e-mail stating that the trending autoregressive model is “simply inadequate”. Indeed, that would be obvious to anyone who has studied statistical time series at the undergraduate level. Note that this implies that a statistician at the Met Office has stated that the Answer given to the original Parliamentary Question (HL3050) is unfounded.
Lord Donoughue’s fourth Question was, as before, refused an answer. Afterwards, I received the following message from Hirst.
I would like to assure you that the Met Office has not refused to answer any questions. The questions you refer to were answered by Baroness Verma, Parliamentary Under-Secretary of State at the Department of Energy and Climate Change.
I note that in her response to HL5359 and HL6132, and a number of other questions from Lord Donoughue, Baroness Verma has offered for him to meet officials to discuss this and related matters in more detail.
Afterwards, Lord Donoughue asked the question a fifth time. And I sent the following message to Hirst.
I do not know whether your message is serious or just your way of telling me to get lost. In case of the former, some elaboration follows.
The question that Lord Donoughue has been asking requires the calculation of a single number. The calculation is purely arithmetical: there is no opinion or judgment involved (nor is background in climate needed). Furthermore, the calculation is easy enough that it could be done in minutes, by someone with the appropriate statistical skills. You could think of it as being similar to finding the total of a column of integers.
The number that Lord Donoughue is asking for is 0.001, according to my calculation. (Yes, it is that simple.) Lord Donoughue, though, would like the number calculated by an official body. He therefore tabled Parliamentary Questions asking HM Government for the number.
Lord Donoughue has now received Written Answers to four such Parliamentary Questions: HL4414, HL5031, HL5359, HL6132. None of those Answers give the number. Instead, the Answers make excuses as to why the number is not given. The main excuse seems to be that the number is not important. The importance of the number, however, is a separate issue: even if the number has no importance at all, the arithmetical calculation can still be done, and the number can still be given.
HM Government has been relying upon the Met Office, to supply them with the number; the Met Office has refused to do this. In other words, the Met Office has refused to answer the question—contrary to the claim in your message. What reason does the Met Office have for refusing to supply the number? The required time would be less than the amount of time that the Met Office has spent in refusing.
Parliamentary Questions have a history going back centuries. I do not have expertise in this area, but it is my understanding that HM Government is obliged to either provide an Answer to a Question or else give a valid reason for not providing an Answer. The refusal of the Met Office to supply the number would thus seem to be leading to a violation of a centuries-old parliamentary convention. Indeed, I have now talked with other members of the House of Lords and the Commons about this: there is real concern, and apparently also by parliamentary officials.
Lord Donoughue has now asked for the number a fifth time. The tabled Question is as follows (HL6620).
To ask Her Majesty’s Government … whether they will ensure that their assessment of [the number] is published in the Official Report; and, if not, why not.
The Answer is due by April 12th. My hope is that if the Met Office continues to refuse to supply the number, HM Government will get the number from elsewhere.
There was no immediate response to that. I did, however, receive an invitation from Doug McNeall to visit the Met Office and discuss the statistics of trends in global temperatures. I replied as follows.
Kind thanks for this. In principle, such a meeting would surely be valuable. The Met Office, however, is refusing to answer a simple arithmetical question, and moreover, is presenting dishonest reasons for doing so. Given that, I do not have confidence that discussion could be in good faith.
Hence, I respectfully decline. If the Met Office supplies the number, I would be happy to discuss this further.
A week later, the fifth Question (HL6620) was answered as follows.
As indicated in a previous Written Answer given … to the noble Lord on 14 January 2013 (Official Report, col. WA110), it is the role of the scientific community to assess and decide between various methods for studying global temperature time series. It is also for the scientific community to publish the findings of such work, in the peer-reviewed scientific literature.
Thus, in the opinion of the Met Office, Parliament has no right to ask scientific questions of government scientists.
A few days later, I received the following message from Hirst.
I’m sorry for the delay in replying; I have been away from the office.
I’m sorry if my previous e-mail gave you the impression I did not wish to discuss this matter further. That was not my intention. Indeed, if you are not satisfied with the answers that have been given to Lord Donoughue’s Parliamentary Questions, I would be more than happy for us to debate your concerns, as part of a detailed scientific discussion about the statistical modelling of global mean temperatures.
I understand Doug McNeall has offered to arrange a meeting with you and other Met Office scientists who work in this area. I feel this would be a sensible way forward and, although our views may differ in some respects, can assure you we would approach this meeting in good faith.
I look forward to hearing from you.
Hirst is clearly supporting the obstructionism. I decided that there was no point in replying.
Under the rules of Parliament, the person with responsibility for a Parliamentary Question is the government minister who delivers the Answer. In our case, that minister is Baroness Verma. According to the Companion to the Standing Orders and Guide to the Proceedings of the House of Lords, §4.68 Ministerial Responsibility, “Ministers should be as open as possible with Parliament, refusing to provide information only when disclosure would not be in the public interest” and “Ministers who knowingly mislead Parliament will be expected to offer their resignation to the Prime Minister”.
Lord Donoughue then sent a strongly-worded letter to Under Secretary Verma, citing the section on Ministerial Responsibility, and adding “I trust we will not reach that point since you are clearly not behind the wilful refusal to answer the Question”. Indeed, Verma seems to have been trusting that the Answers supplied to her by the Met Office were written in good faith.
Then Lord Donoughue asked the question a sixth time (HL62). The Answer, this time, included the relative likelihood. The full Answer (excluding footnotes) was as follows.
There are many ways to analyse time series, including the use of physical and statistical models. The relevance of any technique depends on the question asked about the data. The Met Office has compared the likelihood of the two specified models for fitting the three main independent global near-surface temperature time series (originating from UK Met Office and NASA and NOAA in the US), using a standard approach.
The statistical comparison of the model fits shows the likelihood of a linear trend model with first-order autoregressive noise in representing the evolution of global annual average surface temperature anomalies since 1900, ranges from 0.08 (Met Office data) to 0.32 (NOAA data), relative to the fit for a driftless third-order autoregressive integrated model. The likelihood is 0.001 if the start date is extended back for example to 1850 (Met Office data). These findings demonstrate that this parameter is very sensitive to the data period chosen and to the dataset chosen for a given time period, for such a statistical model.
A high value of relative likelihood does not necessarily mean that a model is useful or relevant. The climate is a highly complex physical system; to model it requires an understanding of physical and chemical processes in the atmosphere and oceans, natural variability and external forcings, i.e. with physically-based models. Work undertaken at the Met Office on the detection of climate change from temperature observations is based on formal detection and attribution methods, using physical climate models and not purely statistical models, as discussed in Chapter 9 of the Contribution of Working Group I to the IPCC’s Fourth Assessment Report, 2007.
The second paragraph gives the relative likelihood of the trending autoregressive model with respect to the driftless model. The relative likelihood is 0.08, if we analyze years 1900–2012 , and it is 0.001, if we analyze years 1850–2012 (using Met Office data). In either case, then, the trending autoregressive model is much less likely than the driftless model to be the better model of the data. Hence, the statistical model that was relied upon in the Answer to the original Question (HL3050) is untenable.
Most of the third paragraph is verbiage. In particular, the cited “physical climate models”, which the Met Office runs on its supercomputer, do indeed provide some evidence for global warming. Physical climate models and statistical models are both known as “models”, but they are different things. It is only the statistical models that are relevant to the Question. The physical climate models, though impressive in many ways, do not provide observational evidence for global warming.
The issue here is the claim that “the temperature rise since about 1880 is statistically significant”, which was made by the Met Office in response to the original Question (HL3050). The basis for that claim has now been effectively acknowledged to be untenable. Possibly there is some other basis for the claim, but that seems extremely implausible: the claim does not seem to have any valid basis.
Plainly, then, the Met Office should now publicly withdraw the claim. That is, the Met Office should admit that the warming shown by the global-temperature record since 1880 (or indeed 1850) might be reasonably attributed to natural random variation. Additionally, the Met Office needs to reassess other claims that it has made about statistically significant climatic changes.
Lastly, it is not only the Met Office that has claimed that the increase in global temperatures is statistically significant: the IPCC has as well. Moreover, the IPCC used the same statistical model as the Met Office, in its most-recent Assessment Report (2007). The Assessment Report discusses the choice of model in Volume I, Appendix 3.A. The Appendix correctly acknowledges that, concerning statistical significance, “the results depend on the statistical model used”.
What justification does the Appendix give for choosing the trending autoregressive model? None. In other words, the model used by the IPCC is just adopted by proclamation. Science is supposed to be based on evidence and logic. The failure of the IPCC to present any evidence or logic to support its choice of model is a serious violation of basic scientific principles — indeed, it means that what the IPCC has done is not science.
To conclude, the primary basis for global-warming alarmism is unfounded. The Met Office has been making false claims about the significance of climatic changes to Parliament—as well as to the government, the media, and others — claims which have seriously affected both policies and opinions. When questioned about those claims in Parliament, the Met Office did everything feasible to avoid telling the truth.
Reader Comments (163)
Is Phil D. Jones our man at the Met? If so this may interest you
http://onlinelibrary.wiley.com/doi/10.1002/grl.50425/abstract
Now there’s an interesting line or two from the abstract -
We have ignored all air temperature observations and instead inferred them from observations of barometric pressure, sea surface temperature, and sea-ice concentration using a physically-based data assimilation system called the 20th Century Reanalysis. This independent dataset reproduces both annual variations and centennial trends in the temperature datasets, demonstrating the robustness of previous conclusions regarding global warming.
Hey, lets all run-off and infer our data indirectly because all that temperature measurement error stuff plays hell with the theory.
Lord Donoughue has left a long comment at WUWT. He says that “In 28 years in Parliament I do not recall such obfuscation”, as occurred here with the Met Office.
Additionally, he states that if anyone would like to suggest further Parliamentary Questions for him to table, he would welcome such.
Doug - thanks for highlighting that comment. Please can I suggest that Lord Donoughue tables Rhoda's unanswered question requesting the specific evidence for AGW?
Rhoda,
Thanks for an authentic answer. I appreciate it.
Now that feels like a bit of history. Excellent, Doug and Lord D.
I have been under the weather for a few days and have not been able to study this very carefully, nor read all the preceding comments, but I will do so as soon as I can. In the meantime, if it is what I think it is I think it is very important. First I note that the Met Office has not been able to refer to or produce out of its filing cabinets at least an internal report showing their analysis leading to claims of statistical significance - there seems to have been instead some kind of scurrying to placate the appliers of pressure via PQs. Second, if there is only weak observational evidence for believing that something extraordinary has been happening to such as global mean temperatures in recent decades, another leg is knocked off the stool upon which the preachers of alarm stand to harangue and persuade the gullible, the vulnerable, and the venal.
The Met Office response to Doug Keenan's post:
The last response from the Met Office seems somewhat lacking.
Because the Met Office does not make an assessment of global warming solely on statistics – let alone the statistical models referred to in Mr Keenan’s article, this exercise is of very little, if any, scientific use.
This is a shame as the 6 questions had a theme, is global temperature rising or falling.
It requires no understanding of the reasons for the numbers in the dataset to change, just a judgement on the change.
To find whether numbers in a dataset have a particular meaning, a number of methods have been developed by staticians over the years.
With data that is believed to contain long term trends hidden by 'noise', a subset of methods are recommended by 'experts' to tease out very small trends from a mass of 'noisy' data.
Mr Keenan suggests that literature puts forward the two methods concerned in the HOL set of questions.
It appears to me, that it is a simple mechanical task to process data by these methods and other methods, to acquire the answers that these methods can give.
It does the Met Office a disservice to suggest that these questions, relating to these methods, are not worthy of discussion.
Perhaps, only people who work at the Met Office, who can/try to 'see' other patterns in the data are the only ones who can undertake a 'scientific' discussion.
If you are seeking simple answers to movements within a set of data, is it not wise to confine your quest to just that data?
If you do not 'like' the answer you get, then by all means try other methods, but the data gives the answers it can give when operated upon by standard methods.
I must add that if the trend lines are so small as to be hidden within the noise, then is it reasonable to expect the trend to be anything usable?
Hi!
Been away for sometime and wandered by to see how ZED was doing. Nothing much changed, I see, but if there is Global Warming, I am thankful for it because without it, I would have totally froze me arse in Ireland this spring.
And the poor sheep were getting lost in the snow that the MET promised would never come!
Poor sheep -- Look at them shivering.
Sheep lost in Irish snow
Back to Florida for me!
The admission, if any, that the data do not show statistical significance is not any evidence - let alone strong evidence - that the warming, if real, is not anthropogenic. In other words, it is possible that the argument has been overplayed (though I doubt it), but do you want to be the one caught up in a war over diminishing food supplies as a result of saying - "prove it" to some climate scientists who, by their own admission, are dealing with an immensely complicated system?
I, for one, would rather not bury my head in the sand. After all, what have the scientists got to gain by supporting this?
Would now be a good time to formally ask the Royal Statistical Society for its comments?
The RSS has been very sensible so far, in avoiding any input into this politically dangerous topic. But there is a groundswell of changing opinion now - I wonder if they can be persuaded to nail their trousers to the mast?
http://upboardresult2014x.in/
The RSS has been very sensible so far, in avoiding any input into this politically dangerous topic. But there is a groundswell of changing opinion now - I wonder if they can be persuaded to nail their trousers to the mast?
http://www.friendshipday2014s.in/2014/07/friendship-day-sms-messages-2014_13.html and http://www.friendshipday2014s.in
Your app needs to be flexible. Otherwise, your app may acquire unnoticed among the massive quantities of competitive apps. A cell app may also be a valuable advertising tool on a broader scale. Building a cell app isn't practically coding. One other important step in creating a healthcare mobile app is testing.