Bishop Hill Climate correspondents
Jan 7, 2013 Climate: Statistics Climate: Surface Doug Keenan has followed up on his observations about the long-term rainfall records for England and Wales with an exchange of emails with Julia Slingo, the chief scientist at the Met Office. (Note that images can all be enlarged by clicking on them).
Dear Julia,
On November 12th, I sent an e-mail, in which you were Cc’d, about the statistical analysis of (observational) climatic data that has been done by the Met Office. My e-mail stated that some of the analysis is so incompetent that it “is not science”. It then asked if scientists at the Met Office had training in the relevant branch of statistics—i.e. in time series.
You did not reply to that e-mail. In consequence, on November 29th, Lord Donoughue put the following Question in the House of Lords.
To ask Her Majesty's Government how many scientists employed by the Met Office spend at least half of their working time conducting work closely related to global warming; how many of those scientists have taken an undergraduate-level course in statistical analysis of time series; and how many of those scientists have taken a graduate-level course in the statistical analysis of time series.[HL3705]
The Answer does not seem to have been published, but Lord Donoughue has told me that the Met Office claims that it has tens of scientists who have at least some training in the statistical analysis of time series. Specifically, the Answer claims that there are tens of scientists who understand that modeling autoregression is essential in such analysis. That is very good to learn. I find it difficult to believe, though, given the analyses previously produced by the Met Office.
Yesterday, the Met Office issued a news release about 2012 weather statistics:
http://www.metoffice.gov.uk/news/releases/archive/2013/2012-weather-statistics
The news release indicated that rainfall is increasing. It gained considerable attention, e.g. at the BBC and Channel 4.
The news release is largely based on a statistical analysis of rainfall data. Few details are given, but it is obvious that the analysis did not properly model autoregression. When proper modeling is done, I am confident that there will not be a significant increase in either rainfall or extreme rainfall events. Indeed that would be obvious to anyone knowledgeable in time series.
I ask that the Met Office address the statistical issue responsibly—specifically, via the following.
1. Promptly issue a news release stating that the statistical analyses that underpinned yesterday’s claims about increasing rainfall were invalid.
2. Implement procedures to ensure that news releases based on statistical analyses are vetted by someone competent in the relevant statistics.
Sincerely, Doug
Slingo's response was as follows:
Dear Doug,
Thank you for your email.
Yesterday’s announcements report on the UK’s observed rainfall records. What was announced was that provisional data show that 2012 was the second wettest year in the UK national record, along with preliminary research on the frequency of 1 in 100 day rainfall events over the last 50 years. A few points in particular stand out:
- 2012 is one of the wettest years on record (since 1910).
- Two months last year (April and June) were the wettest in a series back to 1766.
- Annual rainfall totals for UK are higher in recent years than during the 20th century.
- 4 of the 5 wettest years have been since 2000
- We have observed more wet days (>99th percentile) in recent years than a few decades ago.
These are basic conclusions from the observational record and make no attempt at defining trends, or attributing causes, not least as the observational record of daily rainfall records is not sufficiently long to derive robust statistics.
The widespread coverage of the observed data reflect the real impact that the rain has had on people and the nation’s infrastructure in recent years.
Regards,
Julia
Keenan has now responded in turn:
Dear Julia,
Your message cites “a series back to 1766”. I did not previously realize that the Met Office had precipitation data going back that far. Having such data is highly valuable, of course, and much facilitates analysis.
The cited series is not for the whole UK, but rather for England & Wales. That is fine though, especially as nine out of ten UK residents live in England and Wales—so this is what most people will be concerned about. I appreciate that data for the earlier years might be somewhat inaccurate. Using the annual data, though, should help to smooth out problems (i.e. the errors probably partially cancel each other out). Thus, this all seems to be fair. A plot of the annual data is below.
From the plot, it appears that we should not have concern about recent annual precipitation being overly high. Moreover, during the last decade, 2011 (787 mm) and 2003 (761 mm) are particularly low.
By omitting the above plot, and any mention of the information in the plot, the Met Office would seem to have misled the public and the media. Consider this: if a listed company reported on its financial position, and omitted data that was as material to its position as the above plot is to the state of precipitation, the people who were responsible would probably face prosecution.
The precipitation data seems to be a somewhat like (fractional) Brownian motion, which is a type of random process. Some plots that were generated using Brownian motion are appended, for comparison. I am not claiming that precipitation actually is Brownian, merely suggesting that if the Met Office wanted to do research on the precipitation data, that might be a place to start.
Your message also cites some more recent observational data and indicates that such data supports the suggestion of significant increases. That is not possible, even in principle. In statistical analysis, inferences are not drawn directly from data. Rather, a statistical model is fit to the data, and inferences are drawn from the model. We sometimes see statements such as “the data are significantly increasing”, but this is loose phrasing. Strictly, data cannot be significantly increasing, only the trend (etc.) in a statistical model can be.
If we use a Brownian process as the model for the annual data, then there is no significant increase. Perhaps some non-Brownian models would be better, however, and would support an increase. If so, then it is the responsibility of the Met Office to select models to substantiate that. The Met Office, however, apparently has nothing to substantiate that. It has not even selected any model. And the same is true for the daily data.
It is because the Met Office has repeatedly had such serious failures in its statistical analyses that Lord Donoughue put his question in the House of Lords. In answer to the question, the Met Office apparently assured Lord Donoughue that it does have the requisite statistical expertise. Yet now, it issues a news release based on work that is so incompetent that if the work were submitted in an introductory (undergraduate) course, the student would flunk.
The answer to the question put by Lord Donoughue specifically acknowledged that modeling autoregression is essential. Did the “preliminary” research by the Met Office consider autoregression? No. Is it plausible that research that does consider autoregression will support an increase in precipitation? No. Was the Met Office aware that there is no valid evidence for an increase when it issued its news release? Either it knew, and thus acted fraudulently in issuing the release, or it was misleading when it told Lord Donoughue that it has the requisite expertise. (I suspect the latter.)
Taxpayers pay for the Met Office to provide expertise with climatic analysis. They are not getting anything close to what they pay for.
Sincerely, Doug
Sample plots generated via Brownian motion (the variance and dimension are as per the precipitation data):
;)
;)
;)
;)
Reader Comments (69)
Has the future Dame Slingo any knowledge of statistics, let alone Brownian motion?
Very well done, Doug Keenan. I hope you will also make public any further response from Julia Slingo or indeed other Met Office scientist. This is precisely the kind of question that the Met Office should be prepared to answer.
Hell, dealing with time series is so easy that even economists can do it. Shame on the met office!
As far as I am aware, Julia Slingo is of the same crowd who were giving dire warnings of an impending drought following the heat-wave of March last year (2012). (Who can forget the doom-merchants on the BBC uttering comments along the lines of: "... this rain will just dry up or be absorbed by plants..." as the first of the rain following that warm spell pitter-pattered down?) Can she not see any irony in what she says?
"i.e. the errors probably partially cancel each other out"
No they will all be adjusted in the direction of alarmism***. Gauges will have been reading too high in the earlier parts of the series and too low in the later years.
*** a well known statistical technique used widely in climate science, especially with temperature data.
Thank you Doug Keenan for all your efforts. What an appalling situation!
How long can this charade continue?
Has the Met Office produced any coherent rationale for its apparently arbitrary chopping and changing between the 1766 series and the 1910 series? Given 2012's ranking in the 1766 series, and the clusters of wet years that occurred from time to time in the 18th and 19th centuries, as well as in the 1950s and 1960s, does La Slingo really believe that there is anything unusual about recent weather patterns?
"We have observed more wet days (>99th percentile) in recent years than a few decades ago."
Now here's the thing: if you check the daily rainfall figures for the English regions the smallest recorded rainfall for any day is 0.01mm from January 1997 onwards but 0.1mm before that. So no surprise if we see "more wet days in recent years". Surprisingly though the smallest rainfall recorded for "England and Wales" is 0.01mm. I wonder where the extra precision came from when the regions were aggregated.
This should keep amatuer weathermenand statisticians busy for the unforeseable future
"Hello, this is Julia. Tell the PM - we're going to need a bigger computer."
I must presume that Julia Slingo has absolutely no inkling that every utterance she makes, lands her deeper in Brownian motion(s). This lady has absolutely zero credibility.
The principle of generating small amounts of finite improbability by simply hooking the logic circuits of the new Met Office computer to an atomic vector plotter suspended in a strong Brownian Motion producer (say a nice hot cup of tea) were of course well understood.
With apologies to Douglas Adams.
Brilliant work Doug. And, if I may say so, your tone is also spot on. ( I'm widely known to be the expert on that!)
A little statistics is a dangerous thing! Especially with modern stats software being so easy to use. How many statisticians are there would also be of interest.
Julia@the MetOffice: "The widespread coverage of the observed data reflect the real impact that the rain has had on people and the nation’s infrastructure in recent years."
What I presume this means is that the Met Office anecdotal analysis of the rainfall record has been reflected back by more anecdotal observations by the media. No harm in stoking the fire and getting more heat if it is all in a good cause.
I like this. An example that I can explain to people which clearly shows how relevant data can be mid-interpreted or used to mislead. I hope a few people will realise they have been tricked, and will react accordingly.
'are there' in the Met Office that is (fix for 5:42PM comment)
While I have the stage, if they do have them there, it would be of interest to hear if they try to replicate the recently published co-integration study which confirmed, once again, the difficulty of finding anything meteorological linked to rising CO2 levels - in this case the notional global mean temperature. Some discussion, later de-railed by some feuding, here: http://wattsupwiththat.com/2013/01/03/agw-bombshell-a-new-paper-shows-statistical-tests-for-global-warming-fails-to-find-statistically-significantly-anthropogenic-forcing/
Perhaps the more important statistic followed by the Met Office is the hiring of women and previously disadvantaged cultural and minority groups in order to have a workforce that better reflects the population.
To save me the trouble of finding it and (more importantly) to ensure I don't trip up over my own ingnorance, can someone point me to an internet resource which gives an idiot's guide to time series analysis? Preferably starting simply and pointing out in words of half a syllable why it is important.
Most of the things I've found have started complicated and got worse! Often far worse!!
Maybe it will help those at the Met Office as well.
"Lord Donoughue has told me that the Met Office claims that it has tens of scientists who have at least some training in the statistical analysis of time series. "
"Some tens" - that suggests 20? 30? 40? Met Office staff who have "at least some" training in the statistical analysis of time series. How many staff do they have in total? 1500?
What is the Met Office's core business, if not the statistical analysis of time series? [and prediction of time series, for which analysis is a pre-requsite].
Close it down and replace it with 30 - 40 competent meterologists plus an appropriate support staff.
Julia Slingo should be reminded the Met Office is part of the Governments Department for Business, Innovation and Skills.
and that there is a criminal offence of Misconduct in Public Office
The elements of misconduct in public office are:
A public officer acting as such.
a) Wilfully neglects to perform his duty and/or wilfully misconducts himself.
b) To such a degree as to amount to an abuse of the public's trust in the office holder.
c) Without reasonable excuse or justification.
I'm sure that is a "formula answer" and not a real one - for a start these people rarely reply to the "general public" and secondly the wording is more or less a match for her "easy-go" interview on 5-live last week with the increasingly dumb Ms Derbyshire.
I wonder if they use seperate internal email systems too...
Alan Bates (7:10 PM). From my later years in industrial stats. I found this a good site to refer clients to if they wanted to dig more deeply into statistical methods: http://www.itl.nist.gov/div898/handbook/index.htm . Good, clear explanations (albeit brief) and lots of illustrative data sets. You can get to the main section on time series if you click on 'Monitor'. But be warned, unless you have a specific problem in mind, one that is important for you, it is generally hard going to wade through stats materials in the hope of finding some general enlightenment! Another approach is to Google 'statistical methods tutorials for X', where X is a topic or field with which you are familiar.
Sadly Lord Donoughue did not ask for evidence in his question. I guess you must not do that in the hallowed hall of the british parliament. Oh no no mulud
Brilliant work Doug. And, if I may say so, your tone is also spot on. ( I'm widely known to be the expert on that!)
Jan 7, 2013 at 5:35 PM | Richard Drake
I thought I was. Ask the Bish. :)
If I was a scientist working at the Met Office I would be utterly ashamed. Surely there are some decent folk in the organisation who will speak out about this.
Martin A:
My, you're softening Martin, compared to the rightful fate of the BBC :)
Stephen Richards: so your tone is more expert than mine? NO IT F***ING AIN'T, SUNSHINE.
I jest in both cases, in case any callow youth should stumble on these words.
I studied statistics as part of a subsidiary maths course while taking a BSc in chemistry at university 40 years ago but my knowledge of it now is so rusty that anything more complicated that basic probability, and calculation of the mean, mode and median would be beyond me. Perhaps on a good day I could cope with standard deviations.
It would not surprise me if the Met Office does employ tens of people who studied statistics while specialising in other subjects, but that would be a fat lot of use if they have forgotten as much as I have. What matters is how many of them are competent to carry out statistical analysis now.
More does not mean increasing
Higher does not mean increasing
Lower does not mean decreasing
Less does not mean decreasing
As Doug Keenan points out so skillfully, a trend line is established through a time series statistical analysis; trends so derived can illustrate increasing/decreasing slopes over time.
Also identified by Doug:
Observation of specific datums are not trends!
Well done Doug! Hold them to their job skill requirements!
Sadly they had but a cloudful of dots
NOAA and Slingo said make us some plots
We need a soundbite for suckers and clots.
Duly they laboured like slaves and robots
Cranked all the handles - filled all the slots
Finally found how to hit the jackpots
Till they fell foul of Doug Keenan and Watts
Anthony Watts:- 'So for the seven months between April and December, that forecasts are available for, the Met Office forecast drier than normal conditions in six, and normal in the seventh. They failed to get any month correct, and for the seven months in question, rainfall averaged 36% above normal levels, (which are based on 1981-2010.) It is very kind of Julia to tell us now that she knew all along it was likely to be wetter. It is just a pity, though, that she forgot to tell us at the time.'
http://wattsupwiththat.com/2013/01/06/slingo-pretends-she-knows-why-its-been-so-wet/#more-76993
Pharos: Great quote but those are the words of Paul Homewood, not Anthony, in a guest post. (And I may have ruined the rhyme of your final line. But Watts hosted it, after all.)
John Shade 7:35 pm
Many thanks for the suggestions. I am following them up.
Richard Drake
Thanks for correcting the quote. But I think Anthony did all the leg work for the post on the dodgy NOAA data?
http://wattsupwiththat.com/2013/01/06/does-noaas-national-climatic-data-center-ncdc-keep-two-separate-sets-of-climate-books-for-the-usa/
Pharos: Yeah, good point, the poem was about NOAA and the Met. My objection is withdrawn. It rhymes, it scans and now it's peer reviewed!
Well done Doug; sure glad I'm not paying British taxes....
This comes to mind:
http://okfirst.mesonet.org/train/casestudies/18aug97/return.html
Any hydrologists lurking?
In my experience, in successful businesses the key competencies are held by the leadership team, not just by juniors. Thus, a sales-oriented organization needs to be led by a red hot salesperson, an engineering company by a very good engineer, and so on.
In the present case, it seems that knowledge of statistical analysis and modelling, which is at the very core of any understanding of climate, is not held by Ms. Slingo. And surely, if it were held by one of her management colleagues, she would not dream of writing such tosh without checking with him or her.
It would be interesting to test this hypothesis with a follow up question requesting the seniority of the Met Office's alleged statisticians, and, conversely, the skill sets of its management.
FWIW, this lack of competent oversight is apparent in the Climategate emails and the Hockey Stick saga.
As my old father says, we live in the age of incompetence.
Of course Julia Slingo has had a staring role at WUWT before
http://wattsupwiththat.com/2009/06/20/the-met-office-brings-doom-to-a-place-near-you/
This is Post-Normal Science™.
Only Post-Normal Science™ considers how people feel about the rain. I detect the influence of Hulme and Ravetz.
Thomas Gibbon
It would be interesting to test this hypothesis with a follow up question requesting the seniority of the Met Office's alleged statisticians, and, conversely, the skill sets of its management.
I'm a long-time lurker and very intermittent poster and you may already be aware of this, but the met office has a website that lists a bunch of their scientists and gives a brief description of their area of expertise.
http://www.metoffice.gov.uk/research/our-scientists
I don't have the knowledge to assess their capabilities, a review of some of them may point you in the right direction.
The impact on people of rain? That would be, they get wet, if they happen to be out in it. At least, that's what happens when raindrops 'impact' on me.
The sloppy language (I know, I know, but 'impact' is a pet peeve) is symptomatic of a bunch of people who have had their assumptions unchallenged for too long. According to Ms Slingo, there are only two things that matter - public infrastructure and 'people'. I would have been sent back to write it again with a stern warning about lifting my game if I had proffered such rubbish when I was a public servant.
All this is a great example of what happens when evidence based policy making gets overtaken by policy based evidence making.
Those in charge (until very recently including Robert Napier from WWF UK) are responsible for this lamentable situation.
Well done Doug! Keep at them, as you say, it is tax payers' money.
And then the Met Office has the temerity to campaign for a new bigger, faster and stunningly expensive computer system. From what you have shown here, why should we? The basics of science and statistics is obviously misunderstood at this office and such increased spending I feel would be a wasted of money. As far as I can see such a computer system would give them an excuse to put out more stories such as these quicker and to a higher degree of imprecision.
The BBC Today (6am 8-7-13) programm has reported today that the Met Office has cancalled global warming until 2017.
Apparently its all down to the SUN refusing following Julia Slingo's computer models
I first heard of Doug Keenan when I saw him at the Grauniad Climategate debate. And my takeaway message from that was that it would be very unwise to tangle with him on one of his specialist subjects.
The contrast between his incisive and confident delivery and the bumbling AGW stooges - Bob Watson and Trevor Davies - was marked.
Slingo should watch her step if DK is on her case.
ssat
That's exactly what came into my head when I read "Brownian motion". I expect Ms Slingo will be needing plenty of Brownian motion generator if she continues the correspondence with Doug.
“The widespread coverage of the observed data reflect the real impact that the rain has had on people and the nation’s infrastructure in recent years.”
Maybe she also has some tens of minions who could correct her grammar.
Or was this a mishtake in the transcribing?
Dr. Slingo is an unashamed alarmist, but we have to take on board that she is indeed alarmed and wants to share that alarm with the "complacent". However, she is now deceiving the public to keep them alarmed. That is a very serious matter. For years the Met Office has warned of rising global tempertures but when they, belatedley, realised the temperatures had, to all intents and purposes, stopped rising, they unashamedly started to use "the warmest year on record," "the five warmest years on record," simultaneously keeping the alarmism up, while hiding from the public the fact that temperatures, despite their forecasts, had stopped rising.
Now after a year in which they forecast drought for 11 consecutive months and rain in January, followed by a January that had some 50% of its average rain, instead of knuckling down and trying to understand why an organisation whose primary task is to forecast the weather had so lamentaby failed its Chief Scientist goes on record telling us that more and fiercer rain storms can be expected. Awed by her title, or in thrall to the alarmist meme, no journalist has asked her why, if they expected more and fiercer rainstorms had they just forecast 11 months of drought.
I'm with Martin A. ( I usually am, it's a safe place to be) it's research arm should be closed down so that it can focus on forecasting the weather. The obsession with global warming and climate research is distracting it from its important role of forecasting the weather. On second thoughts it might be better to close it down altogether and rely on weather.com whose forecasting skills are, beyond doubt, streets ahead of the Met Office, I'm ashamed to say. Sorry Richard Betts you, and I'm sure, many more of your colleagues don't deserve it, but politics and science don't mix well together as we can see from the lamentable performance of the Met Office forecasts over the last decade. Forecasts that appear to have been merely stunts to promote CAGW.