Bishop Hill Information Commissioner on academic data
Feb 7, 2012 FOI Just in time to distract from my blushes (see previous post), comes the news that the Information Commissioner has spoken out on the efforts by academics to get themselves exempted from FOI laws.
"Is academic research really threatened by the prospect of premature release of data sets? Are ministers living in fear?" Graham asked a government computing conference last week. "The Chicken Licken version of the FOI that the sky is falling is just that: it's a folktale – and the trouble with folktales is people start reacting to what the think is the case even when it isn't."
He didn't name names, but two particular chickens spring to mind: chiefs in Whitehall and academics such as those whose work has helped give climate research a bad name.
Reader Comments (47)
I don't understand this part
I really don't think anybody in any part of the world should be able to use the FOIA and this includes Steve McIntyre, although there's no reason why SMc can't ask willing UK subjects to ask for him ;)
UK subjects should most definitely exercise their rights under the FOIA, it's a pity that certain "ctrl c / ctrl v" journalists don't use it to check the fodder they're being fed
I had 3 FOI requests refused the other day. Not that they didn't have the information. They just didn't want their dirty washing in public.
So they simply made them a state secret.
You aren't allowed to know what the house of lords told the Peers who have been fiddling expenses because it would interfere with their rights and privileges.
Interesting Nick. Did you make you reveal your full name on doing the FoI requests? What kind of evidence do you think it is for us when you don't do the same here?
OT - but the bit I've emboldened made me scream!
Link: http://www.independent.co.uk/news/science/science-behind-the-big-freeze-is-climate-change-bringing-the-arctic-to-europe-6358928.html
Yep. Full name. No problem there.
It's an absolute exemption. You can't appeal it. Non elected civil servant declares that its a state secret what crimes peers have been committing.
What I'm thinking of doing is going after lots of emails about the request under the law on personal data. I suspect they have lets lot slip.
Chicken Little perhaps?
There is a strong possibility here that the ICO is simply putting in a public appeal to keep its' position (and employment)
It does not appear to take its' responsibilities as thoroughly perhaps as one may wish
See Don Keiller's comments from Feb 6, 2012 1:16pm here:
http://climateaudit.org/2012/02/06/acton-tricks-the-ico/#comments
@steveta_uk
There are several problems in the paragraph. You've highlighted just one of them: thinking that a computer model can verify a hypothesis.
One of the other problems is deciding what is cause and what is effect in weather. "The weak winds cause the high pressure". Maybe it's the other way round: "The high pressure causes the weak winds".
The whole subject is full of "umbrellas cause wet streets" thinking.
(Apologies to Michael Crichton who used the "wet streets cause rain" analogy in his Gell-Mann Amnesia essay)
Feb 7, 2012 at 3:56 PM | steveta_uk
I can see why you object to the word "verified", but that aside, can you suggest a better way to explain the causes of particular weather events? You can't do controlled experiments on the Earth, but numerical models provide the next best thing - by varying different aspects of the atmospheric system in the model, you can see what the model suggests should be the consequence of this. And since the models have been able to reproduce the kind of thing Adam talks about when doing weather forecasts (i.e. predicting in advance how pressure systems and other aspects of the atmospheric circulation will behave) then this suggests the linkages between things like upper level winds and surface pressure are represented fairly realistically.
Richard - in weather models, are the linkages from upper level windspeed to surface pressures derived from first principles or are they tuned to take account of empirical observations?
Also re: validation here is a copy paste of a comment I made on the Protomodels thread in response to your comment of Feb 1, 2012 at 2:03 PM. I'm not sure if you saw it and I'd welcome your comments:
Richard - re: "However the evidence suggests that a reasonable representation of the long-term statistics is achievable (eg: the long-term mean and, to a lesser extent, the statistics of the variability)."
What is your "evidence" here? From blogs that have caught my eye I have the impression models are overestimating the long term temperature trend:
http://rankexploits.com/musings/2009/models-over-predict-using-another-version-of-a-long-trend/
http://judithcurry.com/2011/09/18/climate-models-as-ink-blots-2/
And the conclusions of Koutsoyiannis et al are that the models are useless for long term prediction of temperature and precipitation:
http://itia.ntua.gr/getfile/850/2/documents/2008EGU_ClimatePredictionPr_.pdf
Feb 1, 2012 at 3:31 PM | Unregistered Commenternot banned yet
Even Napoleon should have known better in 1812. Before 'climate change' and 'climate models' all you really needed was a standard text book.
Stuff like this.
'Regions of the Earth that have continental climates include Siberia and central Russia, and much of North America. Siberia, Canada and the northern states of the US in particular can exhibit very large differences between summertime and wintertime average temperature of up to 40°C. This compares to the more maritime climate of the UK, where the annual average temperature range is only 10°C.
Air masses that originate from continental interiors sometimes influence continental fringes that usually experience maritime climates. In the UK, continental polar air in winter is very cold and temperatures associated with this air stream are usually well below average.
Air masses that originate from continental interiors sometimes influence continental fringes that usually experience maritime climates. In the UK, continental polar air in winter is very cold and temperatures associated with this air stream are usually well below average. The air mass is basically very dry and stable but a track over the central part of the North Sea supplies sufficient heat and moisture to cause showers, often in the form of snow, over eastern England and Scotland. During summer, the airflow is usually warmer, since even northern parts of Europe experience high temperatures during this time of year.'
Richard
It's not just "verified" that's the issue, calling model runs "experiments" sticks in my throat even more. I'm not saying the weather models are without use, but regardless of how you use them, the authors of this piece clearly felt they were doing an experiment which proved something.
They weren't.
Feb 7, 2012 at 9:47 PM | not banned yet
Hi, thanks for responding, and sorry I missed your earlier question!
The linkages emerge as a consequence of solving the model equations of large-scale fluid dynamics and thermodynamics, which are based on existing understanding of these processes at smaller scales - but they do require some approximations to be made to represent the overall effects of other small-scale processes such as cloud microphysics. So, I guess my answer to your question is "a bit of both". Seems to work though - the models can successfully forecast things like the positions of the jet streams in advance, and the movements of fronts, depressions and high pressure systems.
Just to give you more of an idea of the kind of thing this involves, a fairly random example of a paper on general circulation modelling is here. (NB This particular paper is not chosen for any particular reason other than to illustrate a bit better what I actually mean - it was just the first vaguely relevant paper I could find, but I hope it gives you some idea of how these things are done).
In response to your earlier question, I know it may be a bit of a red rag to a bull to cite AR5 here, but I do think the chapter on model evaluation is worth reading, along with references therein.
Hope this helps!
Cheers
Richard
Peter, in a word, you mean GIGO !
Sorry Bish, off topic but a must read:
http://www.independent.ie/opinion/columnists/kevin-myers/kevin-myers-energy-policy-based-on-renewables-will-win-hearts-but-wont-protect-their-owners-from-frostbite-and-death-due-to-exposure-3012098.html
It's the Irish Independent but the Independent nevertheless.
Feb 7, 2012 at 9:51 PM | Peter Wilson
Sure - even climate modellers argue amongst themselves about the use of the word "experiments".
It probably would have been better to say "A number of simulations with computer models have suggested that this leads to high pressure across northern Europe and cold winter conditions in the UK as we see now."
"It probably would have been better to say "A number of simulations with computer models have suggested ..."
It would have been altogether more accurate and honest.
Richard Betts
Are they the same models that lead to the 2007 IPCC report saying that the Northern Hemisphere would have increasingly mild winters?
I worked with a few weather forecasters who would tell you four different stories of how the weather would turn out the following day, and when one of them was close to right - would say "I told you it would be like this!!!!"
Modellers have always had a weird view of how much skill their models have, even the ones where you could verify them a few days later. Until the predicted hotspot appears, climate models cannot demonstrate a more predictive outcome. "We can't get the basic response right - but we can forecast how the pressure patterns will change" - erm. I have come across a few financial advisers with the same delusional view.
Richard - there's the problem right there. The models appear to work over relatively short time periods i.e. when all other natural forcings (i.e. those not yet incorporated into the model) remain relatively static. But the fact that the models tend to drift away from reality over longer periods (and here we must exclude hind-casting unless you can find a way to convince independent reviewers that historic temperatures have never been adjusted and smoothed in order to align more neatly with modelled expectations - and CG emails would tend to suggest otherwise) indicates to me that material variables have been omitted from the model.
Can anyone point me at a reference which describes how the models are defined as predicting the weather successfully and to what degree of accuracy?
It seems to me that this isn't a simple problem.
Richard, you probably didn't read Professor M.J. Kelly's report for Oxburgh's interior decorating team. Well he said, and I paraphrase becasue it's from memory " I wish people would stop calling the output of model runs "experiments". So do I, because they're not, experiments are the implementation and observation of real world phenomena. Model runs are based upon equations put into the model that people believe replicate the real world phenomena. You will notice that "people believe" comes between the the input to the model and real world phenomena. As an, admittedley poor, engineer, I can vouch for the fact that putting people's beliefs before observation is a road to failure.
Jack Hughes (of floating duck fame): "(Apologies to Michael Crichton who used the "wet streets cause rain" analogy in his Gell-Mann Amnesia essay)"
Surely no "apology" needed here, Jack?
A "hat-tip" or other acknowledgement seems more appropriate as I am sure if Michael Crichton were alive to see such good use made of his analogy he would be rather pleased; and satisfied.
I will not apologise to you for good use of the Hughes-Josh duck...
Richard Betts - you also must have missed my question (page 2 of the Climate Authorities thread):
==================================================
Hi Tamsin, thanks for popping in, I know you are busy. I don't know if you saw it but there was what Pharos described as a landmark post yesterday evening, on page two of the Leake on the Temperature plateau thread, by Dr Norman Page (Feb 5, 2012 at 8:01 PM).
When you can find the time I (and I am sure others) would appreciate your response to this, thanks. Richard Betts likewise, and ideally Michel (if he is willing). Thanks.
Feb 6, 2012 at 10:13 AM | Unregistered Commenter lapogus
============================================
(I don't think Tamsin responded either. Apologies if you have responded somewhere and I have missed it).
Feb 7, 2012 at 11:46 PM | Retired Dave
Yes - but on average and further into the future (ie: several decades). As discussed on the POST thread, and explained very nicely by Ed Hawkins, natural variability is more important in the nearer-term.
Feb 8, 2012 at 2:36 AM | geronimo
I completely agree with you. However, what we are talking about here is using the models to explain the observations. The observations by themselves don't actually tell you anything about cause and effect, and we all know that simply using statistical correlations doesn't tell you anything about cause and effect either (the famous "correlation is not causation").
What Adam Scaife was describing was using the models as an expression of our understanding of atmospheric processes in order to propose an explanation for the link between one set of observations (upper level winds) and another set of observations (cold UK winters). Nobody was talking about using the models instead of observations, that would clearly be bonkers - it's about using both things together in order to both see what is going on in the real world and try to explain it.
@richard betts
I'm a lot happier with 'simulations' as the output from models than I am with 'experiments'. To use 'experiments' is just plain lying.
Richard Betts 'several decades' in other words we will be right tomorrow given that tomorrow will always be in the future. I have to say that type of approach in science would be probable be reject, given it offers no realistic ability to be disproved, if it was used to support an undergraduates assessment. Why should it accepted from professional scientists in order to achieve major political changes at massive costs ?
I am constantly amazed that lots of academics working in this area ,work at a standard that they themselves would never accept from their own students . Are we really saying that the ‘best’ science in the area is of a standard lower than a science undergraduates essay. Is it really to much to expect those claiming to be the best scientists in the area , to have standards that actual match that claim?
Richard just one request.
Point me to the model, or models, that prior to 2000, forecast/predicted that global temperatures would show no significant increase by 2012. I will need references from that time to confirm.
I am not interested in any model that can reproduce this hiatus with the benefit of hindsight.
Richard Betts.
I agree entirely with KnR, which saves me a lot of typing - thank you.
I could accept the excuse in your last comment (for that is what it is) if we hadn't been fed "certainty" of frying for the last 25 years, which just slowly morphs into something else as predictions fail - climate change, climate disruption, now freezing cold. None of this proves that AGW doesn't exist - it just proves modellers have no idea of the future and are slow to grasp this.
Of course the pattern of ice in the Arctic has an effect on weather, but it has all happened before when CO2 was a lot less than today. I am getting a little tired of the constant shift and ever widening list of terrible things that plant food can do.
It is at least 25 years since a good physicist told me that while the absorption of CO2 was well known, it was likely to heavily suffer from the Law of Diminishing Returns. I see some are still saying it - from The Hockey Shtick today (also linked in WUWT) -
http://hockeyschtick.blogspot.com/2012/02/new-paper-supports-miskolczis-theory-of.html
When climate models demonstrate even some (any) skill then good scientists will take note, until then scepticism (not denial - that's for religious believers in AGW) will continue to be the position of an increasing number of people.
Richard Betts - thanks for the response and the links. I'll make this the last one on this thread as we will be heading way OT...I'll check the Protomodels thread in case you want to pick up there.
The paper you link to looks interesting and it'll take more than a quick skim for me to get to grips with it, but it doesn't appear to deal with the type of criticism I was thinking of - namely the need for models to be pinned and tuned to real world measures in order to prevent them from chronic divergence problems. The type of posts that I'm thinking of are those re: Gerald Browning's criticisms at CA. The paper you link looks to be a description of a dynamical core approach which is evaluated using the Held-Suarez test. As I understand it this test would not show up the type of issues Gerald was referring to but I'm not an expert.
As far as model evaluations go, I note you didn't respond with specific evidence in support of your claim on long term prediction accuracy, nor did you offer comment on Koutsoyiannis et al. I'll have a look at the AR4 (unless you really have an AR5 already?!) reference you provided - it is not necessarily red rag to a bull. However I do recall following a lengthy and detailed model evaluation thread at the AirVent sometime ago which had many knowledgeable contributors where, IMO, it became obvious that "ensemble evaluation" was of questionable technical merit. Some time ago at BH I posted links to NAFEMS who serve the engineering modelling community with much technical guidance including benchmarking methodologies - I don't know of an engineering situation where ensemble evaluation is used (although I did find some references to it wrt financial modelling). In engineering, models have to deliver real world value and, from what I have seen, my opinion is that GCMs do not.
http://www.nafems.org/about/
Hi Don
To make that specific forecast would probably have required a model that was capable of simulating the year-to-year details of natural internal variability over more than a decade, which wasn't possible then (and still isn't).
However, I think the last decade has been within the range of possibilities simulated by the models - they do include natural variability, and even though the specific details are not expected to be correct date-for-date (due to chaos) the long-term statistics are reasonable.
I think the problem is that an own-goal has been scored in the communication of climate model projections. They are normally presented with the results smoothed over time, or as decadal means, for clarity of presentation. This has meant that it was not at all obvious that natural variability is important in the the shorter term. (doh!)
Retired Dave (11.46pm) - well spotted, a nice example of climate science doublethink. Here's what IPCC AR4 says in chapter 11.
And here is the Independent article:
In fact if you check what the Petoukhov paper says (Rahmstorf is definitely on the list of climate scientists whose claims need to be checked very carefully) it's not a 'prediction' at all. It was, as usual, a 'postdiction', responding to a cold winter 2005/6 with a cooked-up simulation to fit the observations.
@Retired Dave:
'Of course the pattern of ice in the Arctic has an effect on weather, but it has all happened before when CO2 was a lot less than today.'
--------------------------------------------------------------
Yes, and just to complicate things further, the weather has an effect on the pattern of ice in the Arctic.
Complex dynamic systems such as weather and the economy manage to defeat modellers even over very short timeframes. The models are getting better, but very, very slowly. Nobody in their right mind would stake their personal assets on the forecasts of a medium or long term economic model, and even very short term forecasts can be way off. Similar reservations should apply to climate models.
Having some experience of economic modelling in an academic setting, any undergraduate student who described a model as an experiment, or its output as a prediction (the correct term is forecast) would be firmly counselled to review the ABC of what modelling is and what it isn't.
Richard, I agree that "the last decade has been within the range of possibilities simulated by the models". Would you agree that it has been well towards the bottom of that "range of possibilities"?
Richard, that was a rather long-winded way to say "no".
Just what do you mean by "short-term variability"?
2, 4, 5, 10, 15, 20 years?
I seem to remember that until the latest bout of "revision" the modelers were telling us that warming from increased [CO2] had "overridden "natural variability".
Now we are being fed the line that "natural variability is greater than we thought" and it is "masking the underlying warming trend".
It just doesn't wash.
Feb 8, 2012 at 12:13 PM | Jonathan Jones
Yes.
Feb 8, 2012 at 2:02 PM | Don Keiller
Anything from a year to about a decade or so. In climate projections we conventionally use 30-year means when looking at changes due to external forcing.
Feb 8, 2012 at 3:15 PM | Richard Betts
So the interesting question is when we should start to wonder whether (1) the model ensemble is too hot, or (2) there is some other major aspect of climate which we haven't understood, or (3) something else broadly equivalent.
We're not yet at the point where those are the only plausible conclusions, but we're getting close.
Richard Betts
Well, it is interesting to see the 30-year periods, but instead of means, shouldn't you be looking for maxima and minima? What your appear to be looking at is the Pacific Decadal Oscillation (PDO) which has a half cycle of 30 years, and being cyclic, the mean is basically meaningless because that assumes a linear trend. Sinusoidal curve is clearly a more appropriate choice.
Richard, thanks for your courteous reply.
However, you define short term variability as "about a decade or so".
Would you agree that we are already in the "or so" aspect?
Also you are non-responsive to my comment about "revisionism".
With £millions, or even £billions, spent on models and the computers to run them, we do not
appear to be getting value for money.
Tell me, as a taxpayer, why I should continue this subsidy?
In Ed's paper, natural variability is based on results for the period 1950-2000 from an ensemble of models. The results are weighted based on the generating model's ability to simulate the warming from 1971-2000. For each model, the 1950-2000 results are fitted using a polynomial. An estimate of natural variability is then extracted by subtracting the fit to obtain a residual and calculating the variance of the residual. Their resulting estimate over this period is 0.13 K.
They proceed by assuming that this magnitude of internal variability will hold over a period of at least 100 years (their figure 3). As they say, "...the contribution from internal variability falls very rapidly with lead time as the signal of climate change strengthens while the amplitude of internal variability remains constant".
The question mark I have with this reasoning is the assumption that natural variability remains constant over this kind of time scale, because the assumption contradicts the findings of several other research groups. For example, please refer to this paper from the McGill group. From an analysis of observational data, they identify:-
1/ A weather regime for times up to 5-20 days, during which fluctuations increase with time scale,
2/ A subsequent low frequency weather regime for times up to 10-100 years, during which fluctuations decrease with time scale, and
3/ A subsequent climate regime, during which fluctuations again increase with time scale.
Although the low frequency weather regime can be identified with the natural variability described in Ed's paper, for the longer time-scales seen in their figure 3 (10-100 years), the natural variability is likely to again increase, thereby undermining their assumption.
It is probably also relevant that GCM simulations described in the same McGill group paper were unable to reproduce the observed climate regime scaling (i.e. > 10-100 years) in the absense of external forcings.
This has just been posed up at "Watts up with That".
Puts the skids under +ve feedback "mechanism" and confirms why Professor Mike Lockwood was reduced to arm-waving at the Cambridge Climate Conference last year when I asked him to comment on NOAA %RH measurements for the last 60 years and its implications for +ve feedback.
I'm sure Richard Betts will also want to comment on this.
Journal of Climate 2012 ; doi: http://dx.doi.org/10.1175/JCLI-D-11-00003.1
Surface Water Vapor Pressure and Temperature Trends in North America during 1948-2010
Over 1/4 billion hourly values of temperature and relative humidity observed at 309 stations located across North America during 1948-2010 were studied. The water vapor pressure was determined and seasonal averages were computed. Data were first examined for inhomogeneities using a statistical test to determine whether the data was fit better to a straight line or a straight line plus an abrupt step which may arise from changes in instruments and/or procedure. Trends were then found for data not having discontinuities. Statistically significant warming trends affecting the Midwestern U.S., Canadian prairies and the western Arctic are evident in winter and to a lesser extent in spring while statistically significant increases in water vapor pressure occur primarily in summer for some stations in the eastern half of the U.S. The temperature (water vapor pressure) trends averaged over all stations were 0.30 (0.07), 0.24 (0.06), 0.13 (0.11), 0.11 (0.07) C/decade (hPa/decade) in the winter, spring, summer and autumn seasons, respectively. The averages of these seasonal trends are 0.20 C/decade and 0.07 hPa/decade which correspond to a specific humidity increase of 0.04 g/kg per decade and a relative humidity reduction of 0.5%/decade.
Richard Betts
I am still waiting for your response to my inquiry about your 30 year PDO cycle.
Why does it have to be a LINEAR system? Surely with the well known PDO system you should be looking at a sinusoidal (aka cyclic) system. Or am I missing something?
Perhaps you need a bit of insight. May I suggest that you read Don't Sell Your Coat This is an excellent review article, but there is a sample chapter that explains my question. I believe that you will find the book quite educational. It should be required reading of anyone wishing to be a "climate scientist."
Hi again Don
I'll have to read the paper in detail, but on first sight I'm not so sure that puts the skids under anything. The HadCRUH humidity dataset already showed generally slightly declining global mean RH since the early 1970s (with rates varying from place to place, and in some places increasing) and the simulation of humidity by the CMIP3 models seems to be fairly consistent with the observations (Willet et al, 2010)
But as I say, I'll have to read it properly in order to make an informed comment.
Feb 8, 2012 at 4:17 PM | Jonathan Jones
You mention the model ensemble, but the difference between models isn't particularly important early in the projections - again, coming back to Ed Hawkins' paper, it's internal variability that is the key contributor to uncertainty to start with. So even with just one model run many times with slightly different initial conditions (rather than the whole ensemble of different models), there is still a wide range of possibilities for the first decade, simply due to the unpredictable nature of the internal variability (i.e.: the "butterfly effect").
So, it could simply be that internal variability just happens to have taken towards the colder end of the range of possibilities lately.
But you are right that we do indeed need to understand what has been happening.
Feb 8, 2012 at 8:23 PM | Don Pablo de la Sierra
Looking back I see it's only been about 4 hours since you asked, and, as I believe I have mentioned before, I do have a life outside Bishop Hill you know!
I'll consider answering as and when I am ready and have time, which is not now.
:-)
Goodnight!
Hi Richard, I appreciate your reply.
Unfortunately I have had personal experience of Professor PD Jones' modus operandi.
This man seriously considered how he could damage me at my place of work, has deleted emails from me (and others) that he knew were related to FOI/EIR requests, refused to share data for the purposes of reproduction and verification, transferred emails from his work PC to memory stick to evade FOI/EIR requests. I could go on.
It is unfortunate that you chose a paper co-authored by an individual who has done more than most to bring climate science into disrepute to support your point.