Bishop Hill What Jones said about station data
Mar 1, 2010 Climate: CRU Climate: Jones Climate: Parliament FOI Here's the bit from Jones statement that was bothering me:
Stringer: Well I will plug on because I've got one of the quotes from your emails which says "why should I make the data available to you when your aim is to find something wrong with it."...to Hughes. Now that's your email. Now that's the nature of science isn't it, that scientists make their reputations by proving or disproving what other scientists have done previously. Your statement there appears to be anti-scientific and the books that people have written around this issue have persuaded me that you have not provided all the information - the programs, the weather stations, - all the information available so that people can replicate your work, and to say the data is freely available in the United States doesn't enable anyone to go through your workings and agree with you or disagree with you.
Jones: Well the list of stations, we did make that available in 2008, so that has been on our website...
Stringer: How long had people been asking for it at that time?
Jones: Erm
Stringer: You're talking about some papers from 1990 aren't you, that have been kept secret?
Jones: No. There was a paper in1990 and we were asked for the data in that paper, which I was talking about in the previous question, that was made available straight away. The list of stations was made available after about six months, from the first FoI request in about early 2007.
Now I would be grateful if someone would check the transcript for me, but Jones' legendary email in which he rejected Hughes request for data was in early 2005, so I'm struggling to see how the list of stations was released "straight away" in early 2007. Am I missing something here?
Reader Comments (80)
OT, but Acton's statement to the press after the hearing is available at http://www.uea.ac.uk/mac/comm/media/press/CRUstatements/vcstatement2
Regarding the 60ish FOI requests, I thought there was a leaked email explaining that Jones had persuaded the University to ignore the requests. Or am i just dreaming it.
"I would like to end by saying what enormous pride I have in UEA and particularly CRU and its staff who have already played an historic role in advancing humanity's ability to understand and monitor climate change."
- Vice Chancellor Acton
Yes, indeedy!
The BBC has a short video clip which seems to sum up the situation;;
http://www.meto.gov.uk/corporate/pressoffice/2010/pr20100301.html
'Most scientists don't want to deal with the raw station data, they would rather deal with a derived product'
When it was just emails, much of this could always be passed off as careless talk. Now it is public, and on video, it seems much harder to explain away. Awaiting the defensive posts from the PR team...
As to the delay from 2005, and the emails concerning the stonewalling, that is how I recall it, as well. But I'm just a common sense sceptic.
I'm sure that there's several journalists out there in the print media eager to refute those claims.
Wouldnt it be special if old Moonbat was among the first to totally discredit the Alarmists?!!!
Read carefully - "... from the first FOI request". Accurate. And misleading - because he got polite email requests, which he ignored or denied. When forced to by FOI requests that couldn't be denied, he supplied the requested information in a "timely" manner.
So now we know - don't bother asking politely, just hit them straight up with an FOI request. Given the history, who'd blame you?
My view is that things got mixed up at the point you quote. The dialogue did not really make sense.
Separately, there is a write up about the hearing at
http://www.guardian.co.uk/environment/cif-green/2010/mar/01/phil-jones-commons-emails-inquiry
It reports the truth. I think it says a lot about The Guardian's integrity that they do that even though they are strong global-warming advocates.
oops... wrong link! BBC news MPs quiz 'climategate' scientist
Martyn,
Email 1228330629 from the escaped lump of files
Phil Jones writes:
There are other comments Phil writes which also show him in a less than stellar light.
I need somebody to cheer me up. This seemed very much like a sad day for all of us--skeptic, alarmist, lukewarmer. Oh, the questions that were not asked...
Doug, agreed, the best write-up that I've seen so far. But that's because it's Fred Pearce and he's got it in for the AGW crowd right now. Why is that, remind me? Did someone mention he'd had a run-in with Gavin?
The section quoted is pretty much right, except for some ums and ahs, do you know it is still watchable? the recording is still available, and if you go to about 1:16 into the debate you will see Stringers questioning start there
http://www.parliamentlive.tv/Main/Player.aspx?meetingId=5979
Tom, it's much less than it should have been but I think the main point about reproducibility did hit home, thanks mostly to Mr Stringer. We've just gonna have to work from there.
Just a note to thank His Grace for covering this event as thoroughly and as helpfully as he did . . . A credit to the diocese, so he is.
Ah yes thanks Gareth that’s the mail I was trying to recal, but I did totally forget the last sentence. “We are in double figures” Jones must have written it before he was inundated!
@Tom Fuller
I found it very illuminating. I have never bothered to follow one of these Gov media things before, and I wonder how many other UK people have discovered this resource based on the experisnce of engaging blogs like the Bishops. I actually had a lower expectation of what would happen based on the remit and the persons to be questioned. I agree with some of the critcs from both sides who criticise, for example, why have Lawson and Peiser representing scepticism? But they did well enough and I think there will be some understanding that this was a bit of a show, Acton/Stringers exchange was funny being sarky about the ICO, and I liked the line about the "lonesome pine" :) Ironically I think that the future of politics may turn out to based on this techno precedent of the peoples engagement with the "worlds biggest problem" TM
Rejoice in Climategate! Could we have imagined these clowns being questioned by a parliamentary inquiry live on our computer screens a few months ago. I think not.
And let's ignore that the results have been independently reproduced 4 times in the last two weeks - even Roy Spencer managed it for the NH - on top of at least two other other confirming results that existed before then.
Let's hope that nobody notices that the data was in fact available to do that all along.
A bit early in the year for Cherry Picking Frank?
Orchard,
Gesundheit. It takes more than parroting 'cherry picking' to constitute an argument.
Getting the same result via different data, different adjustments and different computations does not verify what CRU has been doing.
Gareth,
Yeah 6+ independent replications from 95% the same base data set using similar adjustments for the same purpose is such a red flag for nefarious goings-on, isn't it?
Obviously as 'auditors' and blog scientists this indicates that you must dig deeper in order to 'replicate'. You will require the source code for excel, windows, the fortran compiler, the microcode for the CPU, and detailed printed circuit board layouts so that you can construct an exact replica of the computer used for the calculations.
In fact as blog scientists you should be satisfied with nothing less than the actual computer used for the calculations.
Frank
There's an interesting thread over at Lucia's on releasing the code. Ron Broberg gives what I think it a useful typology.
Now I agree that people can perfectly well go away and do 2) but even if that's so, why should they not also have the option of doing 1)? Are you arguing in principle against people having the option of doing 1) and, thus, against releasing the code? If so, I'd be interested to hear your thoughts.
@FoD
Cherry Picking
reproducibility
I believe in some cultures, individuals independently interbreed with their siblings.
Cherries yup, (mostly windfall), but certainly not worth picking.
Frank, when I talked about reproducibility I was referring to the lamentable record of Climate Science in that area, including the Hockey Stick and all that fed into it, since 1990. I certainly enthusiastically agree that the situation's getting better - as I implied in my second message to Glyn Moody the week Climategate broke (before I knew that it had). And it's going to get better again. In fact, I'm sure that you and I will be dancing through the daisies together before we're done.
RichieRich,
I don't think there's anything wrong in principle with releasing the code but the data is far more important. A description of the high level methods and algorithms should be sufficient for the code - it is normal practice to use a high level vocabulary when communicating concepts and algorithms and for example if somebody says to calculate an average or standard deviation there is no need to demand their code in order to understand what that means. I have implemented crypto algorithms from published papers before. In that field for example, a common piece of vocabulary is modular exponentation - you just go implement that, you don't demand the author provides you with the code. (In fact when I did that, their code would have been no good to me as my task was to write a more efficient implementation of it)
The code should only be of interest if the algorithms aren't clear or there is some dispute about them. That happens sometimes - there is some case like that over on realclimate. But before you can say that there should at least be a good faith attempt to replicate and some kind of wild discrepancy that requires explanation, and a failure to reach understanding based on simply describing the methods.
I'd also argue that it's not entirely obvious what 'releasing the code' means and there is no end to that kind of nitpicking. To take an example, let's say you have crunched some data using excel. If you give me the .xls file, have you 'released the code'? How do I know I'm using the same version of excel you are? Indeed I don't even have MS excel - I'd run that using openoffice or iwork for the mac. Independent implementations of the same thing. Should be good enough, right, until there is a problem? Do I need to demand the source code for excel? If not then why not - maybe there is a lesson there?
I would also say that if everyone used the same code all the time that would be a very bad idea. That would make the code a common point of failure as every 'replication' would have the same failure modes.
Examples of the sort represented by (2) and (3) above are far more powerful and convincing replications, because they are not mere repetition.
Frank, those of us who have worked in software for over thirty years don't believe the crap about "description of the high level methods and algorithms". We want the source code first and the suite of unit tests that define its operational specification second. We don't believe anything else we read until we have both of those and the ability to run the second against the first.
In the case of climate science I haven't heard test suites mentioned at all but every single open source project has one as a matter of course and very soon, we believe, all climate science modeling, reconstructing, adjusting and what-have-you will be open source so we can expect major changes.
I'm speaking for a few other people there who've been in software a while but I think it's along the right lines. In fact you're touching on one the areas I'm most passionate about.
Okay, then--stiff upper moustache and onwards and outdowngorightwards.
I have had a recurring thought while watching Phil Jones' testimony. Is he committing seppuku in the hope that his act will somehow divert attention from the fiasco in which he played his part?
O'Dwyer writes:
"And let's ignore that the results have been independently reproduced 4 times in the last two weeks - even Roy Spencer managed it for the NH - on top of at least two other other confirming results that existed before then.Let's hope that nobody notices that the data was in fact available to do that all along."
The moral commitment in scientific method to repeatable results means exactly that. It does not mean doing parallel research about the same phenomena. So, same data and same results do not make repeatable research. The research should be published in such a way that one can literally repeat the work. That means "show all of your work."
I have a slightly different take on it.
Let us assume Jones is a scientist first and foremost. We want such people to stay within their scientific fields, do a good job and not venture into political waters.
Well I cannot imagine too many real scientists wanting to be in Jones place yesterday, shaking hands and all. Irrespective of the outcome.
Those 'scientists' whose egos drive them onto the political/media stage will always do so (post-normal scientists?) Those scientists who are primarily scientists will think twice. So the event served a purpose, because I am sure that academics across the world and country took an interest..
I think the most telling statement Jones made was when he said that he had never been asked by other scientist to provide raw data and code. This, if taken at face value, would mean that his studies have never been replicated by other scientists. If the studies have never been replicated, then can they truly been said to have been peer reviewed?
Frank O'Dwyer And let's ignore that the results have been independently reproduced 4 times in the last two weeks - even Roy Spencer managed it for the NH - on top of at least two other other confirming results that existed before then.
What the hell are you talking about? What 4 times are you talking about? Roy Spencer did not manage to reproduce it.
The results (meaning the warming trends) have to be shown to be legitimate. The only way to do it is to get ALL the RAW DATA used. Find out which raw data WAS NOT USED and why not. Find out HOW the adjustments were carried out and WHY.
Correct me if I'm wrong, but I understand that much, if not all, of the RAW DATA HAS BEEN LOST AND GONE FOREVER (Dreadful sorry Phil Jones).
Reproducibility does not mean here is the adjusted data I used, this is the program I used to adjust it - do it and see you get my results.
Dr Spencer used the International Surface Hourly weather data of NOAA's, did not make ANY adjustments for the UHI effect, which Jones claims he has accounted for, and got a trend from 1973-2009, 20% less than Jones's. Which is odd because not adjusting for UHI SHOULD MAKE THE TRENDS GREATER NOT LESS.
Moreover Dr Spencer used only those stations operating over the entire period of record. In other words stations that did not require any more "adjustments" (read fudging) for the period.
Hi Frank
Thanks for your thoughts. Here's some more stuff from Lucia's blog. What follows is a rather long post, but I think (hope!) of interest.
Mosher writes
A little later in the thread, carrot eater writes
Mosher replies
In response to Mosher's earlier point about the Hansen paper, Carrot Eater writes
Mosher replies
Your perspective and that of Carrot Eater seems to be that one should (is morally obligated?) to try to reproduce the results in a paper from a high level description and only if one has problems doing so, should one ask to see code. And only at this point is the author obliged to release code.
Mosher's postion, and one I think I have sympathy with, is: if you want me to believe your work, show me your detailed workings (which includes code). I'm not obliged to try to reproduce your work but if you want me to take your work seriously, you are obliged to put your workings out there.
If not identical, this is very close to Theo's position above, when he writes
Frank O'Dwyer
The issue is why you do not want to see data and code released. What is the argument against doing it?
After all, the future of the planet is at stake, not to mention think of the children, trillions, perhaps 100s of trillions, are being requested to be spent on very controversial projects whose outcomes are forecast based on various studies. Before they stump up the money, and before they engage in projects which may have negative side effects, people are invoking the precautionary principle and saying they want to be sure. If even a little doubt is removed by releasing the data and the code, it should be done.
Your argument in reply to this is on a purely personal level. Its to do with whether the people making the requests are nice, reasonable, could do the work over....etc Why it will be a lot of work. Why the researchers may find it stressful and time consuming.
Who cares?
There is not and cannot be any argument why, if people really are advocating the spending of trillions and the modification of the climate of our planet, they do not have to supply EVERYTHING. They have another option if you like, do not engage in political action. Simply assert that the science is as they say, that their studies are legitimate, but accept that it is not a basis for investment. Most of us would have little problem with that.
Our problem is some guys saying 'trust me' to the tune of trillions and to the tune of dramatic interventions in the functioning of the climate of the planet. When you make arguments like this, you have to expect intense and detailed scrutiny.
Its like, I am building a 16 foot sailing dinghy for my own use. Go to it. I am going into business and will be selling the things by the dozen? Comply with health and safety. Show your stability calculations.
Why do you think there are going to be, or should be, exceptions for CRU and for proposals to forcibly modify the climate of the planet?
@RichieRich, good points.
If you said to a Mathematician that you could confirm the Proof a theorem by just printing the final page of Proof but without the 'solution', they would laugh in your face.
The whole of science is based on proven building blocks. Even a simple building arithmetic block like 1+1 is always 2 has had to proved before it too can be used in further proofs.
In engineering a black box technique where you do not care what is in the box (usually supplied by a supplier) as long as you get the EXPECTED results from it can work, but ONLY if you know EXACTLY what is expected beforehand. That is, YOU CAN TEST THE RESULTS against a baseline.
What is it with Climate 'Science' that even these basic principles do not apply?
@michel, also good points.
Science has historically shown itself to be amoral. If it was me in that position I would be going to nth degree to ensure that my results were sound. Because I believe it would be a moral responsibility to do. As an engineer it is my duty to find fault.
So I do not buy any of the 'personal' defence based arguments. This is not about personalities. Science does not care. Science is impersonal.
When I see a Scientist I do not think 'good guy' or 'bad guy'. it is irrelevant. History has always shown that scientists will follow paths not always dictated by morals.
This rigourous approach is missing from Climate 'Science', that is why the vacuum is being filled by the Sceptics.
Correct me if I'm wrong, but didn't this all start with an attempt to reproduce the end result, and not getting anywhere because the 'obvious' guesses as to how each step could proceed not being correct? Where a method was mentioned in the literature, it was a trivial phrase which could be interpreted as being accurate, but was not sufficient to allow precise replication. The reason for the obfuscation - see Mann's response to MM03. The obfuscation provides a number of 'elementary errors' in the critical analysis which are used to dismiss the whole effort. At that stage, all M&M could say was 'there is something odd going on'. How are we all expected to make a judgment on the oddity if all the tricks are hidden?
Precisely, the argument that it is sufficient to provide the high level stuff and a statement of methods was blown out of the water by the study of the original MBH paper. It proved not to be possible to reconstruct the result from the information given in the paper, because they had used a strange, undocumented (and flawed) method of PC analysis.
I agree with Mosher, if the published data is to be used to drive policy or for major decision making it is essential that the code is made available. We all know that mistakes are endemic in computer programmes which is why those of us who do real science (I work in the pharmaceutical industry) have stringent Comval processes for ensuring appropriate documentation, version control and change validation procedures in place.
One of the most astonishing admissions at yesterdays hearing came from Dr Slingo of the Met Office who, with a straight face, said that their forecasting code was thousands of lines long but was validated twice a day by their forecast output. I wouldn't take any drug or fly in any aeroplane designed by her and her colleagues.
Frank O'Dwyer said:
Strawman. People are not wanting to use the code they are wanting to check it.
Maybe the analogy isn't quite perfect but, consider the reaction of a athlete upon finding that (s)he's failed a drug test but then being told by the doctor that the original sample had been lost.
Would you think it fair that the only possible recourse for defence should be to argue against the test result and details of the test method? Should the statement "trust me, I'm a doctor" be sufficient to justify a conviction?
Frank's position would be, why should she see the original sample? Let her go do her own tests and replicate the results if she can. The same would be his argument about drug testing. Why should they reveal the details of their sample? Go collect your own sample and do your own statistics, if you think you can do better.
Right. And until you do release, you don't get to sell the drugs. You don't have to release data for audit, we don't have to approve your drugs.
Same with climate. You don't have to release, we don't have to spend. One or the other. Same by the way with financials. You want to be a listed company, produce accounts. You don't have to produce accounts. In that case, you can't sell stock.
Prof Jones's response to the athlete would be:
You're guilty. We're not going to tell you how we worked that out, and we've lost your original sample. But you're still guilty. Honest.
But it's far more important than this in the context of climate "science".
If I do my own stuff and come up with a different answer, the team, the media and the IPCC ignore it entirely. The sceptics, even those of us funded by the billions put into the sceptic side by big oil (natch), simply could not get any air time at all. There were and are sceptics who have been able to publish papers that go against the consensus but they are simply ignored. (except to be used now by the team, similarly disgracefully, to deny that the peer review process has been goosed - its only the people trying directly to undermine the team that have been pushed out)
The ONLY way to strike at the team and the consensus is to demonstrate that the consensus scientists have arrived at their results incorrectly. You have to demonstrate flaws in THEIR work.
The utterly disingenuous dismissal of the M&M work by the team shows that absolute and complete reproducability must be the start point. Sceptics must demonstrate complete mastery of the team results because it is only after that that they can show why it is flawed. Without the first step, the demonstration of flaws has no traction.
Frank is being completely disingenuous to sideline this.
Dave Salt: very fair analogy.
Arthur Dent: What's a Comval process? That's new to me. I fully agree that 'mistakes are endemic in computer programmes' and to deal with that version control is one of the important tools in the kitbag. Others not so well known in a software team are pair programming (or co-coding) and continuous integration. Most vital of all, well ahead of documentation for me, is a comprehensive and readable set of regression tests. And all these processes depend on each other. Despite the immaturity of software development, there is accumulated wisdom to draw on by now. (The experts of course disagree to some extent - but the factors I mention have been coming to the fore in the last decade at least.)
Gareth:
This is quite a deep point on both sides. You're right that the first and most basic issue is replicating and checking results. And Frank's right to point to the common point of failure. But it's quite wrong to use this to argue against open source. Because open source doesn't preclude anyone starting from scratch, to try and do a better job. In fact, it makes it a great deal easier to do so, which could accelerate advances in climate science no end.
An interesting example would be the introduction of the Chrome browser in Sep 2008 after Google had invested so much effort and funding in Mozilla Firefox. They got to the point where they felt that they could do better by starting again. Improved security was a key reason they gave. But of course when Chrome arrived people at once found new security holes, that Firefox didn't have. For over the years Firefox has built up a loyal base of testers and contributors who very quickly fix such things. Chrome didn't start with such a large 'ecosystem' around it - but it looks to have a lot of momentum by now.
The really cool thing is that both are open source and Google is actively encouraging Mozilla or anyone else to come and use the best bits of their efforts. And Mozilla is committed to moving to the WebKit rendering engine used by Chrome but largely developed by Apple for Safari. Thus there's a mixture of healthy competition and a desire not to duplicate resources too much. This all plays out very nicely in the open source world as we currently experience it in such core areas of the Web.
We desperately need this kind of culture in climate science. There's no reason not to. There are many trillions of dollars of reasons for every human being on the planet to want it to be so.
To follow on from the Pedant-General, a further exchange from Lucia's. Carrot Eater again.
Mosher again.
Richard "ComVal" is the shorthand for "Computer Validation" a requirement under the GLP/GMP (Good Laboratory/Manufacturing Practice Regulations). Any climate scientist would probably have hysterics at the very notion of Good Laboratory Practice one of whose requirements is that any study being undertaken today should be capable of reconstruction (and replication) solely from the archived raw data at any point in the future.
This requires scientists to maintain and archive laboratory notebooks cross referenced and signed together with all the relevant statistical manipulations and computer codes and even the calibration and maintenance records of all the instrumentation used.
"The dog ate my homework" excuse is non viable and could lead to a requirement to run the complete study again or ultimately see your product withdrawn form the marketplace. The methodology of the climate scientists in academia and government institutes make me ashamed to be a scientist.
RichieRich and others have hit it on the head. It vastly increases the work to check the results to not include the code (and data!), for no reason other than to make it harder or obfuscate the process. I admire McIntyre for even trying. The paper is the documentation, and any programmer will tell you the documentation neither perfectly matches the code nor is ever complete. Then, when there are differences, the reimplementation can be dismissed for its incorrectness, even though it is correct to spec.
The code is one of the places where mistakes are the most likely to occur. If there are mistakes in the code, and even the most trivial code has bugs, refusing to release the code means you are hiding mistakes and making verification difficult or impossible. Doing this for publicly funded work from which government policies are set world-wide? That is unacceptable.
Phil said no reviewer had ever asked for the code or data. What does this say for the state of peer review in climate science? Reviewers just glance at it and say "'looks OK to me"?
Perhaps it's worth focussing on this point a little:
"The ONLY way to strike at the team and the consensus is to demonstrate that the consensus scientists have arrived at their results incorrectly. You have to demonstrate flaws in THEIR work."
This approach appears quite combative and perhaps not very constructive: it does not advance the science except in as much as it hauls the science back from what might be a wrong turn. It is therefore open to - and indeed has been the subject of - criticism from warmists.
If warmists did not want to have people crawling all over them and trying to destroy their work, they shouldn't have invested so much time and effort in it without getting it really properly validated. It is the warmists who have contrived to prevent such validation and now there is real and really pressing work to unpick the entire canon of literature before anyone can begin to make any progress at all - there is simply no confidence whatsoever in any of the foundations - there is nothing solid to build on.
In short, IT'S THEIR OWN BLOODY FAULT.
All that's happening now is that we are seeing this lamentable state of affairs.
Did Jones/CRU commit to releasing the data, metadata and code? By when? If not, what is going to be done to compel it to be made public? This is the only issue that really matters.