Bishop Hill What Jones said about station data
Mar 1, 2010 Climate: CRU Climate: Jones Climate: Parliament FOI Here's the bit from Jones statement that was bothering me:
Stringer: Well I will plug on because I've got one of the quotes from your emails which says "why should I make the data available to you when your aim is to find something wrong with it."...to Hughes. Now that's your email. Now that's the nature of science isn't it, that scientists make their reputations by proving or disproving what other scientists have done previously. Your statement there appears to be anti-scientific and the books that people have written around this issue have persuaded me that you have not provided all the information - the programs, the weather stations, - all the information available so that people can replicate your work, and to say the data is freely available in the United States doesn't enable anyone to go through your workings and agree with you or disagree with you.
Jones: Well the list of stations, we did make that available in 2008, so that has been on our website...
Stringer: How long had people been asking for it at that time?
Jones: Erm
Stringer: You're talking about some papers from 1990 aren't you, that have been kept secret?
Jones: No. There was a paper in1990 and we were asked for the data in that paper, which I was talking about in the previous question, that was made available straight away. The list of stations was made available after about six months, from the first FoI request in about early 2007.
Now I would be grateful if someone would check the transcript for me, but Jones' legendary email in which he rejected Hughes request for data was in early 2005, so I'm struggling to see how the list of stations was released "straight away" in early 2007. Am I missing something here?
Reader Comments (80)
Arthur Dent: That's one thing I needed to know about, thank you. Well, potentially two things - or two names for pretty much the same thing is it? Is Wikipedia a reliable guide in this area do you think?
http://en.wikipedia.org/wiki/Good_manufacturing_practice
http://en.wikipedia.org/wiki/Good_laboratory_practice
And within those two Computer Validation clearly has a strict meaning, as you'd rightfully expect.
It is very weird how Climate Science has been given a free pass on such things when the bureaucrats have been so busy in the US and EU in such areas. (I'm not saying anything against such bureaucrats right now. Just pointing out the utter discrepancy.)
Now I am probably saying something against the bureaucratics. I think the open source world probably provides an even better model than the highly regulated one.
But we should learn all we can from ComVal as understood in your neck of the woods, those of us concerned about climate computing. And I'm a pragmatist in the end. Whatever gets the job done and ensures Climate Science becomes real science. Reproducibility for all. Free the data, free the code.
OT - sorry - but important.
Richard Drake said:
Mozilla is committed to moving to the WebKit rendering engine used by Chrome but largely developed by Apple for Safari.
Richard, can you please link to an announcement or blog post from Mozilla that confirms this? Thanks.
Dr Peiser told the committee: “OF COURSE, IF YOU DO NOT HAVE THE DATA SETS OR METHODS THEN YOU HAVE TO TRUST THE WORD OF A SCIENTIST. “YOU CANNOT EVEN SEE IF HE HAS DONE THESE CALCULATIONS DIRECTLY ON THE BASIS OF SOLID DATA, AND THIS IS THE CORE OF THIS PROBLEM ...”
–BBC News, 1 March 2010
But the results of these calculations are not just of passing interest. These are the basis on which the end of the world has been predicted and on which trillions of dollars are being committed to 'solve the problem'. The weapons of mass destruction for which we have to go to war to save mankind.
All this based on the testimony of a bunch of crooks who are willing to send us down the googler for their pound of flesh.
Is there a URL where I can watch the whole thing? I was only able to watch a few snatches at the time for various reasons.
Martin
http://www.parliamentlive.tv/Main/Player.aspx?meetingId=5979
Jerry, re: Mozilla moving to WebKit, I've just done a quick google (as no doubt you have) and then a search of my personal notes (in wiki form) and I have to say that it looks as if this is an unsubstantiated rumour, something that was discussed but rejected. That's very bad - I like to stay within the facts! It was obviously something I remembered reading or thought I had. But if I can't find it within 10 minutes of search then one's gotta say at least 75% probability it was misremembered. Thanks for picking me up on it. Don't want to start an urban myth!
Richard Drake said:
Don't want to start an urban myth!
Thanks for the clarification Richard ... that would have made a *big* difference to some of our planning :-)
http://www.parliamentlive.tv/Main/Player.aspx?meetingId=5979
Many thanks - watching it now.
Dave Salt,
False analogy. Firstly it is based on atoms which cannot be copied, while the real issue is one of bits which can be copied indefinitely. Secondly, it's more like six experts, including one for the defence, have tested based on the same sample and all got the same positive result.
Then her lawyer (Dee Nihilist, attorney at law) complains that she wants check the result of expert 1 but for that she needs to have the sample that expert 1 used. Expert 1 says he doesn't have it anymore, but you can still get the material he used from the people who took the sample. Dee then says no I need to get the sample from Expert 1. Eventually Expert 1 gets the sample for Dee. Having got the sample Dee then says I need you to give me the same scientific instruments that expert 1 used. Expert 1 says well, here is the spec for those instruments, you can get instruments like that anywhere or you can make your own - that's how these 5 guys over here did it.
I will respond on the other stuff later.
RichieRich,
A quick response on Mosher's 'customer requirements', which is all it merits.
Science is in some sense a process for reliably seeking the truth about the world. Philosophers of science and scientists themselves have struggled down the ages to define exactly what that consists of. However to my knowledge none from Karl Popper to Richard Feynman have ever defined its requirements as 'whatever Steve Mosher pulls out of his ass'.
Whatever science is, it's worked well for hundreds of years without meeting Mosher's requirements. Now along comes Mosher to tell us that while he himself is not doing science, science is full of problems and he's got a better idea. Forgive my scepticism.
Also, Mosher when says he isn't doing science, I certainly agree. The only sense in which he is doing science is the same sense in which Debbie did Dallas.
Frank O'Dwyer, are you saying that all of the salient 'raw' data has now been made available to McIntyre, Hughes and the others who requested it under FoI?
The argument that Mosher is I think putting forward is not about science per se, nobody other than scientists really give a toss about science. What Mosher is talking about is the point at which science is applied to deal with the real world. At this point it is no longer sufficient for scientists to say "trust me I am a scientist". What is needed is much more categorical demonstration of the robustness of the output, and Professor of physics might say if you build this it will fly because my investigations say so, but people require a much larger degree of engineering certainty, quality control etc. etc before trusting that opinion.
Climate science would have been allowed to pursue its bumbling course without hindrance, were it not for the decisions now being expected on the basis of its output. The discipline could have meandered along like many scientific disciplines falling down holes and driving up blind alleys but eventually ending up with the right answer and nobody would have cared.
However if you want the world to completely transform its economy and lifestyles and spend trillions of dollars on the basis of a few scientists just saying trust me you are insane. Note absolute certainty is not required but much greater rigour is and if the problem is this serious it is quite frankly outrageous that climate scientists should engage in the sort of prima donna behaviour shown by Professors Jones and Mann.
Dave Salt,
According to Jones yesterday 80% of it anyway. All of it is available the same place Jones got it. Even an implementation of the code is available.
Also that people have now replicated - in the scientific sense - the important result using the GCHN etc, in a variety of different languages, and with a variety of analysis approaches.
Frank
Let's go back to the tripartite division:
1. Same data, same code
2. Same data, different code
3. Diferent data, different code
Mosher is arguing that, on important matters, it is necessary (or at least highly advantageous) to the process of science that researchers should produce both data and code. It strikes me that he is not alone in this. The IoP and the Royal Statistical Society both made the same sort of point. (Questions have been raised re the IoP, but not as far as I'm aware, re the RSS.) And I remember reading that some journals now also insist that code and data is archived prior to a paper being published. So if Mosher is trying to re-invent science, he is clearly not doing it single-handedly.
I can't claim to be familiar with the writings of Feynman or Popper but surely you're not suggesting that both they and other philosophers of science would consider the idea of requiring researchers to release both data and code an outrage? But if they would, why would they?
I entirely appreciate that you believe the 2. and 3. are more powerful weapons, so to speak, in the scientific armoury but I still don't understand what you have against making it a default position that researchers should produce data and code. What's the downside? What harm can it do?
I don't see what is so outlandish about Mosher's "customer" perspective? Policy is in a clear sense a "customer" of science and if, as a customer, I (a) have an option of checking your data and workings personally or (b) have no option to check your data and working but have to rely only on peer review or your title, surely (a) is preferable. Or perhaps I'm missing something?
Hi Frank
I've just had another quick look at the essay by Jerome Ravetz, professor of the philosophy of science, on WUWT. I'm not indulging in argumentum ad verecundiam here, but post this quote simply to reinforce my point that Mosher is a lone rogue agent!
Whoops! Freudian slip or what?!
Should read "simply to reinforce my point that Mosher is not acting as a lone rogue agent!
Frank O’Dwyer, whatever you say Jones said, the fundamental problem is that there is obviously no structured approach to verifying the results; at least, not the type of verification expected in a hard science like, say, physics. Jones himself said that it's not normal in climate science to give out data and code, while he also said that peer reviewers never ask for data.
You then say that “people have now replicated - in the scientific sense - the important result” but all that suggests to me is that your understanding of science and, more importantly, the Scientific Method is very different than mine. This is why I’ve always tried to discuss the issue based upon the IPCC WG1 information and why I’ve been seeking (but have so far failed to find) evidence for unambiguous predictions that can be verified/falsified against real-world observations. Actually, I’ve come across several things that suggest the theory is false but tend to keep an open mind because of the associated levels of uncertainty within the data.
To me, the current focus on CRU or the IPCC Summary relates to the way the message has been sold/spun. As such, it is unrelated to the core scientific issues and is simply distracting attention away from the basic fact that catastrophic AWG theory has never been verified by the Scientific Method. My original suspicions have been bolstered not only by the lack of evidence but by the fact that so many of its proponents now argue the case for radical action in terms of ‘post-normal science’ and the ‘precautionary principle’.
This is why I find it quite understandable, though highly regrettable, that so many now consider climate studies as more akin to a ‘cargo cult science’ than a ‘real science’.
RichieRich,
Re your tripartite division.
Frank is carefully avoiding the entire basis of the climategate kerfuffle, namely that:
- anyone using methods 2 and 3 are either ignored or dismissed if they arrive at different results. Different results = you're doing it wrong.
- The team have been all they can actively to PREVENT anyone from doing 1 and further that anyone who attempts to reverse-engineer 1 is prevented from publishing and has the warmist attack dogs in the press set on them.
Arguing for more 2 and 3 is thus utterly missing the point since this post is entirely about point 1: verification that what Jones did was correct. If we find that it has major flaws - and given the attitude of the team to different results in 2 and 3 this can ONLY be done with method 1 - then we need to start looking to see if those same major flaws exist in the other allegedly independent studies.
Please respond to the substantive issue here Frank.
Sorry - there's another essential point. This isn't a physical science. It is entirely about the data and the precise mathematical methods actually used.
You need the EXACT data that was used (noting that the MBH papers for example used data sets that they knew to be out-of-date versions, because the updated versions didn't give the same results) so that the reasons for selecting those precise sets can be understood and/or questioned and so that the sensitivity to those sets can be shown.
You need the EXACT method - in fact you need the code - that was used, because the method described in the paper uusally doesn't give you enough detail to replicate (see objections to 2 & 3 above) or may be incomplete or misleading. Note that the MBH papers did not mention at all two of the most contentious methods: short centering and the step-wise PC function.
Fundamentally, the attitude and behaviour of the Team et al to those using 2 & 3 to produce different results and, worse the abuse of process to frustrate method 1, essentially forfeits them any right to anyone's sympathy.
We know that the Proxy studies in particular are incredibly sensitive to minor changes in infilling data etc. This, in a sane world, would be enough to have them entirely discredited as unreliable, but in this bizarre parallel Frank-wrold, this seems to be our fault.
General
I think there are three issues here.
First, there is, as you point out, the political issue. If you as a sceptic use their data, but your code and get a different result it can lead to accusations from warmers that you are wrong.
Second there is the issue of ungentlemanly conduct. McIntyre, in line with Frank's favoured approach, tried to replicated Jones' results using his own code and couldn't do so. So he asked Jones for his code. As I understand it, according to the unwritten rules of the game, Jones should at this point have (happily) handed over the code. But he refused. So if hand-over remains discretionary then one can always be frozen out. (Mates get it but you don't.)
Third, and perhaps most fundamentally, surely - even in the absence of the possibility of politicking or ungentlemanly conduct - the default position regarding important policy issues should be that if Person P says she did X to Y and got Z, you should be able to check her workings. Having this option, even if not taken up, gives more reassurance than having to trust her qualifications or the peer review of Reviewer R and, if taken up, may throw up mistakes that wouldn't otherwise have been spotted. For instance when the Met Office released data, John Graham-Cumming found some errors that the Met Office hadn't discovered.
Isn't the fundamental difference here between the interested community having the opportunity to prove to themselves that a piece of research is sound rather than being forced to rely on the views of others or try to replicate the work themselves?
"Having this option, even if not taken up, gives more reassurance than having to trust her qualifications or the peer review of Reviewer R"
Most particularly especially when Person P declares, with glee, not only that Reviewer R has not asked for either the code or the data, but that NO-ONE EVER has.
"So if hand-over remains discretionary then one can always be frozen out. (Mates get it but you don't.)"
Most particularly especially if Person P refuses to do so because "we have 25 years invested in this work. Why should we give it to you when all you want to do is find something wrong with it".
Frank: you have to address these points.
You cannot continue to evade sensible, moderate and ultimately reasonable questions.
Having got the sample Dee then says I need you to give me the same scientific instruments that expert 1 used...
Not quite, Frank.
The problem is that the "instrument" used by various climate investigators was largely software - it didn't really exist other than in the code, and had never been seen before. The analogy is not to an often-performed drug test, but an entirely new and unproven method of determining eligibility.
Consider results of a chemical experiment: If I were to reference results from a mass spectrometer, I can be reasonably confident that any other experimenter can reproduce those results; "mass spectrometer" is a reasonably well-defined product. If they cannot reproduce my results, we might look at their lab techniques, process temperature control, timing and so on before we suspected that a difference in spectrometer is to blame. Eventually, though, if that was the only uncontrolled variable, we would investigate there, as well - and if different spectrometers gave significantly different results, we would learn something, either about my claims or the state of our instruments.
On the other hand, if I produce a result, but base it on a newly-developed 'mass spectophotometron' other investigators have no way to verify or disprove my results, particularly if my 'high level' description of what the device does is either inadervertently or deliberately incomplete. In that case, in order to be taken seriously, I have to demonstrate that my 'mass spectophotometron' actually measures some relevant property, and that it does so in a consistent and meaningful way.
This is analogous to the CRU / Mann / etc. results - based on the descriptions in their papers, capable investigators could not reproduce their findings, and could not verify that the black box code (their 'mass spectophotometron') functioned as decribed. Rather than demonstrate that their analysis was correct, they hid their methodolgies and prevented independent access to their raw data. As shown by the various e-mails, they knew that their results were irreproduceable - so they blustered, appealed to authority, and attacked anyone who actually wanted to see their 'mass spectophotometron' and see evidence that it actually worked.
What dcardno said.
Wonder what happened to Frank?
Just to hammer dcardno's point home: a mess spectrometer is not just a well understood piece of kit: it has well understood mechanisms by which it can be calibrated, that when you put in sulphuric acid, it gives result x and when you put in uranium hexafluoride, it gives result y or whatever.
The problem here is that the actions of the team have been actively to prevent anyone from performing that calibration.
The team's response, with support from the warmmongers, has been to say that M&M had made mistakes when they built their own "mass spectophotometron".
P-G - I am not ignoring anything, carefully or otherwise!
There are a number of points here that I will return to, probably at the weekend.
Frank,
Smashing. Appreciate that real life often intrudes and it's a horror when you return to a threat to find x hunderd comments angrily demanding a response... Also appreciate that that applies doubly when you are not exactly "playing on home turf".
But..... for avoidance of doubt, when I said that you have been carefully ignoring the key issue, I was not referring to your lack of any comment whatsoever. It was that when you did comment, it did not deal with the substantive issues.
Will look out for your response.
PG
P-G,
'The problem here is that the actions of the team have been actively to prevent anyone from performing that calibration.'
Or perhaps that when they discovered that their calibration was incorrect, they replaced their calibration with another, leaving their incorrectly calibrated data.
P-G and others, I have finally gotten around to (I hope) answering more of the points made here and elaborating a bit on my own position on this.
It became rather long and also a bit difficult to format in the comment box so I have made it into a post on my own blog (http://www.frankodwyer.com/blog/?p=378).
Feel free to comment here or there whichever you prefer. If you do comment there I have a spam filter on my blog so comments from first-time commenters would take a little time to appear.