Wednesday
Jan262011
by 
Bishop Hill
Bishop Hill Shub on cancer and climate
Jan 26, 2011 Climate: Jones FOI While we're talking about cancer, Shub Niggurath has an article about problems with the availability of data and code in another field of scientific endeavour, with close parallels to the case of Phil Jones and the Chinese station data.
Reader Comments (14)
An interesting compare & contrast story.
Agree with Adam, very apposite and relevant to Nurse's cancer claims.
Duke, Albany, Penn State, Virginia, UEA, Colorado. We need tree ring analyses of these groves of Akademe to find out what ails them.
==============
Very nice essay by Shub Niggurath indeed!
Perhaps it is worth pointing out that having data/methods/codes made available, be it in medical studies or in climate studies (or in deed any other scientific discipline) is not simply about the reproducibility of the experiments.
The far more terrible consequences are that other scientists will build on those accepted papers, never mind not being able to replicate the original findings - and thus year upon year false data are being propagated and taken as valid.
The famed hockey stick is just one example.
When a paper is withdrawn, after a few years - what about all the other papers which built on this? What about the good names of the scientists using those data on trust? What about the patients who have undergone a therapy based on fraudulently reached results, especially in the case of cancer, where time is of the essence?
I wish those who think this whole issue is just a bit of squabbling between some scientists would take a look at the far-reaching consequences.
The Potti et al Nature Medicine paper is cited 347 times in the literature.
Nice write-up. More evidence that scientists can be as arrogant, duplicitous and error prone as the rest of us and that groupthink occurs in academia and research labs as it does elsewhere. Congratulations to the tenacious biostatisticians.
"A request that Nature publish a formal statement of correction therefore seems logical."
Ahhhh, that hit the spot ... A pleasure to read.
Wow! This makes the point more clearly that many disciplines are required to intersect to get good results. Scientists need to know about their specific fields, as well as statistics and computer programming. All three are major disciplines that require great skill. It makes sense that it would be hard to be a master of all. Scientists need help...back to my point about open science models and methods.
In my experience when folks don't want to share their code or go through a code review process; it means they are in over their heads, don't know what they are doing and their code is crap. Then they try to cover themselves with politics.
I'm thinking of Chris Mooney right now with his Scientific Illiteracy campaign. It appears some scientists need to read his book.
If my memory serves me correctly, weren't we all supposed to be already dead from AIDS, CJD, SARS, bird flue and swine flu by now? If I had any of these conditions, then I would certainly take medical advice, but when it comes to predicting the future, the doctors seem to exhibit a similar level of skill to climate scientists.
A timely article, and another feather in the cap for applied statisticians working to reproduce published results of great potential significance, and finding serious errors or omissions. M&M did it for the Hockey Stick, and now these folks have done it for that cancer study. These are very impressive pieces of gruelling, painstaking work with almost nought but indifference or hostility waiting at the end of them. When in fact they deserve great acclaim. They are protecting us from harm and protecting the noble ideal of scientific integrity. If the Royal Society were not so engaged with politics and government, then they might have been a source of recognition. Instead, one imagines spin-doctors fussing around the professors if they even began to consider such a possibility. Perhaps the Royal Statistical Society might do the honours instead.
Climate science was once (and will perhaps become again), esoteric. For those that practice it to allow policy decisions of some enormity to be based on it without verifiable proof shows a level of arrogance on their part that beggars belief. So Prof 'eerily calm' Jones, if you want policy to go along with your findings, show your workings, for without those you are merely a publisher of literature.
The parallels between the how M&M work's and Baggerly et al's efforts with the microarray statistics was recieved is simply freakish. Of course, in the cancer world, there is geniune 'competition' so things run their course much faster, whereas in climate you have the usual stonewalling carried on and on, for years.
For example (from document: The history of cisplatin and pemetrexed predictors)
The difference between the two cases:
We must keep focusing our attention on practical outcomes.
The Shub article is pertinent and important.
It should be drawn to the attention of a certain member of a certain UK parliamentary sub-committee.
Faulty conclusions contained in official reports cannot be allowed to stay in the record.
I also suggest that all UK residents reading this post, draw it to the attention of their local MPs.
Surely at least some MPs are capable of independent thought.
@ausiedan
It;s a nice thought but I think most are busy lining their pockets and the others know that unless they "go with the flow" they'll never have a chance of making a difference because they'll be shafted by their own party never mind the opposition using the media in some dastardly way.
Yours Mr cynical and i am sad that I am