Bishop Hill Scientists behaving badly
Times Higher Education has a cover story about scientists behaving badly. The focus is on biomedical research and, in particular, the story of how two dogged biostatisticians named Baggerly and Coombes struggled to expose the errors in a paper on chemotherapy by Potti and Nevins.
The Climategate parallels in the story are obvious.
As well as the article (which is by Darrel Ince of the Open University) there is an accompanying editorial, which looks significant.
We may struggle to change human nature, but we ought to be able to ensure that journals, as Professor Ince says, "acknowledge that falsifiability lies at the heart of the scientific endeavour" - they must be less quick to dismiss challenges to their published papers and more willing to admit mistakes.
Duke itself has acknowledged that in work involving complex statistical analyses, most scientists could benefit from a little help from the statistics department before publishing.
Professor Ince goes a step further, arguing that all elements of all the work (in the Duke case, the full raw data and relevant computer code) should be made publicly available so that others can replicate or repudiate the findings.
In this age of information and the internet, that can't be too difficult, can it?
Reader Comments (16)
I see so many articles like this and TV programs where you can hear them say exactly this and then in the next breath they say Global Warming science is settled.
Prof Cox of BBC fame did exactly this in his programs. Then on an interview for the BBC he said "Consensus is that man is causing global warming". They simply can't hear themselves talking.
One big difference is that in this case there was eventual succesful pressure to correct the mistakes.
Why? I think the answer lies partly here:
Only after a funding agency, the NCI, saw that it had direct responsibilty and, I think more importantly, liability if the cancer trials were suspect.
The fact that no entity can suffer the same liability in the open ended, untestable, climate world explains why there will never be any motivation to clean out the climate science stables in a similar way.
See anything similar between the article above and this one:
Data availability and consequences in cancer and climate science
and its follow-up piece:
The code of Nature: making authors part with their programs
:)
Reader should also recall Ince's Climategate article in the Guardian.
It's not even behaving badly necessarily, it's what Feynman talks about in his cargo cult science talk. You have to have that special kind of integrity where you go *out of your way* to find out what is wrong with your own work, so that you don't end up fooling yourself, something which is oh so easy to do, or others.
And pretty much straight away you get to questions about what exactly the source code of your extremely complicated statistical analysis program is doing, and it is obvious that with software you have to show people the source code fo it to be possible to not fool them about it.
Why haven't all scientists read that talk? It just makes everything so *obvious*.
In the physical sciences, if a result can only be communicated by statistics, then it is not ready to be communicated. Statistics cannot ever be allowed to drive the physics. Period. In my own research I have established a working standard of confirmation along at least four independent lines of attack to prove a positive assertion of physical cause and effect. (Of course, only one definite result that is directly counter to the assertion is enough to disprove it.) And any regression with an r-square value less than 0.9 should provoke more suspicion than satisfaction.
It is not true that climate science is untestable, but the test has to be confronted, not avoided, and, if crucial, its results must be accepted by all. I have proved that there is NO greenhouse effect as promulgated by the climate "consensus" at
Venus: No Greenhouse Effect
but "Physics Today" has declined to publish my letter on it. Worse than that, they showed no signs of recognizing it as a crucial, definitive test of the greenhouse effect, that the science community needs to confront, professionally and publically (given the universal indoctrination in the greenhouse effect).
Harry: Good article. I'm sure most physicists would agree with you. The exceptions (Hansen, Trenberth, Allen) gave up doing objective physics in favour of advocacy and money, many years ago.
Rob Fisher
I've just quoted your remark about integrity in a posting on my own blog.
Integrity (in my view) has been one of the casualties of moral relativism, which I know sounds a pompous phrase but does seem to sum up the way in which, over my lifetime, the idea of maintaining certain absolute standards of behaviour has become rather "quaint"!
The idea of a trade-off between being honest and being effective and the suggestion that even if we're wrong about global warming we'll still have been right becuase we'll end up with the "right" result (ie - the end justifies the means) are just two examples.
As my small protest against moral relativism I plan in future to come out from behind Sam the Skeptic and post in my actual name which is Mike Jackson. No comments, please. I've heard them all and few were funny :-)
I am so very pleased to see the article by Darrel Ince and I look forward to reading his book The Cracks in Science. I teach my students that scientists are duty bound to write articles about their research that make it reproducible, including publication of data and statistical methods. After reading Ince's article, it has dawned on me that scientists are duty bound to gain the assistance of professional statisticians when writing articles that use complicated statistics and novel methods. Ince's article will provide a wonderful exercise for my students. As Karl Popper explained in The Logic of Scientific Discovery, scientists should write in a way that encourages criticism of their work from their peers. Yes, falsification is the heart of science.
Wow, what a great feast of comments.
Thanks to all who provided comments here. Very refreshing indeed. Scientists are not all evil funding-grubbing fiends. Most need funding of some sort, but many have deep integrity born of love for understanding. I recall that Solomon said something like, "A man's face shines when he understands a thing...", from Ecclesiastes 8:1. I cannot seem to find the exact translation that says that: read it when I was young and probably misremember it somewhat, but I always understood exactly what it means. Discovering that you understand something is one of the most exciting things in life (the excitement can be brief when you realize that you didn't get it exactly right, but there it is: understanding is an exciting fulfillment).
Rob Fisher's comment was particularly welcome. He said, "You have to have that special kind of integrity where you go *out of your way* to find out what is wrong with your own work." Too right. A team of scientists I work with is a good example of that spirit. While testing an hypotheses of the Team's Lead Author, I was unable to verify the results of one relatively minor equation. I checked my work and could not find any room for error, so I reported my failure to the rest of the team. I told them that I rather thought that the failure was my fault but I couldn't discover where I had gone wrong. Surprisingly, they took this very seriously. They assigned an independent investigator to look into the problem and it was discovered that there was a real but very small error in the equation. Perhaps more surprising, they took the error, small as it was, very seriously: they reported themselves to the journal in which they had published the equation and submitted a correction. I learned of this in a meeting covering other topics and when I expressed some surprise, the Team Leader said to me, "Well, if it's wrong, it's wrong." You have to admire people of such integrity no matter how surprising it is to meet them. It's even better if you get to work with them.
Here is what blogger omnologos says that I think relates to this topic:
the IPCC–led propaganda bandied about as “scientific consensus”), scores 7 out of 7 on the Freedman scale and therefore should lie at the bottom of anybody’s trust level:
1. dramatic (having reached the computational power needed to project future climate just as CO2 emissions got to a previously-unknown “dangerous” level)
2. a tad too clear-cut (with climate change almost completely due to a “thermostat” called CO2)
3. doubt free (the IAC spent an inordinate amount of time complaining about the absurd IPCC policy of underplaying uncertainties)
4. universal (everybody will feel the (bad) consequences of climate change, and everybody is guilty of it)
5. palatable (as it happens, the usual evils of capitalism and freedoms are the underling cause of climate change)
6. receiving “a lot of positive” media attention (shall I really comment this?)
7. actionable implications (every ha’penny worth of a politician understands how many things can be pinned upon the bandwagon called “climate change”)
And I find one sentence by Tetlock as especially relevant to the climate debate:
Whatever may be the merits of the underlying science in the peer-reviewed literature, in the public forum, the ratio of pseudoexpertise to genuine expertise is distressingly high."
http://omniclimate.wordpress.com/2011/03/27/andy-revkin-points-to-the-end-of-the-line-for-the-ipcc-and-its-lot/comment-page-1/#comment-7822
The site is full of interesting looks behind the mask of AGW.
I am currently in on a course of chemotherapy so I hope you guys can understand the utter outrage I feel towards the people involved right now!
There has been excellent research that has led to drugs that have much milder side effects but nothing can be more true in the article than...
"What lessons should be learned from the scandal? Their (the journals) embarrassing lapses stemmed from two tenets shared by many journals that are now out of date in the age of the internet."
It would appear that Keith Baggerly and Kevin Coombes have been the medical worlds version of Steven McIntyre and Ross McKitrick and thank you to Darrel Ince for his excellent article on this scandal.
I really do hope that Jones, Mann and the gang read this and maybe then they will understand a little better the outrage people feel about their refusal to release all their code and research.
At least Potti admitted responsibility for the problems with the research and resigned. Maybe Gavin could cover this story on the "other place" and compare the behaviour of the two groups of scientists!
Pluck
Were these university scientists?
I seem to have taken some credit for what were Richard Feynman's words, paraphrased. I should have made that more clear, but assumed everyone here would be familiar with those words.
Wise words from Professor Ince. I knew him years ago and he is a good guy.
Yes i agree with the above comments made by the (which is by Darrel Ince of the Open University)............
-----------------------
John.