Thursday
Mar102011
by 
Bishop Hill
Bishop Hill How to publish a comment
Mar 10, 2011 Journals H/T to DI for this article by Rick Trebino, a physicist from Georgia Tech.
...in this article, I’ll share with you my recent experience publishing a Comment, so you can, too. There are just a few simple steps:
1. Read a paper that has a mistake in it.2. Write and submit a Comment, politely correcting the mistake.3. Enjoy your Comment in print along with the authors’ equally polite Reply, basking in the joy of having participated in the glorious scientific process and of the new friends you’ve made—the authors whose research you’ve greatly assisted.
Ha ha! You didn’t really believe that, did you?
Read the whole thing.
Update on Mar 11, 2011 by
Bishop Hill
Bishop Hill
Some users were reporting a virus at the Scribd document linked. I've changed to an alternative site (H/T Doug Keenan) but this is not responding at the moment.
Reader Comments (30)
That is an amusing good natured way of describing how journals behave. It should remind people that a lot of the frustrations about peer review are not just peculiar to climate science, sometimes this fact seesm to get lost in the noise.
The bottom line is that all journal want a nice healthy lifespan to their publications. So, while peer review at its best is a lengthy and rigorous process, the corrective process can be made even lengthier and more tedious.
Not a rhetorical question, but I would be curious to know if anyone could point to an example of a science journal (of any subject) that was told of an egregious - paper breaking - error, and pulled the paper within a period of say, less than 6 months?
Close to the bottom of a long but enjoyable read, where have we heard this one before ?
Okay, so the system is badly broken. How would I fix it? 1. All data and parameters associated with any open publication
should be available to anyone interested in it. The NIH has
mandated this for its grant recipients, but sharing data and
parameters should also be a required condition for a
publication in any journal, no matter the funding source.
Refusing to do so after a paper is published should be
considered scientific misconduct.
I opened the link and was at the bottom of the first page when a hoax virus warning came up. Whether or not it came from the site I know not. I am protected by Norton 360.
I closed it down with Windows Task Manager.
A free PDF of the document is at
http://scienceblogs.com/catdynamics/upload/2009/08/how_to_publish_a_scientific_co/How%20to%20Publish%20a%20Comment.pdf
Unbelievable. What a frustrating hassle.
I've seen this before, though I don't remember were it was linked from (possibly CA). It is an eye opener as to how science really works. Sweetness and light don't seem to be major major components of the system ;)
Fascinating
I wonder what Paul Nurse would say about this (it is in his function to comment on such things rather than jet around to Nasa and drink tea with Singer)
PN gave the impression he lived with his "scientists" in a kind of cloud cuckoo utopia, in Horizon.
Scientific Journals are toast.All of them. Dinosaurs. The only thing they are kept afloat by is the establishment. They should be replaced one for one by a wiki. Run by a rabbit.
Anyways lets put out tinfoilhat on and presume we believe in Scientific Journals: Well, i could understand they have some rules as to avoid littering the journal with "comments" from Peter and Paul who are working in the quango with too much time on their hands and think it would be nice to see their name printed in Nature.
But sure the Author of a paper should be allowed to Comment when he gets attacked?
That looks like very basic?
I think all involved should be named and shamed.
The Journal should been put out of circulation of taqxpaidfor scientific reports if 10% of this story is true.
Tried again. No problems this time.
The problem (I might even go so far as to say naiveté) remaining in his list of recommendations is that, because there is behavior that is clearly wrong, we should make a rule against it, and then it will be set right again.
That plainly will not work. How many papers already have policies governing archiving of datasets and code? How many enforce them? Outcry is better than policy, in that sense.
Suggestions such as: "Reviewers should be required to stipulate any conflict of interest in reviewing a paper, even if it’s simply that they don’t like the authors."
Are plainly unenforceable. Like or dislike someone, I may be the best qualified to do a review. Likewise, if I am out to get someone, for my own private reasons, I am hardly likely to reveal that.
One must always think of these things in reverse too. Can't one stymie reviews they don't like by asserting some conflict in the reviewer, causing them to prove they are not, in fact, conflicted?
Many of these recommendations setup avenues for a host of future "denial of service" attacks on the paper, reviews, or comments very similar to the current possibility of submitting an unacceptable reply for the express purpose of killing a comment.
A proliferation of new rules, combined with arbitrary enforcement such as already demonstrated in his case with comment length, is not likely to make things better.
The problem is that modern science is corrupt. I think that is ingeniously captured by some blog writers when they refer to "Science!" and previously by Hayek in his complaint against "scientism."
The will is absent, and no amount of rules will bring it back. Science has lost its soul, as it were.
For my part, I wouldn't attack the journal rules as much as I would the culture, and the incentives. The culture needs to be taught shame (how, I admit I don't know), a value for truth, and the meaning of the old motto "nullius in verba." For incentives, the governments need to get out of funding.
Governments are inherently politically institutions. True science is apolitical. It is ridiculous to assert that political institutions can lay aside all politics in this one area and allocate funds apolitically.
Politicians have a strong incentive to assert that whatever plan they have is "founded on scientific principles" and if they need to buy those results, it will happen. It can even be "innocent" in so far as both the politician and the scientist believe themselves to be correct, but that doesn't prevent it from being corrupt. Tell the politicians what they want to hear and you get money, and perhaps a hand in shaping policy. Tell them something else, and you get ostracized, your grants are turned down, and your papers don't get published (journals know which way the winds are blowing too).
That is not a good system, and despite many technical advances in the 20th century, I don't imagine there are many objective minds that would deny problems stemming from the government "takeover" of science that began in earnest in the '30s in Europe, and the '40s in the US.
Be careful. I had some horrible infected messages come up then it tried to get me to download a setup.exe file supposedly to clean my PC.
Great article. Yes, humans trained and working in science are definitely still humans, with all that goes with that. Anyone see the movie 'Brazil'?
Hilarious but disturbing too if as the author says it's true. I particlularly liked "Global Tilting" and I had no idea science was so well organised that it can be divided into chunks 1.00 pages long!
I got hoax virus as well, beware.
follow the money.
the public in Britain and Australia need to do something about taxpayer-funded broadcasting. amazingly this part of the NPR sting in the US got virtually no coverage at all. even this is in a blog:
8 March: Washington Times Blog: Kerry Pickett: (video) NPR officers compare deniers of climate change to birthers and flat earth believers
Senior Vice President of NPR Ron Schiller met with individuals he believed to be potential donors. However, undercover video was running during this meeting. In the following clip, Mr. Schiller and his co-worker Betsy Liley describe how NPR covers those who deny climate change is happening.
Ms. Liley talks about a donor who would only give to NPR if the outlet did not talk to those who believe climate change is not happening:
"This funder said to us, ' you know you would like us to support your environmental coverage, but we really don't want to give you money if you're going to talk to the people who think climate change is not happening,'" Ms. Liley recounted.
She continues to say, "It is a complicated thing, though. There's a political question and there is a scientific question and we were talking to him about supporting the science desk. And so we've gone back to the science editor and asked how have you planned to cover this thing? Our coverage, if you look at our coverage, you would say that science coverage has accepted that climate change is happening and we're covering it. But in politics, our Washington desk, might actually cover it should it resurface as a political issue...this debate."...
"I think the challenge in our society now is that we are questioning facts. It's not opinions we are debating. I mean, what are the facts? Is the world flat? Is that the next question we're going to debate?" Ms. Liley wonders.
Mr. Schiller chimes in later saying, "The main point here is that it is not our responsibility to present the opinion of a non-scientist through our science desk. All educated scientists accept that climate change as fact. On the political side, however, where it is not accepted as fact, and the fact that debate is happening is news and it's really important news. And our point of view requires that we cover that debate, if for no other reason than to have Americans understand there are still people who believe that it is not fact."...
http://www.washingtontimes.com/blog/watercooler/2011/mar/8/video-npr-sr-vp-climate-change-deniers-wont-be-cov/
(from a comment on WUWT)
- This has gotten almost no press coverage. It’s almost 20 minutes of explanation
about the NPR news desk’s position on climate science. (See about one hour
in to about 1:20.) Here:
http://www.theprojectveritas.org/nprjudge
Virus Warning here too.
A retired physicist relayed a story to me about another physicist's and friend's discovery, I believe it was about the Higgs-boson. His friend's paper got tied up in the journal gate-keeping angle and, subsequently, he was not one of the people credited with discovering the Higgs-boson. Politics can be painful and life-changing.
Kev, No one has discovered the Higgs Boson.
I have friends like that though.
Had to reboot and scan my machine after getting that virus massage, but seems OK.
Ok, maybe it was something else or just the APS prize winners. I can ask my friend again. At any rate, whatever it was, he got tied up in journal gymnastics and was not credited. Thanks for the clarification.
http://en.wikipedia.org/wiki/Higgs_boson
Kev,
I didn't clarify anything I just stated something.
Hoax virus for me too - no damage done, but I didn't get to read the piece and I'm not going back there.
Leopard
steady now
I just left a comment in a WUWT thread claiming scientists live wretched lives because they lack sense of humour. I guess this example disproves my hypothesis. Some scientists can be funny some of the times.
A politics lecturer of ours once pointed towards the library building during a lecture and said: "Let me tell you a secret. Eightyfive percent of all the information stored in that building is trash." I thought at the time that that remark could only apply to social sciences, not to natural sciences. I guess I was wrong.
Ok, thanks for the clarification. :-)
Use the link supplied by Douglas J. Keenan, no pop up virus stuff at all.....
http://scienceblogs.com/catdynamics/upload/2009/08/how_to_publish_a_scientific_co/How%20to%20Publish%20a%20Comment.pdf
Maybe the good Bish can change the original link
comedy and tragedy at the same time.
I have come to the conclusion
governmentCorporatocracy doesn't want the system fixed, clearly right now under the current system information can be controlled, a strict peer review protocol would impinge that, so it'll never happen really, they'll try to make it look that way, but in reality it will be window dressing IMO.Peer review has been broken for years, as we know from Steig09 even the most respected and apparently reputable journals like Nature cannot be trusted. I would argue that this is to be expected, and that cynical perspective is both rational and scientific - research and papers are written by humans, and humans do not always play fair when there are financial and career incentives. Here are some more examples which illustrate the problem is not new:
- Scandal of scientists who take money for papers ghostwritten by drug companies
Doctors named as authors may not have seen raw data
Sarah Boseley, health editor, The Guardian, Thursday 7 February 2002 09.40 GMT
- http://www.guardian.co.uk/uk/2002/feb/07/research.health1?INTCMP=ILCNETTXT3487
- Revealed: how drug firms 'hoodwink' medical journals
Pharmaceutical giants hire ghostwriters to produce articles - then put doctors' names on them
The Observer, Sunday 7 December 2003
http://www.guardian.co.uk/society/2003/dec/07/health.businessofresearch
- Questions over ghostwriting in drug industry
Analysis claims papers drafted by medical writers downplayed risks of hormone replacement therapy.
Ewen Callaway, Scientific American. September 7, 2010.
- http://www.scientificamerican.com/article.cfm?id=questions-over-ghostwriting-in
- Stem cell research. Peer reviewers accused of nepotism
“It’s hard to believe except you know it’s happened to you where papers are held up for months by reviewers asking for experiments that are not really fair or relevant,” says Austin Smith, director of the Wellcome Trust Centre for Stem Cell Research in Cambridge, UK. Smith, who was speaking this morning on BBC Radio 4’s Today programme, is concerned that reviewers can no longer remain objective when there is so much at stake with these publications. “A paper in Nature or a paper in Cell is worth your next grant - it could be worth half a million pounds,” he says.
Pallab Ghosh (Today, BBC Radio 4, Feb 2 2010) - http://news.bbc.co.uk/today/hi/today/newsid_8492000/8492662.stm.
- http://physicsworld.com/blog/2010/02/researcher_accuses_peer-review.html
- political and corporate influence and control of journals - suppression of peer-reviewed study on vaccinated primates -
http://www.ageofautism.com/2010/03/joan-cranmers-fateful-decisions-and-the-suppression-of-autism-science.html (warning Bish - Wakefield connection!)
- climate science corruption and intimidation, various examples collated by Melanie Philips - http://www.spectator.co.uk/melaniephillips/archive/2010/July/
and some analysis:
Peer Review in Health Sciences, Drummond Rennie, BMJ.
- http://resources.bmj.com/bmj/pdfs/rennie.pdf
- peer review reviewed and found lacking: "Just a small number of bad referees can significantly undermine the ability of the peer-review system to select the best scientific papers. That is according to a pair of complex systems researchers in Austria who have modelled an academic publishing system and showed that human foibles can have a dramatic effect on the quality of published science." James Dacey, physicsworld.com, September 2009. (warning - some modelling but the assumptions look okay)
- http://physicsworld.com/cws/article/news/43691
- Lies, Damned Lies, and Medical Science
'Much of what medical researchers conclude in their studies is misleading, exaggerated, or flat-out wrong. So why are doctors—to a striking extent—still drawing upon misinformation in their everyday practice? Dr. John Ioannidis has spent his career challenging his peers by exposing their bad science...'
By David H. Freedman, November 2010, Atlantic Magazine.
- http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/8269/
I am sure there are many more examples and problems, these are just the ones which I have come across in the last few years. And there will be likely many more that did not get any media exposure. Consequently I am now wary of all peer-reviewed research, and more open to ideas which will find it very difficult to be accepted by the mainstream journals or orthodoxies - such as Oliver K Manuel's 'Iron Sun' thesis and the worth of good blogosphere review e.g. http://chiefio.wordpress.com/2011/01/03/iron-sun/ - and more recent research by David Chandler, Prof Steven Jones and Prof Niels Harrit et al on the mysterious collapse of WTC7 and the twin towers e.g. http://www.youtube.com/user/DavidChandler911 and http://www.bentham.org/open/tocpj/articles/V002/7TOCPJ.pdf
Forgive my naivety, but given that the errors were reportedly gross, and that the very first contact with the editors showed a willful determination to block the correction, why not simply publish elsewhere, on a blog if necessary, and shame the editors and the authors of the paper?
If you can’t get a hearing in the cathedral, nail it to the cathedral door.
..."Let me tell you a secret. Eightyfive percent of all the information stored in that building is trash."... --- sHx
That reminds me of what my boss said when I was working on the Saturn S-IVB: "Eighty percent of aerospace research today has already been reported in the German Journal of Physics before 1927."