Peiser on journal and media bias

Benny Peiser's review of the way scientific journals and newspapers cover global warming is a must-read:

The integrity of the science media will depend on whether they will encourage critique and fault-finding analysis by consensus sceptics - or whether they will continue their course towards unbalanced campaign journalism. Given the well-documented reluctance of mainstream science media to accept submissions by critical scientists and the aversion to report on critical papers published elsewhere, I remain unconvinced that science journalism will moderate its blinkered attitudes in the near future.

Reproducibility

Here's something interesting, from the Chronicle of Higher Education:

If you’re a psychologist, the news has to make you a little nervous—particularly if you’re a psychologist who published an article in 2008 in any of these three journals: Psychological Science, the Journal of Personality and Social Psychology, or the Journal of Experimental Psychology: Learning, Memory, and Cognition.

Because, if you did, someone is going to check your work. A group of researchers have already begun what they’ve dubbed the Reproducibility Project, which aims to replicate every study from those three journals for that one year. The project is part of Open Science Framework, a group interested in scientific values, and its stated mission is to “estimate the reproducibility of a sample of studies from the scientific literature.” This is a more polite way of saying “We want to see how much of what gets published turns out to be bunk.”

I don't think readers will be surprised if I suggest that it's not just psychology that is worth checking in this way.

(H/T Alan, by email)

Wunsch on Nature

This from a reader:

I saw a talk by Carl Wunsch at Wolfson College, Oxford this evening.  He's probably best known outside of his field (oceanography) for disagreeing with how his views were represented in "The Great Global Swindle".  He's somewhat equivocal on the certainty of AGW, maintaining that anyone who claims to be able to forecast the climate even a decade or two ahead doesn't know what they're talking about.  Of course that cuts both ways so "deniers" (he included the quotation marks) can take no comfort in such ignorance and certainly not use it as the basis for inaction.  He's broadly in favour of precautionary measures.

Anyway, that was fairly general ho-hum.  The money-quotes came late on when he talked about "the Nature-Science problem".  He seemed faintly disgusted by the lengths to which some climate scientists will go to get published in Nature or Science with the attendant publicity, media appearances and so on.  He sometimes found it difficult to tell which of the Daily Mail and Nature was the peer-reviewed journal and which the tabloid.  Nonetheless, he said, his colleagues  reassure him that just because something appears in Nature doesn't necessarily mean it's wrong.

Show us yer code

John Graham-Cumming, Les Hatton and Darrell Ince are all well known around these here parts. John Graham Cumming was quite prominent after Climategate for his critiques of the standard of CRU's computer code. Darrell Ince wrote an article making similar criticisms. Les Hatton was mentioned as the author of an article criticising the IPCC's handling of hurricane data.

The three of them have just published a new paper in Nature, looking at the issue of code availability and trying to address the question of how we can move to a world in which academic computer code is routinely made available.

Nature: British science needs integrity overhaul

Nature magazine has picked up on the BMJ survey of research misconduct in the UK - I posted about this a couple of days ago. The article carries the bold title British science needs `integrity overhaul'.

British scientists are fundamentally failing to deal with research misconduct, which is widespread in the country, leading experts have warned.

At a conference in London yesterday, participants were united in calling for more action on the issue.

“There is a recognition that we have a problem,” said Fiona Godlee, editor-in-chief of the British Medical Journal (BMJ) and one of the driving forces behind the meeting.

I think it would probably help if journals like Nature stopped trying to cover up for the miscreants.

Richard Smith: peer review still useless

Richard Smith, the BMJ editor whose fulminations against pre-publication peer review have done so much to inform my own thinking on the subject, has returned to the fray:

Recently a paper that I wrote with several others was reviewed by another journal of global repute. Again there were three reviewers (the Holy Trinity), and I'm not very unkind when I paraphrase their reviews as: Reviewer A: "Please reference my work"; Reviewer B: "Pay more attention to my specialty"; and Reviewer C "The authors should have written the paper in the gnomic language that I use."

Last time I mentioned my views on peer review, Bob Ward said my views were biased and misleading.

Look out Dr Smith.

Hulme on BEST and peer review

Mike Hulme has some perceptive comments about the BEST team's approach to peer review.

So what does this do to the conventional journal peer-review process?  Those asked to review these manuscripts for JGR will now conduct their personal reviews in the full knowledge of the parallel public review which is on-going.  And unless they shut-off all their communication platforms for the duration they will hear and see what others' judgements on the manuscripts are.  Whether for better or worse it's difficult to see how this will not change the (conventional) peer-review process.

The press and scientific papers

I've been enjoying the back and forth on the BEST thread about the way the publicity for the team's papers was handled, with some people concerned about the team going to the press before peer review had taken place.

Circulating drafts of a paper seems unobjectionable to me - this is surely an everyday occurrence in the academy. Going to the press before those drafts have been examined seems somewhat more questionable. That said, given my own views on peer review - namely that it's not worth a whole lot - then some interesting questions are raised.

Click to read more ...

Opening up research findings

An announcement from the Department of Business Innovation and Skills

Science Minister David Willetts has today announced a new independent working group to look at how UK-funded research findings can be broadened for key audiences such as researchers, policy makers and the general public.

The group will be made up of representatives from the higher education sector, research investors, the research community, scholarly publishers and libraries. It will examine how access to research findings can be made more transparent and accessible.

Wagner speaks to IoP

Physics World has managed to extract some brief comments from Wolfgang Wagner, the editor of Remote Sensing who resigned over the Spencer and Braswell paper:

I saw several basic problems [in the Spencer–Braswell paper], including that correlation does not imply causality, the fact that 10 years' of satellite data are not enough to come to such strong conclusions about the subtle and long-term changes in climate, and that, indeed, too little quantitative evidence was presented to support these strong claims,"

He also denies having any pressure put on him. Which is odd when one recalls his apology to Kevin Trenberth.

(H/T Jonathan Drake)

Peer review pickle

It is rapidly becoming a commonplace that peer review doesn't work. An article in Times Higher Education looks at the problems its use is causing at the research councils.

"Independent expert peer review" is contradictory. One submits a proposal and the councils ask experts to assess it. But these experts are likely to include proposers' closest competitors, even if they are selected internationally, because science is global - and real pioneers have no peers, of course. How then can the councils ensure that reviews are independent? To make matters worse, these experts can pass judgement anonymously: applicants don't know who put the boot in.

I suggest that the misuse of peer review is at the heart of the research councils' problems. Before about 1970, they largely restricted its use to the assessment of applications for large grants or expensive equipment. Scientific leaders protected the seed corn, ensuring that young scientists could launch radical challenges if they were sufficiently inspired, dedicated and determined. Today, the experts whose ignorance they would challenge might also influence their chances of funding.

There is obviously a suspicion that research funding is directed towards projects that will help the green cause and away from those that might question it, although it has to be said that evidence is thin on the ground. With a process like peer review involved, we suspicious members of the public are hardly going to be reassured though.

Maths do better - Josh 118

H/t Chuckles who, on the Make haste more slowly post, wrote:

"Does this mean that John Abrahams comments, reported in the Guardian and Daily Climate, about Dr. Spencer constantly having to correct errors and revise work, are in fact correct?

Just not in the way Abrahams originally intended?"

Cartoons by Josh

Is AR5 finished before it begins?

Roy Spencer has penned some further thoughts on the campaign being waged by the Team and he is worried:

We simply cannot compete with a good-ole-boy, group think, circle-the-wagons peer review process which has been rewarded with billions of research dollars to support certain policy outcomes.

It is obvious to many people what is going on behind the scenes. The next IPCC report (AR5) is now in preparation, and there is a bust-gut effort going on to make sure that either (1) no scientific papers get published which could get in the way of the IPCC’s politically-motivated goals, or (2) any critical papers that DO get published are discredited with any and all means available.

Click to read more ...

Autumn fireworks testing - Josh 116

It is the story of the week - how Wolfgang Wagner may or may not have been pressurised to resign over the publication of Spencer & Braswell. I wonder how the Team feel now?

Cartoons by Josh

Critiques and responses

There is still huge interest in the Remote Sensing affair and quite what this means for the climate debate is still unclear.

One aspect of the story that has attracted a great deal of comment is the fact that Remote Sensing has not retracted the paper. As Retraction Watch puts it:

We are not in a position to critique the claims. But we are curious: If Wagner feels he published the article in error, why not simply retract it? Was it really necessary to fall on his sword to make the point that he now feels he made a mistake in publishing the paper? It’s a noble gesture, and not unprecedented for editors of climate journals, but is it best for science?

Remote Sensing has now made it clear that they will not be retracting the paper.

Click to read more ...