Bishop Hill David Henderson on GWPF reports
Sep 9, 2012 GWPF David Henderson sends this note on the review of GWPF reports.
One of your commentators has posed the question: ‘If short journal articles are peer reviewed, why not longer GWPF pieces?’
There is a misunderstanding here. The ’longer GWPF pieces’ have taken the form of reports: up to now, nine of these have so far been published, with Peter Lilley’s as the latest. All of them have been peer reviewed by members of the GWPF’s Academic Advisory Council, of which I am chairman. The members of the Council are publicly listed. I have personally reviewed all nine reports, and commented in writing on all but one.
The GWPF procedure differs from that of a journal, in ways that in my opinion are advantageous.
- More potential reviewers are involved. Although no Council member is under any obligation to comment in any specific case, the number of substantive comments has typically exceeded what would normally become available through journal-style peer review.
- The process is not anonymous. The identity of the author is known to the potential reviewers, who make their comments in a personal capacity and may correspond directly with the author. It is up to the author to decide whom he thanks in the eventual published version: the list may go beyond Council members.
In every case, authors have made revisions to their draft texts, sometimes substantial, in response to comments from Council members.
Final responsibility for publication rests with the Chairman of the Trustees, Lord Lawson, and the GWPF Director, Dr Benny Peiser. In every case, publication is accompanied with the following formal statement:
“Views represented in the publications of the Global Warming Policy Foundation are those of the authors, not those of the GWPF, its Trustees, its Academic Advisory Council members, or its Directors”.
Reader Comments (31)
I have total confidence in the GWPF however I think it would be appropriate for reviewers to be from outside the organisation so that although justice is being done it can also be seen to be done.
http://www.independent.co.uk/environment/climate-change/tories-dash-for-gas-risks-climate-target-8120153.html
The dash for Gas is going to put Deben out of business more like.
Thank God
The commentator in question is me. Many thanks to David Henderson for addressing the issue.
My original post was motivated by Richard Tol's remark that
If one reads the preface and acknowledgements of Stern, it's abundantly clear that it was reviewed pre-publication by shed loads of people.
So I took Richard Tol to be suggesting that Stern wasn't formally peer-reviewed (i.e. subject to journal-type anonymous review). So my question was this:
David Henderson think that GWPF's system of peer review is preferable to formal peer-review (though of course it doesn't have to be either/or). But does he, I wonder, think that formal peer review should have happened with Stern?
My own take on the GWPF's system of peer review is that the draft seems only to have been sent to people who would be broadly sympathetic. I sometimes wonder if there isn't added value in sending a draft to folk on "the other side" so that one is subjected (possibly) to more challenging feedback.
The problem with RichieRich's suggestion of sending papers to those "on the other side" for feedback is that they would be likely to reject most of the document as there is unlikely to be any common ground. I think we must face the fact that those who agree with Stern's analysis will reject Lilley's. In the end it is up to the reader to decide which has the most credibility
Yeah, sounds like what you might call "pal review", doesn't it? Wait a minute, that is a familiar term...
Derek
My understanding is that effective journal peer-review (as opposed to "pal review") involves sending a paper to referees with varying degrees of sympathy for the paper's argument. Unsympathetic referees don't have a veto: the final decision rests with the editor who judiciously assesses the referees' comments and the response of the paper's author(s) to those comments.
I don't see how one can reasonably criticize Phil Jones et al for trying to enforce "pal review" and at the same time claim that GWPF is forced to go down the "pal review" path.
mmm...so Bitbucket - I look at the cvs of the GWPF - Lord Lawson belonged to the Conservative Party and conflicted with Dennis Healey of the Labour Party many times. Also on the GWPF board is Lord Barnett, who was Dennis Healey's number 2 at the Exchequer in the 70s and against whom Healey made many attacks. Barnett's unwelcome role was to appear on TV to defend the unpopular budgets that his overlord had declared.
No hint of balance there, is there?
Your response, as always, will be to say that you know nothing of these things.
As so often, you will be right.
A group of us were at the Glasgow meeting. Some warmist starts up with "peer review says".
As soon as someone like that opens their mouth and mentions peer review you know it is corrupt.
Why is it that sceptics seem to say "the evidence shows" (or words to that effect) but alarmists talk about "peer review". It seems to me you become obsessed with peer review, when you just do not have the evidence to back your views.
Indeed. As we all know Einstein wasn't peer reviewed. Or pal reviewed for that matter...
RichieRich said:
The former is claiming to do science. The latter is a think tank making use of science. There is no reason why both should practice identical publication methods seeing as they are not alike.
Proper peer-review in the lobby group sector may or may not be beneficial but everyone can appreciate that what comes out of a lobby group is coloured by the values that lobby group holds. The effect of either method is of no consequence.
Pal-review in science, however, *is* damaging because it aims to artificially strengthen a particular idea, prevent competing ideas from being considered and perverts the traditional review process. It turns science into advocacy with the added power of being able to diminish the voices of your opponents. With pal-review science there is an opportunity to crowd out others that doesn't exist in the lobbying industry regardless of the review process that think tanks, pressure groups and whoever else might use.
It seems to me that this concern arises from an unfounded view of what peer review should be (and is, in most areas of science). Peer review is essentially a vetting to ensure that a paper is worthy of publication, not a stamp of approval that says it is right. The corruption of peer review in some climate science circles has led to a situation where peer review is inflated to present it as an official blessing on the truth of the paper's assertions.
Despite peer review, even the most august journals publish papers that are unworthy of publication, and end up having to be withdrawn.
A lobby group such as the GWPF is free to publish whatever it likes. It does not have an obligation to the readers of its papers to make sure they are fit for publication. Its only obligation is to itself, to make sure that the papers it publishes enhance rather than diminish its reputation. Even with that proviso, it needs to be recognised that much or most of what it publishes will likely be regarded as rubbish by its opponents. Nevertheless, it has a strong interest in ensuring that whatever it publishes is as soundly based on facts as possible, and that its arguments can withstand challenges.
It is difficult to conceive of how the GWPF could obtain review of its papers from parties strongly opposed to its views. With a journal, an opponent of a particular view has an incentive to review a paper they do not agree with - they can present arguments why it should not be published, which may be successful in preventing publication. An opponent would not have the same incentive to review a GWPF paper, because they would know that their review would be very unlikely to prevent publication.
So it seems that GWPF must continue to rely on friendly reviewers playing "devil's advocate". This could, of course, include people outside the GWPF board, but is still not going to have the same effect as journal peer review. It is unlikely to add much to people's perception of whether the papers they produce justify publication.
I don't see how one can reasonably criticize Phil Jones et al for trying to enforce "pal review" and at the same time claim that GWPF is forced to go down the "pal review" path.
As pointed out above, you are not comparing like with like.
If Greenpeace issue a report in their name, even one collating other science, then one would not expect it to be peer reviewed. We know they are partisan, and only a fool would trust Greenpeace without asking their opponents for a view. (But if a Greenpeace paid scientist publishes a journal paper on a straight scientific question, then one would expect it to be reviewed.)
There is no double standard being shown by sceptics on this. It isn't practical to let a party who has every reason to slow down the reports and make life awkward to have access, even if they agree to vet them in good faith and promise not to tell others what they contain.
RichieRich doesn't appear to realise that in any quantitative discipline, numbers are sacrosanct. What Stern did was dissemble by choosing an undeclared discount rate. The fact that this was let through by that 'peer review' shows that those people were either incompetent or part of that dissembling.
The GWPF peer review appears to be based on objective science, as it should be. A rider to this discussion is that all climate science papers in the subjective journals must be re-reviewed by independent reviewers subject to stiff control to make the process objective.
Within the peer review process should be a quality control standard. Reviewers should ask are the conclusions substantiated by the science or arguments. They will suggest changes that will help improve the quality.
It needs a combination of expertise, objectivity and empathy.
When comparisons are made with peer review, it is not words but standards that should be the basis. I have looked at two papers this year with lead authors Gergis and Lewandowsky where basic checks were not made in peer review. The GWPF review procedure seems to set a much higher quality standard.
Alex Heyworth is exactly right. Peer review is in most disciplines a fairly low hurdle, and the reviewers are emphatically not re-doing, checking, validating the author's work. They are answering the question, "Is this paper worth publishing" not, "Is this paper good/correct". If a paper is published and later found to be utter balls that is the author's problem, not the reviewers and to a lesser extent not the journal's either. Because a plausible argument would only be found to be utter balls by re-doing the work the paper reports, which is not what reviewers do. Re-doing the work is what sometimes happens to a published paper: and sometimes the author's results cannot be replicated, which is when people decide the paper is balls.
It seems a particular trick of Leftists to change the meanings of formerly commonly understood terms. 20 years ago no-one engaged in science would have said that peer-reviewed equates to good, correct etc, and current practitioners know any such interpretation is balls. Far be it from me to suggest that certain scientists have a political agenda, and by changing definitions seek to sway the minds of the less well informed.....
@RichieRich
Drafts of the Stern Review were seen by many people, but not by experts in the economics of climate change (its topic). Indeed, many of the more prominent climate economists published post-publication reviews - with Dasgupta, Heal, Nordhaus, Mendelsohn, Weitzman, Weyant and Yohe all coming out less than positively and only Quiggin batting for the defense.
If you do not want to believe me, you should read Stern's contribution to the Yale forum, where he confirms that the Stern Review was not peer-reviewed.
The whole idea that pre-publication review by whatever method is an indicator of anything is daft. All the reviews in the world won't help if you are wrong, no reviews at all are needed if you are right. In a world where any of us can publish anything, the item must stand on its own merits. If academic journals have some strange pre-requisites in their cloistered world, they don't apply to anybody else. They are going the way of the Dodo anyway, their racket is going to be bypassed and left in the dust.
mmm...so Bitbucket - I look at the cvs of the GWPF - Lord Lawson belonged to the Conservative Party and conflicted with Dennis Healey of the Labour Party many times. Also on the GWPF board is Lord Barnett, who was Dennis Healey's number 2 at the Exchequer in the 70s and against whom Healey made many attacks. Barnett's unwelcome role was to appear on TV to defend the unpopular budgets that his overlord had declared.
No hint of balance there, is there?
Your response, as always, will be to say that you know nothing of these things.
As so often, you will be right.
Sep 9, 2012 at 11:26 PM | diogenes>>>>
As I opined on another thread regarding the character who tries but can't hide his/her warmist prejudices:-
"We also know when we're dealing with a blinkered zealot who knows just a little bit about their favourite subject and doesn't realize it."
rhoda; climate science is complex; the models hide fake science. It was done by piggybacking Aarhenius' 'GHG blanket' debunked by Bohr and Angstrom, hiding false claims in correct physics. The first case appears to be 1981 [Hansen et al] who claimed 33 K present GHE when it's really ~9 K.
The IPCC established this as peer-reviewed writ when any professional scientist knows it's wrong. 15 years of constant, now falling temperature, plus ClimateGate alerted the rest of science to the fraud. When pal-review failed, journal editors were strong-armed, some were sacked: not nice.
The Warmist obsession with peer-review is because it can be controlled.
If you publish (and the paper contains nonsense or mistakes) you can point and say, "Ah, but it was peer-reviewed. Therefore your pointing out the nonsense/mistakes herein is invalidated." Harrabin, Revkin, Black et al can wave the peer-reviewed flag as a super trump card that wins any argument and go back to their offices smug in the knowledge they have defeated the evil deniers. It's a bureaucratic trick: stacking the committee.
However, if you publish to the world and say, "Help us find what is wrong with this paper," and let the world pick it apart, you are being a true scientist.
Guess which one the Team prefer.
Stuck-record: these people hate it when you publicise the correct science, still just emerging. The real GHE is very subtle because it has to explain how the Earth survived the 'quiet sun'. This means that not only do you have to explain the GHE, but also how the process offsets changes in tsi. There is only one logical outcome and it doesn't involve CO2!
I would like to echo the comments made above by Alex Heyworth and by bill. Peer review is or should be merely a considered comment by someone with (preferably) some understanding of the subject matter as to the suitability of a paper for publication in the journal concerned. A paper can be new, interesting and well written and still correctly be deemed unsuitable to be accepted for publication if its subject matter does not fall within the specialised field of the journal. Conversely the same paper should normally be considered acceptable for publication in a journal whose specialism it does fall within. It should make no difference in either case whether the conclusions of the paper are convincing to the peer reviewer or editor or not. Of course in some disciplines, such as mainstream climate science, it doesn't always work that way. Publication may be dependent on whether the conclusions of a paper support an existing paradigm. So much the worse for the journals concerned.
Regarding the GWPF, I might find its publications more interesting if they included well-argued contrary views.
Coldish
A journal editor worth his salt (and a reviewer with an open mind) should be keen to give preference to new work which actively challenges the received wisdom, always provided that the submission isn't so off the wall that the writer is patently on the edges of insanity. And even there the edges of scientific insanity have kinda shifted over the last half-millennium!
The refusal of some journals to publish anything that does not slavishly follow the current paradigm and conversely to publish any old rubbish that does (provided they can find excuse for the face-saving argument that it does say something new) really marks them out as no longer deserving of the description scientific.
I also agree with Alex and bill, especially the suggestion that "Leftists" — though I dislike the word since there are some on the "left" that would never countenance the shenanigans that bill refers to — resort to semantic tricks to make sure that only their version of reality is available. Both Lewis Carroll and George Orwell warned us against this sort of behaviour!
Mooloo, I agree completely:
Geronimo, are you suggesting that we should trust GWPF members to be objective because they are politicians from opposing parties? Ignoring your touching degree of trust in politicians, that makes as much sense as saying that they can be trusted to be objective because they have strikingly different tastes in neckties. It is clear from reading this blog that "skepticism" crosses party lines.
RKS, I don't try to "hide my warmist prejudices"; they are plain to see. BTW I have already received the "blinkered zealot" badge from you so I guess I can't count that. Or maybe I should count it as, "blinkered zealot and bar" in my chestful of badges. Expecting some new ones from you in return...
RKS, I don't try to "hide my warmist prejudices"; they are plain to see. BTW I have already received the "blinkered zealot" badge from you so I guess I can't count that. Or maybe I should count it as, "blinkered zealot and bar" in my chestful of badges. Expecting some new ones from you in return...
Sep 10, 2012 at 1:01 PM | BitBucket>>>>>
Argument for the sake of argument with the intention to disrupt the thread.
Even arguing it's perfectly ok to force our industries abroad causing unemployment, and force the less well off and pensioners into fuel poverty, through the implementation of politically motivated carbon taxes and resultant exorbitant fuel prices.
Yes, we know exactly where your prejudices lie.
DNFTT
GWPF and Greenpeace are not scientific bodies they are lobby groups who use scientific data to try to convince the people they are lobbying of the correctness of their position. I think the IPCC has slipped from being a scientific review panel towards a lobbying group and you can see the results..
Peer Review is a respectable scientific process with a long pedigree. It is not without its faults but it's meaning (and value) is being distorted by these same lobby groups, particularly moving the position that "peer review" is good to the position that publication in a "peer reviewed journal" is the only acceptable form of peer review and that this confers truth upon the publication.
For an analysis of the actual process of "peer review" see http://www.wca-environment.com/peer-review/
@Richard Tol
Thanks for your response. Can I clarify something?
In the acknowledgements of Stern it states
So, is it the case that, although Stern sought advice from these folk, they didn't get to see drafts?
@RichieRich
I can't vouch for Heal, but the others did not review the Stern Review. They may well have been in fleeting contact with members of Stern's team.
I was a consultant to Stern. I helped prepare stuff on the impacts of sea level rise. I protested, pre-publication, on a rumour of a single discount rate. But I sure was not shown the final draft with a chance to comment.
It seems to me that good or bad, the more peer reviewed journal articles a scientist has to their name, the more likely they are to recieve more funding. Being correct is less important than being published.
If the GWPF publish lots of repeatedly discredited reports (shown to be wrong, not just dismissed) they will not prosper the way a bad scientist will.
As I understand it, the GWPF may publish scientific papers, but it is primarily an economic and policy foundation. Using the best science available, which it may also publish, especially when it goes against received opinion, it is primarily concerned with the economic costs and implications of policy.
Richard Tol; Stern assumed the IPCC correct yet it assumes [2009 Trenberth Energy Budget] 5 times real IR energy absorbed in the lower atmosphere with temperature corrected by grossly exaggerated cloud cooling [G L Stephens 2010]. Then it assumes the absorbed IR is directly thermalised when that is impossible because of 'quantum exclusion'.
The IPCC was warned this was wrong in 1993 when Will Happer resigned from the US DoE rather than lie about it. Stern's Report is based on scientific quicksand.