Bishop Hill Bob Watson sees the light
Feb 7, 2010 Speaking about the new Africagate story - the best telling is at EU Referendum - Bob Watson has been telling Jonathan Leake about his views on how claims in the IPCC reports need to be substantiated.
Watson said such claims should be based on hard evidence. “Any such projection should be based on peer-reviewed literature from computer modelling of how agricultural yields would respond to climate change. I can see no such data supporting the IPCC report,” he said."
Peer Review Josh '10Well, of course. Anyone who believes in the integrity of science would expect the conclusions to be made on the basis of such hard evidence.
Presumably then, when Bob Watson was in charge of the IPCC, everything was based on hard science?
Well, not quite. While my impression is that there was less input from advocacy groups on Watson's watch, searching the Third Assessment Report for the word "Greenpeace" returns 52 results, including for example, references like this:
Hoegh-Guldberg, O., 1999: Climate Change, Coral Bleaching and the Future of the World's Coral Reefs. Greenpeace International, Sydney, New South Wales, Australia, 27 pp.
Gibson, M.A. and S.A. Schullinger, 1998: Answers from the Ice Edge: The Consequences of Climate Change on Life in the Bering and Chukchi Seas. Greenpeace Arctic Network, Anchorage, AK, USA, pp. 32.
Perhaps Professor Watson is a new convert to the cause of scientific integrity in the IPCC reports.
Reader Comments (21)
The Second Assessment Report of the IPCC was a watershed. There were two flaps. Ben Santer and the discernable influence; and David Pearce and the value of life.
WG1 tightened its standards as a result.
WG2 and WG3 loosened their standards. The "value of life" problem was a problem of the immaturity of the peer-reviewed literature. There were solutions in the grey literature and in related peer-reviewed literature -- but the strict standards at that time did not permit us to write about those. So, standards became more lenient.
In fairness to Bob Watson, although standards slipped on his watch and the IPCC slided towards advocacy, he resisted that. Things have gotten much worse since he left.
Bishop,
So what should the reports reference? Are you saying they should only reference peer-reviewed literature?
Frank O'Dwyer: I thought the IPCC had rules as to what it was allowed to reference; to my knowledge, the Bishop has no input to the IPCC rules.
Major contributors seem to work/consult more or less interchangeably between their real jobs, the IPCC, and advocacy groups like WWF and Pew
As detailed in the attached email from Wigley to Hulme, there has been a great sensitivity to maintain the illusion that the IPCC is other than an advocacy group itself.
http://www.eastangliaemails.com/emails.php?eid=152&filename=941483736.txt
Wigley's explanation of Watson's concern misses the larger issue that IMO it is inherently a conflict to try to triangulate so many positions, whether based on IPCC officially rubber stamped spin, or IPCC preliminary draft spin
IPCC Science Designed For Propaganda
http://canadafreepress.com/index.php/article/19702
Watson places unjustified faith in computer models - "could" not "would".
OT but do keep your snow shovels handy.
In case you haven't heard, the East Coast of the US is buried with 61 CM (about 24 inches) of snow, with more on the way. This in Bloomberg who can be trusted to report the real news.
You should see the storm in the UK in a few days.
http://www.bloomberg.com/apps/news?pid=20601087&sid=a24E7G24WYdw&pos=8
So Frank, what's yer spin?
Philip,
"to my knowledge, the Bishop has no input to the IPCC rules."
So what?
What do you think? Should the reports only reference peer-reviewed literature?
Frank: Let us know what you think.
My opinion, based upon long experience, is that any information contained in a document of such importance should be from a source which has not only been published as a peer-reviewed paper but for which all supporting information is available and which has been independently checked and if necessary replicated. That means all raw data and all codes and methodology. It certainly should not contain information which has been issued just in time to be included in the IPCC reports before anybody has had a chance to study it. It is clearly evident that happened with AR4.
Gosh, if it were restricted to stuff that had been genuinely peer-reviewed, rather than crony-reviewed, there wouldn't be all that much to say, would there?
dearieme: and a good thing too.
Surely the problem with much of the AGW 'case' is that it is, indeed, based on computer models and has been 'peer reviewed'?
This is idiocy of a pretty high order.
Frank
Yes, I would have thought peer-reviewed only was a reasonable criterion. I must say, I thought that was the rule.
The point about materials availability is also well made though. Knowing what we know about the undermining of the peer review process by the Hockey Team, it may be that we need to move to a world of open review and open notebooks (pace Paul Dennis).
@GCooper
While I admit it is a work only a mathematician could love, the samples of code that came to us through the leak have been seriously examined by a number of people, including our own Derek O'Connor. He has published an examination of the code here:
http://www.scribd.com/doc/26135665/Two-Simple-Statistical-Calculations-and-ClimateGate
You can download the PDF and read it. It does require some knowledge of computers and math, but is surprisingly easy reading for those who do have the background. It is meant for the graduate student turned DIY programmer, and explains what can go wrong.
Basically, he demonstrates where the code goes wrong and why the answers are bogus.
And to His Grace -- you should look at it carefully regarding what can go wrong with spread sheets, something I am sure you use in your "day job". I would hate to hear you got a "Hockey Stick" while balancing the books of your clients. :)
@Phillip Bratby
Your explanation of what the Scientific Method consists of that I was taught is right on. Exactly correct. Point by point.
Bob Watson could still be in what's known as the "negotiation" phase; alternatively, his conversion to the faith might only ever have been an opportunistic one.
Either way, there seems to be an overwhelming case in favour of ignoring everything he says. There is certainly little, if any, evidence that the Bob Watson we see today holds any to principle beyond self advancement.
And is computer modelling "hard science"?
I have a computer model which tells how many mice it would take to eat the Moon if it were made of green cheese - so the Moon must be made of green cheese.
"What do you think? Should the reports only reference peer-reviewed literature?"
No, for this is argument from authority. It should only report results that have survived the best attempts to discredit them that science has to offer, in circumstances that would reasonably be expected to discredit them if they were in fact false.
Peer review is an editorial function, a filter to see if a paper is sufficiently interesting, novel, and well-constructed to be of interest to the journal's readership. It is not the function of peer review in the physical sciences to actually check that the results are correct, although one would be expected to check that they weren't obviously wrong. (And as such, one cannot assume that peer reviewed papers are correct simply by virtue of being published. Maths does tends to be a bit different.) The actual checking process is supposed to be done by other researchers in the area reading the paper and replicating the results. And journals are no more than a device for researchers to more efficiently locate results of interest to try to replicate, refute, or extend.
You not only have to be able to cite the literature, you also have to know it in enough depth to know what has been replicated, what is in dispute, and what has been subsequently reported to be wrong. It should not be a simple keyword trawl of the literature, which any librarian or graduate student can do.
If Steve McIntyre publishes an analysis on ClimateAudit that RealClimate have tried hard and failed to knock down, and this isn't because of data hiding or anything like that (or vice versa, of course), then you ought to be able to cite it. What matters is the strength of the failed attempts made to discredit it. Anybody who even suggests using peer review by supportive journals for this purpose should be immediately fired.
Bishop,
"Yes, I would have thought peer-reviewed only was a reasonable criterion. I must say, I thought that was the rule."
Me too. It also seems that many in the IPCC thought so also, which suggests that the culture was normally to use peer reviewed material, especially for WG1.
"Knowing what we know about the undermining of the peer review process by the Hockey Team"
But that's a load of trumped up nonsense, isn't it? It is clear that 'sceptic' (and probably, other) papers which never should have survived peer review got published anyhow. This does not support the notion that peer review had the bar set too high. And anyone can set up their own journal, such as E&E - the issue is simply getting experts to pay attention to it.
Ultimately I suspect this is the issue that deniers have with peer review - they don't get enough respect from experts because the vast majority of their arguments deserve none. Those that have some merit have minor impact, and putting them through peer review would draw even more attention to that fact.
"it may be that we need to move to a world of open review and open notebooks (pace Paul Dennis)."
Nothing wrong with that, except that practical IPR issues and costs that come with it. Replication of results is not a simple matter of rote repetition of algorithms and methods, either, is it? If anything a consistent result achieved using independently developed software and methods is more impressive, and the more independent the better. It's also worth noting that scientific advance has not exactly been crippled by the lack of this so far.
Plus, even with open notebooks you still need some form of peer review system so that genuine experts don't have to waste time figuring out which self-appointed 'auditor' has met some kind of minimum quality hurdle and which has not. There are only so many papers anyone can read.
Nullius in Verbia,
"No, for this is argument from authority. "
Argument from authority is fine when the person really is an authority.
"If Steve McIntyre publishes an analysis on ClimateAudit that RealClimate have tried hard and failed to knock down, and this isn't because of data hiding or anything like that (or vice versa, of course), then you ought to be able to cite it."
Failed according to whom?
"And journals are no more than a device for researchers to more efficiently locate results of interest to try to replicate, refute, or extend."
Exactly so. The rest can be published in E&E, or CA, or WUWT where they will seemingly publish anything except a correction.
Frank O'Dwyer:
"Ultimately I suspect this is the issue that deniers have with peer review -"
This is disingenuous, isn't it?
Because the constant refrain about the sanctity of peer review has been used as a fallacious argument by the FOIA evaders and the IPCC grandstanders to the extent that the term 'peer review' has become synonymous with 'argument ad autoritatem'.
Peer review has been cited in exaltation of the warmist wisdom from day one- even as they conspired to 'rewrite the peer review process' in order to suppress dissent. Their vital concern with peer review was to astroturf it, the better to use it as a logical fallacy in persuading people to accept predation.
The problem I have is with the highly organized, institutional, scientific parasitism.
Real bottom line:
Anything true is provable.
Cause precedes effect. < you CRUballs really need to begin HERE.
"Argument from authority is fine when the person really is an authority."
Ha! And division by zero is fine when there's nothing wrong with it. Affirming the consequent works fine because you can deny the antecedent. And correlation really does imply causation so long as it's all done in a Good Cause.
If the United Nations says two plus two equals five, then it must, because you can't get a higher authority than that.
Tch. Authoritarians... :)
The meaning of "authority" in logic is not the same as its meaning in normal discourse. An authority in logic is someone who makes the rules. A child's parents are the "authors" of its bedtime. Science has no authorities in this context.