Bishop Hill Me and the NAS
Jan 6, 2012 Climate: MWP In the comments to the last thread, Richard Betts is amazed by my statement that the NAS panel's version of events is closer to the truth than the IPCC's and wonders if I now accept the Hockey Stick.
I'm not sure why. The NAS's idea that it is "plausible" that modern temperatures are warmer than medieval ones is surely unarguable. Nobody is suggesting that such a thing is implausible, just that there has been no demonstration that it is the case.
Saying that it is plausible that modern temperatures are unprecedented is therefore essentially to say nothing very much, and is clearly much less objectionable than saying that modern temperatures are "likely" unprecedented.
That said, the NAS panel did still conveniently fail to report the use of bristlecones in most of the temperature reconstructions they discussed, despite North apparently being aware that at least some of them did so. For him to argue that these studies in some way justified the Hockey Stick seems, ahem, less than than straightforward.
In some ways, I think Richard and I are getting caught up in the backdraft from all the spin that went on in the wake of the NAS report. When journalists present the word "plausible" as exoneration of the Hockey Stick, you can end up talking at cross purposes.
Bishop Hill
A later thought - does the word "likely" have any place in civilised discourse on the state of our knowledge of temperatures of the last millennium?
Reader Comments (73)
Richard
Has anyone ever done any research on what probability ordinary people attach to the term "likely" ?
It seems to me that there is a gap between "evens" and "likely" for a term such as "more likely than not".
Is it "more likely than not" that current temperatures are unprecedented?
I think that state of our understanding would be better explained as "we can't say one way or the other".
As Michael Mann has, as far as I know, signally and surprisingly failed to show any interest in or make any comment on the conclusions of the HSI, I am quite surprised that Richard B. thought it worth sending him a tweet.
per,
This is from Gerry North's written response to supplementary questions in the record of the 19 and 27 July 2006 Congressional Hearings on the HS - my bold.
Is is really possible that Prof. Slingo was unaware of Gerry North's correction? Was Nature unaware? Citing NRC, 2006 in AR4 Chapter 6 and subsequently relying on its non peer-reviewed press conference (trumpeted by Nature) to justify the use of "likely" (which the IPCC define as a 2/3 probability) in the face of Gerry North's 'on the record' correction is as big a scandal as relying on a WWF paper in Glaciergate.
Messenger, Richard Betts is fundamentally a nice guy. He still sees hope in the case of people that the rest of us wrote off years ago.
Thank you Richard for taking the time to answer, much appreciated.
Richard Betts: Tell me you're pulling the Bish' leg? I've always found your contributions enlightening, and, I believe to everyone else does. You're not doing a BBD on us are you? That's spending years cultivating yourself as a moderate warmist willing to listen to all sides and then one day changing into Captain Alarmism? Please say you're not I have too many questions I want answered, the first one is can you put the papers where you've examined the benefits of global warming back on this thread I've lost the place where you'd put them on and hadn't had time to read them.
"Richard Betts is fundamentally a nice guy"
Couldn't agree more, at least based on my conversations with him here. However, I've still not had any reply to arguments I left on Dec 14th and 17th over on the Evidence, confidence and uncertainties thread - including arguments relevant to today's topic. And I'm still interested to know Richard's response to the scaling argument I mentioned above (@10:00 AM) regarding the Hockey Stick. What's the problem with it?
Eduardo Zorita observed at the time that he thought that the published NAS criticisms of MBH were as severe as possible given the politics of climate science at the time.
Much of the subsequent spin by Nature came from comments by North at the press conference which were not drawn from the agreed text of the report itself. I've been informed by a member of the panel that this created considerable behind-the-scenes controversy with at least one member (not Christy) threatening to give his own press conference rebutting North.
At the subsequent AGU, another member of the panel told me under drop-dead confidentiality that his view was that we had conclusively demonstrated the pointlessness of the Mann-type studies and that the only way for the field to advance was the development of better proxies - a project that he thought might take 10-20 years.
On no occasion did the NAS report dispute any MM finding.
I corresponded with Slingo on the provenance of her statement to the Parliamentary Committee. She said that it was based on IPCC AR4. This assessment, as widely discussed, inserted Eugene Wahl's language (through surreptitious correspondence with Keith Briffa) between the final draft sent to external reviewers and the final report.
I'm just a clown and no one need pay any attention to anything I say. (Indeed, my last contribution to the International Journal of Clownatology was rejected, but that's a story for another day.) However, I have consulted both the OED (1971 ed.) and Webster's New World Dictionary (NWD) of the American Language (College Edition, 1966) in an attempt to elucidate the differences between the British Standard English usage and American Standard English usage of the word 'plausible.'
The first two definitions given by the OED are marked "obsolete," so I pass to number 3: "Having an appearance or show of truth, reasonableness, or worth; apparently acceptable or trustworthy (but often with implication of mere appearance); fair-seeming, specious."
Webster's NWD says "1.seemingly true, acceptable,etc.: often implying disbelief; hence, 2. specious. 3.seemingly honest, trustworthy, etc.: often implying distrust, [example omitted]. The NWD's further discussion of this under "synonyms" implies that 'plausible' lies on a continuum between 'specious' and 'credible' but somewhat removed from the 'credible' end of things.
Inasmuch as the NAS and its members are American, and are arguably at least as well-educated and literate as the average clown, the inference one draws is that they placed little credence in the methodology and/or conclusions of the paper under consideration. Speaking as a clown, I would not be satisfied with a review of my work which categorized it as merely 'plausible.' Obviously, climatology has a different set of standards.
An interesting Clintonesque discussion.
Steve McIntyre
Good to see you here.
Do you have any thoughts on the use of tree ring records as indicators of vegetation change, as I mention above, or any papers you recommend on this subject (the point here being merely to establish what the different strands of evidence show for change / no change, without worrying about attribution to causes at this point). I realise your focus has been on their use as climate proxies, but given that you have clearly done a lot of reading on the subject I wonder if you have some thoughts on the wider issues.
Also may I invite you to review our first order draft when it comes out in June. WG2 will have a similar open call for reviewers like WG1.
Hi Philip
Ah, sorry, hadn't seen your comments on the discussion thread, I was at the IPCC WG2 lead authors meeting at the time and didn't have much time for blogging! I'll take a look.
Hi geronimo
Thank you. Yes I was a bit - at least, I was genuinely interested in what I thought was a change of position (see above for clarification), but I chose to highlight that in a provocative manner because I was in a mischievous mood :-) I didn't seriously think the Bish would have accepted the Hockey Stick.....
Did you mean this and this? They are about both pros and cons.
Cheers
Richard
Dr. Betts's last link in the prior post got mangled. Try this instead.
Your Grace
I think this is still the problem, and the problem Dr Curry's been trying to slay with the uncertainty monster. How to communicate effectively between scientists, politicians and the public. To me, likely is >50% probability of some outcome. It's a low certainty outcome and as an engineer, it's a beer mat risk assessment at best.
If it's to support a big job, like saving the planet or just decarbonising it, I'd expect a lot more certainty. It may mean something different to a scientist, like asking for more research funding to narrow the uncertainty. I'd never get a business case passed just by saying something is likely to work.
The problem with the Hockey Stick is it was promoted way beyond a level to which it can be supported, and used to provide the supposed context to our climate history. So our current climate is allegedly unprecedented, despite the problems with the methodology and data. If other data suggest it's not unprecendented, then it's hard to justify policy action and we need to understand what may have caused abrupt, natural climate variations.
I still think it is quite amazing that all this discussion is generated on a historical temperature construction on the basis of basically 'Tree Rings'. Unless the person proposing this history can come up with a hugely convincing argument on why tree ring widths would be able to do this then I think the average person would just laugh at the thought that this might be accurate. They don't even grow in the winter do they?
We could test this tomorrow if someone could propose a location where they think good trees are located and it has a good thermometer record nearby. Just predict in advance the widths every year and take some cores and see if you are correct. At the same time you could also then analyse whether rainfall or sunniness or maybe temperature in the spring is more important, if you have that data to work with.
Mooloo
It is the imprecision of the word 'likely' in a scientific report that people are objecting to - both its meaning (somewhere between 50 and 90%) and how it is determined
Then "people" are wrong.
All scientific reports will use words like "likely". Good ones will actually quantify that amount: 95% confidence is usual. "Likely" has a standard meaning, it is 67% confidence, and generally any paper will actually show the uncertainty calculation, although a political paper will likely bury it.
What people are objecting to is that they actually don't think that the supposed warming scenario is actually likely. I don't either. But I understand that while I think Sligo et al are wrong, that there is nothing wrong with the language being used to express their incorrectly held belief.
Let's not make our opposition to the current AGW theory, either scientific or political, become so pig-headed that we start to say stupid things like: "likely has no place in a scientific report". That would make us exactly the stupid anti-intellectual, anti-science bunch of loons that many Greens like to portray us as.
In fact the danger of AGW is too little of it has been couched with "likely", instead being couched in terms of certainty. We should be welcoming a bit of uncertainty in these reports.
You don't think that dangerous global warming is likely. Fine. Then object to the science.
You think warming is likely, but not that we should decarbonise. Fine. Then object to the economics.
But don't let your dislike of incorrect science or poor economics make you start saying stupid things about how the world needs to work.
The moment every report to the government has to be certain before it is accepted is the moment science dies. People here bang on and on about how science is always open to challenge, blah, blah, blah, ... Then as soon as someone puts "likely" in a report they start to go mental about how "likely" has no place in science.
dnftt
Twitter gets us all in trouble eventually :-)
Jan 6, 2012 at 12:49 PM | Richard Betts
As Milliband just found out ;-)
Mooloo,
You are flailing against a perception of what people are saying, that exists in your own mind.
Firstly, the words 'likely', as derived from a statistical test has its standard meaning. What if you find scientists using the same language to support non-statistically derived quantitative probability estimates? (for e.g., estimates of probability based on expert opinion).
Compare the NAS statement with the IPCC-derived statement: the NAS statement uses more precise language couched in the proper context. The IPCC, by design, plays fast and loose. It uses statistical language in a non-statistical context. Does the IPCC provide the statistical test it performed to evaluate how likely it is that the current temperatures are the highest in the past 1300 years?
In other words, it is the IPCC which uses the word 'likely' in a non-statistical sense.
Secondly, Stephen Richards used it in the right context. It was you who conflated unrelated issues of vaccination and muddled the issue. Lesser levels of certainty are always enough to take beneficial action, if the expected rate of unintended adverse effects is very low. This is the case with vaccines. Taking action that would be vastly harmful in the anticipation of meagre benefits requires higher levels of certainty. Even if statistically derived, a mere 'likely' is not enough to form the basis of actions that are sought to be carried out in the name of the IPCC.
Thirdly, you say, "Then as soon as someone puts "likely" in a report they start to go mental about how "likely" has no place in science."
Who said this? You are saying this, and no one else really. You are imagining the thumbless coterie of science ignorant deniers who would say something like this, and that is what you have seen. To my reading, it is clear what Stephen Richards said ('likely' is not enough for stunt that the UN is trying to pull'; and 'how it is determined). It is clear to me what the Bish is saying in the post update as well
Tree-rings-as-thermometers has already been tested in big greenhouses - see the climex.doc document in Climategate2.
The IPCC definitions of likelihood and confidence are subtle. Confidence is the assessed probability of a statement actually being true. Likelihood is the probability given the models, assumptions, or current scientific understanding. Thus one would say one has very high confidence (90%+) that it is about as likely as not (33%-66%) that a coin will come up heads.
This means you have to be very careful interpreting statements described as 'likely' in IPCC language when they are derived from models and assumptions. The stated probability does not include any uncertainty about whether the scientific theory is correct. This is most notable in the famous IPCC attribution statement: that it is very likely that more than 50% of the warming since 1950 is anthropogenic. It's not the same as saying they have very high confidence that this is so. If we're going to start parsing words like 'likely' and 'plausible' as being in IPCC-speak, we need to be very cautious.
As for the NAS statement on the Hockeystick, it's a masterpiece of true-but-misleading.
"The basic conclusion of Mann et al. (1998, 1999) was that the late 20th century warmth in the Northern Hemisphere was unprecedented during at least the last 1,000 years."
True. That was his basic conclusion. But with a wrong method and messed up data, the conclusion is unfounded.
"This conclusion has subsequently been supported by an array of evidence that includes both additional large-scale surface temperature reconstructions and pronounced changes in a variety of local proxy indicators, such as melting on ice caps and the retreat of glaciers around the world, which in many cases appear to be unprecedented during at least the last 2,000 years."
This is partly true. The conclusion has indeed been supported by large-scale reconstructions - again by bad methods and data, and hence with no more foundation than the original. But if I understand correctly what they mean, the pronounced changes in local proxies are in the main short-term, local, and subject to other factors, and so don't weigh in on the global, millenial question being considered.
"Not all individual proxy records indicate that the recent warmth is unprecedented, although a larger fraction of geographically diverse sites experienced exceptional warmth during the late 20th century than during any other extended period from A.D. 900 onward."
This isn't known. The data isn't good enough. And that word "extended" is tricky. The late 20th century is about 25 years of warming, but many older records don't have that resolution.
"Based on the analyses presented in the original papers by Mann et al. and this newer supporting evidence, the committee finds it plausible that the Northern Hemisphere was warmer during the last few decades of the 20th century than during any comparable period over the preceding millennium."
On the basis that we have no data with which to reliably determine the global (or even NH) temperature to that accuracy, and hence no proof that it wasn't, it is indeed plausible. But "Based on" is misleading. You can't base a conclusion on bad science. You can't base a conclusion on results you haven't checked, or that you know to be full of errors.
It's a reflection of an attitude apparently pervasive in some parts of climate science - that getting the right answer by the wrong methods counts as a vindication. When teachers ask you to show your working, it's because they understand that the method is more important than the answer. But when Ed Cook got to review a sceptical paper which showed the dendrochronology methods they used to be rubbish, proved it mathematically, and then proved it again in a practical sense with Monte Carlo trials, what he wanted to do was try to get the paper rejected or altered on the basis of what it meant for one particular real-world reconstruction, for which the actual truth was not known and therefore not testable. If he could show that "the errors don't matter" then they could keep the conclusions without worrying that they was based on errors or invalid methods.
In a sense, whether the 20th century is unprecedented or not is not important - what really matters is whether science as a matter of principle is going to insist that all scientific conclusions have to be justified by correct, valid reasoning and data, that methods and results have to be checked before being accepted, and that known errors automatically invalidate the conclusions until the work has been re-done, re-checked, and re-submitted using methods known to be correct. On this far more important point, the NAS failed utterly, capitulating to the politics.
That's my view. I'd be interested in whether it accords with either of the opposing views of the NAS conclusions being discussed here.
I'd always assumed that the word 'likely' was chosen to avoid anything more precise, since it can mean any degree of certainty from 50-100%. I wouldn't expect to see it in anything authoritative.