Bishop Hill Zorita on McShane and Wyner
Aug 20, 2010 Climate: Mann Climate: Statistics Climate: Zorita Eduardo Zorita has an interesting review of McShane & Wyner's paper on the statistics of paleoclimate reconstructions. He takes them to task for not understanding the actual methods used by paleo people and in particular those used by Mann et al in the Hockey Stick papers. Although one can point to the foggy writing in MBH98 in defence of M and W, Eduardo's point that they should have worked more closely with paleo people is fair enough. That said, Eduardo does spend quite a lot of his review criticising the preamble to the paper rather than the guts of the thing.
When he does get onto the paper, he is less critical, but not convinced. He describes M&W's comparison of reconstructions based on proxies to those from various forms of noise as "interesting and probably correct" but doubts its usefulness because they have used a method that is not used by paleo people. I'm not sure about this - surely if there is a real temperature signal in the proxies, they should outperform noise regardless of the method?
There is a modicum of agreement on the last part of the paper though, with Zorita agreeing with M&W's conclusion that the uncertainties in paleo reconstructions have been underestimated. However, he says that this observation is...
...hardly revolutionary. Already the NRC assessment on millennial reconstructions and other later papers indicate that the uncertainties are much larger than those included in the hockey stick and that the underestimation of past variability is ubiquitous.
I guess the IPCC missed that memo.
Reader Comments (31)
Zorito misses the point. Paleo results from arcane methods are not properly supported by STANDARD statistical methods that may readily be applied to the data and are presented with claims of skill and confidence that are unwarrented.
Your Grace
You’ve imitated a mistake first made on a Guardian thread, i believe, of confusing McShane and Wyner with Marks and Spencer.
Too long fraternising with the enemy, I’m afraid. You’re in danger of going native.
[BH adds: Whoops. Fixed now.]
Agree Peter. Alot of the critics of this paper seem to be making more complicated than it really is. Zorito seemed to going all over the place with his comments -- hard to follow
( maybe that was deliberate).
Briggs's comments are much more readable
http://wmbriggs.com/blog/?p=2773
The problem is that paelo-reconstructionists refused to work with professional statisticians.
Dr. Zorita's critisism of that Stataticians should have consulted with Paleo people on how to do Statistics work sounds right out of the "Team" playbook: "Your not a Climate Scientist so you don't know what you are talking about".
What's next Astrophysicists should ask Paleo people how to do their jobs properly?
There's also this from the Deep Climate guys
http://deepclimate.org/2010/08/19/mcshane-and-wyner-2010/
Again I think they are trying to say that climate scientists are better at statistics than the experts?
If I'm repairing my washing machine, and have to loosen a bolt, then, since I am a highly thought of solenoid winder, my intuitive decision that the correct tool for the job is a left handed three lug tensioning whapple grip is quite clearly correct. When a mechanical engineer tells me that I should use a 10mm spanner, he clearly hasn't understood the complexities of the context within which I have to work.
I thought the object of the exercise was that the stastiticians applied normal statistical methods to the data to see what the result would be. They very low signal to noise ratio and it is difficult to discern a signal in the data so they used the best known published statistical methods available.
The paper is clearly dynamite for the Team it has been around for almost a week now and there's still no technical refutation, just ad homs and now accusing them of not working with the paleo community, presumably because they erroneously believed that the statistics should be done and the output should be what the statistics give us. Of course this isn't how the paleos work, they start with the answer and use statistical methods that provide the answer.
The basic point Zorita makes about not essentially having a paleo specialist as co-author makes sense. Clearly Zorita and his recent co-authors, already recognized the main statistical conclusion of M&W - the proxies do not allow much confidence in any point estimates for pre-instrumental global/NH temperatures.
More puzzling to me is how the Frank et al (2010) paper, which more generally calls into question proxy reconstructions, does not mention McIntyre and McKitrick or Wegman. Methinks Zorita's effort is way too defensive. If M&W are correct, then the steps suggested by Zorita et al to address current limitations in reconstructions are plainly insufficient.
The magic of this is that serious papers are getting serious reviews by serious reviewers here in blogland. It will be interesting to see how these reviews and comments affect M&W.
My take on Zorita's discussion is that it is absolutely clear that the science isn't "settled". As I read his post, it occurred to me that a lot of this is no more accurate than reading tea leaves or examining goat entrails. The process is nothing more than hand-waving "beautified" with academic credentials, buttressed by lots of numbers and multi-syllabic vocabulary.
It is but a tale, told by PhDs, full of formulas and warnings of doom, signifying nothing.
I thought Zorita landed some blows. When I read the paper I thought they went out on a limb early on about detection and models, I thought they overquoted M&M and Wegman, and opened themselves up to criticism on the parts I could understand, let alone the parts I couldn't.
So I never thought this paper was the great white hope of Hockey Stick smashers, but it is a decent brick in the wall.
Zorita has admitted to the real flaw in the of paleoclimate: it is not reproducible by standard statistical means. It can only be reproduced by those who follow exactly what the paleopromoters tell them. Any outside review of paleoclimate leads to showing that it is not real.
MW were doing the right thing to not rely on paleopromoters to advise them on their audit.
That would have made as much sense as asking Bernie Madoff to advise the SEC on how to audit his funds. Oh, that's right: he did.
If, as Zorita suggests, this has led them into errors that will tarnish their conclusions (and it seems they were pushing at a somewhat open door anyway?), then they have underperformed. If they were wrong about ice core air bubbles (paragraph 1), that's weak, for example, and, as Zorita says, evidence of non-expert reviewers too, even though it is an irrelevant mistake in terms of the paper. I would automatically diss someone critiquing my field if they made a simple error of fact.
I understand that it read well, as Jeff or Anthony said, almost like a blog piece in parts, but that ain't the point, which is to be right on the science, in this case stats employed in paleo.
This para I was sad to see in there : "On the other hand, the effort of world governments to pass legislation to cut carbon to pre-industrial levels cannot proceed without the consent of the governed and historical reconstructions from paleoclimatological models have indeed proven persuasive and effective at winning the hearts and minds of the populace. Consider igure 1 which was featured prominently in the Intergovernmental Panel on Climate Change report (IPCC, 2001) in the summary for policy makers1. The sharp upward slope of the graph in the late 20th century is visually striking, easy to comprehend, and likely to alarm." That's just not relevant to the paper, and adds an impression of bias.
As with commenters at CA and WUWT some of you, including Zorito, are missing the point.
This paper is not about paleo-recons, is not about Temp - time it's about statitistics. It doesn't need a paleo, it doesn't need a Mann. They were using the team's data and plugging it into statistical methods that ARE ACCEPTED by the wider and more knowledgeable statitistical community. What they showed was as you have ascertained. The HS has NO, NONE, de NADA, AUCUN acceptable statistical value. ZORITO et al know they have been rumbled and are searching for strawmen with which to attack the paper. Sorry team Mann, you lost.
Roddy Campbell
No, NO NO its not about science. Its about stats. They used the teams rubbish data and still could not produce the HS. Get it straight. They are STATISTICIANS. They did statistical analysis. Not scientific analysis.
If only that amateur guy Copernicus, a mere parish priest, had worked more closely with established astronomers instead of watching stars from the belltower of his godforsaken church in Poland, he would have seen that Ptolomeus oeuvre cannot be so easily dismissed. Fifteen hundred years of impeccable record and hosts of professional astronomers back him. The same goes for that improvised star-watcher with his unreliable telescope, this amateur individual who calls himself Galilei, who confuses aberrations of his instrument with impossible things in the sky such as Jovian moons and lunar mountains. They do nothing but obnoxious noises and hinder scientific work, not to speak of weakening well founded beliefs on the centrality of Earth in the Universe. We should go to town with the Inquisition bosses in Rome, and have them banned from publication, even if we must change the church rules on allowing publicartions. They should never get a Nihil Obstat or Imprimatur from the church, and they should possibly be shown the torture tools to convince them of the convenience of recanting their outrageous views.
Hector M reframes the trap Zorita and apparently all of climate science has fallen into very well.
@Roddy Campbell,
And the authors of AGW promotion papers standing in front of cameras, Congress, Parliament, profiting off of the IPCC, etc. produces an impression of exactly what, pray tell?
And for bonus points, please explain why climate science depends on statistical analysis that is not recognized as meaningful by anyone except climate scientists?
gosh you are an angry lot.
The EFFECT of the paper is, in my view, lessened by:
a) obvious errors of fact (I take Zorita's word for those), even if unconnected with the paper's results, which also means that there was no paleo-expert reviewer (you can pretend that doesn't matter if you want, but it lessens the EFFECT).
b) the 'blog-like' introduction (perhaps written with the spotty geeks in mind who read AAS, who may not be familiar with the paleo stuff, I accept that) gives an IMPRESSION of bias in the McKitrick-style echo.
They have made themselves an easier target for Deltoid and RC and CP, in my view, which is a pity. I would have preferred the paper to be as dry as dust.
PS: btw Stephen Richards, when I say 'the science' obviously I include statistics, especially in this context. Stats is science. So wind your neck in, take a chill pill. :) And rethink your last sentences? 'They did statistical analysis. Not scientific analysis.', as I don't think that's quite what you meant. We all hope they did science.
I think the importance of the paper (McShane and Wyner 2010) is that Academics in other fields (Statistics) are looking at climate science from their expertise and publishing papers on the topic. Maybe climate scientists will look at the paper and say hmmmm, maybe I could use some of these methods (e.g. - LASSO) in my work. Likewise, MMH2010 introduces panel regressions into discussions on climate science.
@ stephen richards, Aug 20, 2010 at 3:31 PM:
"Roddy Campbell
No, NO NO its not about science. Its about stats. They used the teams rubbish data and still could not produce the HS. Get it straight. They are STATISTICIANS. They did statistical analysis. Not scientific analysis."
Indeed so.
If you had written 'climate science', and 'climatic scientific analysis' - as perpetrated by The Team - then your point would have been even punchier.
McS&W used Mann's data, and they did not set out to give us yet another climate model. They showed, as did McIntyre, that the same hockey stick can be got with random data.
Talking about palaeo-dendrology, and telling them they don't know about that misses the point entirely.
And we are told to feel guilty and pay through our noses for the Team's atrocious statistics.
eduardo:
" For instance, for the first part of the millennial reconstruction 1000-1400, MBH used an inverse regression method with 12 proxy indicators and one principal component of the temperature field. This point is so clear in the MBH paper that it really shows that McShane and Wyner actually did not read MBH98."
MBH98 only started in 1400; they did not do anything from 1000-1400. Eduardo's error falls into the category of minor error, as he was probably thinking of MBH99. However, when one sees a minor error, it isn't always necessary to start suggesting that the perpetrator of the error didn't read the paper.
Per
Indeed. It needs to be a signficant error, or a repeated one, before that kind of accusation is warranted.
Folks should really read Frank et al (2010) in order to get the Zorita's broader context. I do think M&W has flaws and flaws hopefully that will be addressed. However, they have demonstrated the full consequences that others have more or less given the nod to - as can be seen in Frank et al.
Several commenters above have it right. It is a paper on statistics.The defence of the faithful is as always to set up a strawman, at the moment this is usually to approach it as a climate science paper, and to attack it as a poor climate science paperwhich brings nothing new to the table.
I'm ot sure whether they are unable or merely unwilling to understand the simple statement - 'It is not about climate, it is about statistics'
For whats its worth, I'm a qualified 'palaeo person' and know sweet FA about statistics. In fact, I shared a lab with a researcher who spent three years x-raying, measuring and computing the internal dimensions of his carefully collected and slide mounted microfaunal samples. After monopolising the entire college computer for days, he finally resigned himself to the conclusion that his three years effort resulted in no significant correlations at all. But then, he wasn't a climate scientist.
Zorita has taken review tips from Bob Ward.
Frank et al says this in its abstract:
"We suggest that much of the sensitivity in the reconstructions, a topic that has dominated scientific debates, can be traced back to the input data. It is perhaps advisable to use fewer,
but expert-assessed proxy records to reduce errors in future reconstruction
efforts."
As an old cynic with many years of reading scientific and engineering papers, the last sentence reads "It is better if the experts cherry-pick the proxy records that give them the desired results."
The paper does not establish the existence of an anthropogenic signal in the temperature records yet the authors state:
"Current anthropogenic activities have led to unprecedented trajectories and states in the
earth’s coupled climate system," and "For at least the next several hundred years, observed
climate variations will be a superposition of natural and anthropogenic forcing."
Which tells us where the authors stand on the issue.
Roddy Campbell
M&W or no more doing science than a proof reader is writing the book. They are in fact proofing the hockey stick stats.
The Hockey Team did not consult qualified expert statisticians during their data manipulations covering a 10 year span. Three peers of "independent" qualified statisticians have now rendered opinions that the Team used improper statistical methods (NAS, Wegman Panel, and McS & W), plus there is also the M & M paper which has challenged the Team's statistical analysis. Am I missing something?
The Hockey Team's peer review tactics have now been exposed via Climategate; i.e., "friend and family" review in sympathetic journals will never convince anyone but the faithful in future publications. So if the Team is to recover any credibility in future publications, they will need to address their critics with a rebuttal in a high level statistics journal. Until the latter happens, the Hockey Team has been relegated to obscurity.
I suspect that the Hockey Team is actively seeking some new players.