Bishop Hill Zorita on Smerdon
Jul 24, 2010 Climate: Mann Climate: Zorita Eduardo Zorita has a must-read post up at Klimazwiebel, discussing a new paper by Smerdon et al. Michael Mann fans will be amused to read of geographical problems uncovered in some of Mann's papers, which will instantly bring to mind favourite episodes from the Hockey Stick story, like the Rain in Maine (falls mainly in the Seine) and the documentary records of East African climate from the medieval period (Mann et al 2008). Here's a sample:
In one case, when interpolating the climate model data onto a different grid, the data were rotated around the Earth 180 degrees, so that model data that should be located on the Greenwich Meridian were erroneously placed at 180 degrees longitude; in another case the data in the Western Hemisphere were spatially smoothed, while the data in the Eastern Hemisphere were not.
Ouch.
As Eduardo points out the implications are rather interesting, since Smerdon's findings imply that Mann's stress-testing must have been too weak to actually demonstrate what they purported to do. Fascinating stuff.
Reader Comments (29)
Read Zorita's account, and then the response linked to by the first commenter, to see Mann et al getting really annoyed by having their errors made public.
Their claim is that, had the problems been pointed out to them privately, "we would have acknowledged their contribution in a prompt corrigendum". Somehow I doubt that any correction coming from Mann et al. would have found room to mention that their programming ended up using points for comparison that were accidentally on opposite sides of the globe...
I don't have the time or patience to do it, but somebody ought to start keeping a list of how many times Mann admits that his inputs were wrong but 'it didn't change the output.'
Mann must have found the remains of the logs used in Thor Hayerdahl's Kon Tiki raft.
Harry Eagar,
That's the beauty of garbage-in-garbage-out.
Mann comments "had the problems been pointed out to them privately, "we would have acknowledged their contribution in a prompt corrigendum"."
Would that be before or after the authors have been smeared as fossil funded shills? Dr Manns record on this sort of thing is unambiguous, and well known to Smeardon et al I presume.
Were Wegmans comments politely acknowledged in a prompt corrigendum? Were MacIntyres? Why would Smeardon expect any different treatment from the good Dr?
Impossible! Even the most primitive proof-reading would have flagged such errors. Mann's appeal to discreet private fact-checkers presupposes climate hysterics, who under no circumstances would have stirred that pot. As for "peer review"-- when Mann invariably presents nothing but a steaming mess, why bother? Just hold you nose, mutter "hide the decline," and offer Bernard another biscuit.
It is not clear to me the significance of this error to the actual conclusions drawn by Rutherford and Mann. Certainly in their response they acknowledge the errors and then assert they are of no substantive import beyond they are embarrassing. Smerdon seems to have a history with Rutherford et al - so perhaps we need to understand the scope and nature of the prior issues.
Bernie may well be right about their effect, but how did such embarrassing and pretty bleeding obvious errors occur in a peer-reviewed paper? Does nobody bother to check the data during this supposed Gold Standard process?
Sorry - I forgot...Prof Jones gave us the answer back in Parliament
'They never asked'
John Blake
Impossible! Even the most primitive proof-reading would have flagged such errors.
Well, you just can't get the help you need in Penn State. All the research assistants are too busy partying!
Penn State Tops the List!
Oh, btw, Bishop do keep track of these errors so you can document them in Breaking the Hockey Stick
Bernie,
Read the Smerdon et al. article linked to above. Notice what they say the consequences are. Those errors have definite effects and they are not likely to be trivial, contrary to Mann et al.'s assertions. Note how they explain pseudo-proxies, and then consider attempting an analysis of a model of a region using a pseudo-proxy data provide by Mann et al. that you believe is derived from the appropriate region. If the data really comes from the other side of the planet, how reliable will your estimate of the accuracy of your model be? You want to compare apples to apples, and what have you really?
I once pointed out at RC how amazed I was that a paper with multiple authors had got through peer review when it contained a major and very obvious error. I was told in no uncertain terms by Gavin that I didn't understand the processes involved in publishing a paper. When I responded that actually I had many years of writing, verifying and reviewing technical papers and so I did know what I was talking about, I was moderated out of existence and have never had a comment through RC moderation since.,
The mistakes never seem to be significant do they? I can't be the only person that wonders how come the answer is always a hockey stick, regardless of the input to the model used.
I've read the paper. In any other science, errors of that magnitude would simply mean that the original paper was worthless. If, corrected, "that doesn't affect the conclusions", that does not meam that the errors did not matter - what it means is that the matter studied is trivial. Which is exactly the case of the original hockey stick.
When first I read a bit about Climate Science, my immediate impression was what a bunch of duds the practitioners were. I have been obliged to modify that impression over the years by the admission that a fair few seem to be crooks too.
Trust is the issue.
If Mann's group can blunder fundamentally on getting the basic data input wrong in simple studies, how can they be trusted to get the right answers in a really complicated study? How can we trust the 2009 Mann "Global Signatures" paper in Science claiming the Medieval Warm Period was warm only in limited regions?
Steve McIntyre says that Jones spent most of his career as a glorified temperature accountant. What does that make Mann? A failed temperature accountant?
Jones and Mann are out of their depth. The work of these lightweights cannot be relied upon as sources of global policy.
I understand the immediate consequences for the papers, i.e., gigo. However, what is less clear to me are the implications for the statistical methods that were being assessed.
To me the biggest thing is that Dr. Mann has shown a consistent streak of very sloppy work at the minimum: Flawed statistical methods in MBH98/99, The Upside Down Fiasco and now he adds a 180° twist to the globe. In any other endevor anyone that consistantly showed this level of shoody work would have been fired, not lauded as one of the tops in their field.
Would you keep on the payroll a cashier that consistently never got his count right at the end of the day? I think not. If you wouldn't keep wasting money on an incompetent cashier, why in the world are we wasting millions in grants on an incompetent scientist that can't even tell the western hemisphere from the easern one.
It is not just Mann, Rutherford, Wahl and Ammann, his co-authors, were all contributors to this embarrassment. How come they all missed it? Does it provide insight into the way they work? I wonder who the reviewers of the article were - for surely they also bear some of the responsibility.
EZ is of BV-VS fame, of which also a book should be written ("The return of VS")
Phinnie,
You want to re-visit the spectacle of a bunch of clueless warmists getting their a** handed to them, check out Tim Lambert's latest thread on Monckton
Bernie
However, what is less clear to me are the implications for the statistical methods that were being assessed.
Garbage in, garbage out. Computers just do what they are told. A number is a number. Meaning is what humans give. I suspect Mann would fail Jean S's statistics course. I know anyone who can't even get the data right would have flunked my course for sure.
Shub
Do you have a URL? I am not as familiar with the world of blogging as you.
Don Pablo
Here you go:
Monckton Vs the House of Lords
;)
"How can we trust the 2009 Mann "Global Signatures" paper in Science claiming the Medieval Warm Period was warm only in limited regions?"
Limited is certainly an appropriate adjective when discussing Mikes grasp of Geography.
It's becoming apparent, however, that this may have been one of his stronger subjects.
Excuse my sardonic observations but I've just got back from a trip to RC.
Shub
Thanks
If you took the sloppiness out of climate science, you'd have nothing left. The hits just keep on coming. The SST wild-ass guess, Briffa's magic single tree, Rahmstorf's embarassing "my pretend numbers say it's worse than we thought!", Jones' CRU code mess, Jones' Chinese data, Gavin's "September, October, whatever", the IPCC's "peer-reviewed" science, the gross incompetence of the recent whitewash efforts, and of course, pretty much everything that Mikey Mann has ever touched.
Sloppy, sloppy, sloppy. That's the best you can say for the work that isn't fraudulent. They can't site thermometers in accordance with scientific standards. It never even crossed their minds to check! They use models to forecast without bothering to satisfy the basic standards of forecasting. They never check anyone else's work. Heck, it's apparent they don't even check their own work.
Clowns.
The paper does not criticize directly, but earlier modeling work that Mann et al used. Mann has replied acknowledging the errors, but pointing subsequent work was not affected. This is the normal cross checking that is part of science.
Mann et al: "Smerdon et al. (2010) describe two technical errors in the model grid data used in Mann et al. (2005, 2007a). They are correct in the discovery of these errors. At the same time, we feel that they have not adequately addressed the fact that both errors did not occur in subsequent publications and that the main conclusions of Mann et al., 2007a, which supercedes Mann et al., 2005, are not impacted."
See: Mann et al: http://holocene.meteo.psu.edu/shared/articles/RMWAcomment_2010_jclim_smerdonetal.pdf
See also, Smerdon et al. (2010): http://www.ldeo.columbia.edu/~jsmerdon/papers/2010b_jclim_smerdonetal.pdf
Bishop - is it my imagination, or is there a change of tone, softer, in the RMWA response to Smerdon?
Two Penn State Related Comments
As someone who has an impressively - well maybe embarrassingly- large number of years under my belt as a Penn State student, Penn State grad student and Penn State research assistant I feel compelled to reply to two comments.
Pharos
Dr. Mann might well have discovered long forgotten Kon Tiki material at PSU. The Penn State Amateur radio club station K3CR- which is still in existence- was one of the stations that maintained short wave communication with the Kon Tiki expedition. The Club archives contain a QSL (confirmation) card sign by all the Kon Tiki expedition members.
John Blake
Penn State and the State College community does indeed have a very serious student drinking problem. The university and the town are struggling to to find ways to deal with it but it has been developing for a long time and is the result of multiple social, economic and cultural trends that are difficult to influence.
I can tell you science and engineering grad students and research assistants are rather scarce in the downtown party scene. About as scarce as good Arctic region temperature records. Extrapolation both cases produces misleading results.
This is just another example of sloppy work by Mann. What's interesting to me is that Smerdon had already published a comment about this paper. That should have prompted Mann and colleagues to double check their results. Instead they argued and now they are caught with their pants down. Arrogant and sloppy. The tone of Rutherford's comment is par for the course. Note also that Smerdon and Pollack found another unequivocal mistake by Mann in a previous publication.
Pollack, H. N., and J.E. Smerdon (2004), Borehole climate reconstructions: Spatial structure and hemispheric averages, Journal of Geophysical Research - Atmospheres, 109, D11106, doi:10.1029/2003JD004163.