Bishop Hill Flipping bizarre
Feb 6, 2010 Climate: HSI Climate: Mann An interesting little development on one of the story lines from The Hockey Stick Illusion. In Chapter 14, I tell the story of one of Michael Mann's later attempts at creating a hockey stick shaped temperature curve - Mann 2008. This paper is not as well-known as the Hockey Stick itself, of course, but has become fairly notorious because of an oddity in Mann's algorithm. Because of the way it works, the algorithm is unable to detect the orientation of the proxy series in a dataset and in the case of Mann 2008, this failing had some unfortunate consequences, namely that some of the series ended up upside-down, with what would normally have been read as declining temperatures flipped over so that they looked like warming.
This error was picked up extremely quickly by Climate Audit readers, and McIntyre included this point in a formal comment on the paper. The correction didn't, however, prevent an identical error being made in a later paper, Kaufman 2009, which was written by some of the same authors as Mann 2008 (although not the HockeyStickMeister himself).
McIntyre's comment was accepted for publication but received a brief dismissal from Mann et al, who said that his observation was "bizarre".
Now, however, the story has moved on slightly. Kaufman has admitted the error in his paper and has just published a correction (although without mentioning McIntyre). Interestingly though, some of the Scandinavian scientists involved in collecting the original data series have revealed that they have been discussing the problem with Raymond Bradley, an author on both the Mann 2008 and Kaufman 2009 papers. Bradley, it seems, has told Professor Matti Saarnisto that the error was not deliberate, an explanation that the Finn is trying hard to accept.
Well, when Bradley says so to me, I don’t doubt even a slightest moment. I hold him in high regard. He is one of the best paleoclimate researchers, and … a frequent visitor in Finland.
Well, absolutely, but with the same error happening again and again, one does start to wonder. Certainly Saarnisto seems troubled.
It has been turned twice upside-down in Science, and now I doubt if it can be a mistake anymore.
It's fair to say that this position is probably widely shared. Lady Bracknell might have something to say about this: "To flip one proxy series may be regarded as a misfortune, Dr Bradley, to flip two...." And in fact the error has been observed no less than three times now. One hates to think what Lady Bracknell would think of that.
But if this isn't bad enough, there's an even sharper sting in the tail. If Bradley, one of Mann's co-authors on the 2008 paper, says the error was inadvertent, how is it that when he and Mann responded to McIntyre's comment on that paper, they described the idea that some of the proxies were upside down as "bizarre"?
Reader Comments (31)
Your book has just received a big boost on James Delingpole's blog at the Telegraph, what biscuits did he get?
You want bizarre.
I would like to know why it is you feel the need to get an American engineer fired from his job.
A few weeks ago I had a talk with the company Internet monitors over my desire to have a
tot of whiskey because I had typed that into a search engine. They were worried about my
severe drinking problem in as much as a tot (small child in American English) was an awful lot
of alcohol.
So the next time I try to get smarter and go to a website that said it could explain English slang. But when I typed "biscuits" into this websites search function I found out that it was
a dictionary of how apparently you Englishmen can use the simplest word and phrases
as euphemisms for bizarre sexual acts.
So now I have a meeting scheduled with the Internet smut monitors and still have no clue as
to the meaning of "they have no biscuits" or whatever the hell it was you wrote.
Why not go to court if the fraud is so obvious that even a lawyer or judge can understand it? Why not for heaven's sake?
From the correction:
The primary trends of the Arctic tempera- ture reconstruction, however, are not changed, including the millennial-scale summer cool- ing that was reversed by strong warming during the 20th century and (on the basis of the instrumental record) continued through the last decade.
So yet another error that doesn't make a damn of difference, except that:
The original conclusions of the paper have been strengthened as a result.
Funny how that's never mentioned by 'sceptics'. I suppose it goes against the insinuation of fraud when you have to admit that what you allege to have happened makes no bloody difference. Kinda takes away the motive, doesn't it?
Fascinating. Seems to bear out Steve's point that the conclusions are unaffected by the data. Now that's what I call a robust methodology - you get the same answer whichever way up you input the data. Oh, and we can ask that nice Professor Jones to tell us all about the instrumental record, now he has so much spare time.
Ladies and Gentlemen,
You are not going to believe this – http://www.timesonline.co.uk/tol/news/environment/article7017922.ece
[Snip - there's no call for that sort of thing]
Admit I am no expert on this but here's a theory on which I am happy to be corrected.
Could it be that Bradley is telling the truth and Mann at least believes he has a point?
The impression I have is that Mann's method is just a wiggle-matching algorithm.
He stuffs thousands of potential proxies into a programme that compares them with temperatures in a "correlation period". The proxies are just wiggles - deviations from a straight line, and are considered dimensionless in principle, so that each wiggle is tested twice - "right" way up and "wrong" way up - against the temperature curve in the correlation period. If either version of the proxy passes, it is then compared to temperatures in a different time-slice, a "verification period". If it passes a second time, the same way up as it was in the correlation period, then it is taken to be a good proxy of temperature - that way up - and is used to help reconstruct temperatues back in the past when there were no thermometers.
To be generous to Mann, one could say he is trying to get the maximum possible information by shoving as many candidate proxies as possible through the sieve, and using all those that pass. It's his method itself that determines whether thick layers of something or thin, or high or low values of an isotope, mean warm years. That's why he said it was "bizarre" to criticise him for flipping a series - his method doesn't care which way is up, in fact it selects which way is up depending on what best matches temperatures in the correlation period.
Not that I agree with the method. In fact it is statistics without sense. First, you get a huge amount of noise with only two test periods. Second, as I understand it, he took hemispheric temperatures instead of local ones - adding a lot more noise. Third, he SHOULD have pre-specified which way was up in accordance with the physical theory that made each series a candidate proxy in the first place - failure to do so added not noise, but anti-signal in the earlier, reconstruction years. Series that probably were good proxies the right way round got fed in the wrong way round because some recent physical disturbance had fouled up the correlation with the temperature record.
Mann's method was junk as even IPCC stalwarts in this field like Briffa and Cook recognised at the time. They confided this to each other in climategate e-mails, but never spoke up publicly through all the years of controversy over it.
Re will8ace. I have some slight sympathy for Jones, but not this comment:-
"Jones, 57, said he was unprepared for the scandal: “I am just a scientist. I have no training in PR or dealing with crises.”"
As head of a world leading climate advocacy centre and being widely quoted in the media, then perhaps he should have invested in more PR training, or stayed away from advocacy.
For BillS-
1) Don't play the biscuit game
2) At least we can nip out for a fag without getting strange looks.
The latest gate has just been opened:
http://eureferendum.blogspot.com/2010/02/and-now-for-africagate.html
"Even the mildest critics of the IPCC and Dr Pachauri might now be moved to observe that they have eschewed uncertainty, to project the most pessimistic scenario imaginable – with no scientific support and a great deal of embellishment. After "Climategate", "Glaciergate", "Amazongate" and now "Africagate" can either survive?"
Everytime I ask the "scientists" at RealClimate the simple scientific question:
"What peer-reviewed biological process allows Mann to include proxies that show negative tree-ring growth correlation to temperature in his studies." My comment is deleted.
It isn't hard to infer whether that site is actually interested in science from their actions.
It should be noted also, that not one paleoclimatologist has spoken out against what Mann has done and called for a correction or withdrawal of the papers involved. "Climate Science" just isn't Science at all ... and the lead players have zero integrity.
@will8ace
That's not good. It just isn't.
It just demonstrates the underlying problem of academics being forced into the light of public scrutiny.
I may want him to go because he has failed but I do not want to hound the man to take his own life.
Of course, I am a little[lot] naive, so this could be a sympathy stunt - who knows.
Dennis, I too hope he doesn't feel that bad. However, sometimes pr "gurus" advise their clients to make this claim in order to get folk to back off - which devalues the seriousness of a suicide threat.
Perhaps he might find a measure of personal redemption in resolving to put right what he put wrong.
The Kaufman Correction: Before & After
Science has published the “Correction & Clarification” to Kaufman et al 2009, but curiously the reconstructed temperature proxy has changed yet again between the draft clarification and the published clarification, to “hide the decline”? from the temperature anomaly 2000 years ago that in the draft was shown to be very close to the present.
In addition, the highest peak was in year 1998 in the draft, but 2000 in the published version.
http://hockeyschtick.blogspot.com/2010/02/kaufman-correction-before-after.html
Yer Grace
Will8ace's choice of words also concern me. [snip]
It is becoming sadder and sadder for Jones' family.
[BH adds: Sorry I've had to snip this as I don't want Will8ace's words repeated]
From memory I'm not sure that this one was a particularly important series anyway. Was this the one that was contaminated by bridge building (or agricultural changes or something and was never supposed to be used as a modern temperature proxy anyway?)
Just so long as Sheep Mountain and / or Yamal are included I think that one still gets the same result anyway.
Jones believes that the unit was maliciously targeted with multiple FoI requests by climate change sceptics determined to disrupt its work
What a BS sentence from jones he in his wildest dreams never thought they were ever going to be questioned on the science since they held all the cards ( but it happened) i say crocodile tears.
Jones reminds me of the orphan who throws himself on the mercy of the court after mudering his parents.
My Phil's a good boy, 'e is. 'e just fell in wiv a bad crowd. That Wriggly Tom, and that Micky Mann - wouldn't trust 'em as far as I could frow 'em. And don't you mention that Gavin Smirk. They led my boy astray, they did. 'angin's too good for 'em. But my Phil loves 'is old Mum, 'e does. You leave 'im alone.
Your Bishness
Can you reassure your readership you are not in fact in the pay of big biscuits.
[BH adds: big biscuits would turn me into a fat cat. I am declining all snack inducements].
OT but Might be of interest
http://www.dailymail.co.uk/news/article-1249035/How-Met-Office-blocked-questions-mans-role-hockey-stick-climate-row.html
"So yet another error that doesn't make a damn of difference, except that:"
Frank, you should read up on the Kübler-Ross model: You are stage one.
http://en.wikipedia.org/wiki/K%C3%BCbler-Ross_model
There is nothing more horrible than the murder of a beautiful theory by a brutal gang of facts.
-- François Duc de La Rochefoucauld, Reflections; or, Sentences and Moral Maxims (1678)
Don Pablo,
You're quite stupid, aren't you?
Frank,
As usual you are charming and wiity
Tony,
One runs out of patience after about the 5th contentless ad hominem and non-sequitur from the same anonymous loser.
O/T
The Independent is getting desperate. Lead story on their website on Sunday morning. If this is the best attack they can do (and they have obviously been trying to dig) then things are looking good.
independent poor hatchet job
They really do not understand how blogs have done this. What emotions cause me at 7:00 on a Sunday morning to look at a Climate Skeptical Blog? A PR person, lobbyist, journalist would be in their bed. You cannot buy this 'outrage', and they do not understand that.
Unfortunate banner ad placement in the Independent article?
"ExxonMobil said in a statement: "We have the same concerns as people everywhere – and that is how to provide the world with the energy it needs while reducing greenhouse gas emissions."
"Switch your Gas & Electricity and save up to £380 with Independent Compare"
Could I ask Frank and Pablo to calm things down please.
The issue with the proxy is that it has to be flipped upside down to be used properly.
Higher values represent colder, then after about 1700, the numbers become higher for non-climate reasons like farming. So now, Mann's computer sees the higher values, matching with a temperature increase, and uses the proxy as is. There is some dispute as to Bishop Hill's description. Previously Steve McIntyre told me that the proxy doesn't flip direction in the computer, it has to be preset with the correct orientation.
The climategate e-mails show that Kaufman made the error in the same way, he saw that it was getting higher in the recent timeframe, and so made that association. The Kaufmann paper cut off the late farming-influenced values though.
What we desperately need, and so far as I know is not currently available, is a detailed, one-by-one listing of the various/many IPCC/team lies/falsehoods/transgressions. There is heaps of evidence, and I myself have a list of about 20 items. However, it needs someone to pull it together.
If I were to do it (I could, but there is no real point if somebody has already done the work) I would have a separate page for each issue. I would explain the IPCC/team statement. Explain why it is wrong/misguided/misleading and produce evidence in contrast. I would provide references. After it is done, we would have a report that we could send to MSM showing them why the IPCC/team 'work' is a crock.
My question is: Has anybody already done this? CA is full of important references/discussion but they are spread over a lot of posts presented over quite a few years now. Same applies to TAV, Tom Fuller, Lucia, WUWT et al.
Thoughts? Views?
I believe MikeN is correct the proxies were manually flipped. In summary there are 3 main issues with the Tiljander sediments.
1. Manual flipping of the proxies. In my opinion this was the issue that Mike Mann tried to respond to in his PNAS response early last year
2. The modern spurious portion of the proxies was wrongly used for collibration purposes with a couple of undesirable results.
3. The modern spurious portion is "perverse" (as described by Bender) in that it it pinned the valid historical portion to the wrong orientation when the modern part was flipped and callibrated. This led to warm being treated as cold and vice versa.
I have been discussing this with Steve M and others and remain convinced that Steve M has misinterpreted Mike Manns response. I still remain convinced that Mike Mann was not aware of 2 or 3 and was only responding to 1. Steve and Ross were limited in space to provide adequate details and were mainly focused on other issues.
Steve appears to have the view that Mike Mann should have read an obscurely named post at CA called "Its Saturday Night Live" to get the details. Until October last year that was the only place where a detailed explanation of the Tiljander issue was available and the issue with the modern sediments is quite subtle. I know fom Climategate that Mike Mann would rarely read CA and I doubt that someone like Gavin Schmidt who is an avid reader of CA would have necessarily passed the details on in this case.
We also know from Climategate that Darrell Kaufman only became aware in September last year well after the PNAS exchanges.
http://www.eastangliaemails.com/emails.php?eid=1011&filename=1252164302.txt
One other aspect is that Mike Mann did a follow up paper using the Tiljander sediments which was published in Science just as Climategate broke out. In that paper according to Jean S an algorithm is applied which does automatically flip the proxies. In my opinion that was probably done by Mike Mann with 2 fingers raised firmly in the direction of Steve M. I would love to see Steve M submit a response to Science focussing on Tiljander alone in order to chop those fingers off!
Yer Grace
Thanks, you made the right choice. Stupid of me to repeat those sentiments in my comment.