Bishop Hill The scientific firmament
Nov 23, 2011 An amusing exchange of views between Bob Ward and Phil Jones on statistics:
Dear Phil,
I was wondering whether you have seen the article by David Whitehouse in the latest edition of 'New Statesman'? [3]http://www.newstatesman.com/200712190004
It would be great if somebody could respond to the article. I would be happy to do so if somebody can supply me with the ammunition. Any thoughts?
Best wishes,
Bob
Bob Ward
Jones responds swiftly:
From: Phil Jones
Sent: 20 December 2007 13:58
To: Bob Ward
Subject: Re: More nonsense on climate change
Bob,
Quickly re-reading this it sounds as though I'm getting at you. I'm not - just at the idiots who continue to spout this nonsense. It isn't an issue with climatologists. All understand. If I tried to publish this I would be told by my peers it was obvious and banal. I will try and hide it in a paper at some point. I could put it on the CRU web site. I'll see how I feel after the Christmas Pud.I would have thought that this writer would have know better! I keep on seeing people saying this same stupid thing. I'm not adept enough (totally inept) with excel to do this now as no-one who knows how to is here.
What you have to do is to take the numbers in column C (the years) and then those in D (the anomalies for each year), plot them and then work out the linear trend. The slope is upwards. I had someone do this in early 2006, and the trend was upwards then. It will be now. Trend won't be statistically significant, but the trend is up.
This is a linear trend - least squares. This is how statisticians work out trends. They don't just look at the series. The simpler way is to just look at the data. The warmest year is 1998 with 0.526. All years since 2001 have been above 0.4. The only year before 2001 that was above this level was 1998. So 2cnd to 8th warmest years are 2001-2007
The reason 1998 was the warmest year was that it resulted from the largest El Nino event of the 20th century in 1997/8. We've not had anything resembling a major El Nino event since - they have all been minor.
Using regression, it is possible to take the El Nino event into account (with a regression based on the Southern Oscillation Index). This accounts for about 0.15 deg C of 1998's warmth. Without that 1998 would have been at about 0.38.There is a lot of variability from year-to-year in global temperatures - even more in ones like CET. No-one should expect each year to be warmer than the previous. The 2000s will be warmer than the 1990s though. This is another way of pointing out what's wrong with their poor argument. The last comment about CET is wrong. 2007 will be among the top 10 warmest CET years - it will likely be 2cnd or 3rd.
CheersPhil
Ward is impressed:
Dear Phil,
Thanks for responding so comprehensively. I have plotted the data before, and as you observe, the trend is up but the result isn't statistically significant, which I think makes it open to attack. I think the problem is that NOAA made the following statement in its report on the 2006 data:
"However, uncertainties in the global calculations due largely to gaps in data coverage make 2006 statistically indistinguishable from 2005 and several other recent warm years as shown by the error bars on the [1]global time series."
I'm not sure how to argue against this point - it appears to imply that there is no statistically significant trend in the global temperature record over the past few years.Best wishes,
Bob
Reader Comments (79)
I don't see why that's a problem, after all the goal being sought is a global average, and Phil can do that with a single column. Or get the long suffering Harry to do it for him.
Biffra tells how they tailored papers to match the IPCC needs and introduces us to his careful choice of an "additional fudge word "unusual"........" Truly amazing examples of how to achieve high academic standards - not!
Tue Aug 9 17:21:11 2005
from: Keith Briffa <REDACTED>
subject: Section on last 2000-years
to: REDACTED,REDACTED
Given the time constraint , this "final" revision is not as considered as it might have been , but we have tried to take into account all comments available , and have given considerable attention to the IPCC terminology and emphasis on the bullet points .
David's suggestions about re-ordering the paragraphs was particularly difficult to resolve in my own mind , because I do see the logic , but equally , did not want to interfere with the time line approach to describing post - TAR work that underlies the current structure. As you can see I decided to leave the order as it was. It would be great if David and Fortunat could check cross Chapter referencing (eg in relation to forcings and detection chapters). We can revisit this , and the issue of McIntyre and McKitrick (centering of PCs in Mann et al reconstruction - which is clearly unfounded) until such time as the numerous responses are published.
………………… I will give you a continuing argument later about our overstepping the "paleo" boundary. Note also that the bullet on European summer 2004 has been altered to reflect what was a last minute , one-sentence , insertion in the first paragraph regarding Jurg Lutterbacher's Science paper - as there was no mention of it otherwise. We had to remove the reference to "700 years in France" as I am not sure what this is, and it is not in the text anyway. The use of "likely", "very likely" and my additional fudge word "unusual" are all carefully chosen where used.
Thanks
Keith
--
Professor Keith Briffa,
Climatic Research Unit
University of East Anglia
Norwich, NR4 7TJ, U.K.
After this, the only reason a journalist should be interested in interviewing Ward is to take his confession.
In fairness to Phil Jones and Bob Ward the science of statistics existed long before computers were invented. The fact that somebody does not know how to use a particular software package does not necessarily mean that they do not know how to analyse data. It would not surprise me if there aren't some retired professors around who cannot use Excel even though they worked in disciplines requiring a good knowledge of mathematics or statistics.
"It would be great if somebody could respond to the article. I would be happy to do so if somebody can supply me with the ammunition. Any thoughts?"
Yes Bobby, follow the yellow brick road until you find a Wizard. Ask said Wizard for a Brain to replace the PR straw.
Nicholas Hallam, your comment is spot on. It is not just that Jones does not know how to use Excel to answer his question (although that is bad). Things are even worse—Jones is asking the wrong question.
The first question to be asked is this: what statistical model should be used for the series of temperature? Only after that question is answered can we consider questions about significance. Jones is so ignorant, he does not realize this.
Bob Ward on twitter
in complete denial
http://twitter.com/#!/ret_ward
Douglas J. Keenan, thank you, my first thoughts when I read the extract were,'Why has he arbitrarily decided that an increasing linear trend perfectly describes this unknown data?'
Atomic H, It's the 'Texas Sharpshooter' * approach - desired results first, then plot the curve and data required to get that result
* First fire the shots, then draw the bullseye.
We, with our local reference points, know that Jones is a nit-wit: first degree at university of lancaster says it all. Yet this nit-wit is taken seriously, apparently, by our 'political elite' hahahaha, and the big money men; in fact he is a major player in directing what it is hoped will be a global policy with staggering economic implications. So questions - lots of them - arise: are leading climate scientists in other countries as intellectually inferior as Jones? Have (if they are) dim-wits like Jones et al really persuaded the 'policy community' who we could expect to be hard headed, at least on the money side? Or are they just dim dupes being played by the policy community to provide 'intellectual' clothing for ideas already decided on - cash-strapped governments get wonderful new revenue streams, & G-S et al make a packet along the line, happiness all round? WTF is going on?
My point being, while its entertaining reading these emails, watching people who aren;t able to tie up their shoe laces, tripping over same, they give us no clue as to the question, WTF's going on?
The fact that somebody does not know how to use a particular software package does not necessarily mean that they do not know how to analyse data. It would not surprise me if there aren't some retired professors around who cannot use Excel even though they worked in disciplines requiring a good knowledge of mathematics or statistics.
Those people would have better software, and know how to use it.
A quick time series with trend is high school stuff. I teach it, in fact. Even the worse student in my class (last year at high school in NZ) can do that. For a decent mark you need to be able to explain the flaws in the analysis, not just do it.
It should take a decent mind being shown it once to be able to master it, Twice if you are a bit dim. To not learn how to do it when shown is basically pathetic, and I wouldn't accept it from a (poor) 17 year old.
last time, the establishment circled the waggons and showed Jones as a poor bewildered victim. This time, we need to show him as the vindictive scheming swine that he really is. the evidence is there in the emails if someone can bring all his passive-aggressive bullshit together
denialist Nick Stokes jumps the shark Nick Stokes (Comment #86064)
November 23rd, 2011 at 6:29 pm
#86063
Steve, Joking? No. Who do you think RC are suppressing, and how?
diogenes reference is to the latest thread at lucia's Blackboard: rankexploits.com/musings
===================
@ bill
are they just dim dupes being played by the policy community to provide 'intellectual' clothing for ideas already decided on
Some are. It's all in Donna's book.
But others are clearly political activists first and third-rate scientists a long way second.
We have seen Jones dismissing Climate Audit simply because as far as he's concerned it's "right of centre". He thinks that's all that needs to be said on the matter, he simply cannot imagine that anyone who disagrees with him could possibly be doing so except for political reasons, and that's his very best argument against CA.
He's completely bought in to the idea that his job is to fabricate ammunition for the cause. The left is seen relying on cod-scientific arguments from spurious authority here because if they had to debate off the facts and economics they'd lose.
It's a Trojan horse and always has been. Nobody freely votes for tax, repression and the dismantling of civilisation, so creeps like Jones figure they'll have to be deceived into doing so.
Luckily he's so thick he writes it all down. University of Lancaster? Pffffffffft. Dunce.
Try searching for "BBC impartial" at foia2011.org and up pops 4844. Oops!
Sorry 4894 - fingers!
Has Ward ever directly and explicitly replied to any of Whitehouse's multiple articles on the topic? I tried to get something out of him on Twitter once but only the usual generic GWPF-is-bad stuff came out.
What the hell is a Climatologist when he or she is at home. From my humble understanding there are around 80 different subjects that go to make up climate science. To be an expert Climatologist therefore, assuming generously that say from a 4 year degree, 3 years overlap each course. Let's assume that only six months of additional study are required from then on, that makes another degree course requiring a further 6 months x 80 subjects = 480 months or 40 years in my reckoning! Surely that means nobody under at least 62 yoa can be a Climatologist? Surely they'd retire by now? :-)
I agree with Roy. Not surprised if these professors don't know excel. They're probably all using matlab or some similar (and probably 2 decades outdated) program that they learned during their initial education and stuck to ever since.
Wow! Statistically insignificant can still mean climatologically significant
2. We cannot help remarking that it is very surprising that research in an area that depends so heavily on statistical methods has not been carried out in close collaboration with professional statisticians.
Oxburgh was wrong - no way these gentelmen could benefit from "professional statistcans" , they would need Adding and Subtracting, level 1.
Nov 23, 2011 at 4:14 PM | Mac
Hi Mac,
Sorry about the slow response - it's hard trying to get through the comments on the various threads on this, it's a fast-moving discussion!
As TBYJ says, 0738 is from Simon Tett to co-authors (including me) of a paper we were writing together. I can see why you have homed in on the word "tricks", but hopefully it should be reasonably clear that Simon is using the common parlance of "missing a trick" to mean "missing an opportunity" or "not being sufficiently alert".
If you read the email you'll see he was suggesting different options for papers arising from the work. When he asks if he's missing any tricks, he means is he failing to spot opportunities to do something else, such as link to some other work and exlore other avenues, or possibly look at something really novel with a view to possibly getting a really high profile paper in Nature or something.
I guess the real interest in 0738 is that it's to do with a climate model simulation of the last 500 years. My role in that was to sort out the land cover change forcing.
BTW when you said at Nov 23, 2011 at 3:53 PM that "there are 52 references to Betts in these emails", I don't think they are all to do with me are they? There are other Betts's in the world - some of the emails are from/to/about other Betts's (none of whom are any relation AFAIK!!)
Thanks for looking me up though....! :-)
Cheers
Richard
A generic science comment (long) here. Snip if you must...
Jones' comment "A simpler way..." brought this to mind:
I've observed over time that scientists try at every turn to simplify everything. It is part of the scientific method, and quantifying is part of all of that - resolve things to a formula/equation. The philosophy of "The whole is the sum of the parts" means that they break everything into as small of bits as necessary, then analyze the bits, then hopefully when they put it together the gestalt of the whole is the sum of the analyses of the bits.
But what that means is that in extra-complex phenomena science cannot do "science," because they can't break it into small simplistic bits. Climatology is so far like that. Because they cannot determine where one bit ends and another begins, they are stumped. There are some areas where they are able to identify bits, but without the other bits around that one being identified and analyzed, any overall conclusions are half-baked. The chaotic nature of climate - with everything being in flux all the time - means that when they try to stop the flow to look at one of the bits, its place in the whole has changed.
They think that they understand about CO2 and re-radiation, because someone in physics has published a paper which says that this is how CO2 molecules should work. But in the real world they don't have one iota of proof that the theory actually matches the reality. Instead, they argue ad infinitum that the reality IS what the physicist says it is and everyone is supposed to accept that as fact. The history of science is replete with ideas that seemed to make sense on paper, but which did not prove out empirically. That is supposedly what separates modern science from the ancient Greeks, who had all sorts of things worked out in their heads - but that we now know are wrong ideas.
The models are attempts at breaking climate into small bits and then reassembling them. In one of the emails (sorry I can't recall which) one climate guy states (effectively) that they have all these models but that we don't understand any of the processes, meaning that the models are not inductive reasoning, but instead are hindsight deductive reasoning, patches upon patches, simply to try to come up with something that has similar output to real climate - but whose individual components are not even bits of the real whole.
When science tries to break up a whole - a whole that is real - and then put the bits back together, that is one thing. But climate science is hiding behind the models - simply because they can get them to make pretty maps that seem to look like meteorological maps. But if the bits are - in EVERY mathematical term and constant - based on real, identified, bits of the climatic whole, they are just faking it.
Phil Jones has found out how (like managers the world over) to get a quick, simplified, version of what is going on. So, he can fake it, pretending that he understands what is going on. And he has fooled his co-scientists and the policy makers.
Similarly, the modelers have found out how to fake it - to make models that seem to give the approximate right output. I read a year ago about back in the early days of modeling, when they had one feedback that, past a certain point, always led to an infinite slope, and that they brought in an Oriental modeler who was able to add some jury-rigged term into the equation that would stop the anomalous output. The other modelers were ecstatic - but the term was just faking it. It had nothing to do with the reality they were modeling.
I lost all respect for climate modeling, when they didn't care if terms matched the phenomena or not, as long as it gave them the output they wanted.
So, it matters when breaking phenomena or processes into bits, whether the bits are really parts of the whole. It also matters if they identify - transparently, for all the world to see - each different quantum of the whole. Because the model only represents the whole found in nature when the bits - ALL of the parts - have been identified and properly represented by formulae. The model is - if correct - the mathematical representation of the phenomenon/process. If it cannot be shown - in every mathematical term - to do that, then the model is garbage (not science, in any event).
As such, not only must the models' math be shown to be correct, but also that the code - one step further removed from the reality - even represents the math, much less the reality.
All of this is fundamental science: The bits have to be part of the real whole, and the math has to represent the bits properly, and the code has to represent the math properly. That none of the models can hindcast worth a diddly, this is proof that the models are all faking it.
What does that mean? In their history, models came along before the science had broken out the bits properly. Over the last two decades more bits have been discovered, and more are yet to be worked out. Any argument NOW that the models THEN were correctly done would be laughed at. Yet the conclusions and arguments somehow are the same as then.
This is not possible. Without the bits identified and quantified, they could not possibly have come up with the right science then. Random chance/lucky guesses? That isn't science.
It all hinges on CO2 re-radiation or the greenhouse effect, we are told. That sounds like Phil Jones saying, "Well, there is this simpler way of looking at this..." when he can't even do an Excel chart.
Phil Jones is a microcosm of both models and climate science - he is faking it, and so are they. It isn't even just my opinion. It is all through the emails here - those around them know the Team is faking it, know that the models are faking it, know that the science is overstated, know that Michael Mann's statistical capabilities are not up to a 200 class level, know that they will all be busted someday.
Because it isn't science.
@Mooloo Nov 23, 2011 at 11:27 PM
I am with you. I was thinking that learning Excel graphs is something akin to long division, maybe even easier. I mean, the software is intuitive and leads you through, step by step, for chrissakes. To find the trend line, you have to grope around a bit, but if someone shows you once and you have a mind sharper than a kumquat, you should be able to retain it. The only way to be pathetic at Excel is to blow it off intentionally - or REALLY, REALLY dim.
It is incredible how much lower my opinion of Phil Jones is, after having read that email.
I'm looking forward to more of such hacked email databases.
If one could hack the US diplomacy completely, it shouldn't be too long for some more juicy material.
The science been all settled or so we are told by Susan Watts, it is interesting to see that one man sites like Climate Audit , also this one, cause so much discussion amongst the settled scientists.
[Snip}
Why am I reminded of the expression: "the blind leading the blind" ?
Whoever got these emails into the light of day deserves a Nobel Peace Prize.