The Kelly paper

The most interesting of the attachments to Professor Hand's emails is a paper by one of his fellow panellists, Michael Kelly, who is Professor of Electronics at Cambridge.

The paper records Professor Kelly's impressions as he reads through some CRU papers. I can't really do justice to this paper as a short summary. For the moment, I've extracted the relevant pages from the full download and you can just read it. I think Steve M will probably have something to say on Kelly's work, which seems fairly frank.

The download is here.

Kelly on CRU

Update on Jun 22, 2010 by Bishop Hill

McIntyre's take on the Kelly paper is here.

Comments

[Peter B]

I read the whole thing.

He seems to have concluded that:

- dendrochronology (at least as practiced by the CRU people) consists of dubious uses of statistical methods to extract any signal from very noisy data, which he is not sure how they were selected;
- while "malpractice" may be too strong a term, he thinks there is a great risk of biased conclusions;
- he seems appalled at how such noisy data have been converted into "certainties" when cases are made regarding policy,

[ZT]

If this was inquiry input - how could the 'clean bill of health' report come to be its output?

Did someone convince Kelly his views were incorrect or did someone decide to leave Kelly's input on the cutting room floor?

It would be interesting to hear Kelly's comments on this.

[Lazlo]

[Snip - venting]

[John Shade]

One can detect some shock and dismay in Prof Kelly the scientist, as he pored over the artwork and the data preparation as if done by artists searching for just the right effect. Their hazy and tentative characterisations to be later transformed by the IPCC, using chiaroscuro better suited for the tastes of policy-makers.

[Pharos]

I see Judith Curry was the first to comment on the CA thread, with 'Michael Kelly’s comments are absolutely spot on'.

[Don Pablo de la Sierra]

One of the lessons Prof. Kelly and many others writing in this arena need to learn is "Communication via Written Messages 101."

To wit:

You start off saying in few lines (at most) what you are going to say. Then you say it, point by point. Finally, you end with "I have said that which I said that I was going say."

For example.

"Climate change caused by human activity is pure bunk for the following reasons:
1) reason one; 2) reason two; 3) reason three; 4) reason four; and 5) reason 5.

What I mean by reason 1 is .....

Moving to reason two, ....

Looking at reason three, which is ...

Of course, many of us miss reason four, but it is obvious when you notice that ....

Finally, moving on to the most egregious reason, the fifth on listed, we see ....

So in summary, you can see that given the evidence of the data above, it is clear that ..."

Folks, this is elementary. Or may I say alimentary.

Let us get some rigor in our rhetoric. We really need it. Professor Kelly needs to kiss the Blarney Stone several times if he expects to do anything with is his wishy-washy rhetoric. I was sadly disappoint in it.

[royfomr]

I feel sorry for Dr K. All he did was put parts of the CRU under the cold light of scientific scrutiny and now he's, probably, on a black-list!

[David44]

Good work Andrew! Coming from a physical scientist, Kelly's comments are a devastating assessment of the weaknesses in the work of these highly influential IPCC climate scientists. I especially identify with his taking offense at them calling computer runs "experiments".

[GrantB]

"I take real exception to having simulation runs described as experiments (without at least the qualification of ‘computer’ experiments). It does a disservice to centuries of real experimentation and allows simulations output to be considered as real data. This last is a very serious matter, as it can lead to the idea that real ‘real data’ might be wrong simply because it disagrees with the models! That is turning centuries of science on its head."

One wonders how this didn't make it into the final report. Presumably this was a minority view amongst the esteemed scientific panel.

[HaroldW]

I appreciate the quotation cited by GrantB above, also this one about the IPCC:

Up to and throughout this exercise, I have remained puzzled how the real humility of the scientists in this area, as evident in their papers, including all these here, and the talks I have heard them give, is morphed into statements of confidence at the 95% level for public consumption through the IPCC process. This does not happen in other subjects of equal importance to humanity, e.g. energy futures or environmental degradation or resource depletion. I can only think it is the 'authority' appropriated by the IPCC itself that is the root cause.

and this one about dendrochronology:

My overriding impression that this is a continuing and valiant attempt via a variety of statistical methods to find possible signals in very noisy and patchy data when several confounding factors may be at play in varying ways throughout the data.

Dr Kelly's insights are very reminiscent of my own impressions as I started learning about climate science.

Thank you Bishop for obtaining these. They contain an infinitely greater amount of information than the Oxburgh Report itself.

[Sir DigbyCS]

MJK is a good guy (physicist) and he doesn't mince words. In a carefully framed, caveated and precise way he's identified all the shortcomings in the work.

As has been obvious for a long time, much of this work wouldn't even merit a Pt II project.

SDCS

[dearieme]

"the starting data is patchy and noisy, and the choices made are in part aesthetic, or designed to help a conclusion. rather than neutral." Scathingly accurate. Well said, Kelly.

[James P]

But he's a professor of electronics. He won't know anything about climate!
:-)

[Roddy Campbell]

Bish - why do we not have Jones and Briffa's responses under the same FoI?

And well done, by the way, really good stuff.

Does what you have found make Oxburgh's response to McIntyre denying the existence of anything at all really a little 'economical with the truth'?

[Chuckles]

Dr. Kelly is to be commended, both for his insight, and his restraint.

I would certainly have certainly have extended the comment noted by HaroldW above, about humility and uncertainty morphing into 95% confidence, to enquire why the authors did not loudly protest about this unwarranted conclusion drawn from their results.

The following review message recommendation is beautifully phrased -

'I think we should consider using the opportunity to make entirely positive
recommendations that would improve the situation, such as
(ii) more effective engagement with the end-users of their findings beyond politicians and policy makers.'

Ouch!!

And as an engineer, I can hardly disagree with -

'Engineers are more sceptical that others on the implications of the findings to date.'

[PaulM]

I worry about the sheer range and the ad hoc/subjective nature of all the adjustments, homogenisations etc of the raw data from different places.
...
There is no evidence of overt scientific malpractice. That is not to absolve the authors of conscious or unconscious bias in making all the choices referred to above.

I think most of us would agree with this.

[Michael Ozanne]

"Let us get some rigor in our rhetoric. We really need it. Professor Kelly needs to kiss the Blarney Stone several times if he expects to do anything with is his wishy-washy rhetoric. I was sadly disappoint in it.

June 23, 2010 | Don Pablo de la Sierra"

It seems clear from the context that this is a summary of Professor Kelly's notes and impressions in pretty much "Running Commentary" style, not a formalised academic paper or indeed, heaven protect us, an op/ed "advocacy" piece. As such it carries more weight I think with most people than a rhetoricised pile of flim flam would.

[Douglas J. Keenan]

It is notable that Kelly is an engineer, rather than a scientist. Engineers cannot easily lie about what works: e.g. if a bridge collapses, no one will try to say that it is still standing, according to the consensus view of leading engineers.

If a bridge collapses, not only can this not be denied, but also, this tends to publicly tarnish the reputation of bridge engineers everywhere. More generally, a real-world failure due to a mistake in any branch of engineering tends to tarnish the reputation of all engineers in that branch. The result is that engineering societies put tremendous pressure on their members to act with high integrity. The societies will sometimes censure members for tiny, even trivial, infractions—in order to stress to all members how important it is to maintain strong quality.

In science, truth is not determined by direct comparison with reality. Rather, it is determined by proclamation in prestigious journals. If a group of researchers get together, they can claim almost anything to be true, and get away with such. This occurs in many fields (not just global warming). I have seen up and down claimed to be the same. I have seen four dots in a figure claimed to be five. There is no effective mechanism to hold scientists to account for dishonesty. It is all very different than engineering. Hence the ethos in science is very different from the ethos in engineering.

[Gilbert]

And these were the handpicked papers. Makes one wonder what his response would have been to the contested papers.

[Q]

"In neither of these papers is there any overt malpractice, but one can't eliminate the possibility of conscious or unconscious bias in the choices of data. I just do wonder if a different hypothesis was being tested whether the same approach could give a very different answer."

Agenda/postnormal science

[Libby]

Much of the twisted approach and market driven attitude can be understood by reading the so called climategate emails. A good review of these emails can be read here http://bit.ly/b1KC7Y its a link to the free ebook in pdf format - hope this helps shine some light in the right dirrection!

[Richard]

He writes "I find no evidence of blatant mal-practice".

How would one define "blatant mal-practice" I wonder? Would a "valiant attempt via a variety of statistical methods to find possible signals in very noisy and patchy data when several confounding factors may be at play in varying ways throughout the data" qualify as malpractice? or "turning centuries of science on its head"? or drawing positive conclusions from a null hypothesis?