Bishop Hill That error
May 16, 2014 Climate: Sceptics Climate: sensitivity The Bengtsson paper would have to have been very bad to be worse than, say, Kummler and Dessler, but at the moment we just don't know because we haven't seen it. However, the ERL editor claims that Bengtsson's offering contained errors. Unfortunately she doesn't actually identify any; the only concern in the reviewer report published to date seems to be with Bengtsson's temerity in thinking that observations and models really ought to match up, and of course the concern that sceptics might be keen on the paper.
But there are some errors floating around that are worth a look - as I mentioned earlier a cursory glance suggested to me that the reviewer's report itself included a bit of a boo-boo. I've now been away and done some fact checking and confirmed that I was right. Actually, I'm righter than I thought I was, as I shall now explain. Here's the paragraph in question:
Even more so, as the very application of the Kappa model (the simple energy balance model employed in this work, in Otto et al, and Gregory 2004) comes with a note of caution, as it is well known (and stated in all these studies) to underestimate ECS, compared to a model with more time-scales and potential non-linearities (hence again no wonder that CMIP5 doesn't fit the same ranges).
What struck me - a humble blogger, a mere accountant, a grubby scribe, as my detractors are occasionally wont to say - is that the Kappa model is not actually used in Otto et al (or indeed Gregory 2004). So here we have an expert reviewer who seems to be less familiar with the details of the relevant studies than I am. The reason I know about all this is that it was discussed at Ed Hawkins' blog some months ago. Here's Nic Lewis in the comments thread:
I spent weeks trying to explain to Myles Allen, following my written submission to the parliamentary Energy and Climate Change Committee, that I did not use for my projections the unscientific ‘kappa’ method used in Gregory and Forster (2008)...
...going on to explain that by 'unscientific' he meant that it didn't conserve energy.
And this quote reveals the other problem with the reviewer's remarks. The kappa method is an approach to estimating ocean heat uptake, which assumes that this is proportional to surface temperature change. Everybody seems to agree that the approach is unphysical because it doesn't adjust for 'warming in the pipeline' and indeed the Gregory and Forster paper notes that 'its validity is restricted'. But the important point is that it is only relevant to projections of future warming anyway. It has no relevance to energy budget estimates of climate sensitivity, which are backward looking.
But the reviewer who recommended the rejection of Bengtsson's paper did not understand this.
Scary eh?
Reader Comments (81)
`Scary? Scary to me is that there are hundreds, perhaps thousands of "educated" scientists who believe they can foretell the future.
Sort of caught "kappaing"? Or "not kappaing"? Or plain don't know, just don't like it, black ball, not in my club, the exact antipathy of the so called "progressives".
no no the scary aspect is that neither Dr. Nicola Gulley nor anybody else at the esteemed journal bothered to check what the reviewer said
this would mean they trusted the reviewer fully and unconditionally. Now, what scientist very near Dr Gulley and ERL and very trusted by them has singularly obsessed in mentioning Kappa and especially in making sure the whole world knew he had not really understood whom and when it had been applied by?
Nice observation, Bish.
The topic of ECS etc is complicated by the choice of calibration temperature. There is quantitative evidence in assembly to show that temperatures in the conventional record have a significant influence from rainfall at the time. Rain cools the Tmax.
Given that rainfall should not affect ECS calculations -but it does - surely makes the ECS results wrong.
This comment is preliminary. Confirmatory work is in progess at more sites.
Bish, you have caught the essence. I posted more details at length over at Judith's. Long or short, result is the same.
This is a consensus fail of epic proportions, needing as much exposure as possible.
Highest regards.
Interesting to see that none of the experts contacted by Fiona and her Science Media Centre appear to have spotted this problem.
a big WOW, great catch, Andrew!
Now the journal will need to withhold the other reviews, or at least go over them with the finest-toothed comb ever, lest they humiliate themselves further!
That review seemed quite ineptly biased anyway, with the statements disparaging analysis of discrepancies between observations and models, plus the hilarious claim that the paper threatened a large public impact against the consensus.
I'm still trying to get over my naive view that discrepancies between observations and models should matter, and that actual empirical observations are more scientifically fundamental than any model projections. Silly me, silly us!!
Skiphil - as I asked at CA - if it is so difficult to go from models to observations, what use models in policymaking??
According to Steve McIntyre:
IOP: expecting consistency between models and observations is an “error”
I was curious about the other bit of that phrase "compared to a model with more time-scales and potential non-linearities".
But according to Willis, the models are all equivalent to simple lagged-linear filters applied to the forcing, and observations have far more non-linearity than the models. So what do they mean by "potential non-linearities"? And what "more time-scales" does a lagged-linear equation give?
I'm not sure if this is talking about the same thing, but it isn't clear to me that the models do have any non-linearity, so it would probably be worth asking the reviewer/journal to clarify what non-linearity they were talking about.
http://wattsupwiththat.com/2013/05/21/model-climate-sensitivity-calculated-directly-from-model-results/
http://wattsupwiththat.com/2013/12/18/the-fatal-lure-of-assumed-linearity/
Yeah, you can't expect church dogma to match up with observed events. Come on, that's not the point.
You gotta have faith, faith, faith...
"that the Kappa model is not actually used in Otto et al (or indeed Gregory 2004)."
Do you really know that? As I understand, the comment from Nic Lewis that you quote is referring to the report that he and Crok wrote for the GWPF, not Otto et al.
I managed to dig out Otto et al, and I see that they do use annual estimates of ΔQ, without assuming proportionality. So it seems the key question is whether Bengtsson used it. If he did, and Nic L says it is unscientific, then maybe the referee had a point, even if he was wrong about Otto. I guess we'll have to hope LB produces the paper.
Thanks Nick. The reviewer was wrong about Otto. The Bish was right about that.
Richard,
Indeed. The key question, though, was whether he was right about Bengtsson. He's not reviewing Otto.
If the Times wants to put LB's rejected author gripe on the front page, and then not tell us what was in the paper, then I guess we're stuck.
The reviewer was also WRONG, not just wrong, about what the purpose of peer review is. It's not to figure out which papers will and will not have a desirable political impact. He ought to be pilloried and run out of the peer reviewing profession for that alone.
The Bengtsson example has been branded as McCarthyism. That, perhaps for political correctness, ignores that other more terrible, parallel in history where science was subverted by politics in the 1930s with appalling consequences for mankind.
Even members of the German Meteorological Association now make carefully oblique reference to this in expressing their grave concern over the subversion of climate science to meet political aims. I must brush-up on my history as I think a contrast and compare between science under the Nazis and science under the political control of the 'team' might be illuminating . Others are probably far better placed than I to do that ...... any takers?
The question is, and its one for Nick Stokes as much as anyone, is how all this would be treated if exactly the same arguments and events took place in another field.
At the moment there is a controversy over statins - some feeling that everyone over the age of 55 or so should be preventively dosed, others feeling that no-one should take statins unless they are clearly suffering from CHD, and then only under careful supervision.
We have studies which show what the death rates are in populations taking statins. These are purely observational. They seem to show some fairly small positive effects. No-one in responsible health policy positions appears willing, on the basis of these studies, to move to population dosing. Not to recommend it, and not to finance it.
Now suppose we had a series of models of the effects of population dosing. Some show it reducing death rates dramatically, some show it increasing them. It turns out that the ones that can accurately model the effects of statins on cholesterol in the body go very far off on death rates. The ones that track observed death rates closely turn out to to make non-physical projections of some other variable.
We run our different models lots of times, and we end up with a spaghetti chart with very wide variablity. We then average the spaghetti (in one or two cases abbreviating and splicing in the well known way to hide an inconvenient decline), end up with a projection, scenario or forecast, call it what you will, that predicts between 5 and 50 lived saved per thousand treated.
Would we now on this basis be advocating mass dosing, and calling all dissenters denialists?
May 17, 2014 at 7:51 AM | Robin
Here is a link on Nazi science policy:
http://archive.adl.org/braun/dim_14_1_nazi_med.html
I think Lysenkoism might be a better example. But history is full of delusions of groups reinforced by group controls. There is a fine analysis by Irving Janis of 'groupthink' which analyses how social controls work. Janis was concerned with a policy question: how could the best and brightest be so deceived about Vietnam?
But the question could also be asked about financial bubbles, various mad and bad 'treatments' of the mentally ill.
We are prone to group irrationality, and the more the evidence conflicts with a group held belief of this sort, the stronger the denials that it is falsifying. The very interesting study 'When Prophecy Fails' is a case history of how this works - I have mentioned it before on these threads. If you look at Nick Stokes postings here and on CA on the Bengtsson affair, you can see the mechanism in action.
If Festinger were alive today he would be studying Climate Science as it moves into its terminal phase of denial of the reality or significance of contradictory observations.
michel
"Would we now on this basis be advocating mass dosing, and calling all dissenters denialists?"
Fluoridation might be a better example. Or immunisation. They are ongoing. And there are denialists.
But what is the alternative? Is it impossible to make a decision unless it is unanimous?
Nick Stokes
If your reread the reviewer's report you will see that Bengtsson is comparing different estimates of ECS in AR4 and AR5 and within AR5. He is not creating a new estimate. Therefore the reference to the Kappa model is incorrect unless he is highlighting Gregory and Forster 2008 in some way. (I don't think AR5 cited G&F2008, but I haven't checked).
"But what is the alternative? Is it impossible to make a decision unless it is unanimous?"
I think we are currently seeing the robustness of civil society in the face of ambiguous science. For a variety of reasons no one is taking the big leap. In the end the population at large in democratic societies are pretty pragmatic, until the evidence becomes sufficiently clear cut they are unlikely to allow too much of a wholesale intrusion on their rights.
This is of frustration to those who believe the science is clear cut, but is likely a racial survival instinct that has seen us right in the past (notwithstanding excursions to extremes within some societies all of the time).
The science of muddling through adaptively has seen the human race pretty well - if the only cost is a bankrupt Spain and our host with a big windmill in his front yard we will have little to worry about.
Even West Islanders (a technical term) and Nick S have a role to play.
> the only concern in the reviewer report published to date seems to be with Bengtsson's temerity in thinking that observations and models really ought to match up
No. There's a previous, fatal concern: “The overall innovation of the manuscript is very low”. Its curious that you managed to miss that. Perhaps you need to read it more carefully.
Bishop,
Yes, I expect that is right. It looks as if the reviewer is mis-describing the Otto ΔQ usage as a kappa model. He's right about the note of caution though (Otto et al):
"Our results match those of other observation-based studies and suggest that the TCRs of some of the models in the CMIP5 ensemble10 with the strongest climate response to increases in atmospheric CO2 levels may be inconsistent with recent observations — even though their ECS values are consistent and they agree well with the observed climatology. Most of the climate models of the CMIP5 ensemble are, however, consistent with the observations used here in terms of both ECS and TCR. We note, too, that caution is required in interpreting any short period, especially a recent one for which details of forcing and energy storage inventories are still relatively unsettled: both could make significant changes to the energy budget. The estimates of the effective radiative forcing by aerosols in particular vary strongly between model-based studies and satellite data. The satellite data are still subject to biases and provide only relatively weak constraints (see Supplementary Section S2 for a sensitivity study)."
That's pretty much saying "don't do it". Bengtsson seems to have compared Otto et al with CMIP5 over some short term and found a discrepancy. The reviewer said, well, that's what Otto said to expect. What's new?
It is not helpful to attempt to review a paper none of us has read by dissecting one reviewer's comments. Waste of time. But interesting how many people (and who) are trying to do so.
Dr Stokes - is mis-describing the same as wrongly descibing? ie wrong. An error.
Dr Connolley - shouldn't you be over at Wikipedia editing something? Perhaps not, there isn't a "Cleansing of Bengtsson" article up yet but no doubt you'll be keeping an eye out.
GrantB,
Yes, it's a wrong description. But rhoda's right. There isn't much point in reviewing the review without the paper.
His basic point seems right. Otto et all said use of this model, misnamed or not, can't be expected to give the same results as CMIP5. So a paper that repeats that is not attractive to the journal.
Thanks are due to William Connolley for an unexpectedly useful contribution.
By grasping at the weakest straw in the review - he has neatly highlighted another example of the absurdly doctrinal approach to this branch of science.
When the obituaries are finally written for this massive, unscientific folly, the expression “The overall innovation of the manuscript is very low” will surely deserve a place of honour.
One of the most farcical results of throwing almost limitless funding at a scientific "non-problem" has been the mass production of scientific papers recycling the same scraps of stale, tired data with low-grade amateur statistical trickery designed to keep the "consensus" intact and the funding flowing.
Has there ever before been branch of science where the expression “The overall innovation of the manuscript is very low” could be applied to 99% of the literature?
In reality, Bengtsson's pointing out the vast & growing gulf between the output of the computer models and the real observations is probably the only truly original idea we have seen in climate science for years.
Foxgoose: I sometimes feel the same way about blog comments but yours hits the spot. :)
HAS: "Even West Islanders (a technical term)..."
Chortle chortle....one of my favourite references to Oz.
Seriously: Some cracking comments on this thread. Most illuminating.
The other classic mechanism is to move from an assertion of fact about the climate to a policy prescription as if they were the same thing</>.
Exactly, Foxgoose. And some practitioners of the discipline, if I may use that word in this context, appear unaware that very little novelty ever came about as a result of simply running a computer program. (That's not to say that they are never useful. Sometimes they are.)
I think that reviewer contradicts himself on two points (apart from not pointing out any errors, and instead rumbling on about what h thinks should be believed or done research on instead).
The inconsistencies are:
1a) He says the that model-observation discrepancies are irrelevant, that they shouldn't be discussed, definitely not in a publication (as this is unheplful, even harmful ... to the 'The Cause™' ?)
1b) But it would have been very helpful if these discrepancies, and more importantly their causes had been investigated by a careful, constructive and comprehensive study, also detailing the underlying problems.
Secondly, he claims:
2a) As his major, decisive objection: That the paper does not advance knowledge in the field significantly, but concedes
2b) That it has/would have a very high impact on that field, albeit a 'negative' one.
He is however both adamant and very emotional about that this paper should not see any daylight.
Draw your own conclusions ..
William Connolley said:
No. There's a previous, fatal concern: “The overall innovation of the manuscript is very low”. Its curious that you managed to miss that. Perhaps you need to read it more carefully.
If "the overall innovation of the manuscript is very low" then the climate science community must already be familiar with the arguments in the paper and will have either confirmed or refuted them. Can William Connolley, or anyone else, give references to the publications in which that has been done?
Does W Connolley pretend to have read the paper? I mean if he can assert and ascertain that "The overall innovation of this manuscript is very low", and especially that others should have come to that conclusion too?
Well, I haven't and I can't. But I can assess the overall quality of that review, and it doesn't strike me as coming from somebody in the habit of 'making significant advancements in the field' in every paper, nor for that matter keeping up 'a high overall standard of innovation'</> in them. But that's of course just speculation. :-)
Maybe however, his publications really 'are helpful .. as they open the door for oversimplified claims of "certainty" and "settled science" from the gullible climate media side' .. Who knows?
There's an interesting parallel with the BMJ and a paper on the side effects of statins:
"Dr Godlee said this figure was based on data from an "uncontrolled observational study" and was incorrect. However, it had not been picked up by the journal's editors or the experts who peer reviewed the work."
From the BBC
In the current climate is it possible for Prof. Bengtsson et al to resubmit the paper or even try a different Journal ? Or even publish it anyway on a Blog so we can all see what we are discussing ?
re reviewer's comment: “The overall innovation of the manuscript is very low”. My observation, including comments that Ross McKitrick and Steve Mc have gotten on their submissions, is that applying a simple technique to test a model/forecast/ theory such as the tropical hotspot or Antarctic warming is viewed as not innovative enough. After doing the work in a fancy pants way, it then becomes publishable. The reviewers are hung up, as are the journals, on innovation and have no interest in testing hypotheses or paradigms, even if it is something big.
"...going on to explain that by 'unscientific' he meant that it didn't conserve energy." I once wrote a short note explaining that the model in a paper by Professor, let us say, Smith failed to conserve mass. One referee took exception, declaring that Professor Smith was a scholar and a gentleman. FFS.
Didn't McIntyre's additional Nature reviewer also say 'not of interest to the readers of Nature?' Could it be the same person?
We're nicely building up a list of addenda for Bengtsson's paper to get published.
1. Add obscure statistics to claim innovation
2. Insert reference to urgent need to cut emissions
3. Disparage skeptics
4. Describe need of being cautious with its own methods and results
more am sure will be added...
What's this become? The Bishopaedia?
"because the paper itself is Sekrit."
Not yet published. They're not the same thing.
Does it not bother you to complain about lack of openness in climate science? Why aren't you pestering all your friends to release their data -- because publishing papers without data to check is worse than not publishing at all.
May 17, 2014 at 9:10 AM | Registered CommenterGrantB "Dr Connolley - shouldn't you be over at Wikipedia editing something? Perhaps not, there isn't a "Cleansing of Bengtsson" article up yet but no doubt you'll be keeping an eye out."
Now that, is one of the funniest posts I have seen here in years.. :)
Sorry Connolley but your not in your realm surrounded by sock-puppets any-more, hear others are allowed to ask question you rather not answer and 'sceptical' a word normal in science but hated by AGW promoters , is actual the norm.
If you cannot play nice perhaps you should not play at all .
Enough with the skandal of free passes to non-compliant minds!!!
My vote for William Connolley to get admin rights on this website so the content can be so much cleaner and scientific!! Sieg William!!
Bengtsson shows the models are fantasy. The response seems to be 'Of course they are; why would you object'?
Connolley emphasizes that that is not innovative, reinforcing that it has been known that the models are fantasy.
So much for policy.
================
Connolley/Weasel,
Review papers by definition aren't 'innovative' in that they generally don't bring new data to the table. What they can offer is a fresh insight by comparing publications in a similar area. The only implication that would lead to the censorship of this paper is that the alleged scientists don't like it when the truth holes their conjectures below the waterline.
And the reviewer admitted this.
And somehow this means Andrew Montford is lying? Pychological projection, anybody? Connolley, you're nothing at all but a serial deceiver. We know your game, and you're not amongst gullible saps here.
He's like a hooded KKK member trying to comment in a NAACP blog, wondering why none's taking him seriously.
Keep burning all information you can get your keyboard on, Billy!!