Bishop Hill The nature of scientific advice
Nov 21, 2013 Bureaucrats Climate: McIntyre Greens ;)
Writing in Nature yesterday, David Spiegelhalter and two other eminent scientists tried to explain to ministers how to understand the advice they get from scientists. It ranges from the worthy (No measurement is exact; Bigger is usually better for sample size) to the much racier (Bias is rife; Scientists are human). It's hard to disagree with any of this although I'm not sure that it really portrays the problems with academia as a reliable source of advice for policymakers.
For example, when the authors tell the reader that scientists are human and that peer review is fallible, you get no sense of the failings uncovered by controlled studies of peer review (as described in The Hockey Stick Illusion), which suggest that it is nearly useless for ensuring that the conclusions of a paper are correct (although that obviously doesn't prevent peer review being useful as a way of improving a paper).
Nor does "scientists are human" quite get over abuses like those we saw in the Climategate emails and which the official inquiries carefully chose not to investigate. These too were explained away as scientists being 'human', and so it is clear that this heading potentially undermines the integrity of all policy advice.
I thought of this problem while reading Steve McIntyre's latest post on the Cowtan and Way paper, in which he discusses some of Robert Way's contributions to the Skeptical Science discussion board. Steve's post is largely non-technical and is therefore quite accessible to the lay reader, and I encourage people to take a look. Anyhow, Way turns out to have been enough of a scientist to tell other users of the discussion board that Steve was right about Mann's papers, including the hockey stick, that he was right about 'hide the decline' and that O'Donnell et al were right about Steig's Antarctic paper. But he also suggested that he had been advised to steer clear of Mann's papers:
I’ve been shown before by even climatology profs in my university time that it might be best to stick clear of Mann’s reconstructions until the dust settles.
And in the comments to Steve's thread, Way protests that his positive comments about Steve have been publicised:
Do you feel that it is fair to do this to a grad student? To choose to highlight examples that may someday lead to negative repercussions for an individual?
So we need to be clear that in the climate field at least, mere agreement with a position that erodes, however slightly, the alarmist position, is expected by scientists working in that field to have negative repercussions.
How then can policymakers trust the scientific advice they receive and, perhaps more importantly, why should the public trust the policymakers? The answer is that they can't - either of them. Without some external check on its integrity the advice is worthless. We need red teams, and quickly.
Reader Comments (44)
a by the way observation, why trust Robert Way?
Why was Robert Way too shy to put his affiliation with SkS on the Cook et al Consensus paper.
http://iopscience.iop.org/1748-9326/8/2/024024/article
he is a regular author, contributor and inner forum member
http://www.skepticalscience.com/posts.php?u=2360
All the authors of the Cook97% paper, had a role at Skeptical Science, being authors/contributors there and/or part of the inner forum,
but Robert Way, Sarah Green, Mark Richardson & Peter Jacobs did not declare it..
Robert Way for example has more articles written by himself at SkS. than I do for Watts Up With That.
Were they simply hiding from the peer reviewers that they wear ALL involved at Skeptical Science.
(thus by not declaring their affiliations with SkS giving the peer reviewers the misleading impression, that at least 4 outside (non tribal) academics were involved with the paper)
What worries me is that the word scientist has adopted the quasi religious aura of infallibility. It is the first point of attack by the enemies of rational thought that the Science is actually in agreement with them, making it evident that they are right. The notion of the null hypothesis and the statistical probability that several outcomes are possible as against one is alien to them. I would question whether climate science ( which in my student days was a cumbersome statistical dirge), is a science in the true sense or for that matter environmental science, the uncertain nature and debasement of both in common understanding makes them akin to the early fumbling of the alchemists. .
Looks like excellent advice. I think it would be improved if some comments were added to caution ministers et alia that computer models can suffer from problems due to any or all of the areas of possible concern listed, and that the model output need not necessarily reflect any of them. The risk of dubious use of model outputs as if they were scientific evidence comparable to observational evidence is grounds for including some advice on this topic.
"So we need to be clear that in the climate field at least, mere agreement with a position that erodes, however slightly, the alarmist position, is expected by scientists working in that field to have negative repercussions."
Sorry Andrew, this is nonsense - absolutely against my experience of climate science, and I expect most other climate scientists too, if you asked them. Robust challenge of established knowledge - where the data warrants it - is just as established in climate science as in any other field.
Doug
I agree with Barry. Who's this Way guy? What makes him more truthful in public today than in the past?
Unfortunately the only people his behaviour reminds me of are Albert Speer (who he doesn't know about) and Eugene Terre'Blanche (who he does know about). I presume that's why I am currently in moderation at CA ;)
Doug McNeall
Have you actually read any of the Climategate emails?
Apart from the fact that Way himself appears to believe that having McIntyre say nice things about him is the kiss of death, we have from their own mouths evidence of conspiracy to sack journal editors who publish material that challenges the consensus; we have Jones admitting that he will do his best to ensure that critical papers are kept out of IPCC reports; we have acknowledged experts on subjects as diverse as polar bears, sea-level, and hurricanes sidelined or "uninvited" or simply accused by implication of being nutters because their standpoint on global warming doesn't fit the theology.
"Robust challenge" sums it up nicely but not in the way you mean.
@omnologos
Are you being ironic, or do I get to call "Godwin"?
Doug - there is no Godwin any more, not after Cook's own nazi pics.
Spiegelhalter
99% Hero of statistics
1% Disgrace : He explained in the Telegraph this IPCC 95%, means they are sure that 19 times out of 20 this will happen ...he did not explain the 95% number just came out of thin air with no evidence and calculations to back it up.
(I note he normally he refrains from mentioning climate change issues, I believe if he applied his expertise to warmist dogma he could smash it apart)
A more worldly-wise and depressing list of cautions was given by John Bignell How We Know They Know They Are Lying.
His post begins with this:
and concludes with this:
(hat-tip Bob Koss)
The tracking in detail of the scientific advice received in private and promoted in public on climate variation and its possible causes will surely be studied for generations to come as people work at trying to understand just how we came to the sorry pass of seeing a weak hypothesis about increases in ambient levels of CO2 being a major driver of climate take centre stage and have an astonishing amount of influence.
I'm a little surprised that you haven't made a full post over the latest one at Climate Audit:-Robert Way's apparent distress, maybe fear, seems palpable.
But fear of what? Fear of the revelations about Way standing up for the scientific arguments, concurring with McIntyre's assessments of how deeply flawed the hockey-sticks are? Or fear about the consequences of McIntyre being nice to him?
With regards to scientific advice and in the context of the C&W paper, one prerequisite is that you make people aware of your assumptions. What you know and what you don't.
The C&W paper is one in a long line of temperature reanalysis where statistics are used for data with more larger error bars.
Which basically means that this is all mathematical playing when in reality anomalies and trends of 0.1 degree are meaningless.
Now I'm sorry that people have made careers out of analysing temperature data but in the real world getting accurate measurements is bloody hard. You need consistent methods and you also need to state your errors from start to finish.
In the C&W paper we don't see 0.1 with an error of 2 degrees which is what it actually is realistically. Instead we see the usual assumed accuracy based on a heap of assumptions.
The only reason we are in this position is because AGW predicts temperature rises of the order of 0.2 degrees C per decade. So people are desperately trying to conjure a result from data they have, arguing about trivialities and basically arranging the deck chairs on the Titanic.
To be proper science it all needs to be caveated with assumption. And then the results stated with that assumption.
The problem is that to do so would make the results much less important and destroy a billion dollar industry in the process.
Peer review is but one layer of the onion skin that an academic paper travels before it either reaches the majestic heights of a paper surrounded in awe, or it descends into the hopefully forgotten, or somewhere in between.
Nothing can give it the stamp of 'divine truth'. Science isn't like that at all.
It might follow the path of: a draft is written, it is revised by the author, then revised by colleagues in the same team, revised by colleagues in the same field or establishment, revised by colleagues in associated fields and then put out for peer review. After peer review it is published, and the process begins again, and continues for ever!
The earlier a problem, mistake or an error is found the sooner it can be STUDIED and appropriate changes made. It might be the start of a discovery.
How embarrassing, for some!
What still puzzles me (and you lot, no doubt) - is how the 'original' hypothesis is still taken as gospel by the media and politicians....
I suppose I could answer myself by the following simple procedure...
Follow the money...
How about a bit of carrot for climate models which actually work? An "X-Prize" type award.
1) First take the trouble to actually carefully define some key physical metrics. (I'm referring to actual measurable quantities such as temperature here, not abstractions like climate sensitivity.)
2) Hand out some cash prizes to teams that are able to make models predicting these metrics better than their competitors, within rigorously pre-defined limits, conditions, and dates. I'm sure Richard Branson, or even Al Gore, could dig in to their pockets for this. It could be the best $1Million or so that they ever gave away.
This could have been done years ago, but we are not yet even at the start line yet. I mean, who even decides what models get included in the IPCC reviews, and why? The number of models only ever seems to increase as their collective performance gets worse. The current prediction range still seems shockingly broad as well as too wrong to be credible. Will AR6 include models forecasting falling temperatures while the SPM still insists they will only go up, irrespective of what happens in the intervening years, and their confidence has laughably increased again?
If there is to be an AR6 in another six years time, they could, potentially, have six years credibility in the bank. Or instead, they could still have a situation where they can fiddle the graphs every six years, including all the latest most alarming ones they can think of, and discarding previous ones that are now an embarrassment. And credibility still effectively zero.
I did notice this, in Steve McIntyre’s article: “…gap between scientists and skeptics…”
Is this a admission that the latest purview of science is that to be sceptical is not to be scientific?
The Nature article is also being discussed at Judith Curry's blog.
She picks out most of the climate-relevant points:
"Confirmation bias arises when scientists find evidence for a favoured theory and then become insufficiently critical of their own results, or cease searching for contrary evidence."
"Scientists have a vested interest in promoting their work, often for status and further research funding, although sometimes for direct financial gain. This can lead to selective reporting of results and occasionally, exaggeration."
"Data can be dredged or cherry picked. Evidence can be arranged to support one point of view. The question to ask is: ‘What am I not being told?’"
You may feel this doesn't go far enough, but it is a useful step in the right direction.
Doug McNeall in private your partly correct in the public your are totally wrong and 'the Team ' have made dam sure of it .
You only have to read Mann's own words to see what happens to those that 'step out of line'
Much effort has been but in by climate 'scientists’ into PR , spin and smearing . none of which have been the requirements of science
As with management consultants, 'everything OK' is not what they expect to hear, or will ever admit.
Doug McNeall -
At CA, Way wrote, concerning the publication of his posts on the SkS forum, "Once again – I must reiterate my disdain at finding more of my private discussions placed on the internet. Including instances where I disagreed with other scientists and individuals using language that is meant for private discussions. Do you feel that it is fair to do this to a grad student? To choose to highlight examples that may someday lead to negative repercussions for an individual?"
If, as you suggest, science is somehow above personalities, why do you think that Way believes "negative repercussions" will ensue from his posts?
Robert Brown has commented at length at WUWT (Cowtan & Way off course - Monckton). This is an extract
This suggests to me that what is being practiced here is not science
Interesting. Way seems to be following advice to steer clear of the hockey stick yet finds nothing wrong with anything authored at SKS?
MikeC - Way has been advised to steer clear from criticizing the HS. So he spent time at SkS.
ps last time I check Steve McIntyre hadn't been born yesterday so...I am not sure how to say this nicely but I find it unlikely he didn't know what was going to happen to Way after CA's revelations. Perhaps he (McI) holds the boy at the same level of contempt I do.
If Robert Way agrees with the sceptics, why are the sceptics not accepting his results?
Entropic man - your meaningless question implies a degree of partisanship you should be thoroughly ashamed to show in public. Thank goodness for your anonimity.
Scottish Sceptic has a post on a similar theme, personal indemnity insurance and whether climate scientists who advise governments and get paid, for example the Met Office, should should have some kind of insurance for when their advice is incorrect and costs money as a result.
do climate scientists have personal indemnity insurance
I've just waded through that thread over at CA and have to say that Way sounds like one seriously wounded individual. It's almost as if he has viewed SteveM's positive comments as a death wish or something and all of his comments just reinforce that impression. He is doing done serious damage limitation!
Why, it's almost as if he believes he will be punished in some way?
Mailman
omnologos
Robert Way co-authored a paper and is now getting personal attacks from sceptics who disagree with his results.
Attacking the author because his results disagree with your beliefs is no way to do science.
Entropic man - your new comment makes a completely different point than the previous one. However I cannot fail to consider that you're still implying motives that you cannot demonstrate, and therefore you still have no argument.
I for one have made it very clear that my point is about Way's duplicity. His results have zero to do with that.
OTOH if you can demonstrate your abilities in internet-based telepathy, please confirm which number between 1 and 50 I am thinking at 00:29GMT. thanks.
omnologos -
42. It's always the answer.
Entropic man
'Robert Way co-authored a paper and is now getting personal attacks from sceptics who disagree with his results.'
Actual the normal way to do science is to challenge the claims research makes ,although I understand that climate ‘science ‘ regards is as both unnecessary and unwanted . In case people find things wrong with your work.
Although your right on the personal front , you be in much better position if you could equally condemn the smear technics of ‘the Team’ which have extended to trying to get journal editors sacked for the ‘crime’ of publishing accurate but ‘team ‘ unfriendly research. And not to mention the repeated attempts by the fat Mann to defame any and all who dare not agree with him.
But oddly those personal attacks , amongst many others , get a free pass , strange how things work is it not.
Stewgreen
"I believe if he applied his expertise to warmist dogma he could smash it apart" Agree 100%.
David Spiegelhalter is a longstanding (40 years) friend of mine, a genuinely nice guy and very good indeed at statistics.
Can I suggest the readers here pick their favourite peer-reviewed nonsense and at the end we can give David a reading list of perhaps 6-10 papers that satisfy the following criteria:
1) They are dependent on statistics to support their argument (don't ask him to opine on atmospheric physics)
2) They contain flaws that are sufficiently serious to invalidate their conclusions (ie no nitpicking)
3) They play an important part in the CAGW narrative.
Obviously if he chooses to ignore the evidence when it is given to him, that would be a disappointment, but it has to be worth a try.
The Entropic Straw-Man strikes again.
Sceptics are not, and do not need to be, of a single monolithic opinion. They do not have to agree with each other, with you, or with Robert Way. It is the substance of the argument that matters, not a consensus.
As knr points out, science is not about one person being either right or wrong all of the time. No one is. Robert Way's biggest worries at the moment are probably about how to deal with hockey-team backslash resulting from a prominent 'sceptic' being publicly nice to him. Nicer to him than he apparently deserves.
@David S Yes OK, but my guess is he avoids talking about the subject as to keep peace with his social circuit (like maybe greenie wife or kids)
Anyway This ties in with a kind of Comfirmation Bias ; Motivated Numeracy BBC Radio : More or Less
: A study suggests that our ability to do maths plummets when we’re looking at data which clashes with our worldview.
- Of course in the US the usual left wingers in the states used it to say "see Republican's are dumb", but the effect applies across all sides and all abilities (even uni profs) in the sample of 1000. HuffPo reasonable coverage
- Personally I see this as a "confirmation bias" effect I see all the time. When people just jump to requote figs like 97%, or IPCC 95%confidence, it's like their IQ suddenly dropped, cos surely with things which are "too good/bad/strange" to be true you have to go back and look for problems with the sources.
To paraphrase Way: "I agree many sceptics and particularly with everything Steve M has to say about Mann's Hockey Stick but he's still a 'wackjob' because I'm scared about what the bigger boys - i.e. Mann and (for heaven's sake) Cook will do to my career."
How did it come to this? Snivelling fraudsters and serial fantasists get to impose their warped outlooks on the whole field of environmental science and bully young scientists into submission on the basis of... will somebody please tell me?
And there's somebody here who used to have responsibility for challenging young minds actually defending this idiocy?
Yes you, Whybray.
Policy makers are modern politicians. That is, they exist to identify "issues" that can keep them employed, if, that is, the word is meaningful when discussing politicians. Issues guarantee constituency. Constituency guarantees votes. The idea that "good" advise might matter to the typical politician is absurd. They wouldn't known if they tripped on it. What a politician wants is advise that will open the door to some "plausible" issue that will allow him - or her - to differentiate themselves from the remainder of the mob. AGW is plausible, ideally so. No one disputes the laboratory work that revealed that CO2 absorbs LWIR. Proportionately few voters understand the difference between energy and heat and assume that the terms are equivalent. Consequently, it is an ideal platform for accumulating an easily alarmed constituency - including purported scientists. And last, sadly, sincerity is often mistaken for honesty. Regardless of the "scientist" label far too many scientists in all fields are more sincere than they are honest.
The plot thickens, with McI refusing more comments. For some reason he's been sucked into the warped world of modern climatology, where nothing happens as usual.
At the end of the day I have the strong impression that Way is being treated paternalistically, as some kind of naive child, by his SkS friends, his professors and then now Steve McIntyre. I would be quite worried, if I were him.
"Sceptics are not, and do not need to be, of a single monolithic opinion. They do not have to agree with each other, with you, or with Robert Way. It is the substance of the argument that matters, not a consensus."
michael hart.
This is very noticeable. The term sceptic covers a great diversity of opinion, and much of it lacks substance.
At the rational end are scientists such as Curry and Pilke, who accept the physics and the data, but are unhappy with the forecasts. I disagree with them on the forecasts, but accept their position as legitimate and they are prepared to argue it in the scientific arena
At the other end are those such as AlecM and PSI whose thinking diverges widely from the generally accepted laws of physics, and those who refuse to accept the measured data.
The biggest scientific weakness of the sceptic movement is its inability to present an alternative paradigm to carbon dioxide induced climate change, one which fits the measured data and the physics.
In the long run, scientific theories are accepted because they explain observations better than any alternatives. Until a sceptic paradigm demonstrably outperforms the current climate change paradigm, climate scepticism will remain a minority view among researchers in the field.
Omnologos,
I’m not sure I would read too much into SM closing comments. As you know, he has always had an aversion to what he terms “piling on” and I’m guessing that this, coupled with the fact that all the newer comments seem to be pretty much reiterations of previous ones, is what has prompted the closure of the comments on that post. There may well be an element of paternalism there, or it may just be that he has spotted a crack in the wall and he wants to see if it’ll widen. As for Way, well it seems the boy has talent, but he is at the crossroads. The direction he chooses to go now could well set the pattern for the rest of his life. I don’t for one moment expect him to come over to the dark side, but I sincerely hope, for his sake - not mine, that he can at least move away from the bunch of charlatans that is SkS and the group of activist “scientists” they kiss the arses of.
And yes, I can see the paternalistic nature of this comment ;)
EM,
For goodness sake, give it a rest man. You’re beating a drum for a “paradigm” that is disappearing up it’s own backside so fast that soon all you’ll be able to hear is it’s farts. The rest of us will have moved so far away from it, we won’t have to either hear them or smell them, but you, still superglued to the left cheek, will have no choice. We’ll just be stuck with the bill for the toilet paper.
With yet another conference achieving absolutely nothing, with the science being shown to be more and more inadequate day by day, with the general population finally realising just how much this is all costing them and not liking it and with more and more politicians beginning to see the light, it’s all but over. So it really, really doesn’t matter what the “researchers in the field” think. It’s irrelevant and has been since they themselves allowed the thing to become political dogma somewhere between the FAR and the TAR. That you seem incapable of seeing that speaks volumes.
The biggest scientific weakness of the sceptic movement is its inability to present an alternative paradigm to carbon dioxide induced climate change, one which fits the measured data and the physics.
---------------------------------------------
I get the distinct feeling we've been here before, EM. It is not incumbent on sceptics to come up with any alternative paradigm other than a null hypothesis. In fact, it's not even incumbent on sceptics to do that. That is the job of the scientist proposing the paradigm under discussion.
In this instance their inability to disprove the null hypothesis wouldn't pass muster at the undergraduate level...and they have not set themselves a particularly challenging height at which the bar is set as far as the IPCC is concerned.
Omnologos, like LC, I wouldn't read much into SM closing comments on the thread. I've seen it on at least one other thread when he thought that sufficient had been said or that a particular comment was a very good one to finish with. And it is his blog.
Unfortunately the null hypothesis you put forward does not explain the measurements. It answers a variety of questions less well than CO2 induced warming.
For example.
Why is there a dip in outward longwave radiation at 15 micrometres?
Why is there a corresponding downward radiation of similar wavelength and intensity?
Why is the downward radiation increasing?
Why is there an imbalance of ~1W/M^2 between total incoming radiation and total outgoing radiation.
Why has the minimum Arctic sea ice extent and minimum volume decreased since the 1950?
Why is the volume of the Greenland and Antarctic icecaps decreased by ~500 cubic kilometres/year?
Why is the sum of the annual latent heat of fusion from melting ice and the annual increase in ocean heat content similar to the incoming energy from the radiation imbalance?
Why is Winter Antarctic ice extent increasing while Winter Antarctic temperatures also increase?
Why is man sea level rising by 3.2mm/year?
Why is the average intensity of typhoons increasing?
This is what I mean when I talk about paradigms. All these processes are connected by energy states and energy flows in the climate system. They are part of the same pattern, which the climate change paradigm recognises and , in large part, explains. There is considerable evidence to support it
If this were a scientific site, rather than a propoganda site, I would expect you to present counter arguments in support of the null hypothesis. I do not expect it here, just old arguments rehashing the output of lobbyists like the Heartland Institute. Have you no originality?
I have never seen anyone demonstrate that your null hypothesis can come anywere near to understanding the interactions of climate, let alone explaining them. It is a propoganda sham, designed to cast doubt as similar arguments regarding leaded petrol, smoking etc were designed before
EM I'm not going to go through your "whys" one by one. Some of them are questionable assertions; others have no demonstrable physical connection with a defined climatic outcome. Several such as sea level rise and gradual global increases in temperature were happening before there was any material increase in CO2 levels. One, the one about typhoon strength, is demonstrably false.
To bundle all these up and say "I assert the cause is CO2 and the prognosis is catastrophic warming - prove me wrong" is complete nonsense and betrays your total ignorance of the scientific process. If you make an assertion, it is for you to prove it, and to prove it you need a replicable demonstrable physical mechanism that links the parts.
Otherwise you are selling flying teapots and spaghetti monsters.