Bishop Hill Science is often flawed
That is the message of this long piece at Vox.com
Recently, the conversation about science's wrongness has gone mainstream. You can read, in publications like Vox, the New York Times or the Economist, about how the research process is far from perfect — from flaws in peer review to the fact that many published results simply can't be replicated. The crisis has gotten so bad that the editor of The Lancet medical journal Richard Horton recently lamented, "Much of the scientific literature, perhaps half, may simply be untrue."
That science can fail, however, shouldn't come as a surprise to anyone. It's a human construct, after all. And if we simply accepted that science often works imperfectly, we'd be better off. We'd stop considering science a collection of immutable facts. We'd stop assuming every single study has definitive answers that should be trumpeted in over-the-top headlines. Instead, we'd start to appreciate science for what it is: a long and grinding process carried out by fallible humans, involving false starts, dead ends, and, along the way, incorrect and unimportant studies that only grope at the truth, slowly and incrementally.
I'm not sure that this is meant to apply to climate science though.
Reader Comments (37)
Surely he's wrong.
After all, we know that if something is 'Peer-reviewed' (Blast of holy trumpets, clouds part and God-rays stream down as the Celestial choir sings, ALLELUJAH!) it is true.
Climate scientists have told us stupid people this.
I am a retired physician-scientist and some years ago wrote an essay for our local medical journal trying to make the point that decisions, including those based on science depend on evidence bu not all evidence is of the same quality. What follows is a summary of types of evidence and how reliable they are. Evidence is derived from information but information is not necessarily evidence. Our ease of communication often makes it easy to blur this distinction.
Climatologists often seem not to care whether an observation is information or evidence.
The following is a list from the essay mentioned above and seems pertinent to the present post.
Information as Evidence (From Least- to Most-Reliable Sources)
1. Advertisements (including political speeches)
2. Hearsay
3. Testimonials (including most internet sites)
4. Testimonials from Friends or “Authorities”
5. Sworn Testimony
6. Recorded Observations
7. Recorded Systematic Observations
8. Recorded Results of Interventional Experimentation with Randomisation and appropriate Controls (including blinding of the observer(s) and subjects when necessary)
9. Recorded Results from Replications of Experiments as in 8
10. Successful Predictions Based on a Model derived from Recorded Systematic Observations e.g., Tide Tables and Eclipses
11. Successful Predictions Based on a Model derived from Experimentation e.g., Atomic fission and fusion
Only information derived from the sources referred to in points 8, 9, 10 or 11 allows causal relationships to be defined. Information from the sources indicated in points 1-7 can give clues but not conclusions.
What is truly frightening are the policy decisions which have been based on flawed science. Scientists are not gods, yet regulators and legislators treat them as such. Therein lies the problem. The practice of science has been sloppy since it was first invented. But in the recent decades. legislators have abandoned their responsibilities -- to employ common sense -- and latched onto dubious scientific claims because it is relatively safe.
'Half of it is not true'
'I am not sure it applies to climate science though' BH
Is that because in climate science, 97% is not true?
No different to Board room decisions, about 50% are correct, rest are wrong.
Morley, a question for you, on a quick run through none of those 11 evidence sources include Climate Models, ie all the ones that do not match observations. Care to put them in your list, looks like a 4.5 to me.
As this unfortunate development has all to do with the reverse of the scientific approach: to disprove a hypothesis it is unfortunately transformed into a bureaucratic " science" in which the mores appears to be to prove the hypothesis. This is not unlike Lysenkoism or Eugenics with their disastrous consequences.
As mentioned before, in the 10th series of the BBC's "QI", it was noted that of all the new "facts" introduced in series 1, 5/6ths of them had since been shown to be wrong.
New facts referred to facts recently reported in the scientific literature, which the QI gnomes were fond of reading to extract information with which to confuse the panellists.
I was wondering how long it would take...
This statement has a long history. Recently, the most obvious researcher to have his head above the parapet was John Ioannidis in 2005. Here is his paper - which should be given much better publicity than it has got...
http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124
Dear Breath:
Correct predictions from models are good evidence that you more or less understand some system e.g., being able to predict tides or stellar eclipses (Number 11)
However, ALL models must be tested and are not evidence until shown to have "skill". Being able to hindcast (mimic the past) is not proof that a model is equivalent to reality. In the case of Climate Models", they can be tested and shown to have "skill" only be making reliable observations of future climate.
And using computers does not change those requirements. Nor is writing a computer program the same as doing an experiment.
Dear Breath,
My proof-reading skills are non-existent: There should be quotes around "Climate Models" and "be" should be "by" in the phrase, "only by making reliable observations".
Morley Sutter, as someone who never knew the correct way to fold a floppy disk (I did try), I do understand your explanation of computer models and verification. Thank you.
Now could you please explain it to climate scientists, some of whom seem to rely on photocopying their Tippexed computer screen, to hide the mistakes.
Climate Science is an oxymoron designed to fool real morons.
It has succeeded brilliantly.
Discuss.....:o)
'I'm not sure that this is meant to apply to climate science though.'
oddly your right and that is becasue either its standards are so low that its almost impossible to fail or that in a very 1984 style 'fail' is not an available option, they either pass or double plus pass.
We are after all dealing with a 'science' that thinks 'heads you lose , tails I win' is acceptable scientific approach that seeks to reverse the null hypotheses ideal and claims that 'scepticism' is both impossible and bad when it comes to its theories . So in the mind of its 'faithful ' it simple cannot fail .
And sadly in playing the three wise monkeys the rest of science has merely reinforced that view .
I am an organic chemist and when I was going through college and grad school, one of the big developments was the Woodward-Hoffman rules on molecular rearrangements. In their seminal paper on these rules, the authors (who subsequently shared a Nobel Prize) wrote a short section just before the conclusion titled "Exception". The first line of that section was "There are none." This line was not put in their to stifle debate but was essentially a throwing down the gauntlet saying prove us wrong. It sparked an intense amount of experimental work over the next 10 years to do just that. I think of science as a king of the hill battle where the dynamic is to always try to come up with a better description and understanding (theory) of the system you are studying. There are a lot of hills to try and hold, some of these are key terrain, others few really care about and sometimes, things just go obsolete. For climate scientists to claim "the science is settled" is akin to creating the Maginot line, just when the main battle tank and the tactics to employ it came into its own.
Science can only deal with FACTS, not "information". The above list of "information as evidence" is just a circling of that central fact at the heart of true science: facts. Objective facts: both physical and logical observations can be so. Remember that, please. And this generation's discussions about "truth" in science are a lot like the continuing vain debates about "climate"; everyone is mistaking the forest for the trees (there IS NO "global climate change", throughout the history of Man on Earth, only local). A true scientist knows he never discovers ALL that is true, nor does he try to; he tries to ADD to the true knowledge of mankind (and in the process, he is aware that he/she may very well correct past mistakes, or even make new ones that will need to be corrected; nevertheless, true science--which is very rarely seen in the public debates/"information wars" of today, because the entire system is broken--IS self-correcting, although not much in the last two generations, and not even in the last 156 years, in the case of evolution theory). Don't listen to those science wannabees who assert confidently that "science never proves anything". That is a bald lie--an OBVIOUS lie, people!--and there are too many bald lies threatening the people of the world now.
Science can only deal with FACTS, not "information". The above list of "information as evidence" is just a circling of that central fact at the heart of true science: facts. Objective facts: both physical and logical observations can be so. Remember that, please. And this generation's discussions about "truth" in science are a lot like the continuing vain debates about "climate"; everyone is mistaking the forest for the trees (there IS NO "global climate change", throughout the history of Man on Earth, only local). A true scientist knows he never discovers ALL that is true, nor does he try to; he tries to ADD to the true knowledge of mankind (and in the process, he is aware that he/she may very well correct past mistakes, or even make new ones that will need to be corrected; nevertheless, true science--which is very rarely seen in the public debates/"information wars" of today, because the entire system is broken--IS self-correcting, although not much in the last two generations, and not even in the last 156 years, in the case of evolution theory). Don't listen to those science wannabees who assert confidently that "science never proves anything". That is a bald lie--an OBVIOUS lie, people!--and there are too many bald lies threatening the people of the world now.
I'm not a climate scientist but I do not believe climate science has any information wrt your points 8-11.
Sorry, just thought of another comment. I believe a lot science is now backwards. It seems like it is more beneficial to support a popular hypothesis than falsify it. And the reason to me is fairly clear; money and/or politics and/or power.
I supported separating the state and the church and I now feel our next task is separating the state and science.
From the article, "Indeed, another review found that researchers at Amgen were unable to reproduce 89 percent of landmark cancer research findings for potential drug targets."
Imagine that 89 percent of automobiles failed to start in the morning. Imagine that 89 percent of the milk sold was sour. Imagine that 89 percent of airline flights failed to arrive at their destination. Imagine that 89 percent of climate models failed to predict "the pause."
What makes cancer research and climate models different from automobiles, airlines and milk?
They used to say that half the money spent on advertising was wasted, but nobody could tell which half.
I think this could be recycled and applied to Climate "Science".
Mike:
Climate, as with astronomy does not lend itself to interventional experiments so point 11 above does not apply to Climate scientists.
Point 10 applies, but only after the passage of time. That is one reason why the "hiatus" (I prefer "plateau") of the rise in global temperature is of so much import with so much concern about its statistics and whether it is occurring.
"Climate Science is an oxymoron designed to fool real morons. It has succeeded brilliantly. Discuss.....:o)" --NCC1701E
Thank you, I shall discuss, 1701-san.
There are legitimate climate scientists. The number is uncertain, particularly when so many cave in to Lysenkoist demands and append the shibboleth to their papers saying, in essence, "The fact that this study indicates there is no evidence of global warming in the area we studied doesn't mean that there isn't some or maybe even a whole bunch. Send money. We ♥ Michael Mann..." One doubts the intellectual legitimacy of such people, if not their conclusions. It is sad.
As for morons, yes, Climate Science as mostly practiced has converted large numbers of them to the Faith. But it has also fooled huge numbers of otherwise intelligent people. People of the so-called "liberal" persuasion are often intelligent, but lack some other quality, such as common sense. Putting it another way, they believe in magical thinking. They do not, typically, fully believe in laws, rules, history, logic, reason, tradition, or established methods (such as the scientific method as previously understood).
All these things and more are, in their minds, subservient to their own will, to their own desires. They excuse themselves for rejecting established order in favor of their own self-will on the basis of some noble goal loosely connected with what they want, the particular end that will justify their means. Part of this may be on an unconscious level.
For example, some Believers in their hearts wish to blow up children. Their license to do so consists of having consciously put these children (and/or their parents) into the class they call "Deniers." This form of judgmentalism and its associated processes are known as "Othering." Having successfully (and culpably) othered these children, the Believers can then in good conscience set aside the children's rights and carry out the perverted desires of their own Shadow, if only in film at this point in time.
Morley, thank you for the reply, between 4 and 5 is looking too optimistic then ;)
Dr Kelly was equally scathing in his report too.
Sean, in their book Woodward and Hoffman followed "There are none!" with "Nor can violations be expected of so fundamental a principle ..."
I think they (or perhaps only W) thought "the science was settled" :)
NCC1701E In primate science tests, most of the primates sat on their desks, scratched their armpits, and made grunting noises eating a banana.
In climate science tests, all the scientists did all the above, but only after securing a five figure advance from a leading climate science journal.
It has long been a mystery to me why climate scientists en masse refer to a new paper by Newbie and Sprog as proving something, peer reviewed or not, two days after its publication as though the conclusions of said paper had any worth until the clici community had tried to replicate it, or better still prove it wrong. Of course if a paper by Experienced and Tested that doesn't support the CAGW meme is dismissed as rubbish on the day it is published.
A scientist publishing a paper should be simply someone saying "Look, this is what we found, can anyone see where we went wrong?"
Researchers will just have to produce more policy-based evidence to keep their paymasters happy.
And if the science ain't right...............it can be
adjusted upwardsfixed - right?I mean don't get me wrong, coz we are believers and all.............there would be no need - after all the science is settled! But then, that pesky lot called um - the public - God they are a set of unbelieving bast**ds - know what I'm saying?
"So, we gotta keep 'em coming....."
Recent [EU - ergo no arguments sonny!] Agence spatiale européenne - satellite altimetry is the be all and end all in oceanography. And it now says that, universal sea level is rising by so much, even by more than you think cos the western Antarctic ice sheet is melting and Lordy so is the Greenland ice cap too [haven't heard that one since......... for a bit so get it out there again - Paris coming up] and don't forget it's all peer reviewed stuff - so that's covered by 97% of all scientists - without question and stuff - innit.
It's religion = not science matey.
It also includes the condescending, one eyed - purblind propaganda, its risiblly false reportage is what many of us - we all find so bloody annoying.
Yes, the observations apply to Climate Science.
No, the observations do not apply to politics and, consequently, to the politics of Climate Science.
graeme no 3 @ 6.13: In preventative medicine there is a concept of "numbers needed to treat" which is the number of people you need to treat to prevent ONE specific adverse effect. Your 50% would equate to a NNT of 2, a figure which would certainly earn a Nobel Prize (well these days it would). In the field of Statins figures are over 10 meaning 90% is "wasted", though I doubt if the 10% would agree, if we new who they were. As it is I dutifuly take my meds each morning reflecting that I am probably saving someone elses life!
I think our approach to the "scientific reality commons" will fundamentally change from "discovering the truth out there" , to "defining reality"
note how relativity makes from 3D something utterly incomprehensible with "time" , whatever that is, being just another space dimension. a 4D world to which Kurt Goedel immediately objected. Nowadays 11D worlds are opined etc.
Very likely it will become a sort of "tell us how many Ds you want, and we fabricate you something where you feel comfortable in"
It is ridiculous to pretend there is anywhere a consensus but ok without consensus "sir" paul cannot keep going on organizing 1 checkbox baloting. You need a "consensus" for that , fought over with all the marxist sociopath tricks of the book.
Kim also has consensus in his government. Probably even better than 97%..maybe 99%..but the future will determine how reliable a 99% it is..
Morley Sutter 3:15PM
As you likely know, the definition of causal relationships (criteria of causality) was well described by Bradford-Hill and requires a number of minimal conditions not least among which is plausibility. As we all well know, association is not causation. Models may make fair or good predictions and be judged as 'successful' but they do not necessarily demonstrate causality, though they may highlight association. To be 'successful' you will surely require the assurance that ALL universal possibilities have not only been predicted but tested...
Manfred,
As you undoubtedly know, causality is difficult to define and even more difficult to demonstrate.
The simplest approach is to classify putative causal agents it into 3 types: non-causal (unnecessary), necessary but insufficient, and necessary AND sufficient. The Moon's gravity is a necessary but insufficient cause of the height of tides at a particular location and some 60 variables are involved in producing the tables of tides - a mathematical model is used. So you have a point and my use of the term causal was loose.
If removing or altering a factor believed to be a causal agent results in a different outcome, that is good evidence that the factor indeed was part of the causal system. One is testing a model when one does the intervention.
You are quite right in pointing out that one must know exactly what the intervention does before a causal role can be assigned to the intervention.
In my view, CO2 certainly has not been demonstrated to be the controlling factor of climate temperature.
@golf charlie and jorgekafkazar: thank you for your replies. I have just enrolled for my Climate Alchemy PhD by examination (my first PhD was an awfully long time ago in a hard science). Apparently, the modern version requires that you investigate any physical or biological phenomenon and show how increased temperature and associated humidity change will vary it.
Therefore, I have decided to investigate the growth rate and habitat of bananas, and how increasing [CO2] will lead to the starvation of Climate Alchemists.
But I may not survive......:o)
To say nothing of attempts to actively bias it, and so give some preconceived prejudice or vested interest the false prestige and status of "science".