Bishop Hill More critical science journalism required
May 18, 2013 Media Jalees Rehman, a medical professor from the US, reckons we need more critical science journalism.
Critical science journalism takes a different approach and focuses on providing a balanced assessment of the work, one that highlights specific strengths but also emphasises specific limitations or flaws. It is no big secret that the majority of research findings published in peer-reviewed scientific journals will probably not hold up when other groups attempt to replicate them. This lack of replicability can be due to research misconduct, systematic errors or other cognitive biases, which commonly occur even in the most conscientious and meticulous scientists.
Therefore, critical science journalism requires a careful analysis of all the data presented in a paper and is likely to uncover key limitations and flaws that scientific researchers themselves do not readily divulge. This form of science journalism can also encompass some degree of investigative journalism. Journalists lack the resources to check the validity of scientific data by performing experiments themselves, but they can track scientific research in a certain area over the course of months and years as multiple research groups attempt to replicate published scientific findings.
In the climate debate, critical commentary is of course par for the course, at least among the blogs. It's the newspapers that feel they have to act as cheerleaders, usually because the journalists have no scientific background and therefore struggle with any kind of critique.
Reader Comments (27)
Australian journalists have been accused of being lazy.(not by me!)
How much "news" is really regurgitated press releases.?
Esp in the business press.
Just lifting the phone and making a call or two could often add a bit of warranted cold water to the regurgitation.
And I was about to have a nice cooked breakfast.
Umm
Isn't the good Professor merely describing what we are asked to believe 'peer-review' achieves already?
Perish the thought that it doesn't!
I will wash my mouth out with soap at the mere idea......
Don't forget that some of the loudest noises in journalism are on the team. Seeking out and developing counter points is not in their brief.
"In the climate debate, critical commentary is of course par for the course, at least among the blogs."
///////////////////////////////////////////
I question the accuracy of that statement.
By using the expression "at least" it conveys the impression that in climate science critical commentary is not just limited to the blogs. Is it not the position that for the main part critical commentary is only found in the blogs, and even then some blogs act as cheer leaders and do not, or extremely rarely, objectively criticise. If that is so, it is difficult to see how one can assert "In the climate debate, critical commentary is of course par for the course..." Personally, I consider the opening sentence to require reworking.
The comment by Rehman is about journalism. Whilst journalism is potentially a wide expression, I am unsure whether blogs are journalistic in the sense being discussed by Rehman.
I consider the observation made by Rehman to be particularly apt to climate science, especially given the pal review that often takes place which appears compounded by group think and where the boundaries of science and politics have become blurred with politics and MSM polluted by activists wedded to a cause both because of perceived moral grounds and often financial self interest.
The BBC are so disappointing when it comes to science reporting. When I was at uni some 20 years ago I thought they were very good and had cracked it but now I realise it was down to one person presumably fighting the system. Being the BBC I don't suppose they actually know how hopeless they are and if I were to tell them I would be 'wrong.'
This has been said before
http://www.huffingtonpost.co.uk/dr-david-whitehouse/science-a-new-mission-to-explain_b_1122560.html
This may not seem relevant: Archaeologist claims to have located site of Roman battle
but I said "crop marks" not "crop circles".
"It is no big secret that the majority of research findings published in peer-reviewed scientific journals will probably not hold up when other groups attempt to replicate them."
//////////////////////////////////////////////////////////////////////////////////////////////////
So cut funding by 50% and again and again until it is no big secret that the majority of research findings will hold up. If Rehman's statement is correct then we are being right royally fleeced.
Or move science journalism to the crime desk.
@ Mike Haseler May 18 2013 and his reference to Tacitus; I came acros this quotation by Tacitus (Agricola 30.4) some years back: "The Romans, great robbers of the world, after all the lands have been diminished by their devastation are exploiting the sea, greedy if the enemy is rich, arrogant if he is poor. They cannot get enough of either the east or the west; They alone to desire or possess with equal madness the richness and the mysery of nations. Under the false name of empire they pass off robbery, murder and pillage:and when they have achieved desolation, they call it peace"
Nowadays it is called AGW
@ LA 2013 at 11:07 AM
Latimer, if you thought that you thought wrong.
I know quite a bit about peer review in my own field of medicine. It is solely about the one particular paper that has been submitted - does the methodology appear sound, are the results reasonable, are the conclusions logical? (Note - NOT are the results legitimate.)
What it does not do - or at least never should - is recommend rejection if the findings are contrary to current scientific wisdom. If that were the case the causes of stomach and duodenal ulceration by Warren and Marshall would never have seen the light of day. Not to mention Einstein, and a few others in different fields.
The real test of a "discovery's" legitimacy comes from it's repeated failure to be falsified, and by the emergence of confirmation studies. This can take many years. In medicine, almost all "discoveries" suffer the fate of falsification at worst or severe modification at best, and the world is better off for that.
"Infotainment" science journalism ruins my breakfast at least twice a week here, where there is breathless radio anouncement of yet another ground-breaking medical breakthrough by *Australian* researchers. At least 90% of these breakthroughs are never heard of again.
In Rehman's otherwise sensible article there was a glaring (with bias showing) inconsistency:
"This critical analysis can be either based on the science journalist's own assessment or interviews with dissenting scientists. It is not common practice to quote unnamed sources in science journalism. If investigative science journalism becomes more established, unnamed sources may become more commonplace as dissenting scientists might fear reprisals if they were to comment publicly about the work of their peers."
Sits pretty uncomfortably with, in the next paragraph:
"If 98% or 99% of scientists agree that humans contribute to global warming, it would be wrong to give equal weight to the views of the 1% fringe scientists who deny climate change and pass this off as a "balanced view"."
OK, it may not be a balanced view, but it is a view. If that always happened would Warren and Marshall's findings ever have seen the light of day?
I'd prefer 'analytical' to 'critical'. The latter's acquired enough baggage to founder its meaning.
===============
Mike Haseler:
And with one bound they made him sound like a nutter. Presumably by accident in that case. What can they achieve when they are really trying? :)
Richard Verney's quibbles are important but I think I can still empathise with the Bish in this sense: journalism or not, thank God for climate blogs. Climate science is in no sense ordinary science. It was highly politicised even in 1988, based on what Al Gore was already claiming for it in Washington, goading a younger Richard Lindzen to begin his long, often lonely rearguard action in defence of truth. That made Lindzen unusual even within the science and quickly caused his demonisation outside it. Such a poisonous atmosphere made the job of journalists much more difficult than usual and there's no evidence they were doing a great job of proper criticism in other areas of science.
So I interpret the Bishop as expressing deep gratitude for the climate blogosphere - not just tooting his own horn! Making the Amen from here almost involuntary.
Well said. If one had to rely on 'science' or 'environmental' correspondents in the mass media, what a narrow-minded, politically-loaded, ill-informed view one would gain about the issues around climate variation. I can only presume the people involved were chosen, and were attracted to these new opportunities, in order to help spread light where there was darkness. Their mission was and is to act as helpful intermediaries between the great works of intellectual giants, such as Mann or Jones perhaps, and the benighted public.
Fortunately, there have also been other journalists and other scientists such as Lindzen who have taken a more thoughtful and critical view of the veritable avalanche of papers and press releases produced by the CAGW monster, a monster seemingly deliberately constructed over the past two or three decades.
Here is one example which I just came across minutes ago. It was published in 2007, and is by John Stossel, who is presumably the same man who choreographed that version of the Nutcracker Suite relayed here by the Bish as few days ago (14th May).
http://abcnews.go.com/2020/Stossel/story?id=3751219&page=1#.UZdTN0pXpU0
Now it is very odd that more ordinary journalists have not pursued this field, since it would seem ideal for anyone intent on uncovering controversy, double-dealing, and deceit by powerful groups. The great and pioneering work by Donna Lamframboise shows the scope here - just about any of the pages of her book The Delinquent Teenager contains enough leads and issues to inspire further genuine investigative journalism. In the mass media in the UK, we have Christopher Booker as a fine example of a penetrating writer for the Telegraph, as is David Rose in the Daily Mail. But the blogs have Delingpole for fire, Montford for light, McIntyre for analysis, Watts for everything, Curry for academia, Peiser for policy, Pile for reflection, Pointman for overviews, Nelson for news, and so on and on, with many more I could easily mention off the top of my head.
There is surely scope, and plenty of raw material and ideas about for science correspondents of the kind imagined by Rehman, who would ' track scientific research in a certain area over the course of months and years as multiple research groups attempt to replicate published scientific findings.' and who would not hesitate to highlight 'research misconduct, systematic errors or other cognitive biases'. The grossly and rapidly inflated field of 'climate science' should provide opportunities a'plenty.
A lot of science reporting is "science by press release". It used to happen before a peer review as in the case of cold fusion.
An informed journalist is an unemployed one. The politics of a mainstream outlet is its best selling product.
There is a key misunderstanding/mistake that is pervasive throughout science-infotainment journalism, that Jalees Rehman fails to address adequately:
Science is NOT done by selecting from competing individual scientists working in the field, neither jointly nor severally. It is done by selecting from competing hypotheses.
Going further, it should then be pointed out that there is an infinite number of alternative [wrong] hypotheses that the diligent scientist needs to investigate and eliminate. This is, of course, impossible. So a compromise must be found, assumptions must be made, and the scientist must show in detail how/why this was done. Scratch the surface of the truly diligent scientist and underneath you will sea the effort expended in both finding, and eliminating, plausible alternative explanations.
Glib statements such as "we've looked at all the alternatives" should set alarm bells ringing.
The canny old guy, Adam Smith, saw this kind of thing 2 centuries ago, and recognized that it will continue as long as there is a market for it and little market (or other incentive) for a better product. Substandard nonsense will continue to pour out of lecture halls, journals, and all sorts of other places until then.
The thing I keep wondering is why the customers of this medical research don't start up their own rating list. If Journal A publishes stuff that they can't replicate but Journal B does better, how about encouraging the better ones by publishing that evaluation. It's in their own interest to encourage better work.
Who would those customers be? In the case of medical research, how about the medical companies whose job is trying to convert these marvelous insights into real therapies.
'Jalees Rehman, a medical professor from the US, reckons we need more critical science journalism.'
Well, there's a blog right now on the Telegraph that he might care to review for a start. Particularly the comment thread below. My opinion of the critical awareness of Telegraph commenters has gone up impressively.
http://blogs.telegraph.co.uk/news/graemearcher/100217545/statistically-speaking-how-rational-people-can-disagree/
Corollary to: An informed journalist is an unemployed one.
An informed journalist is like a blogger.
'usually because the journalists have no scientific background'
That however does not mean their not well educated , jounlism is full of Oxbridge PPE's , with oddly the Guardain which sells itself 'of the people ' being more full of them than the Times and Telegraphy.
But when it comes to 'the cause ' even those that do ask questions of what they are told , come over all wooded headed. With many acting has little more than photocopying mechines for anything their feed by the greens .
I seem to remember a once feared breed designated "sub editor". The type that would tear up "cut and paste" copy and feed it back to the reporter shred by shred on a nightly basis in the designated watering hole.
The Fourth Estate has lost its ability to challenge, this media may become the Fifth Estate but at present there is a peculiar sort of vacuum that only appears to suck in sound bites.
Indeed, sub-editors regarded all journalists as illiterate, drunken, opinionated manglers of logic and the English language, and their wearisome -- even Augean -- task was to transform the gibberish sent (usually late) to them into something that was fit to appear in print on the following day.
In such circumstances, the desire to add some public humiliation was understandable. I don't know whether this practice survives in our caring, sharing society.
May 18, 2013 at 10:09 AM | Peter Walsh
/////////////////////////////////
Peter, I like the quote.
The line "...when they have achieved desolation, they call it peace" is a particularly astute and damning observation. Wonderful stuff.
Times may change, but the short comings of man apear stubborn and rather too resilient..
Peter Walsh
Wasn't Tacitus allegedly quoting Calgacus first inhabitant of what is now Scotland to be identified by name? It's one of my favourite quotes. Unfortunately could now be applied to Scotland's current leader and his insane energy policy.
@KNR... Journalism may well be full of Oxbridge PPEs, but after scrutiny of many of the more high-profile holders of said degree one is forced to draw the conclusion that the Oxbridge PPE is the equivalent of lesser establishments' "Media Studies".
"critical science journalism", indeed it could be a fine idea.
How different might things be, if journalists writing for and representing certain media organizations in the outset, instead of blindly adhering to what they were told to believe had peered hard and used a critical eye and an objective approach?
If, Hansen had been got at much earlier and Gore's pathetic film had been challenged by the media, if the UEA and the Met Office [Penn State University et al] were scrutinised and properly investigated, questioned about their clearly dodgy predictions and methods.
But no, oh no, the loons of the left leaning media have ruled the roost, closed down debate and all along it was they who have been ordered by the political elite to set the agenda - thus the media were and are, totally complicit in the great man made CO2 equals global warming and willingly helped along the greatest ponzi scheme known to mankind.
Groupthink through mass propaganda were the only instructions in the MSM editorial threads and studios - thus, "critical science journalism" currently doesn't and never did exist in the mass media.
You have to look to the blogs for any real critique and that's why the EU wishes to censor and cut down to size - all anonymous commentary and 'maverick' bloggers [yes that's you Andrew:*)...].
"Stuff of nonsense!" shouted the eco warrior and swivel eyed Brussels loony: we cannot allow people to know and to tell the truth!
That would never do.