Darrell Ince on the tranny

Darrel Ince is interviewed by Tim Harford about the difficulties of getting corrections made to scientific papers.

Darrell Ince on wrong papers

Comments

[The Leopard In The Basement]

The interviewers reaction is interesting and he asks the important basic questions. I think it would be indicative of most peoples reactions if they knew how lax the verification requirements are in the major journals.

[ZT]

'...one of the Duke researchers accepted responsibility and resigned...' so in the US, the right things occur, eventually. Very different to the UEA/CRU/RS/UK fiasco.

[jorgekafkazar]

Nice interview! But the Duke experience merely shows that there are no megabuck institutions that are willing to prop up bad medical science. Climatology is quite different. I'm wondering how many dollars have been donated to University of Virginia recently to assist them in stonewalling Ken Cuccinelli's anti-fraud investigation.

[Wellers]

Perhaps when Tim Harford he is in Toronto on May 27 to give a lecture he should interview Steve McKintyre : Climate Change is one of the topics he will cover in his talk. I am sure he would find this very enlightening given his surpise at what he heard about major journals (like Science) obstructing objective criticism of earlier published work, and their weak data availability requirements.

[hro001]

Not sure what the "tranny" is (pls. forgive my Canadian ignorance!), but I found the interviewer, Tim Harford, to be amazingly enlightened. His bio indicates that:

In 2011 Tim was named one of the UK’s top 20 most influential tweeters by The Independent newspaper

So I decided to pursue one of his recent tweets:

Use academic research with caution: http://goo.gl/2feg4 says academic researcher @cblatts Wisdom. (cc @bengoldacre )

This took me to a fascinating post on the blog of Chris Blattman (Associate Prof. of Political Science and Economics at Yale), in which Blattman very approvingly cites some excerpts from the post of someone else. Here are the excerpts:

Never put too much weight on a single study. If nothing else, the issue of publication bias makes this an important guideline…

Strive to understand the details of a study before counting it as evidence. Many “headline claims” in studies rely on heavy doses of assumption and extrapolation…

If a study’s assumptions, extrapolations and calculations are too complex to be easily understood, this is a strike against the study. Complexity leaves more room for errors and judgment calls, and means it’s less likely that meaningful critiques have had the chance to emerge…

If a study does not disclose the full details of its data and calculations, this is another strike against it – and this phenomenon is more common than one might think…

Context is key. We often see charities or their supporters citing a single study as “proof” of a strong statement (about, for example, the effectiveness of a program).

It's also worth noting that "Holden", the author of the post from which the above was excerpted, begins his post as follows:

"We often use academic research to inform our work, but we try to do so with great caution, rather than simply taking reported results at face value. We believe that if you trust academic research just because it is peer-reviewed, published, and/or reputable, this is a mistake. [emphasis in original -hro]

Hear! Hear! to all of the above :-)

Holden's post is well worth a read in full, btw. While it pertains to a "debate" on microlending, the parallels to what we have seen in the "hockey stick" debate are quite interesting (if not astounding)!

[Eddy]

Tranny = Transistor Radio, or portable radio, popular back in the days of pirate radio :)

[The Leopard In The Basement]

@hr001

The one comment on that Holden microlending article has an interesting section that caught my eye

Secondly, there are immense issues with releasing data sets. Why? Because the researcher has spent considerable time and money collecting them and they can be used for multiple analyses, multiple purposes. By publishing them you’re thereby permitting others to analyse them not just for the current studies, but to draw unrelated new findings and inferences from them. Why should I essentially “give” that away.

My emphasis. Seems that it is not just climate science where one has to deal with the personal feelings of ownership of the researchers. A phenomenom I don't think Popper adequately covered in his work on the progress of scientific knowledge. ;)

[Paul Maynard]

BBC Radio4 More or Less

The interview with Ince was part of this programme.

There was also a very valuable interview with Jonah Lehrer about his article in the New Yorker last December describing the "Decline Effect" which is the propensity for effects that appear big when first appearing in peer reviewed top quality journals decline to nothing over time as they are unpicked by subsequent work. Of course we have the best example in the HS and the Team.

Both Ince and Lehrer make reference to the lack of archiving of data, complexity and the statistical weakness of modern scientists.

The New Yorker article is here

http://www.newyorker.com/reporting/2010/12/13/101213fa_fact_lehrer

Regards

Paul

PS regarding Harford's trip to Canada, my bet is that he will be a warmist that in the case of MMCC he will have set aside his professional scepticism.

[amavo]

If you don't know what tranny is, DO NOT GOOGLE IT.