Tuesday
Feb082011
by 
Bishop Hill
Bishop Hill Statistical literacy
Feb 8, 2011 Climate: Statistics Economics Education An interesting paper by Gerd Gigerenzer et al, looking at statistical illiteracy and what might be done about it. I'm struck by the emphasis on "statistical thinking" and how it is either taught too late, badly or not at all.
Rather like economic thinking, critical thinking and logical thinking.
Reader Comments (22)
I just tried the link. 'Service Unavailable'.
Ditto. Geoff.
Moi Aussi
Indeed, today we have the usual absurd medical stories: 'Children who eat mainly junk food have lower IQs'! No mention whether this is a mere statistical corollary (which it obviously must be unless biology has jumped to star trek level overnight!), no mention of sampling, no mention of how they control for other factors - what where the IQs of the parents, what where their socio-economic backgrounds etc etc. As it is we can dismiss it without further ado as one more piece of medical bunk - assuming they weren't using children as lab rats, the families that feed their children junk food from infancy, 'mainly', can't be the most intelligent of parents and, therefore, nor are their children likely to be. Nothing, therefore, to do with so called 'junk food'! How this is suited to our preconceived ideas and therefore research funding bodies is another matter.
Another absurd one - sleepwalking 'link to chromosome' - O lord, save us from idiot geneticists!- this reductionist absurdity based on the study of one family!
It's just typical of what passes for research, 90% of the time, in medical academia, amongst others, and it's abuse of statistics and then the statistical illiteracy of journalists for swallowing this bilge!
Following on from Lewis:
There used to be films set in news rooms where the star reporter would be called "Scoop". Remake those films now and he would have to be called "CTRL C, CTRL V".
There is another copy of the paper at
http://www.math.uni-konstanz.de/~dreher/gigerenzer.pdf
I don’t think there is any such thing as statistical illiteracy. It is just another invented condition – yet another name given to a statistical artefact.
The paper admits that the problem is created by nontransparent framing of information. This implies that correct, unambiguous and unbiased framing of information would cure the ‘problem’. So we are talking about misinformation with a range of causes. In other words poor or biased reporting – such as inventing misleading names for statistical artefacts.
Gigerenzer is very talented analyst and writer of lucid descriptions of statistical topics in medicine and healthcare, such as 'false positives', and 'relative risks' (his book is excellent on this, and more accessible than this paper for the lay reader: http://www.amazon.co.uk/Reckoning-Risk-Learning-Live-Uncertainty/dp/0713995122).
This paper illustrates some of the confusion in medical circles about statistical methods, and urges more work to improve on 'statistical thinking'. There is no universal definition of what this means, although he gives an outline of what he means by it in this contect. Industrial statisticians have their own versions, e.g. see http://asq.org/quality-press/display-item/index.html?item=H1060.
Very interesting monograph - and rather frightening in regard to the statistical literacy in physicians ...
Little anecdote:
ages ago I was sitting in at a medical conference, accompanying my sister. One of the doctors gave a glowing report on a treatment he employed, which, he said, had a 50% success rate.
I nearly fell off my chair when he finally came out with the number of patients in his study, n = 2.
I'm not a statistical genius, but I find that having been taught basic stats in the age before personal computers were available, where we had to use pencil and paper, and strictly no pocket calculators, is still very useful when eyeballing the stats published in climate science.
Actually, the real point is it's just about learning to reason - particularly learning how to think logically about causation. Admittedly, you must be born with some aptitude but much of it can be taught and in today's world these should be considered essential skills. But with News 24's disgraceful and typical reporting of the above mentioned story - 'Children who eat mainly junk food have lower IQs' - ridiculous anecdote, brief quote from talking head, no analysis of the paper, no context as regards this paper and the science in general, not even a brief description of its methodology and a little detail regarding it's results - and this is typical of the BBC and the MSM in general - is it any wonder this does not seem to be taken seriously?
Half the people in the West have IQs below 100. What can you expect?
(and yes - it is a statistical joke)
Some years ago I bought and read Prof. Gigerenza's 'Reckoning with Risk' book (different titles in different countries) published in 2002.
Fascinating, clear enough to understand and enlightening.
At around the same time I had been reading similar books on broadly similar matters that, whilst presented is a rather different style, carried the same message. These were written by a retired UK based professor whose career work often relied in statistics although it was not, per se, his departmental reason for existance.
The interesting thing to me was that whilst the message about understanding was so similar and the subject matters discussed - particualrly related to health - were almost identical, one example completely opposite viewpoint stood out. This, unsurprisingly, related to tobacco use withthe underlying recent message of second hand harm to non-smokers as the subtext.
It struck me that for this subject, and this subject alone in this comparison, there was a significant change of, shall we say, 'style' in the writing and application of words. My takeaway impression - a long lasting one - has been that everyone, no matter how smart and how utterly logical they may appear to be, will have some form of single issue blind spot or fixation that shifts their judgements and opinions. I'm not suggesting that such a thing is right or wrong - just that it happens. Regularly. To all of us. Belief systems can take on strange appearances.
Apart from that difference I think the two authors would have had a strong consensus.
Seeking heroes can be a rather frustrating experience in science. I've been lucky enough to know a few smart, mainly scientific, academics and have met some others. Fascinating and very intelligent people and mostly very capable. But instant opinions and judgements they put forward don't always seem to suit fit the information presented. There could, of course, be many reasons for that, including, but not limited to, poor initial communication that goes unchecked by all.
It's a great shame that Sandy Szwarc is no longer blogging on Junk Food Science. This sort of report would have been meat and drink to her (forgive the pun) and she would almost certainly have ripped it to shreds in minutes.
The trouble is that there appears to be no-one around with a vested interest in making sure that a detailed analysis of this sort of pop pseudo-science gets out into the public domain. We are left with press releases from PR departments of pharmaceutical companies and university departments with an axe to grind, a product to sell, a profile to be raised and grant money to be won.
And the message of all them to the general public is: "**** you, suckers!"
Yes, GP, one always, almost, finds that blind spot, somewhere were, either out of prejudice or, indeed, intellectual cowardice, even the best thinkers will not question! Especially in Academia. Or as Nietzsche once said, in a different context, there is always a point at which
The ass comes along
Beautiful and strong!
(Not Hengist again!)
And, indeed, so called passive smoking is one of the most notorious of them! But because it suits and suited the Zeitgeist it went and goes unquestioned. Truly a nadir in Medical Science. And, on the other side, the toleration of a whole plethora of quackery - so called 'alternative medicine'. I sometimes think the end of our 'Enlightenment', if it is ending (let us hope just in abeyance against the barbarian horde) began first with the oxymoron of the so called 'social sciences', then infected the real science of medicine and has from there begun to spread like a virulent virus throughout science in general. Let us hope the forces of rationality can rally and retrieve their rightful place and respect.
BH, you're starting to sound like me grumbling ;-)
Almost nobody realizes that statistics is a mathematical model fitted to data, which is then assigned a probability of matching the data based on assumptions about the size and shape of the population that data are from.
In other words: assumptions, presumptions and guesses.
Only the simplest of statistics, such as mean, median, and average tell you much, and as soon as you start assigning "probabilities" you enter the wonderful world of Alice in Statistical Wonderland.
I just love Factor Analysis and its closely related Principle Component Analysis. Total BS.
As for what Mann et al. did,--- well, better left unsaid.
Statistics is like sausages. We love to eat it, but few of us know what goes into it. I, for one, do not eat sausage or read statistics. Bad for your health.
There are only three types of people in the world; those who can add, and those who can't.
Just in case there is anyone out there who hasn't heard of:
John Brignell, Ph.D., a retired Professor of Industrial Instrumentation at University of Southampton
Author of 2 books on the abuse of numbers and the list of 600+ things that are caused by global warming. And a similar number of things that cause cancer. And many other useful items on his site:
http://www.numberwatch.co.uk/
Sadly, suffering long-term ill health.
Maybe the retired Professor referred to by GP, above?
And, of course, Matt Briggs, "Statistician to the stars!"
http://wmbriggs.com/blog/
Strongly second the comment on Sandy Szwarc, sadly not currently blogging on Junk Food Science. But thgere is a message Jan 2011 hoping to return The material is still all there at:
http://www.junkfoodscience.blogspot.com/
You are too hard on Statistics, Don Pablo. It is, or ought to be, on the front line when it comes to the testing of new ideas, and making sense of observations and experiments, and that front line can get very noisy at times.
Let me add to the applause for
Gigerenzer, G. (2002). Calculated risks: How to know when numbers
deceive you. New York: Simon & Schuster. (UK version: Reckoning
with risk: Learning to live with uncertainty, London: Penguin).
A tremendous book: very good indeed.