Monday
Oct072013
by 
Bishop Hill
Bishop Hill Speed reviewing
Oct 7, 2013 Climate: IPCC Climate: WG2 The IPCC report is, famously, the most-reviewed document in human history or something like that. Which is why I was so intrigued by this graph sent to me by a BH correspondent. It shows the number of review comments received on each page of the Second Order Draft of the Working Group II report, leaked to this blog just a few days ago.
The labels on the x-axis appear at the start of each chapter, and you can see that after an initial flurry of activity on the first page or so interest tails off rapidly.
;)
I think we can say then that this is the report with the most thoroughly reviewed chapter title pages in history.
Reader Comments (11)
Does anyone actually expect any of our politicians or civil servants to do more than read the summary and flick through the rest of the report? They might notice the chapter titles, which is why it was worth devoting time to them.
When Ken Clarke was the Chancellor of the Exchequer he famously joked that he had not bothered to read the Maastricht Treaty. If, in his private life, he had signed a contract which involved a significant chunk of his money, you can be certain he would have been more conscientious. At least Ken Clarke was honest. Most of our politicians would gladly sign anything or agree to anything that gets them approval from the BBC, the Guardian, and other self-appointed arbiters of what is good for us.
An excellent visualisation resulting from a fair amount of effort.
We'll done that correspondent!
There's lies, damn lies and IPPC review statistics. It looks like each chapter has a clearly defined half-life. There's always a good case for ensuring that any topic is covered in a few well-written and concise pages.
It makes sense.
Since nobody reads past the first page or two, why waste effort reviewing the later pages?
I'd assume that the real dips are in the long lists of papers at the end of each chapter, but it is hard to tell from the graphs. Is there a link to a more detailed analysis?
Those are the typical graphical profiles of a pal reviews: Climate science is infamous for it. Just a quick formality for appeerances.
Doug
I've attached the data.
For most documents I read, once I have noticed about four serious errors or glaring spelling mistakes, I stop.
Doug
I think the big dips at the end of each chapter are probably because the figures are at the end (marked in green on the spreadsheet). An individual reviewer often makes more than one comment on a page of text, making specific points about specific lines, but is more likely to only make one comment per figure.
They missed an opportunity to arrange the text so that towards the end of the report there would be an enormous increase in review comments. As it is, although there clearly are wild natural fluctuations and adaptation does appear to be unprecedented, the trend is pretty flat. But of course crisis is in the eye of the beholder, as always ...
I ran a regression.
Average number of comments per page is 21.9±0.6.
Numbers fall by 0.18±0.01 comments per page. The shortest chapter is 38 pages. The longest 151 pages.
Executive summaries receive 17.6±1.0 comments per page more.
Graphs and tables receive 6.5±0.6 comments per page less.
Later chapters receive 0.07±0.04 comments per page less.
Regional chapters receive 1.31±0.73 comments per page less.