Bishop Hill Rip it up and start again
Feb 23, 2010 Climate: CRU Climate: Surface Fox News seems to have something of an exclusive, with a story that the Met Office is proposing creation of a new verifiable set of global temperature data. Something of an admission in there, wouldn't you say.
At a meeting on Monday of about 150 climate scientists, representatives of Britain's weather office quietly proposed that the world's climatologists start all over again to produce a new trove of global temperature data that is open to public scrutiny and "rigorous" peer review.
I am almost certainly an incorrigible cynic, because I just can't get out of my head the idea that this will mean that the existing CRU data and code will remain off limits, while scientists spend a decade or more creating this new temperature set.
Bishop Hill
The text of the Met Office's proposal (or at least the executive summary thereof) is here. Interestingly it requires the data to be publicly available and the methodology to be published in the peer reviewed literature.
I guess this means that we will have access to the data but not the code. Adjustments to remain a secret then, and remember the warming is all in the adjustments (or it is in the US at least).
Reader Comments (37)
More stunning "non admissions" that the data are incapable of supporting the hypothesis.
It is highly significant that this has been proposed.
If it is fully open, I cannot see anyone having any valid reason to oppose it.
The interpretation of the results are another matter, although it will be near impossible to fudge this time.
If only Sir Muir Russell were so understanding!
Given the climate scientists' abject failure to challenge or blow the whistle on the shenanigans at CRU and NASA, they are generally the last people who should be put in charge of the project. Perhaps a group from across the spectrum (eg Julia, Prof Curry, etc) could oversee a public effort to rebuild the database of raw data, including if possible the "Zombie stations" and others ignored by GISS.
Recommend readers click on the Fox News link, it's a very well written synopsis. Nothing new to informed persons, but nicely put and a good article to share with your less informed friends.
They have already decided that they do not expect any substantial changes from the existing trends, which will become even more robust. There's nothing like pre-determining what answers you are going to get. No bias here chaps.
The key here is that the Met office want the business before it is taken away from them... this a pre-emptive strike to retain their control.
The point should also be made the Met Office should not be the Controller of this data, just a user. A user that would have to compete on the new worldwide internet based marketplace of climate studies.
There must be a SEPARATE "OPEN SOURCE" FOUNDATION with a "board" accountable to the people who want to use the data.
Since it was the Met Office's lack of oversight and incompetence that resulted in the existing datasets being flawed, any new dataset should be completely removed from the hands of the Met Office and CRU. It should be put out to competitive tender. It is not a job that needs Met Office or UEA staff to do at great expense. A few competent statisticians and software developers could do the job.
I forgot to add - why should the taxpayers pay the Met Office and CRU again because of their incompetence? The Met Office and CRU should have their funding reduced to pay for the new database.
How are temperature data going to answer the question "what adaptation and mitigation decisions should be taken"? These are social and economic questions. Aren't the Met Office getting above themselves again?
Jiminy: I think we are saying the same thing here.
If you'd been sold a "new" car, and found out that it was made of bodged together write offs, and was totally unfit to use...
would you buy the replacement from the same people?
The Met Office needs disbanding
This is a step in the right direction. We can write our own code. Somebody should start an open-source climate site/warehouse.
All the infrastructure you need including code versioning control, documentation, distribution, wiki's etc is available free of charge at both sourceforge.net and freshmeat.net as long as it is released under an open source licence. sourceforge is the better of the two currently.
Not bundling the code with the database makes sense to me. Keep the data pristine without a hint of taint. Lots of people are going to have different theories on how that data should be cleaned, manipulated and displayed and that should happen through publishing. They can provide a link to where to get their code in the article. After some one publishes with out their code and gets a beat down others will fall into line.
I notice that no-one is exactly highlighting that they are announcing this in Turkey?
More worldwide junkets for the chosen ones.
> We can write our own code.
And this is what needs to be done urgently. Start the project now and release it under an open source license. The code doesn't need to incorporate the adjustments and assumptions, but could include the ability to see the results based on loading different sets adjustments and assumptions.
This will clearly allow us to see the effects of different modelling techniques, and see which results held under which assumptions.
Surely with a genuine interest in science, whether currently holding warmist, luke-warmist or skeptical views, would oppose such a project?
"it requires the ... methodology to be published in the peer reviewed literature": that rather assumes that pal-review will wave it through, doesn't it? If it were to be subjected to a proper critical scrutiny, one couldn't assume that it would pass muster.
...and of course it will all cost a lot of money, which the Met Office and CRU will happily take off our hands.
So the Met Office implies that existing datasets are not "fit for purpose" for the 21stC per the linked pdf. In itself this is something of an admission and could be significant if repeated by the right person at the right time and venue. Cue Lord Lawson next Monday? I suggest that our host points this out to Lord Lawson - if he has not already done so. It could make an interesting session and put those that follow on the spot.
On the CET web page: http://hadobs.metoffice.com/hadcet/ there is a References panel and if you download Parker and Horton 2005, then check the table on page 2 it can be seen that between 1974 and October 2004 they used to add the Blackpool Airport temperature and the Manchester Ringway Airport temerature together and then divide that figure by two! This average (?) was then combined with temps from Rothamstead and Malvern to compile the Monthly CETmean series.
Is that OK taking two airports and averaging them?
Bishop Hill
'Rip it up and start again'
This is not a new story: As reported by the Daily Telegraph
below, the Met Office first canvassed an international
project of preparing a 'new verifiable set of global temperature
data', as you describe it, on 5 December 2009.
However per the Times 5 December 2009:
http://www.timesonline.co.uk/tol/news/article6945445.ece
'The Government is attempting to stop the Met Office from
carrying out the re-examination [ie the above project], arguing
that it would be seized upon by climate change sceptics.'
And duly, per the BBC later on 5 December 2009:
http://www.wsmweather.co.uk/?p=3102
'An MO spokesman denied it would spend up to three years
re-examining the climate change data, and said it had already
planned to publish the material long before the "Climategate"
controversy broke.'
However the Copenhagen Conference is over, so that the
Government need no longer suppress news of the existence
of the project.
So we learn that the denial by the Met Office spokesman on
5 December 2009 was a false denial that a compliant Met Office
misleadingly issued at the behest of an embarrassed British
Government.
The Government intended thereby to 'quieten' excitable
'sceptics'. And no doubt even more so it intended to quieten
impatient--'The planet cannot afford that we wait a further
three years for credible evidence'--'believers'.
But now the Government's tomfoolery has successfully served
its purpose, so that in February 2010 we can all, sceptics
or believers, who call ourselves first and foremost scientists,
'move on'.
Stephen Prower
Stevenage
Tuesday 23 February 2010
----
Document location:
http://www.telegraph.co.uk/earth/environment/climatechange/6732011/Scientists-may-re-examine-temperature-data-to-prove-climate-change.html
Document title: Scientists may re-examine temperature data
to prove climate change - Telegraph
'Scientists may re-examine temperature data to prove climate change
Climate scientists may re-examine 160 years of temperature data
after admitting that public confidence in the reality of global
warming has been undermined by 'climategate'.
Published: 8:00AM GMT 05 Dec 2009
A number of climate models, based on information from weather
stations around the globe, show the world has been warming
gradually since the 1850s.
But the figures have been called into question following the
‘climategate’ affair at the University of East Anglia.
Sceptics alleged that emails stolen from the Climatic Research
Unit at the university show scientists were willing to
manipulate data to show global warming.
They also complain that the raw data for the climate models was
not made available to the public.
To try to restore public confidence the Met Office is talking to
other meteorological organisations around the world about
recreating the model using the same raw data but more modern
computers.
The whole process will also use any new information and be more
open to the public.
However, it could take up to three years for the study to
complete, meaning the scientific world would have to wait until
after 2012 to provide updated proof of the extent of global
warming.
...'
[END]
A data entry firm would take the daily temperature records and have two people independently type them in. Then quality control procedures would be instigated starting with a comparison of the two datasets. Differences would be resolved by returning to the original hand written data to see which typed version is correct.
I doubt if the Met Office is going to do the above. As it stands now, the CRU data has lots of typos such as 7 typed instead of 2, etc. The Met Office won't catch them.
They also need to photograph all the stations and get as much historical information about their sitings as possible. Otherwise, who knows what the temperature readings mean?
Even Roy Spencer has got the same trends as CRU when working with the raw data:
Yet more independent validation of the temperature trend based on raw data, from Tamino here
Golly gee, seems like you can do this work without any vexatious FOI requests at all!
Of course since the results all corroborate each other, all this is merely good science and not good PR, such as you can get when you ask the wrong people for data you can easily get elsewhere and then lie about the implications.
Nature just isn't playing ball with you guys, is it? Damn that physical reality and its liberal bias!
Your update post showing no actual warming in the US pre-adjustments sort of made me sit up. It is the same over here (NZ). All of the warming is caused by adjustments (temperatures are the same as they were over 100 years ago once you remove the adjustments.
I'm not saying that the adjustments aren't there for good reasons but it is interesting that nobody can get hold of a schedule of adjustments to check them - it has been asked for in parliament and still nothing.
Any idea of how many other countries only have "man-made" warming through adjustments?
Frank,
so why's it so bloody cold here, together with cold in the US and China (minus 40c in Daqing the other day lowest ever recorded there) at the moment?
oh, sorry, that'd be the none significant warming trend would it?
and if it was unseasonably warm? what would be causing that? same mechanism by any chance?
With Al Gore's wonderful invention, if i want, I can get daily updates from trusted friends on the ground pretty much accross the world, true they aren't formal weather stations but -40c is still the record for 40 years or so that town has been there.
Bull about other areas being warmer doesn't wash any more, the whole hemisphere is cold, and in the year our wonderful Met office told us was going to be a record mild one.
sack the whole lot of them, it would be no loss.
Keith,
Perhaps reviewing this material may help you.
Meanwhile UAH recently posted the hottest January in the satellite record, and the earth's systems have been warming for decades.
Yes, it is Winter, and it is a colder winter than for many years, possibly a couple of decades, and the exceptional cold for this time of year appears to be pretty anomalous accross the whole northern hemisphere.
The satellite showing warming, yes, but where is the warmth? it isn't at ground level. and I'm sure you recall the email about the "travesty" of not being able to show the warming?
"Earth systems warming for decades", with un documented adjustments yes, without those adjustments and without urban heat islands, the warming trend is pretty tenuous from the mid 90s onwards, if it exists at all.
Keith,
"Yes, it is Winter, and it is a colder winter than for many years, possibly a couple of decades, and the exceptional cold for this time of year appears to be pretty anomalous accross the whole northern hemisphere"
Most of it anyway (not Alaska for example, and not the Arctic).
"The satellite showing warming, yes, but where is the warmth? it isn't at ground level. "
Yes it is - there is anomalous warming elsewhere in the globe, it's not called local warming for a reason. Most of the heat is in the oceans, which is the same place most of the heat has been going all this time.
"I'm sure you recall the email about the "travesty" of not being able to show the warming?"
I recall 'sceptics' misrepresenting that email, yes.
""Earth systems warming for decades", with un documented adjustments yes"
Are you accusing Roy Spencer of something here, or just the scientists that aren't 'sceptics'?
"Without those adjustments and without urban heat islands,"
No UHI in the satellite readings. No UHI in the oceans. It isn't UHI that's melting the ice and making species migrate to cooler climes.
And I already showed links that show the warming is there adjustments or not. It is certainly rather unreasonable to complain about UHI effect and also complain about adjustments to remove it (which would reduce the warming trend seen).
Frank,
wasn't the whole theory of AGW based on the idea of increased CO2 in the atmosphere preventing long wave IR radiating back into space? yet you are saying that the supposed heating is occuring in the oceans?
Now, would that be satellite recorded heating? or is it inherited from coastal stations due to no cell values in the ocean in a computer model?
We really can't be definite on any of the analyses of instrumental temperatures until we have the station's raw data, situation and surroundings topographic setting and any station moves, increases in heat sources etc all out into the open.
We've seen what Anthony Watts' sample audits of the US stations have found, we've seen a few samples of station adjustment that supposedly referred to adjacent stations, yet the aggressive warming trends which had been applied to all the adjacent stations were not present in the raw data, and we have seen how a single adjusted terrestrial station's data is inherited by cells for thousands of kilometers out to sea.
As the good Bishop's post title suggests, a fresh start is needed,
I do not think that the Met Office have shown much aptitude for reliably carrying out that task
Is the data that's being made available raw or "adjusted"? If it's not raw I can't see any value in it whatsoever - other than for hiding the adjustments of course.
Possibly a naive (but genuine) question: if CO2 blocks IR, what temperatures are satellites measuring?
I'm also pleased to see that all the NOAA/GISS/CRU instrumented temperature data auditing efforts of Steve Mc, Anthony Watts, ChiefIO, Jeff ID, hpx83, Verity Jones, NicL, Lucia, Tamino, Chad, Nick Stokes, Giorgio Gilestri to name only a few have finally paid off.
In particular I hope that once the global raw instrumented temperature dataset has been re-established, that full acknowledgement and appreciation is given to the significant warming and cooling trends of the 19th and 20th century.
I also hope that the current obsession with 'anomalising' and 'gridding' the data is dropped as IMO it is not necessary to anomlise and grid the raw data to see the clear cyclic warming and cooling trends of the past two centuries.
I also hope that something is done about the post 1990 'station drop out' issue i.e. that the data is brought up to date, and that the 'missing months issue' is also dealt with in a much more appropriate manner. I also hope that all available temperature data for the rest of the world is found and added to the dataset not just data post 1950 as at present in GHCN. I don't believe for one minute that the sudden increasing in reporting station post 1950 was due to the construction of airports world wide due to expansion in the aviation industry.
It's only after the latter three issues highlighted above are dealt with properly that IMO we will then be able to properly assess whether or not the late (1970 to 2000) 20th century warming trend was any more significant than the 1910 to 1940 warming trend.
In doing so (i.e. ensuring that we can can compare the two warming periods correctly) due allowance must be made for the effects of station moves, instrument changes, land use changes and urbanisation (UHI effect). It is not IMO sufficient to apply an 'algorithm' (as done at present by NOAA/GISS/CRU) to attempt to allow for these changes over time (i.e. to 'homogenise' the data), but rather it is vital that sufficient meta-data be collected for each and every individual station (as done in the surfacestations.org project) which can then be used make due allowances the above highlighted issues on an INDIVIDUAL station basis.
As a UK taxpayer I'll be happy to fund and even participate in this process free of charge, provided it adheres to the conditions I've expressed above. Regardless of the costs of this project, it will be small beer in comparison to the costs we are currently spending on funding organisations like the Tyndall Centre to look at masures to mitigate and adapt to supposed man-caused climate change when in fact at this point we are far from certain as to whether or not man has actually caused the problem and for that matter whether it is even a problem at all.
'Bish'. I disagree with you. The 'warming is not all in the 'adjustments'. It's real and it's in the raw (unadjusted, non-homogenised data). Th edoubt in my mind is as to whether or not the 1970 to 2000 warming trend is any more significant than the 1910 to 1940 warming trend. Due to lack of global station coverage from 1910 to 1940 compare dwith 1970 to 2010, IMO the jury is still out as to whether or not the late 20th century warming trend is 'unprecendented' in teh last 100 years let alone 1000 years as Michael Mann would have us all believe. If the 1970 to 2000 warming trend is not statistical significantly different to that for the 1910 to 1940 period then there really isn't any need to invoke CO2 to explain any differences is there? It just yet another warming trend that is part of a multidecal cyclic trend of warming (1910 to 1940 followed by cooling (1940 to 1970), followed by warming (1970 to 2000), possibly followed by cooling (2000 to present) again.
Questions need to be asked of the Met Office, and of Government.
What is wrong with the current trove of global temperature data? You know, those that the whole AGW bandwagon rests on.
Has the mountain of funding for the current "trove" been wasted, and if not, wouldn't it be much more cost effective to publish the current global temperature data so that it is open to public scrutiny and "rigorous" peer review?
Like what we've been asking for years.
Is there not some public oversight committee prevents repetition of research to prevent frivolous spending of public funds?
No doubt this will swiftly be swept under the carpet.
"part of a multidecal cyclic trend of warming (1910 to 1940 followed by cooling (1940 to 1970), followed by warming (1970 to 2000), possibly followed by cooling (2000 to present) again"
Followed by warming (eventually), don't forget... :-)
"Is there not some public oversight committee prevents repetition of research to prevent frivolous spending of public funds?"
Perhaps we should be asking these guys:
http://www.nao.org.uk/
Keith,
I am saying that the heat accumulates in the earth's systems, which includes the air, land and oceans.
Are you just disputing every claim for the sake of it? No matter what causes the earth to heat, a great amount of that heat would wind up in the oceans.
It has found jack of any importance to the trend.
James P,
I don't pretend to understand the details, however as I understand it they don't measure temperature directly, but radiation at various wavelengths, and temperature is inferred from that. Not just IR but microwave for example. Also CO2 only blocks IR in some bands.
Frank,
Go back and check Watts' station samples.
Formerly green field stations surrounded by tarmac, concrete and buildings- em, do you think that might just add a little bit of warmth? (hint, what would you rather be sat in on a sunny day, a tarmac car park or a grass field?)
hot air vents from AC systems sited right by the Stevenson screen, new boiler houses, might that thermometer be recording slightly warmer temperatures now than previously?
We have seen the reduction in the numbers of stations going into the models, so, such changes now get applied over a larger area.
The station count in the US is now at lower mean altitude ( 1 degree C per 500 feet average change due to altitude) and at lower latitude.
Those changes add up to a trend that cannot be dismissed as:
"It has found jack of any importance to the trend."
Just on the CO2 and its absorbtion lines
I don't think anyone has ever done a spectral energy balance study,
but,
as we are still way short of the temperatures seen for prolonged periods in previous interglacials in Britain (we certainly aren't getting rubification of clay soils, and don't even have the climate to grow grapes that was here during the MWP), and depending on what the instrumental record actually shows sans arbitary aggressive warming adjustments, we may not even have exceeded the early 20th century warming cycle.
- So it is hardly an urgent priority to do such a study.
You have to wonder how much of an effect the tiny proportion of CO2 can have. 385ppm sounds an impressive figure, but everyone forgets (or fails to note) that the 'm' is for million, so the concentration is less than 1:2500 or 0.04%.
The oceans contain far more than the atmosphere, and absorb or release it largely according to temperature, so it seems rather more likely that any cause and effect is the other way round.