Buy

Books
Click images for more details

Twitter
Support

 

Recent comments
Recent posts
Currently discussing
Links

A few sites I've stumbled across recently....

Powered by Squarespace
« A new edition of the Hockey Stick Illusion | Main | DECC consistently misled public over electricity costs »
Wednesday
Nov182015

A $100,000 climate prize

Climatologists often claim that they are able to detect the global warming signal in the temperature records. If they are right then they are going to be having a very happy Christmas indeed, because Doug Keenan is offering them the chance to win a very large cash prize at his expense. Here are the details.

There have been many claims of observational evidence for global-warming alarmism. I have argued that all such claims rely on invalid statistical analyses. Some people, though, have asserted that the analyses are valid. Those people assert, in particular, that they can determine, via statistical analysis, whether global temperatures are increasing more that would be reasonably expected by random natural variation. Those people do not present any counter to my argument, but they make their assertions anyway.

In response to that, I am sponsoring a contest: the prize is $100 000. In essence, the prize will be awared to anyone who can demonstrate, via statistical analysis, that the increase in global temperatures is probably not due to random natural variation.


The file Series1000.txt contains 1000 time series. Each series has length 135 (about the same as that of the most commonly studied series of global temperatures). The series were generated via trendless statistical models fit for global temperatures. Some series then had a trend added to them. Each trend averaged 1°C/century—which is greater than the trend claimed for global temperatures. Some trends were positive; others were negative.

A prize of $100 000 (one hundred thousand U.S. dollars) will be awarded to the first person, or group of people, who correctly identifies at least 900 series: i.e. which series were generated by a trendless process and which were generated by a trending process.

Each entry in the contest must be accompanied by a payment of $10; this is being done to inhibit non-serious entries. The contest closes at the end of 30 November 2016.

The file Answers1000.txt identifies which series were generated by a trendless process and which by a trending process. The file is encrypted. The encryption key and method will be made available when someone submits a prize-winning answer or, if no prize-winning answers are submitted, when the contest closes.

More here.

PrintView Printer Friendly Version

Reader Comments (129)

Typical Richard, defending denialist nonsense.

Nov 19, 2015 at 7:48 AM | Unregistered Commenter...and Then There's Physics

If people really want to know why it's not a random walk, they could read Tamino's post.

Nov 19, 2015 at 7:49 AM | Unregistered Commenter...and Then There's Physics

@Green Sand back in 2011

One of the troubles with "climate science" is that it is in its adolescence: It thinks it knows everything. Maybe it will grow up some day.

Dec 2, 2011 at 12:46 PM | Jimmy Haigh

Nov 19, 2015 at 9:01 AM | Registered Commenterstewgreen

For "There's no physics", let me explain how this works. There is random data, to which has been randomly added a trend larger than added to the climate, which is either positive, negative or none. You and your stupid buddies claim you can detect such trends with 95% confidence. Mr Keenan has called your bluff and asked you to prove this (false) assertion.

The reason he is confident you cannot is because unlike you, he understands the statistics involved. And unlike the climate data it isn't possible to fudge this test by inventing fraudulent new datasets to pass the test.

Nov 19, 2015 at 9:02 AM | Registered CommenterMikeHaseler


You and your stupid buddies claim you can detect such trends with 95% confidence.

No, we don't, which is why one description for this challenge is "savaging a strawman", one of Doug Keenan's specialities - well, that, and making unfounded accusations of fraud.

Nov 19, 2015 at 9:15 AM | Unregistered Commenter...and Then There's Physics

I am mightily impressed with Tamino: On 11 March 2010 he cracked a challenge that was posted on 18 November 2015!

I do hope that our dear friend Ken Rice is not involved in the teaching of causality to the students at U Edinburgh.

Nov 19, 2015 at 9:24 AM | Unregistered CommenterRichard Tol

Is it really that difficult to understand that you should only apply a method once it has been properly validated by benchmark testing? Applying a method to the actual temperature series before proving it is the issue that the test reveals. And adding seasonal data doesn't help at all; pretending so undermines anything else you might say about stats.

Sure you need a tested physical model of natural variation to determine any manmade trend - as the Met Office finally admitted. Everyone should know that! That fact doesn't seem to prevent conclusions about manmade warming from statistical analyses being published every year. It is upon such unscientific dross that the consensus is maintained.

As for the temperature series being indistinguishable from a random walk - that has long been established in the actual literature by people who know a lot more than pause-denier Grant Foster about both meteorology and stats.

The big problem with statistics is that it is too easy to introduce bias. You can save some time by ignoring any paper than mentions statistical significance as arrant tosh. Applying frequentist methods to non-random models is ridiculously common in climate spheres but nowhere else. Arguing moreover that wider error bars somehow make a model better should be punishable by defenestration.

Nov 19, 2015 at 9:28 AM | Unregistered CommenterJamesG

Richard,


I am mightily impressed with Tamino: On 11 March 2010 he cracked a challenge that was posted on 18 November 2015!

I do hope that our dear friend Ken Rice is not involved in the teaching of causality to the students at U Edinburgh.


Oh dear, Richard, are you having trouble with basic reading comprehension again. I'd like my null hypothesis to be that you don't make elementary errors, but the evidence is against you. Tamino's post referred to earlier claims of it being a random walk, not to this one (as should be obvious). I posted the link so that people could understand why it's not a random walk, as I'm pretty sure you realise yourself - even though admitting it would probably be frowned upon by your GWPF associates. The debunking in 2010 applies as much today as it did then; the last 5 years isn't going to change our understanding of whether or not it could simply be random natural variations. This random walk idea is simply a rehash of ideas that have been debunked time and time again. That Doug Keenan does not seem to appreciate this is no great surprise, but that doesn't mean that those who promote this are not illustrating their extreme ignorance of this topic. Of course, that you would associate positively with climate science denial is no great surprise. That AM would promote this is also no great surprise (and he wonders why people regard this as a science denial site?). I'll refrain from commenting on what this might imply with respect to what we might hope you don't teach at the University of Sussex.

Nov 19, 2015 at 9:39 AM | Unregistered Commenter...and Then There's Physics

Proving the naivety of bad statistics by publishing the encrypted answers in this way holds no small amount of irony. I could give you a 'key and encryption method' that would produce the opening lines of the bible from that encrypted text, just as I could give you another 'key and encryption method' to produce the opening lyrics of 'Bohemian Rhapsody' from that encrypted text. In short a key and encryption method could be produced at any time in the future to 'demonstrate' any given permutation of answers to the given challenge.

Nov 19, 2015 at 9:53 AM | Unregistered Commenterbobbyboy

Could everyone please take a deep breath and try to keep the comments calm and civil.

Nov 19, 2015 at 10:12 AM | Registered CommenterBishop Hill

@ bobbyboy

In principle, you are correct. In practice, that would be extremely difficult to achieve. In any case, the encrypted file, Answers1000.txt, is intended only to be a check. The computer program (including random seed) that was used to generate the 1000 series will also be made available when the contest closes.

The relevant paragraph has been revised to make things clearer. Kind thanks for bringing this up.

Nov 19, 2015 at 10:34 AM | Unregistered CommenterDouglas J. Keenan

About submitting a contest entry, this should be done by sending me an e-mail. The contest web page has been revised to state that.

Nov 19, 2015 at 10:41 AM | Unregistered CommenterDouglas J. Keenan

@brandon

I'm pretty sure that's not true, but I suppose it's also not really important for this challenge. This challenge can stand on its own merits. I do have to ask though, do people attempting this challenge get to have any idea of approximately how many series fall in each category? Normally a person might expect the distribution to be about 50/50, but the challenge doesn't actually say.

It would be really cheesy if it turns out there was only one series created with a trending process, or some other crazy answer like that.

It doesn't take much thinking to realize that while we can't know the split it's not going to be a cheesy/crazy 1 either way. It takes $20 to submit 2 answers, all have a trend added and none have a trend added. If the split is over 90% either way you instantly win US$100k and I don't expect Douglas is that silly.

So no it's not 1 series and has to be at least 90/10, I'd assume at least 80/20 to allow a margin for some random guessing.

Nov 19, 2015 at 11:03 AM | Unregistered Commenterredc

redc:

It doesn't take much thinking to realize that while we can't know the split it's not going to be a cheesy/crazy 1 either way. It takes $20 to submit 2 answers, all have a trend added and none have a trend added. If the split is over 90% either way you instantly win US$100k and I don't expect Douglas is that silly.

So no it's not 1 series and has to be at least 90/10, I'd assume at least 80/20 to allow a margin for some random guessing.

That is a good point. Still, it'd be nice to know if I sort the series by their likelihood of having had a trend added to them, what portion I should pick. Even if I could get the sorting completely correct, I could get my answer wrong simply by thinking the distribution is 50/50 when it's really 70/30, and that seems unfortunate. But I guess that might be something he wants as part of the challenge?

I don't know. I doubt I'll submit an answer. If I knew the distribution was 50/50 I'd be confident I could get at least 750 right, but getting 90% right without knowing the distribution seems beyond the amount of effort I'd be willing to put into it.

Nov 19, 2015 at 12:02 PM | Registered CommenterBrandon Shollenberger

I would have put $100,000 on ThenThere's to be the first to attempt a debunk.
Go to it Physics. Go get loaded!

Nov 18, 2015 at 4:26 PM | Geckko
====================================================================
Nice job tho' - paid to pester people on anti-social media :-)

Nov 19, 2015 at 2:18 PM | Unregistered CommenterJeremy Poynton

The problem has always been that the latter-day Lysenkoists are trying to tease a very small putative signal out of extremely noisy data. Their methods, to date, have not been limited to ever more bizarre statistical contortions based on less and less reliable proxies, but have necessarily included blatantly outrageous "tricks."

Nov 19, 2015 at 5:27 AM | jorgekafkazar
=================================================================================
Yes, "tricks" such as - if the data doesn't fit the message, change the data.

Nov 19, 2015 at 3:01 PM | Unregistered CommenterJeremy Poynton

"Nov 19, 2015 at 7:49 AM | Unregistered Commenter...and Then There's Physics "

"If people really want to know why it's not a random walk, they could read Tamino's post."

As usual, Tamino goes just far enough to support his pet hypothesis, and confirm his bias. What he actually showed is that the test for significance very much depends on what model you choose for the data. And, without a reliable guide for selecting your model, you are just guessing.

Nov 19, 2015 at 6:04 PM | Unregistered CommenterBart

" Each individual trend averaged 1°C/century" Anybody care to enlighten me on what it means to say the trend of a single time series has an average? I'm still thinking a trend is the slope of a linear regression - a single number. I'm even more confused that this trend can be negative and still have a positive 'average'.

Nov 19, 2015 at 9:17 PM | Unregistered CommenterRich

Trends can be deterministic or stochastic (this is standard in time series analysis). I agree that the issue of averages should have been worded better. Kind thanks for pointing that out. The paragraph has been revised.

Nov 19, 2015 at 10:01 PM | Unregistered CommenterDouglas J. Keenan

@doug here is a short Python script which will print out the first few lines of Genesis Chapter 1, generated by the provided key from your answer text. Just as easy in practice as in principal. If you would like the Bohemian Rhapsody version too just let me know.

answer='3OT0JEN95MslFL7iZvt99rG8s9uaAUP/q9e87KDdjBb9ja5WgJm3cF0GNedfd9XhAXUz4SZEzqJSIXrQrMu6JL2+GXMzkYccPLAF3ghLEtqDMQSXnz1eCU1VhIk6g8dx891u87HBFNobq02IKBdijfL79FC92f0qtFwO5FlJMq7Eidrz7W/zPY68j/sqIv3U4LsMllnUuQooIlZmZH2Wt4s4V2Js3AAl449xrrUkFfvZD5tGUCvmf99blvHQBW/204jDV/UUiMLOcEtkQjhx1AOiQb12zeDLtanxYC78hAkqItDpnidEa+4P3kTzPrPzQi+ihTuuNqSE35eRZ65VamN+husVQ7eRwhHoi+Qi+SZlJBELzj8tvCstPJo46IZZXcAggAFhvgL+VntG2Xl/TOFpPmjoLUG6xCjSq81RlxmLb77Vjb87FG9QJZOZpvxKXt6srDToH9iCCKQ0zhzWcwsCHJC4dUJ9gj+o5uDweG8Crv/U8Fwf'

key = '\x7a\x21\x74\x44\x22\x20\x6e\x5b\x50\x2a\x1a\x02\x28\x25\x59\x0e\x7a\x31\x1b\x5d\x19\x11\x35\x5d\x12\x4d\x10\x05\x61\x21\x38\x4a\x51\x51\x00\x59\x41\x2e\x2a\x17\x4a\x23\x0c\x5d\x4a\x15\x5d\x32\x47\x2f\x0c\x41\x17\x2e\x1e\x67\x00\x0a\x13\x46\x10\x51\x3d\x48\x24\x39\x27\x0e\x5c\x73\x2d\x24\x09\x51\x2c\x3c\x3b\x35\x1e\x34\x01\x3e\x55\x57\x24\x28\x12\x4e\x2a\x28\x39\x03\x47\x79\x07\x02\x22\x27\x2f\x23\x40\x14\x48\x3b\x24\x07\x51\x2b\x3b\x34\x21\x78\x1a\x12\x54\x45\x30\x20\x43\x30\x09\x2a\x0e\x16\x08\x5e\x44\x0c\x50\x5c\x11\x11\x5d\x52\x38\x6e\x66\x2f\x01\x06\x51\x44\x5a\x2c\x6b\x11\x14\x00\x18\x0f\x38\x17\x56\x20\x63\x7e\x5d\x02\x10\x06\x15\x35\x57\x27\x5a\x30\x09\x38\x24\x1f\x50\x65\x06\x12\x17\x08\x17\x23\x47\x1f\x70\x2e\x57\x4c\x0f\x5d\x00\x5f\x69\x37\x5d\x31\x14\x0b\x1c\x29\x4c\x1f\x0f\x3c\x11\x7d\x4f\x4d\x05\x09\x2e\x4d\x2e\x20\x57\x25\x11\x14\x11\x51\x76\x5e\x23\x14\x5b\x35\x6d\x4e\x14\x55\x57\x1c\x52\x06\x3d\x0e\x34\x03\x56\x2d\x25\x46\x54\x2b\x3c\x24\x1e\x19\x48\x19\x7e\x0d\x08\x56\x3b\x30\x35\x77\x5b\x5a\x51\x40\x4a\x30\x3e\x4a\x75\x39\x00\x2a\x24\x3b\x43\x32\x15\x18\x71\x0d\x07\x17\x55\x6d\x6f\x08\x3f\x06\x11\x5a\x1f\x45\x37\x29\x04\x00\x1c\x19\x2d\x26\x53\x18\x1c\x29\x0e\x51\x25\x1d\x23\x18\x1a\x49\x02\x37\x0e\x46\x14\x24\x5b\x0e\x74\x1e\x31\x00\x3b\x14\x34\x1a\x58\x47\x48\x13\x1a\x11\x6e\x12\x32\x29\x5f\x50\x01\x72\x2e\x5e\x50\x76\x0d\x04\x29\x43\x1c\x55\x51\x32\x30\x4e\x49\x70\x57\x09\x26\x0b\x49\x5f\x39\x0c\x0b\x37\x3b\x1e\x21\x2c\x20\x3f\x09\x4a\x50\x11\x56\x20\x12\x18\x3c\x2f\x0b\x14\x14\x27\x33\x3d\x30\x17\x6f\x45\x47\x00\x28\x0c\x56\x13\x24\x4e\x24\x0b\x54\x30\x53\x2b\x4c\x4a\x22\x2a\x28\x19\x3e\x0a\x46\x4f\x2d\x3b\x23\x16\x0c\x2b\x0f\x21\x14\x18\x46\x33\x1f\x58\x00\x23\x10\x59\x5e\x38\x0d\x42\x15\x17\x32\x2f\x5c\x71\x2c\x33\x3d\x29\x04\x56\x1c\x2a\x21\x5a\x16\x32\x1c\x20\x74\x28\x27\x5d\x49\x30\x22\x22\x35\x1c\x5a\x4a\x36\x32\x17\x57\x07\x2b\x2d\x38\x26\x14\x06\x30\x6a\x58\x47\x1c\x4a\x1a\x59\x01\x64\x15\x00\x33\x4f\x26\x17\x18\x0f\x21\x50\x23\x57\x11'

for i in range(0, len(answer)):
print chr(ord(answer[i]) ^ ord(key[i])),

Nov 20, 2015 at 11:35 AM | Unregistered Commenterbobbyboy

@ bobbyboy— Then it is obvious that the thing was rigged. Doing it in a way that did not seem rigged would be extremely difficult. (In any case, the Answers.txt file is only there as a redundancy check for the program output.)

Nov 20, 2015 at 2:23 PM | Unregistered CommenterDouglas J. Keenan

I've looked at the distribution of the 'standard deviation' of the 'auto-correlation' relative to a 1 degree/century linear trend. There is no clear cut-off between those series with a trend and those series with zero trend. If you knew a priori the number of series containing a trend then you can probably solve it. Otherwise it is impossible to distinguish between all of them. I think this means that stochasticstic noise has been generated across a wide frequency range up and including the full 135 year period.

see: http://clivebest.com/blog/wp-content/uploads/2015/11/Doug-stat.png

Doug's money is safe!

Nov 20, 2015 at 2:30 PM | Unregistered CommenterClive Best

"And why am I being included in that list? What have I done to be grouped with Anders and Russell Seitz?"

Brandon Schollenberger's point is well taken- a Science Direct search indicates he's never published anything.

Nov 20, 2015 at 5:02 PM | Unregistered CommenterRussell

Brandon Schollenberger's point is well taken- a Science Direct search indicates he's never published anything.

Ah, yes. We can always count on Dr Russell Seitz of Harvard University to class up the place with a completely irrelevant non sequitur.

Nov 20, 2015 at 5:24 PM | Unregistered Commentertarran

@doug Thanks for the condescension. I hope your Statistics is better than your Cryptography. The irony is killing me.

Nov 20, 2015 at 6:04 PM | Unregistered Commenterbobbyboy

@bobbyboy Doug is correct on this, if he produced a key and encryption method akin to what you posted then the contest would indeed be obviously rigged.

Nov 20, 2015 at 6:32 PM | Unregistered CommenterPaul

Doug: Andorra is in France
Bobby: Andorra is not in France it borders France
Doug: in principal you are right Bobby but very hard to show in practice
Bobby: <shows map>
Doug: Andorra is nearly in France
Bobby: I hope your Stats is better than your Geography
Paul: Doug's right Bobby, Andorra is nearly in France

Nov 20, 2015 at 6:55 PM | Unregistered Commenterbobbyboy
Nov 20, 2015 at 7:25 PM | Unregistered Commentermichael hart

Richard Tol, I love the way Tamino ends his analysis:

One final note: there’s an ever-growing number of “throw some complicated-looking math at the wall and see what sticks” attempts to refute global warming. It seems to me that a disproportionate fraction of them come from economists. Perhaps that’s because they fear the loss of corporate profit more than they fear danger to the health and welfare of humanity. Or perhaps it’s just a reflection of the rather poor track record of economists in general. When it comes to predicting the future, it’s well to compare the truly astounding successes of, say, physics, to, say, economics.

Nov 20, 2015 at 9:29 PM | Unregistered CommenterRaff

@bobbyboy Nevermind. Why not put some energy into solving Doug's puzzle? It will be interesting to see your analyses. I'm pretty sure that putting in the information you want to extract won't help. There's probably an wider analogy there somewhere...

Nov 20, 2015 at 11:13 PM | Unregistered CommenterPaul

Unfortunately, I can't see what Doug Keenan's wager has to do with the question of global warming. I see three problems:

1) Everyone knows that monthly temperature data is available and can be used to determine whether a statistically significant warming trend exists. For example, a time series nearly 2000 data points long can be obtained from CRUTEMP4 here. I don't see why a statistician would prefer the annual temperature averages shown simplified graphs in IPCC reports. (The monthly data is noisier and shows significant auto-correlation, so it may not contain much more useful information, but discarding data doesn't make sense to me.)

http://www.cru.uea.ac.uk/cru/data/temperature/CRUTEM4-gl.dat

2) If your time series contain noise that can divergent from a long-term average of zero, then - as best I can tell - that time series isn't a reasonable model for the GMST. For noise that behaves like a random walk, the magnitude of its divergence from zero grows proportionally to the SQRT of the number of steps. GMST has existed for roughly 1,000,000 times longer than the instrumental temperature record and the divergence in a random walk would be 1,000 times bigger. If you had created model time series 135 million records long, rather than just 135 records long, even a statistical ignoramus like myself could tell which are possible models for the earth (without forcing) and which were not. Could Doug have made this wager if noise he used was not allowed to diverge indefinitely from zero.

3) It is obvious that GMST in the most recent decade is warmer than in a decade a century before. There is a perfectly good formula for finding the confidence interval for the difference between two mean. This creates the dilemma that it is "warmer" than it used to be, but we can't prove that it has been "warming". Perhaps it can be warmer without requiring a cause, whereas "statistically significant warming" requires a cause.

I'm seriously (not trollishly) interested in better understanding the answers to these issues.

Nov 21, 2015 at 6:21 PM | Unregistered CommenterFrank

"...but discarding data doesn't make sense to me."

Those data are not useful for diagnosing long term trends. They are high frequency noise.

"GMST has existed for roughly 1,000,000 times longer than the instrumental temperature record and the divergence in a random walk would be 1,000 times bigger."

A very long term view may indeed reveal limiting factors. But, over a relative short time span, a less complicated model is sufficient. For example, an exponentially correlated series with a correlation time of 10000 years, viewed over a 100 year interval, is indistinguishable from a random walk over that interval.

"There is a perfectly good formula for finding the confidence interval for the difference between two mean."

Student's t-distribution is only applicable for independent Gaussian variates. This is the whole point - if you do not have a valid model of the process, your statistical tests are garbage.

Nov 21, 2015 at 6:46 PM | Unregistered CommenterBart

Raff: You've been reading it wrong:

One final note: there’s an ever-growing number of “throw some complicated-looking math at the wall and see what sticks” attempts to refute the lack of global warming. It seems to me that a disproportionate fraction of them come from eco-activists. Perhaps that’s because they fear the loss of government handouts more than they feel concern for the health and welfare of humanity. Or perhaps it’s just a reflection of the rather poor track record of eco-activists in general. When it comes to predicting the future, it’s well to compare the truly astounding successes of just about anybody to, say, meteorologists or climate fanatics.
I 'm sure that was what Tamino was trying to say.

Nov 21, 2015 at 6:55 PM | Registered CommenterMike Jackson

Bart: It is my understanding that monthly temperatures show significant auto-correlation. If so, they aren't "high frequency noise", they are repeat measurements of a "single event" and statistically handled by reducing n (number of data points) in some cases. The extent of autocorrelation drops over about six months, not over 10,000 years.

Bart wrote: "Student's t-distribution is only applicable for independent Gaussian variates. This is the whole point - if you do not have a valid model of the process, your statistical tests are garbage."

Thank you. You are exactly right. I have drawn a statistical conclusion based on an assumption I haven't demonstrated. (I suspect I could do this analysis correctly.) Doug's challenge may be making the same mistake. He may be presenting statistical models for the noise in temperature data that may not make any sense physically (random walk) or for which there is no evidence in observation. IF so, those time series don't prove anything about global warming. Remember, noise (unforced variability) is produced by the physics of our atmosphere and ocean (and probably the sun). Noise isn't something that can be arbitrarily chosen or changed at the whim of the statistician looking at the data. It has a mechanism - or mechanisms.

Likewise, the IPCC's assumption of a linear AR1 for temperature is equally ridiculous. Forcing hasn't been linear and the response to forcing involves lags. The Met Office admitted this problem when they said climate models were needed to demonstrate that the planet is warming - no simple statistical model was adequate and justifiable*. I judged that a victory for Doug. I'm less sure about this challenge.

* Science rarely makes progress through statistical models. Newton hypothesized that the force of gravity varied as 1/r2 and created a theory that withstood observational tests for several centuries and is still useful (if not perfect) today. If we waited until observational data allowed Newton to pick and choose between models of the statistical noise in the observations, we would literally be back in the Dark Ages ... especially if we permitted that noise to diverge from zero like a random walk. However, the noise in observations of planetary position shouldn't diverge.

It makes perfect sense for the Met Office to develop hypotheses relating temperature change to forcing and use them to demonstrate that it has been warming. Unfortunately for the Met Office, there is little observational data convincingly showing that their hypotheses are correct. (We have the equivalent of days of planetary motion around the sun, not many orbits of data.) Seasonal warming (of 3.5 K in GMST before anomalies are calculated) don't produce the changes in OLR and reflected SWR from clear and cloudy skies that their models predict.

Nov 21, 2015 at 8:28 PM | Unregistered CommenterFrank

Frank,


I judged that a victory for Doug.

Why? They basically said he was wrong. He's probably lucky that they have an obligation to respond politely and thoroughly. I guess that could be a victory of sorts, but a somewhat hollow one.

Nov 21, 2015 at 9:09 PM | Unregistered Commenter...and Then There's Physics

"It is my understanding that monthly temperatures show significant auto-correlation. If so, they aren't "high frequency noise", they are repeat measurements of a "single event" and statistically handled by reducing n (number of data points) in some cases."

I.e., they are long term correlated, i.e., you can reduce the sampling rate and still be able to reconstruct the signal (or at least its long term statistical properties), in accordance with sampling theorems.

"It makes perfect sense for the Met Office to develop hypotheses relating temperature change to forcing and use them to demonstrate that it has been warming."

Nobody is arguing over whether it has been warming. The question is whether it has been warming due to the hypothesized influence. There is no problem with testing the hypothesis and seeing where it leads. There is a problem in claiming the hypothesis has been proved by drawing conclusions from inapplicable models, and then advocating wholesale deindustrialization, poverty, and death on that basis.

Doug's exercise is testing whether they can generally draw appropriate conclusions when they do not know the underlying model.

Nov 21, 2015 at 10:05 PM | Unregistered CommenterBart


Doug's exercise is testing whether they can generally draw appropriate conclusions when they do not know the underlying model.

There is no statistical model that can determine what is causing the warming. None. I'll repeat. There is no statistical model that can determine what is causing the warming. Absolutely none. Is that clear? None. Doug Keenan is - at best - savaging a massive strawman. There are less complimentary conclusions/

Nov 21, 2015 at 10:20 PM | Unregistered Commenter...and Then There's Physics

"There is no statistical model that can determine what is causing the warming."

Agreed. So, what else have you got? Cartoon science, shorn of all complexity, saying what should happen, holding all other influences constant? Our most accurate measurements showing global temperatures are currently trending down? What is the basis of your faith?

Nov 21, 2015 at 11:02 PM | Unregistered CommenterBart

aTTP - Isn't the point rather that it is not possible to say if any warming has a cause?

Nov 22, 2015 at 1:05 AM | Unregistered Commenternot banned yet

Frank wrote: "I judged that a victory for Doug."

ATTP replied: "Why? [The Met Office] basically said he was wrong."

ATTP, does reality ever intrude on your thinking? The first scientific statement in the SPM for AR5 WG1 says:

"The globally averaged combined land and ocean surface temperature data as calculated by a LINEAR TREND, show a warming of 0.85 [0.65 to 1.06] °C, over the period 1880 to 2012, when multiple independently produced datasets exist. The total increase between the average of the 1850–1900 period and the 2003–2012 period is 0.78 [0.72 to 0.85] °C."

The IPCC made this determination - including the confidence intervals that demonstrate statistical significance - by applying a linear AR1 model to analyze temperature vs time data. Doug Keenan challenged the validity of this approach. The Met Office statement you cited says:

"There is very high confidence (using the IPCC’s definition) that the global average net effect of human activities since 1850 has been one of warming. The basis for this claim is not, and never has been, the sole use of statistical models to emulate a global temperature trend."

Surely you must recognize that, if the Met Office believed the linear AR1 method were appropriate, they would say that statistical analysis AND climate models provide two INDEPENDENT lines of evidence that statistically significant warming has take place. The linked statement from the Met Office's Chief Scientist is less ambiguous:

"These results [from various statistical models] have no bearing on our understanding of the climate system or of its response to human influences such as greenhouse gas emissions and so the Met Office does not base its assessment of climate change over the instrumental record on the use of these statistical models."

The Met Office's Chief Scientist said that the IPCC's statement about global warming - derived from a purely statistical model - has NO BEARING on our understanding of the climate system.

Nov 22, 2015 at 4:08 AM | Unregistered CommenterFrank

Bart wrote: "Nobody is arguing over whether it has been warming. The question is whether it has been warming due to the hypothesized influence. There is no problem with testing the hypothesis and seeing where it leads. There is a problem in claiming the hypothesis has been proved by drawing conclusions from inapplicable models."

Given the existence of unforced variability (chaotic behavior), one isn't going to be able to reach any robust conclusions about whether GHGs cause warming (and how much) in a one or two decades. That doesn't mean that increasing GHGs don't reduce OLR and thereby warm the planet. The right place to study simple phenomena like the absorption and emission of radiation by atmospheric gases is in the laboratory, where reproducible, controlled experiments are possible. Most of those experiments were performed long ago, before CAGW became an issue.

Nov 22, 2015 at 5:18 AM | Unregistered CommenterFrank

Frank,
The tone of my response is mediated by this


does reality ever intrude on your thinking

Bear that in mind.


The Met Office's Chief Scientist said that the IPCC's statement about global warming - derived from a purely statistical model - has NO BEARING on our understanding of the climate system.

Yes, everyone else has known this for a very long time. No one has claimed that they did have some bearing. That you and Doug Keenan are only starting to realise this now, does NOT make this some kind of massive revelation. It just means a bunch of clueless buffoons have wasted a great deal of taxpayers money getting the Met Office to respond to a ridiculous request from someone who clearly has a great deal more money than sense. Do you get this now? Statistical models ALONE can tell us nothing. This is obvious and should be to anyone who has even a basic understanding of data analysis. Arguing about a better statistical model is ridiculous, given that there are NO statistical models that can tell us what has caused the warming. Doug Keenan's revelations are only revelations to those who didn't understand what others have understood for a very long time and assumed that it was so obvious that they didn't need to actually continually repeat it.

Nov 22, 2015 at 9:01 AM | Unregistered Commenter...and Then There's Physics

not banned yet,


Isn't the point rather that it is not possible to say if any warming has a cause?

Do you think it's magic?

On the other hand, if you mean "can we actually establish what is causing the warming?", then there are two answers. Doug Keenan clearly cannot because he continues to use statistical models only, and they cannot (under any circumstance) determine causality. Others, though, who understand this far better than Doug Keenan does clearly can establish causality (or, at least, probable causality) by using models that are suitable for such an analysis. One should be very careful of imposing Doug Keenan's obvious ignorance onto everyone else.

Nov 22, 2015 at 12:04 PM | Unregistered Commenter...and Then There's Physics

There has been an update to the Contest. Briefly, the 1000 series have been regenerated. The reason is discussed at
http://www.informath.org/Contest1000.htm#Notes
Everyone who plans to enter the Contest should ensure that they have the regenerated series.

Nov 22, 2015 at 12:29 PM | Unregistered CommenterDouglas J. Keenan

Doug,
It's possible you think this is true


The purpose of the Contest is to test researchers' claimed capability to statistically analyze climatic data.

However, it is obvious that this contest will do no such thing. If you do believe this to be true, then you are largely clueless about the analysis of climatic data. If you are not clueless about the analysis of climatic data, then you are knowingly setting up a strawman.

Nov 22, 2015 at 1:20 PM | Unregistered Commenter...and Then There's Physics

ATTP writes: "then you are knowingly setting up a strawman."

I believe all paleoclimatologists knowingly set up straw men although it is a matter of degree as to how much straw and how much man.

A series of measurements of oxygen isotopes going down an ice core is science; anyone suitably equipped can do likewise. Deciding what was the temperature of Earth 400,000 years ago based on that is an invention with the same results unlikely to be obtained by others as it is at least partly "straw", an invention.

Dendrochronologists ought to get the same results everywhere if the Earth had a temperature and that temperature followed expected or desired trends. However it appears that tree rings are highly variable and not well correlated with just temperature. In my opinion there's more straw than man. It is still worth doing but I wouldn't anchor world policy on something made of so much straw.

Doug Keenan's series' are an invention and declared to be so. That's the straw part. The man part is the extent to which techniques to process ice core data, instrumented recordings and tree rings ought also to be able to process series of numbers designed to mimic these streams of data. It becomes a test of technique, not data.

Nov 22, 2015 at 3:55 PM | Unregistered CommenterMichael 2

It appears statistical methods ARE still being used:

"In our new paper, we test the assumption that the recent history of the Earth's climate is statistically equivalent to its response to each individual external forcing"

http://marvelclimate.blogspot.com/2015/10/the-whole-sum-and-parts.html

Nov 22, 2015 at 4:47 PM | Unregistered CommenterMichael 2

Nov 22, 2015 at 5:18 AM | Frank

"Most of those experiments were performed long ago, before CAGW became an issue."

Unfortunately, they are not applicable to a complex Earth with many feedback processes. It is like claiming the lake level will rise when animals urinate into it, while ignoring the river source and the action of the floodgates of the dam and evaporation from the surface. It is that farcical.

Nov 22, 2015 at 6:32 PM | Unregistered CommenterBart

Can you imagine a scientific paper claiming that:

"Since the dawn of the industrial age, Lake Ivanagenda has risen half has high as the estimated sum total urination of migrating animals. Therefore, the animals are causing the rise, and we need to erect fences to keep the animals from migrating past so that they do not bring in unnatural excess waters that eventually will cause the lake to flood our homes if we do not do something about it."

That's pretty close to what is being claimed.

Nov 22, 2015 at 6:40 PM | Unregistered CommenterBart

Douglas J Keenan

A list of the hundred datasets most likely to show a trend will probably contain several random datasets with large standard deviations.

Regrettably that makes it a lottery, in which it is very unlikely to choose the correct hundred except by chance.

Your stake is safe.

Nov 22, 2015 at 6:44 PM | Unregistered CommenterEntropic man

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>