Monday
Nov232015
by 
Bishop Hill
Bishop Hill A change to the playing field
Nov 23, 2015 Climate: Statistics ;)
Doug Keenan has posted a note at the bottom of the notice about his £100,000 challenge, indicating that he has reissued the 1000 data series. This was apparently because it was pointed out to him that the challenge could be "gamed" by hacking the (pseudo)random number generator he had used.
Brandon Shollenberger emails to say that this is a terrible thing, but I can't get terribly excited about it. Presumably it doesn't make any difference to those who think they can detect the difference between trending and non-trending series.
Reader Comments (89)
Keenan wants a mulligan but a bet is a bet.
Bart:
Seriously, what are you smoking? I haven't personalized anything. I haven't focused on individuals or made any personal remarks. The only things I've said about individuals are statements which were conclusions drawn from the evidence I presented.
Unless you want to claim me saying "what are you smoking" is personalizing the problem? I guess that kind of is, but... really, what other response is there to someone saying things that have absolutely no connection to anything they're responding to? I guess I could just respond to your comments by saying, "This comment appears to have no connection to or bearing on anything that has been said, has happened or has existed at any point in the history of mankind," in the future.
But that seems more harsh than, "What are you smoking?"
Bart, I wonder if ATTP realises he's essentially conceded.
Brandon, too, Shub. Yet, he keeps going on, and on, and on. I'm not bothering to read it all. I wonder if anyone is?
Nov 24, 2015 at 12:01 AM | Eli Rabett
Thank you for sharing that gravely profound tautology, Eli. Do you agree with Brandon and ATTP that, as Brandon states it, "No statistical model can determine the cause of anything"?
Brandon,
Of course the problem got harder! As Doug Keenan acknowledged, he had inadvertently made it too easy, so he corrected the technical details to match his unchanged description of what he had intended. Your complaints don't dispute or even address this fact, so I and others cannot take your complaints seriously.
ATTP,
If statistical analysis of data fails to reject the null hypothesis of no anthropogenic signal, then why do you not simply and clearly state that anthropogenic global warming (to any extent at all) lacks substantial statistical support? It is certainly news to me that no one claims such statistical support. Won't you please tell President Obama how mistaken he is?
This matters for me, because I have been thinking until now that there has been at least some anthropogenic warming mixed in with natural fluctuations, and I need to know if there is no firm statistical basis for that. I had thought the only open question was whether the anthropogenic component was a small or large part of the total. I take the physics of CO2 as a greenhouse gas to be settled theoretically, but I rely on statistics to address empirically the less clear issue of overall climate sensitivity.
Dr. Doug:
What are you talking about? Keenan didn't say he had inadvertently made the challenge too easy. He didn't say he "corrected the technical details to match his unchanged description of what he had intended." I don't know where you got this idea from, but it isn't true. He didn't say anything like that. Nobody did. You and other may not take my complaints seriously, but if your reason is you believe people said things that they never said... I don't know how to react. I'm trying to be polite here, but what you've just said is completely and utterly baseless.
What Keenan said is he changed the RNG he used. That should not change any statistical properties of the underlying data set. He did make some vague allusions to other "related changes," but he didn't say anything about what those changes were, and he certainly didn't say those changes were made to correct any technical details to match any description like you claim.
If you want to dismiss what I say because some figment of your imagination tells you Keenan said and did things he never said or did, that's your prerogative. Our host has already shown it's okay here to not only dismiss indisputable evidence, but pretend it doesn't exist, while making backhanded remarks about the people who provide it. Given that, I'm sure your decision would not be considered remarkable.
Still, I would hope you could read what Keenan actually wrote and see it is nothing like what you claim he wrote.
Dr Dung
The null hypothesis is not they there is no anthropogenic signal. The null hypothesis is that the observed trend is due to random variation.
The temperature record shows a trend approaching 1C in 135 years. The standard deviation for this data is 0.05C for Hadcrut4. A difference of 20 standard deviations is highly significant. A difference of 4 standard deviations is enough to 95% confidence that the trend exists.
Bart, shub
What foolishness. The question under discussion is the confirmation of the trend by statistical methods. It is not whether the same methods can identify the cause. Causality has to be identified by other scientific methods.
If your bank balance shows a descending trend, you can demonstrate the trend statistically, but not determine what you are spending the money on.
entropic, I have discussed attribution with you before. There's no use going there.
ATTP states: "Neither of your links are examples of studies that do attribution using statistical models ONLY"
Wrong!
This is the supplemental data showing the methodology for my first reference showing there is nothing other than statistical analysis involved.
http://static-content.springer.com/esm/art%3A10.1007%2Fs10584-015-1495-y/MediaObjects/10584_2015_1495_MOESM1_ESM.pdf
The press release for the second reference states "Rather than using complex computer models to estimate the effects of greenhouse-gas emissions, Lovejoy examines historical data to assess the competing hypothesis: that warming over the past century is due to natural long-term variations in temperature". There is no consideration whatsoever of natural variation beyond assuming it doesn't exist and he specifically eschews physical models which the Met Office admitted were necessary for attribution!"
ATTP further stated: "The 95% attribution statement is based on null hypothesis testing"
But the IPCC did not give any reference or calculation for this 95% number. It was written in the Summary for Policymakers prior to the finalisation of the main report and behind closed doors. In the light of the reality of lowering sensitivities and clearly failed models an objective observer must conclude that the certainty increasing from 90% to 95% has no relation to science and must be purely political. Perhaps a whistle-blower will someday enlighten us.
ATTP just makes stuff up! So his acronym should really be JMSU.
Hadcrut4 fits right in with Keenan's series, according to variance, linear trend and ratio of variance to linear trend.
If Hadcrut4 between 1880 and 2014 were dropped randomly into Keenan's series, could you find it? Would it stand out in any way?
Paul
JamesG,
Okay, are you concentrating now? Read this slowly, and think. That paper does not do attribution. It is a statistical analysis that makes - as far as I can see - no claims as to why we're warming or why the trends are what they are. Is that clear enough for you now?
As far as the 95% issue is concerned, try reading Chapter 10. Maybe start at 10.2.4
Objective. Don't make me laugh.
Shub,
Indeed there isn't. You're unlikely to have learned anything since the last time you discussed this.
And why would it matter if it did, or it didn't? I know, it would matter one jot. Finding HadCRUT4 in Keenan's series is entirely irrelevant. Why? Because HadCRUT4 is based on observations of the real world; a world in which energy, mass, momentum,... are conserved. Keenan's series are entirely made up and are based on a world where even the concept of a conservation law seems to be beyong those who live there.
ATTP
You are saying that mass, energy and momentum are conserved in the real world ... and I don't think anybody disagrees (if you are talking about a bounded, isolated system).
But what does that have to do with trying to with anything at all? Particularly with attempting to detect a signal in quite noisy temperature data.
Temperature most certainly is not 'conserved', and neither are once believed/modelled values of climate sensitivity.
What, if any, is your point here?
Jonas H.,
Firstly, noone is trying to detect a signal in noisy temperature data. That claim is a Keenian strawman. Secondly, if we do want to understand why the data looks as it does, we need to use more than simply statistical models; we need to use models that are developed on the basis of basic physics and chemistry and that are intended to represent a real system. Using those, we can start to understand what is causing the temperature data to look as it does. Giving me some randomly generated time series that happens to look the same as the actual temperature data tells me absolutely nothing (well, not quite, it might tell me something about the person who gave me the randomly generated data).
I guess my point is that you can keep on knocking down that strawman and I - like many others - will wonder why people who claim to have some understanding of this topic, clearly do not.
Paul,
Rather, the relevant(*) question would be that:
If you dropped the Hadcrut4-series among a number randomly generated trendless ones having the same statistical properties (according to an assumed statistical model), but without them already knowing(*) its shape ..
.. would they then be able to identify (with any significant certainty) which one it is?
Given the amount of noise some are making here (and elsewhere) about entirely irrelevant other suff, I think we know the answer to that.
;-)
To be fair, if one threw in a couple of other (generated) series artifically 'enhanced' by a trend of +/-1, it would become even more difficult. Keenans challange was to identify (with at least 90% confidence) which ones were which. If the Hadcrut4 series were among them, that problem would be slightly simpler, as only 999 series remain to be categorized.
(*) I am certain that they could spot the Hadrcrut4 series among the others, if they were told it's in there.
I repeat:
Why did you bring up 'conservation of energy(*), mass and momentum' here?
What does that have to do with anything at all!?
Your attempted answer does not address this ('conservation') in any way. At best I take it that you think physical models should not violate those laws (nobody thinks they should, so that would indeed be a strawman). Why would your 'conservation laws' matter when comparing actual data with generated random (trendless) data to see if they can be distinguished?
(*) And I recall you've tried similar stuff before: That 'the missing heat' wasn't an issue because of 'conservation of energy'!?
Jonas N.,
Because a made up graph that represent nothing and hence is associated with no conservation laws has no relevance to the surface temperature dataset which represent an actual system that does obey those conservation laws. Why do you find this so complicated?
That isn't what I said, but at least I now know who you are.
Paul
"Traces like the extremes of Keenan's set are consistent with about the right amount of natural variability coupled with a 0.01/year trend."
The largest change I observed in Keenan's datasets increased from -0.23 to 2.39, an increase of 2.62.
That would be a random variation of 2.62 if there is no trend. The variation either side of the mean is at least +/- 1.3 .
Alternatively that would be a 1C trend superimposed on random variation of +/- 0.65.
I know of no statistical test which would distinguish between these two alternatives.
I am not sure what you mean by the "right"amount of variability. For the actual HadCRUT4 data the variability is +/-0.1
ATTP says no-one is trying to detect a signal in noisy temperature data.
In a post entitled "The Real Global Warming Signal", the blogger Tamino referred to a then recently published paper "Global temperature evolution 1979–2010 – Grant Foster and Stefan Rahmstorf 2011 Environ. Res. Lett. 6 044022 doi:10.1088/1748-9326/6/4/044022."
In his introductory paragraph, he claimed: "Natural factors cause temperature fluctuations which make the man-made global warming signal less clear, fluctuations which are often exploited by fake skeptics to suggest that global warming has paused, or slowed down, or isn’t happening at all."
Much as I dislike Tamino's zealotry, I wouldn't accuse him of being "no-one".
igsy,
"Using statistical models ONLY". That Foster & Rahmstorf paper attempted to remove the influence of ENSO events using the ENSO Index. Now if Doug Keenan wants to give me a second dataset that represents the "noise" that he's introduced to his time series, then I could probably extract the signal. Giving me a time series with no other information (including what's even being measured) means that this is probably impossible, which - I assume - is the intent.
ATTP, you are damaging the cause (in more ways than you can imagine). Even the former leader of the Labour party knows that global temperature is the highest in the past 3000 million years.
Nov 24, 2015 at 9:41 AM | Entropic man
"The question under discussion is the confirmation of the trend by statistical methods."
And, the answer is, you must utilize a proper statistical model in order to derive proper statistics regarding existence of a trend. If you employ the wrong statistical model, you very likely get the wrong answer regarding existence of a trend.
The data evidence very clear and distinct low frequency peaks in the spectral density to the limits of the observational interval. There are likely even lower frequency peaks which we simply do not yet have sufficiently clean and long term data to detect. As a result, they have very pronounced cyclic autocorrelation which completely annihilates attempts to establish confidence intervals by assuming independent, or even first order correlated, variates.
When I see you, and others, attempting to do this, I immediately know I am dealing with amateurs.
ATTP you are still not answering the question.
When trying to extract a signal in noisy data, possibly because one assumes there migh be one, nobody argues that the noise must be an attempt to explain everything that is not included in the model. Including random noise. The attempt to quantify the noise is not a model of the real world, or an attempt to model it without the hypothesized effects investigated. It is merely an ad hoc attempt to quantify how much things fluctuate quite naturally.
Because the very second you are trying to extract a signal from your noisy data, and attempting to extablish whether or not you may say it is clearly there (significant) you are indeed implicitly making an assumption (pure statistical) about the noise. Mosts data handling packages do this for you.
And maybe that's why you are unaware of that a statistical model is required when looking at empirical data, and trying to assess how it matches your hypothesis.
Keenan, and several others have explained this many times. The other slightly finer point is that your 'significance' is highly dependent on what actually statistial (noise) model you assume. You aren't just allowed to pick one that suits your beliefs, and then proclaim: Look, high significance!
That would be pseudoscience:
sorry, some formatting error.
In between there shoud have been version of that psedoscience, paraphrasing:
@Entropic man
I'm struggling with,
"I am not sure what you mean by the "right" amount of variability. For the actual HadCRUT4 data the variability is +/-0.1"
The 'right' amount of variability is clearly something compatible with what we measure. And that's why I think that if you put HadCRUT4 into Keenan's set you wouldn't be able to pick it out with a statistical method. If you can, then that immediately shows his challenge is irrelevant.
I think you're confusing a property of the measurement with a property of the data, or at least not making yourself clear.
sorry, same mistake again (and 403-error when reposting yesterday)
In between there should have been a version of that pseudoscience, paraphrasing:
But as I and others have said, it doesn't quite work that way.
Paul
" If you can, then that immediately shows his challenge is irrelevant."
That is exactly my point.
But as you clearly cannot separate HadCRUT4 from Keenan's series by its properties then your point is pointless.
EM ...
It's hard to see what you may think 'is exactly your point'.
If you claim that can pick out those series where there actually is a (artificially added) trend, from the trendless ones without (at 90% confidence), you would have a good shot at getting 100.000USD from (almost) free.
The contention among the other wafflers however seems to be that that the challenge is too difficult. That it therefore should be disregarded as 'irrelevant' ..
Based on 'conservations laws' someone argued ignorantly, that tests for establishing significance for an assumed signal i noisy data aren't required. In this case that empirical observations should not be tested against an assumed version of noise without a prescribed trend.
It seems they are whining because they cannot do what 'climate science' claims it can do easily ... ie establishing those claimed certainties.
I assume you mean me, so - just for the record - this is not what I said. Strawmanning is, of course, par for the course here, but I thought I would just point this out. You seem to be a particular specialist in this. I think you've done this every time we've ever exchanged views. Quite a master. Maybe you could try not doing it, or would that completely destroy whatever arguments you think you have?
ATTP, this is what you wrote:
I cannot read this in any other way than that you argued that:
Trying (with some level of confidence) to find a series with an assumed signal (ie. hypothesized, based on models, 'understanding', conversation laws etc) among others (created) trendless series with similar noise levels ...
.. that that would be "irrelevant" (a meaningless exercise, I take it). And this because the modelled hypothesis contains conservation laws.
Sorry ATTP, but the words and phrases "irrelevant" and "Because" .. "HadCRUT4.. of the real world [with] energy, mass, momentum, ...conserved" have certain meanings to me.
I also asked you how and why you think 'conservation laws' demonstrate this, when comparing to artificial/modelled noise from some appropriate statistical process.
There was no meaningful answer. Just waffle about other things.
So sorry, not one single strawman either. Not even a misunderstanding as far as I can see. The enitrety of you last comment once more was nonsense (or projection).
In a later comment, you wrote:
I very much doubt that you could do that. And once again, it seems that you are missing the point.Keenan gave you a large number of series (with trendless noise) and said that there were a bunch in them with an artifical trend added. And challenged you guys to find out which (at 90% confidence)
How do you mean that that would be different than asking you whether you can single out one which has a trend among all those (that could be generated) resulting from a trendless process (with similar noise-levels)?
The only difference would be that if you knew it was only one, you chances att succeeding (by chance) would improve. Wich has no bearing on the method needed to establish that a perceived signal also is 'significant'.
I still maintain, this (ascertaining that the models indeed get it right) is at the very core of the whole climate conundrum. And I am just amazed how many of you seem to be (almost willfully?) ignorant of what this is all about
And yes, the attempts above in this thread come across as otherworldly as when you tried to explain that to Paul M that the missing heat must be there because 'energy is conserved':
Amazing stuff, frikking unbelievable for someone claiming to be familiar with physics, even.
Jonas,
Yes, I know what he's done. Explaining it again doesn't make it not stupid.
Willfully ignorant seems appropriate, but maybe not as you intended.
I realise that you are probably not really interesting in thinking, but for those who might be (although the number who read this blog is probably small) the energy conservation point is this. If you have an understanding of the various energy fluxes then you can estimate how the energy in a system changes with time. If your observations/measurements don't match this expectation, then you will want to understand why. Of course, your understanding of the fluxes might be wrong, but if there are regions for which you don't have measurements, you will want to check if maybe energy is accruing there, rather than in the regions where you do have measurements. Energy conservation is a pretty fundamental concept.
Now, I realise that those who prefer to base their science on stolen emails taken out of context may find this kind of thing rather surprising, but most people who actually do science find it perfectly normal and good practice. Consider everything, in other words.
Can't see that there is any real content here. Trying insults instead ...
You argued that 'conservation laws' makes testing for significance 'irrelevant'.
And earlier, that 'conservation of energy' makes 'the missing heat' issue a strawman
The former involves an (artificial) trendless noise (assumption). Conservation of energy has nothing to do with that part.
The latter is pure nonsense. It's the other way around: The assumption of 'knowing' the total heat accumulated (together with its conservation) casues the presumed 'missing heat'.
Further:
"Energy conservation is a pretty fundamental concept"
Well, yes. And not disputed by anybody. And more importantly, not relevant at all wrt to both actual issues. That's probably why you keep pounding it ...
Stolen emails? Yet another strawman? Geesh ..
I don't believe I said either. Are you sure you understand what strawman means? I would say "try again" but I can't imagine it's going to get any better.
I've been reading through some of these debates with some interest, and thought this might be appropriate to the discussion (especially the bit starting at 2:30):
http://www.cc.com/video-clips/hzqmb9/the-daily-show-with-jon-stewart-large-hadron-collider
Hopefully some will understand the relevance of the points made there...
"The 1000 series were generated as follows. First, 1000 random series were obtained (via a trendless statistical model fit for global temperatures)."
what does this mean ?
there are many many statistical models, which one was taken, a flatline with noise, a pure random walk which takes the next step from the previous step?
they are all unfit for global temperatures :)
Brandon says "What Keenan said is he changed the RNG he used. That should not change any statistical properties of the underlying data set."
With respect, I disagree. I am not a statistician but I am a computer programmer. If the random number generator was initialized to the same seed for each series of only 135 numbers what you will be testing is RNG+Something; where the Something is any of several algorithms. Presumably these algorithms are deterministic, meaning if you start with a particular value it will always produce exactly the same series.
Assuming the RNG was deterministic (unseeded algorithmic RNG's are always deterministic), and the algorithm is deterministic, you could calculate with absolute correctness not only the entire 1000 series of numbers but the next million, billion or whatver until the whole thing loops back around.
I sense among climate scientists a remarkable ineptitude at computer programming, way too much dependency on "Excel" for instance. I do not fault anyone for this; you have your specialties and I have mine, but it does create the occasional marvel of high science and low computing.
http://rosettacode.org/wiki/Random_number_generator_%28included%29
PRNG that is repeating, visual representation from https://www.random.org/analysis/
https://www.random.org/analysis/randbitmap-wamp.png
ATTP
Yes. I too reckon that it's mostly about what you believe. You seemingly don't even know/remember/understand your own comments and aeguments.
And this is the second time you've referred to 'conservations of energy' in a completely inappropriate way. And does not even understand this after it is explained to him. That's quite amazing. In particular for someone claiming to be knowledgable about physics and even science ...
So I guess, there is some twisted logic to it, since your beliefs came down on the CAGW-side ...