Bishop Hill Keenan on McKitrick
Sep 29, 2014 Climate: Statistics Climate: Surface Doug Keenan has posted a strong critique of Ross McKitrick's recent paper on the duration of the pause at his own website. I am reproducing it here.
McKitrick [2014] performs calculations on series of global surface temperatures, and claims to thereby determine the duration of the current apparent stall in global warming. Herein, the basis for those calculations is considered.
Much of McKitrick [2014] deals with a concept known as a “time series”. A time series is any series of measurements taken at regular time intervals. Examples include the following: the maximum temperature in London each day; prices on the New York Stock Exchange at the close of each business day; the total wheat harvest in Canada each year. Another example is the average global temperature each year.
The techniques required to analyze time series are generally different from those required to analyze other types of data. The techniques are usually taught only in specialized statistics courses.
Assumptions
The calculations of McKitrick [2014] rely on certain assumptions. In principle, that is fine: some assumptions must always be made, when attempting to analyze data. A vital question is almost always this: what assumptions should be relied upon? The question is vital because the conclusions of the data analysis commonly depend upon the assumptions. That is, the conclusions of the analysis can vary greatly, depending upon the assumptions.
The problem with McKitrick [2014] is that it relies on assumptions that are wholly unjustified—and, worse, not even explicitly stated. Hence, I e-mailed McKitrick, saying the following.
The analysis in your paper is based on certain assumptions (as all statistical analyses must be). One problem is that your paper does not attempt to justify its assumptions. Indeed, my suspicion is that the assumptions are unjustifiable. In any case, without some justification for the assumptions, there is no reason to accept your paper's conclusion.
The issue here is not specific to statistics. Rather, it pertains to research generally: in any analysis, whatever assumptions are made need to be justified.
McKitrick replied, claiming that “The only assumption necessary is that the series [of temperatures] is trend stationary”. The term “trend stationary” here is technical, and is discussed further below.
There are two problems with McKitrick's claim. The first problem is that trend stationarity is not the only assumption made by his paper. The second problem is that the assumption of trend stationarity is unjustified and seemingly unjustifiable. The next sections consider those problems in more detail.
The assumption of linearity
McKitrick claimed that his paper only made one assumption, about trend stationarity. In fact, the paper also assumes that all the relevant equations (for the noise) should be linear. Hence, I e-mailed McKitrick back, saying the following.
Stationarity is not the only assumption. Your paper also includes some assumptions about linearity … I do not see how [linearity] can be justified….
McKitrick did not respond. Five days later, I sent another e-mail, again raising the problem of assuming linearity. This time McKitrick replied at length. His reply, however, did not mention linearity.
The climate system is nonlinear. This is accepted by virtually everyone who has done research in climatology. For example, the IPCC has previously noted that “we are dealing with a coupled non-linear chaotic system” [AR3, Volume I: §14.2.2.2]. Hence the assumption of linearity is very dubious. There might be occasions where it is suspected that a linear approximation is appropriate, but if so, then some argument for the appropriateness must be given.
The assumption of trend stationarity
For technical details of what it means for a time series to be trend stationary, see the Wikipedia article. This section considers issues that do not require those details.
McKitrick's first e-mail to me acknowledged that trend stationarity “makes an enormous difference for defining and interpreting trend terms”. Simply put, if the trend in global temperatures is not assumed to be trend stationary, then the calculations of McKitrick [2014] are not valid.
The abstract of McKitrick [2014] states that the calculations used in the paper are “valid as long as the underlying series is trend stationary, which is the case for the data used herein” (emphasis added). The emphasized claim seems to imply that trend stationarity of the temperature data is an established fact.
The body of the paper says that the temperature data is “assumed to be trend-stationary”. The paper makes no attempt to justify the assumption. At least, though, the body of the paper acknowledges that trend stationarity is an assumption, rather than a fact.
McKitrick's first e-mail to me said that “decisive tests [for trend stationarity] are difficult to construct”. Thus, McKitrick seems to be acknowledging that he has no decisive statistical tests to justify the assumption of trend stationarity.
McKitrick's first e-mail also referred to a workshop, held in 2013, at which “there were extended discussions on whether global temperature series are stationary or not”. Thus, this effectively acknowledges that McKitrick knows trend stationarity is nowhere near being an established fact.
McKitrick's second e-mail attempted some justification for assuming trend stationarity. It said this: “The reason I do not accept the nonstationarity model for temperature is that it implies an infinitely large variance, which is physically impossible, and also that the climate mean state can wander arbitrarily far in any direction, which does not accord with life on Earth”. The first claim, about “an infinitely large variance”, is false; so it will not be discussed further here. The second claim, about how “the climate mean state can wander arbitrarily far in any direction”, is true in principle.
To understand McKitrick's second claim, first note that for “climate mean state” it is enough to consider simply “global temperature”. If the global temperature were truly non-stationary, then it could indeed wander arbitrarily far, up and down; i.e. it could become arbitrarily hot and arbitrarily cold. We know that global temperatures do not vary that much. Hence, global temperatures cannot be non-stationary—this is McKitrick's argument.
McKitrick's argument is easily seen to invalid. Consider a straight line (that is not perfectly horizontal). The straight line goes arbitrarily far up and arbitrarily far down—i.e. arbitrarily far in both directions. A straight line, though, is the basis for the calculations of McKitrick [2014]. Thus, if McKitrick's argument were correct, it would invalidate the basis for McKitrick's own paper.
McKitrick's argument against non-stationarity was raised earlier, by someone else, on the Bishop Hill blog. In response, an anonymous commenter (Nullius in Verba) left a perspicacious comment. The comment is excerpted below.
… everyone agrees that a non-stationary … process is not physically possible for temperature, in exactly the same way as they agree that a non-zero linear trend isn't physically possible. If you extend a non-zero trend forwards or backwards in time far enough, you'll eventually wind up with temperatures below absolute zero in one direction, and temperatures hotter than the sun's core in the other. For the *actual* underlying process to be a linear trend is physically and logically impossible.
However, nobody objects on this basis because everybody knows it is only being used as an approximation that is only considered valid over a short time interval. ….
In exactly the same way, a non-stationary … process is being used as an approximation to a stationary one, and is only considered valid over a short time interval. It arises for exactly the same reason….
Statisticians use non-stationary [models] routinely for variables that are known to be bounded, for very good reason. They're not stupid.
Additionally, McKitrick's argument is an appeal to physics. Yet using physics to exclude a statistical assumption is inherently very dubious. For some elaboration on this, see the Excursus below.
As noted above, several researchers have contended that non-trend-stationarity might be an appropriate assumption for global temperatures. An early paper making that contention is by Woodward & Gray [1995]. That paper currently has 68 citations on Google Scholar, including several since 2013. (One of the latter even presents a physics-based rationale for non-trend-stationarity: Kaufmann et al. [2013].)
There are other papers that do not cite Woodward & Gray, but which also contend for considering non-trend-stationarity; e.g. the paper of Breusch & Vahid [2011]—which is part of the Australian Garnaut Review. Such contending has even appeared in an introductory textbook on time series: Time Series Analysis and Its Applications [Shumway & Stoffer, 2011: Example 2.5; see too set problems 3.33 and 5.3]. Contentions for non-trend-stationarity would not appear in so many respected sources, over so many years, if McKitrick's appeal to a simple physical argument had merit.
It is worth reviewing how McKitrick's story on trend stationarity of the global temperature series changed. First, the abstract of the paper claimed that the temperatures are trend stationary—seemingly an established fact. Second, the body of the paper mentions, in one sentence, that trend stationarity is actually an assumption, rather than a fact—but it gives no justification for the assumption. Third, McKitrick's first e-mail acknowledged that there have been no tests to justify the assumption and also that the validity of the assumption is debated. Fourth, McKitrick's second e-mail, in response to my criticisms of the foregoing, attempted some justifying of the assumption—but with a justification that is easily seen to be invalid, as well as not supported by many other researchers who have studied the issue.
Statistical models
Whenever data is analyzed, we must make some assumptions. In statistics, the assumptions, collectively, are called a “statistical model”. There has been much written about how to select a statistical model—i.e. about how to choose the assumptions.
This issue is noted by the book Principles of Statistical Inference (2006). The book's author is one of the most esteemed statisticians in the U.K., Sir David Cox. The book states this: “How [the] translation from subject-matter problem to statistical model is done is often the most critical part of an analysis”. In other words, choosing the assumptions is often the difficult part of a statistical analysis.
Another book that is relevant is Model Selection [Burnham & Anderson, 2002]. This book currently has about 25 000 citations on Google Scholar—which seems to make it the most-cited statistical research work published during the past quarter century. The book states the following (§8.3).
Statistical inference from a data set, given a model, is well advanced and supported by a very large amount of theory. Theorists and practitioners are routinely employing this theory … in the solution of problems in the applied sciences. The most compelling question is, “what model to use?” Valid inference must usually be based on a good approximating model, but which one?
The book also refers to the question “What is the best model to use?” as the critical issue (§1.2.3).
The selection of a statistical model tends to be especially difficult for time series. Indeed, one of the world's leading specialists in time series, Howell Tong, stated the following, in his book Non-linear Time Series (§5.4).
A fundamental difficulty in statistical analysis is the choice of an appropriate model. This is particularly pronounced in time series analysis.
Note that, in making the statement, Tong does not assume that time series are linear—as the title of his book makes clear.
Concluding remarks
What McKitrick [2014] has done is skip the difficult part of statistical analysis. That is, McKitrick does not genuinely consider the choice of statistical assumptions. Instead, he just picks some assumptions, with negligible justification, and then does calculations.
Realistically, then, McKitrick [2014] does not present a statistical analysis—because the paper is missing a required part. If McKitrick had been forthcoming about this, that would have been fine. For example, suppose McKitrick had included a disclaimer like the following.
The calculations in this work rely on assumptions: about linearity and trend stationarity (and normality). Those assumptions are unjustified and might well be unjustifiable. Relying on different assumptions might well lead to conclusions that are very different from the conclusions of this work. Hence, the conclusions of this work should be regarded as highly tentative.
Such a disclaimer would have been fair and honest. Instead, the paper, especially the abstract, greatly misleads: and McKitrick must have known that it does so.
Finally, methods to detect trends in global temperatures have been studied by the Met Office. A consequence of the study is that “the Met Office does not use a linear trend model to detect changes in global mean temperature” [HL969, Hansard U.K., 2013–2014].
Excursus: Realistic models?
A statistical model does not need to be physically realistic. An example will illustrate this. Suppose that we have a coin. We toss the coin a few times, with the outcome of each toss being either Heads or Tails. We might then make two assumptions. First, the probability of the coin coming up Heads is ½. Second, the result of one toss is unaffected by the other tosses.
The two assumptions comprise our statistical model. The assumptions obviously elide many physical details: they do not tell us what type of coin was used, how long each toss took, the path of the coin through the air, etc. The assumptions, though, should be enough to allow us to analyze the data statistically.
The set of assumptions—i.e. the model—also differs from reality. For instance, our assumption that a coin comes up Heads with probability ½ is only an approximation. In reality, the two sides of a coin are not exactly the same, and so the chances that they come up will not be the same. It might really be, for instance, that the probability that a coin comes up Heads is 0.500001 and the probability that it comes up Tails is 0.499999. Of course, in almost all practical applications, this difference will not matter, and our assumption of a probability of ½ will be fine.
There is also a second way in which our model of a coin toss differs from reality. We can predetermine the outcome of a toss by measuring the position of the coin prior to the toss, measuring the forces exerted on the coin at the start of the toss, and determining the air resistances as the coin was about to go through the air (all this is in principle; in practice, it might not be feasible [Strzalko et al., 2010]). Thus, a real toss is deterministic: it is not random at all. Yet we modelled the outcome of the toss as being random.
This second way in which our model differs from reality—incorporating randomness where the actual process is deterministic—is fundamental. Yet, by modelling the outcome of a coin toss as random, our model is vastly more useful than it would be if we modelled the toss with realistic determinism (i.e. with all the physical forces, etc., that control the outcome of the toss). Indeed, statistics textbooks commonly model a coin toss as being random. Moreover, people have probably been treating a coin toss as random for as long as there have been coins.
To summarize, we model a coin toss as a random event with probability ½, even though we know that the model is physically unrealistic. This exemplifies a maxim of statistics: “all models are wrong, but some are useful”. The maxim seems to be accepted by all statisticians (as well as being intuitively clear). McKitrick, by appealing to a supposed lack of physical realism of non-stationary models, ignores that.
❧ A draft of this Comment was sent to Ross McKitrick; McKitrick acknowledged receipt, but had nothing to say on the technical issues.
See also
| • | Is a line trending upward? |
Breusch T., Vahid F. (2011), “Global temperature trends”, Econometrics and Business Statistics Working Papers (Monash University), 4/11.
Burnham K.P., Anderson D.R. (2002), Model Selection and Multimodel Inference (Springer).
Cox D.R. (2006), Principles of Statistical Inference (Cambridge University Press).
Kaufmann R.K., Kauppi H., Mann M.L., Stock J.H. (2013), “Does temperature contain a stochastic trend: linking statistical results to physical mechanisms”, Climatic Change, 118: 729–743. doi: 10.1007/s10584-012-0683-2.
McKitrick R.R. (2014), “HAC-robust measurement of the duration of a trendless subsample in a global climate time series”, Open Journal of Statistics, 4: 527–535. doi: 10.4236/ojs.2014.47050.
Shumway R.H., Stoffer D.S. (2011), Time Series Analysis and Its Applications (Springer).
Strzalko J.,Grabski J., Stefanski A., Perlikowski P.,Kapitaniak T. (2010), “Understanding coin-tossing”, Mathematical Intelligencer, 32: 54–58. doi: 10.1007/s00283-010-9143-x.
Tong H. (1995), Non-linear Time Series (Oxford University Press).
Woodward W.A., Gray H.L. (1995), “Selecting a model for detecting the presence of a trend”, Journal of Climate, 8: 1929–1937. doi: 10.1175/1520-0442(1995)008<1929:SAMFDT>2.0.CO;2.
Reader Comments (150)
Cycles?
Funny to see how most pundits only vent a bit their sour grapes.
We should stop being so reverent to "scientists" and "professors" these are dying professions/concepts anyway
What do we actually mean when we try to identify a "pause" in a longer time series that may not be trend stationary? First, I think the Ross/Keenan debate is analogous to the debate with the Met Office over trying to apply an inappropriate linear AR1 model to global warming: The Met Office was trying to misude the statistical significance of a trend greater than zero. Ross is trying to identify a trend equal to zero. Is it wrong to look for a short period of simpler behavior in a longer series with more complex behavior? We will, by chance, find pauses in random-walk processes. We will also, by chance find periods that appear to have a fairly linear non-zero trend (such as global warming). The mistake is to assign significance to the detection of such periods. At WUWT, Lord Monckton and others broadcast the length of the current pause in every global record (except ocean heat content, of course) as if the pause has significance - as if it is statistical PROOF that GLOBAL WARMING HAS ENDED. And half of the idiots in the comments section then conclude that this means that AGW NEVER EXISTED. That is embarrassing. The MET Office and IPCC used linear models to PROVE that global warming IS HAPPENING RIGHT NOW. Doug embarrassed the Met Office. (Now I wish he would embarrass the peanut gallery at WUWT.)
As an amateur, I don't object to the identification of regions of simpler or approximate behavior in a more complex time series. The warming rate has been about 0.14 +/- X degC/decade for the last half century (but we know that it is warmer because of the statistically significant difference in the decadal mean temperature from the beginning to end of this period). The warming rate has been indistinguishable from 0 for the last 15 years. The problem arises when the detection of such periods is MISINTERPRETED by the public and some technically knowledgable people. The pause exists. It is important because: 1) Such long pauses are rarely found in the output of climate models. 2) The pause is increasing the discrepancy between models and observations.
Ross,
I don't know if you are still reading this thread, but you say above:
I had a look for the online code and found some here: http://www.rossmckitrick.com/uploads/4/8/0/8/4808045/hr.txt
Is this what you was talking about? You appear to load the library that contains the tests, but I can't see the test itself. Is there more online code elsewhere? Or am I missing something?
Thanks.
Frank,
You pose the question:"Is it wrong to look for a short period of simpler behavior in a longer series with more complex behavior?"
There may be a few exceptions, but the general answer to this question is "In a statistical analysis, yes, it is wrong." If the statistical model developed for the short period is different from the unknown-but-perfect structural model of the longer series, then the noise model is different. Any inference drawn from a hypothesis test based on the short period model relies on probabilities developed from this incorrect noise model, and will in most cases be spurious.
I find Keenan's arguments completely unconvincing. If the Earth's temperature were non-stationary, then at some point over the past billion years or so temperatures would have increased or decreased to where they were incompatible with higher life forms. Stationarity is a very reasonable assumption.
Non- linearity is present most everywhere, but usually is approximated reasonably well over smallish ranges (eg a degree or two out of 290).
Both are non-issues, and Keenan's objections are silly and risible.
"In statistics, inferences are not drawn directly from data; rather, a statistical model is fit to the data, and then inferences are drawn from the model."
----------------------------------------------------------------------------
Doug, I have a lot of time for you. But, as a non-statistician who has worked with statistics (and models) for many years, this statement is tosh.
Sometimes inferences are drawn directly from data, and sometimes the data are fed into a model. Or do you subscribe to Mosher's peculiar theory that any form of analysis is a "model"? While this is a handy get-out for failed modellers, it strikes me as sophistry to the point of absurdity.
@ johanna, 2:39 PM
There is always a statistical model. Here is an extract from the Wikipedia article “Statistical inference”.
The article includes many references, including Cox [2006], cited in the post.
For an example of how this works even in simple cases, see the note “Is a line trending upward?”, linked near the end of the post.
"If the Earth's temperature were non-stationary, then at some point over the past billion years or so temperatures would have increased or decreased to where they were incompatible with higher life forms."
If the Earth's temperature was a linear trend, then over the past billion or so years it would have been below absolute zero, which would violate a few laws of physics. And yet, nobody complains about Ross's use of a linear trend as the appropriate model.
"Non- linearity is present most everywhere, but usually is approximated reasonably well over smallish ranges (eg a degree or two out of 290)."
Not necessarily. The Lorenz butterfly curve was derived from a simplified version of the climate equations. It hops at 'random' between two closely-spaced solutions. It oscillates about one point, then it jumps to oscillate about the other, then jumps back.Stranger behaviours are possible too. The point about chaos is that arbitrarily small differences can have large long-term effects. "The flap of a butterfly's wings..." and all that.
"Doug, I have a lot of time for you. But, as a non-statistician who has worked with statistics (and models) for many years, this statement is tosh."
Statistical models are not the same thing as computer models. Statistical models are implicit in the vast majority of statistical techniques that are used in practice, even the simplest ones, although it's not uncommon for people to be unaware of this. It's true that not every statement in statistics is derived from models, but most of the interesting ones are.
For example, a lot of people know how to plot a trend line through a scatter of points. Do you know what this trend line actually means? It's derived on the assumption that the data arises from a process in which each output is a linear function of the input plus a Normally distributed random value with zero mean, fixed standard deviation, and independent from measurement to measurement. On this assumption, the calculated line is the one for which the observed data is most likely. All that stuff about independent identical zero mean errors is the 'statistical model' we're talking about. If the data isn't generated by such a process, the line output by the algorithm is meaningless.
When you read about a statistical method in a textbook, it's that list of preconditions you always skip over at the start of the theorem. It's the thing you use to calculate error bars on anything. They're ubiquitous.
And there is a huge amount of bad science done by people who know how to press the buttons on the calculator or statistics software, but don't understand the assumptions on which the methods rely. Climate science has more than its fair share of that.
Ross's approach does necessarily depend on assuming particular statistical models. The trend-stationary model is basically a piecewise-linear function of time plus an ARIMA process with d = 0. The Zivot-Andrews test he said he used (of which there are several versions) compares this model against a null hypothesis of an ARIMA process with d = 1, which is described as a 'unit root'. The problem is that it's not the only possible non-stationary / unit root process, and rejecting this particular null doesn't mean that we can't find another non-stationary process that can't be rejected. As I said earlier, I haven't looked at the supporting arguments closely enough to decide - it's not an area I know much about - but I do know that there's something here to discuss.
I don't agree with the slightly confrontational approach Doug takes, and he needs to caveat his assertions with a bit more humility and caution, especially as regards people's motivations, but it's a style that a lot of people here routinely cheer on when it is applied to warmists. Doug is quite right that from a scientific point of view we shouldn't treat our friends any differently, and if we honestly think there is a problem we should say so. Obviously that gets up everybody's nose, and has resulted in some interesting technical issues being completely buried by the emotional flaming from people upset over the way they were put.
But then, that's the story of the whole climate change argument in a nutshell.
There is always a statistical model.
---------------------------------------------
Nonsense. If I take a random survey of 1,000 people about whether or not they agree with something, I get an answer along the lines of X% said yes, Y% said no, Z% said don't know.
Now, like Mosher, you might argue that collecting and reporting these results is somehow a model. But you are only fooling yourselves.
Methodology is not a model.That is like saying that science is a model. Everything is a model!
Well, that might be interesting to philosophers with nothing else to do, but in the real world that just makes every definition meaningless.
For practical purposes, a model is where additional data or parameters or whatever you want to call them are applied to raw data.
Joanna you are only summarising data about 1000 people
That's primitive statistics more reporting really
When you draw conclusions from that data about the larger population from
Which the 1000 people were sampled, THEN you will need a model I think
NiV:
Do I do that? I hope not. In which case I wish Doug hadn't with Ross. I like NiV's consistency, as usual. I also greatly appreciate the host's request for advice. But I'm not sure he put a foot wrong.
I'd put it a bit differently. The meta-discussion has been rather intelligent, by my lights, with the host setting a great example. But I agree we don't seem able to combine discussions at both levels so some 'interesting technical issues' are on hold. That could easily change too.
Fair comment. We should reflect on our reactions in this case and learn the right lessons.
"For practical purposes, a model is where additional data or parameters or whatever you want to call them are applied to raw data."
Science tries to take raw data/observation and make deductions from it about the underlying reality. That the sun came up in the east every day for the last ten thousand days is data. That the sun always comes up in the east, or even that it will do so tomorrow, is a model. That in a particular experiment the calculated energy was the same number before and after is data. That energy is always conserved is a model.
Scientific laws are models. They're not meaningless, though.
@ johanna, 6:00 PM
The theory of statistical inference has been developed over many decades, by thousands of researchers, as you would know if you had consulted the Wikipedia article that my prior comment linked to. You plainly have negligible familiarity with what into that development. Yet now you act with what many people would consider as less-than-ideal politeness, demanding that the work of thousands of scholars be rejected in the face of your wholly-uninformed view.
Ordinarily, I would ignore someone who did something like that. You have been courteous and helpful in the past, however; so I will elaborate a little.
The purpose of statistics is to draw inferences about a (typically large) population from a (typically small) sample of that population. In the example you give, the population might be, for instance, the set of all adults in Scotland, and the sample is the 1000 people who were surveyed. The answers of those 1000 people are of little value on their own. What is valuable is our inference about the opinions of the population.
We cannot say that X% of the population agree, because we have not surveyed the whole population. Rather, we can say that, based on our survey sample, approximately X% of the population agree. Then we use statistics to make that “approximately” more precise. For instance, the organization that did the survey might report the results as something like “the results are accurate to within 3 percentage points, 19 out of 20 times”. Such a report requires statistical analysis, which must be based on assumptions. It is difficult to do that well, as we have just seen in the Scottish referendum.
Douglas, once again you verge on personal insult to those who disagree with you. Did you really have to say: "Yet now you act with what many people would consider as less-than-ideal politeness, demanding that the work of thousands of scholars be rejected in the face of your wholly-uninformed view. "
I didn't "demand" anything. And, your opinion that I am "wholly uninformed" is more along the same lines.
Please stop behaving like Michael Mann having a hissy fit.
To those of us who have difficulty in following all the science or statistics, accusations of bad faith are often helpful - in judging how much weight to put on other assertions of those who make them.
a lot of the aggravation here is about academics who cannot cope with I believe an IT-person (engineers, from the real world)
Here's a message: that university and all the rituals and chiqu etc? chuck in the bin. nobody wants to keep paying for that. zero results. all the so called results of universities the last 50years are to be written on the conto of the computer.
joanna is losing the argument .. :)
as for NiV's continuous blathering , I think a common strain in replacing old physics laws with new ones, is that the newer ones are more devoid of meaning.
Time is but another space dimension
magnetism is but electricity with a delay or something
mass is frozen up energy, or energy is accelerated mass or something
etc
semantics/meaning is a dual of grammar and in fact contains scant extra value
"joanna is losing the argument .."
It shouldn't be about 'winning' or 'losing'. 'Learning' is a lot more important.
"I think a common strain in replacing old physics laws with new ones, is that the newer ones are more devoid of meaning."
Actually, I think they tend to be even more full of meaning than the originals. Although it usually takes a lot of deep study to realise it. Why do you think they're not?
NIV,
I have enjoyed all of your inputs on this topic. Thank you.
DK wrote:-
You more or less repeated this argument in your response to Steve Fitzpatrick:
There is a tu quoque fallacy hanging around here. Having a statistical model of temperature which is unbounded in history should be rejected out of hand IMO. It does not matter whether it is unbounded because the noise model is unbounded or because of the functional form; if it is unbounded, it needs to be thrown out as a null and stamped on.
@ johanna, Oct 4 at 6:56 PM
You referred to my first statement as “tosh”; you referred to my subsequent linked-and-referenced elaboration as “nonsense”. You did that while insisting that you were right on a deep topic that you have seemingly not studied at all. If you had instead said that you did not understand my statement, and asked for an explanation, I would have responded differently. I still gave an explanation. You still have not acknowledged that what you were insisting on is wrong.
Douglas, I made those comments about your statements, not about you.
OTOH, you attacked me personally.
You applied similar tactics to Ross McKittrick.
Disappointing, and not consistent with the usual standard of debate around here.
"There is a tu quoque fallacy hanging around here. Having a statistical model of temperature which is unbounded in history should be rejected out of hand IMO. It does not matter whether it is unbounded because the noise model is unbounded or because of the functional form; if it is unbounded, it needs to be thrown out as a null and stamped on."
See the bit Doug quoted up at the top.
When people fit a linear trend to data that obviously cannot be linear (because a linear trend is unbounded), they're implicitly doing something that is different but related to what they say. Any smooth wiggly curve can be expanded as a power series - constant term plus linear function plus quadratic plus cubic ... etc. The higher-order terms need more data to estimate, and beyond a certain point you just don't have enough data to do it. So you truncate the power series after a few terms, estimate those, and declare the remainder to be unknown. The reason the higher order terms are hard to estimate is that they have little effect on the data in the short term, but that also means they're safe to ignore because they have little effect in the short term. So long as you don't try to extrapolate your regression line beyond the interval for which you have good data, that is.
This is the way people normally go about it. There's an implicit model hanging around in the background, but its parameters are too hard to estimate. So we invent an approximate model that is mathematically more tractable, but is not strictly correct, and cannot be extended beyond bounds we implicitly understand. Expert users start taking short-cuts and not bothering to explain the approximation, but just take it as background. And eventually, they stop even thinking about it, relying on unconscious monitors to raise an alert if the bounds are exceeded.
Physicists are even more prone to this than mathematicians. Consider the flat, smooth, frictionless planes, inclined slopes, perfect spheres, rigid bodies, light, inextensible pieces of string of the mechanics class. Consider ideal gases and black bodies and perfect fluids. Consider point particles and force laws that imply instantaneous action at a distance. Consider the way people neglect relativistic corrections. Consider how often physicists act as if the world was flat, and gravity uniform.
Yes, it's true that there's no such thing as a rigid body, and any physicist will tell you so if you ask them directly. But they'll still do the calculations for a gyroscope as if there was.
This is the same thing. It is commonly understood that linear trends are only being used to approximate a short section of a wiggly line. It is commonly understood that a non-stationary process is only being used to approximate a short section of a (much more complicated) stationary one. There are implicit higher order terms hanging around in the background that we are neglecting, which we are ignoring because we can't estimate them accurately, and because they have no significant effect on the section we're looking at anyway. We just need to keep a watch for any attempt to extend conclusions beyond the short section we can see, because then they will make a difference.
I'm inclined to agree that when presenting stuff to non-specialists, we need to explain this stuff. Because when people are being properly sceptical and looking at things closely, they'll naturally start poking their fingers through the façade and asking awkward questions. Which is good!
But the reason I presented it as a bit of a tu quoque was that I think you already know this stuff when it comes to linear trends, which is why you don't ask those awkward questions about it. I'm inviting you to extend that same understanding to the less familiar world of ARIMA.
Johanna,
If you look closely, he criticised your familiarity with the material and your views, not you.
But really, this back-and-forth is tiresome. It doesn't matter who started it. It doesn't matter who's most to blame. *Both* sides have been less than polite about the other, both sides are now angry and upset about it, and keeping it going will only make things worse.
The best thing to do with rudeness is to ignore it. Only the people using it look bad.
NiV (9:45 AM): Very nice, thanks.
will she apologize now, i wonder?
paulk makes me think of a puppie that learns to behave
amongst taller animals. sometimes too much brownnosing is a bad thing.
Nullers. Learning is more important
Hmmmm I wonder if that platitude is true
Or rather related to your platitudes on meaning
anyway have a nice day all
absurdists out in force this weekend
we must be doing something right
NiV,
Thanks again for your comments.
I have no objection at all to individuals simplifying problems by making reasonable assumptions, and I am all in favour of neglecting anything which can be neglected - just ask my wife.
I tend, however, to make a clear distinction between estimation and inferential models in statistics. You can get away with a lot in estimation, but making unjustifiable assumptions in hypothesis testing leads to conclusions which are, at best, without foundation and, at worst, completely erroneous.
Whether Ross himself accepts the view or not, what he is doing here is setting up a predictive model from which to draw inferences. He claims trend-stationarity, a point I will return to. He sets up a model comprising a trend plus a noise model, which only has validity if the assumption of trend-stationarity is valid. From that he estimates a 95% confidence interval for the trend and applies a hypothesis test with a reversed null. For any given year X, say, he tests the null that the trend has zero gradient from year X forwards. He then chooses the earliest year X for which he fails to reject the null hypothesis at 95% confidence. Predicting backwards in time, there is only about a 1 in 20 chance that a zero trend is "true" for the period he chooses. If he were to increase his confidence to 99% he could obtain an even longer period of zero gradient! I have a slight problem with this, but at the end of the day, it is definitional. I don't like it, but, to be fair to Ross, he has defined his criterion clearly.
The larger problem relates to his willingness to apply a trend-stationary model to the data. Quite frankly, I don't care about the length of the hiatus, and if I want to know how long it is, then I will measure it with a ruler or an eyeball, but the assertion of trend-stationarity in these data - in a citable, published paper - propagates a sad history of statistics being badly applied to climate science data.
A couple of years ago, I ran some stat. an. on GISSTemp and HadCrut3. The HF component is autocorrelated at least up to lag 2 or 3. Using Augmented Dickey-Fuller, the least biased test for this number of datapoints, I found I could not reject a unit root at augmented lags of 2, 3 and 4.
Breusch and Vahid (http://www.buseco.monash.edu.au/ebs/pubs/wpapers/2011/wp4-11.pdf ) found very similar results from a battery of tests. They landed, IIRC, on an ARIMA(0,1,2) model with drift. Doug Keenan from previous correspondence has preferred an ARIMA(3,1,0) with no drift. Both of these models stink for inference testing because of their unbounded nature, IMO, but it does not change the fact that the temperature series tests positive for I(1).
Ross states in his paper:
The data tested apparently included "HadCrut4". While I have not tested HadCrut4 directly, I am moderately certain that if I were to repeat the tests of auto-correlation and integration on this series it would produce a similar result to HadCrut3, since the unit root arises from the longer wavelength (multidecadal) character. Ross's online code does not include this testing, as far as I can find, but it does suggest that he only downloads HadCrut4 from 1900 forwards. My best guess at the moment is that Ross has either applied Dickey-Fuller to long-lag auto-correlated data, which would be wrong, or he has applied Augmented Dickey-Fuller to a shortened interval of the data, which would also be wrong. I can see no other way that he could reject a unit root for HadCrut4 with such certainty. I am hoping, if he is still following, that he might turn up to clarify the point.
In summary, I don't really care about minor errors in the estimation of the length of the hiatus. I do care about someone in the future basing an important inference on a trend-stationary model of temperature and citing Ross's paper as a justification for so doing.
@ Paul_K, 10:01 AM
Your comment says that “to be fair to Ross, he has defined his criterion clearly”. That might at first appear to be the case, but it is actually false. In particular, the model is given by Equation (1): a straight line plus noise. About the noise, the paper says that “may be described as autocorrelated or persistent to unknown dimensions”. In fact, the method used by the paper does not work with persistent data. Indeed, McKitrick's reference for the method [Vogelsang & Franses] “rules out stationary time series with long memory”. For more details, see my comment on Oct 3 at 8:39 AM.
I definitely agree with your point about the results of the Dickey–Fuller tests: if McKitrick had done the tests properly, he would not have been able to reject the temperature series having a unit root, as his paper claims to. And if a unit root cannot be rejected, then there is not much left of the paper.
About the code for unit root tests (Dickey–Fuller and also Zivot–Andrews [Nullius in Verba, Oct 3 at 8:05 PM]), this does indeed seem to be missing. It is especially ironic that the code is missing, given that McKitrick has been championing requiring making code a required part of papers. Is it really a coincidence that the missing code would surely demonstrate that the paper was ill-founded?
Please note that I have never advocated adopting any particular statistical model for the temperature series.
Another issue with McKitrick [2014] is that it uses monthly data. What method did McKitrick use to test if a unit-root model for that data should be rejected?
Pauls the man 4 this: if data is measured per month theirs
Bound to be a cycle in the data (like, with period 12
Months, like)
Paul_K,
"I tend, however, to make a clear distinction between estimation and inferential models in statistics. You can get away with a lot in estimation, but making unjustifiable assumptions in hypothesis testing leads to conclusions which are, at best, without foundation and, at worst, completely erroneous."
You can sometines get away with a lot, but it isn't always obvious when you haven't. Personally, I prefer it when people state their assumptions, so I can credit their conclusions to the extent that I find their assumptions plausible. I can use my own prior.
Frankly, all it requires in this case is the insertion of a sentence to say something like "The method assumes trend-stationarity, which has been found to be at least a plausible approximation by applying the following tests:..." It's a minor tweak, that I'd probably insert mentally anyway while I was reading it. I'd never take a paper as 'gospel', and I'd never expect a paper to be perfect. One makes allowances for human fallibility.
"Predicting backwards in time, there is only about a 1 in 20 chance that a zero trend is "true" for the period he chooses."
There's a 1 in 20 chance of seeing a bigger slope if the zero trend hypothesis is true. Assessing the probability of the trend being zero (the hypothesis) requires Bayesian priors on how likely zero trends are to start with.
"The larger problem relates to his willingness to apply a trend-stationary model to the data."
It depends whether it's just being used as a descriptive approximation, or if it's claimed to be related to the underlying physics somehow. Taken as an approximate ansatz, he can use what he likes. As you say, it's just a definition. If there's supposed to be a connection to the physics, we need to see the argument supporting that.
"While I have not tested HadCrut4 directly, I am moderately certain that if I were to repeat the tests of auto-correlation and integration on this series it would produce a similar result to HadCrut3, since the unit root arises from the longer wavelength (multidecadal) character."
I would expect so too. But I don't know if testing for trend-stationarity with break points would give the same results as testing for stationarity. There is always a risk of circularity with this sort of testing where you use a model to estimate the trend and residuals, and then test the residuals to confirm that the model is correct. And similarly, if you decide what hypothesis to test based on looking at the data and seeing what shape it has (e.g. why consider just one breakpoint?) it again affects the confidence intervals.
Zivot-Andrews I know takes that sort of thing into account. I'm not sure if an ad hoc ADF test would do so.
"My best guess at the moment is that Ross has either applied Dickey-Fuller to long-lag auto-correlated data, which would be wrong, or he has applied Augmented Dickey-Fuller to a shortened interval of the data, which would also be wrong. I can see no other way that he could reject a unit root for HadCrut4 with such certainty."
He said in an earlier comment on here that he applied Zivot-Andrews. He cites the numbers, so I'm sure he has. But he also said that it was in the code, but like you, I couldn't find it.
Zivot-Andrews uses a specific set of unit root series as its null hypothesis, so rejecting the null is indeed positive evidence against that particular form of unit root. The Zivot-Andrews null (I think) is an ARIMA(p,1,q) series plus a piece-wise linear function, where the breakpoint is set at the point where the ADF statistic gives a minimum. You can test against a break in level, gradient, or both. And you can set the maximum number of lagged terms to consider, so I guess that sets a limit on the maximum p and q considered. (I'm no expert at this, so I'm not sure.)
I don't know exactly what models have been rejected. But in any case Zivot-Andrews doesn't include every possible non-stationary series (in particular, it doesn't include long-term persistence), so I don't think you can reject non-stationarity in general.
I'm guessing that Ross might be using the test based on the black box summary "This tests for unit roots" and maybe hasn't looked closely at the details of what that means. I don't for a second think he's being deliberately deceptive. But I think it's possible that he's mistaken, genuinely believing something on the basis of a statistical test he's done.
"In summary, I don't really care about minor errors in the estimation of the length of the hiatus. I do care about someone in the future basing an important inference on a trend-stationary model of temperature and citing Ross's paper as a justification for so doing."
Nobody should do that without checking Ross's working (or at least checking that somebody else has). If they do, that's their own fault.
--
Doug,
If you keep on using that tone, I don't expect Ross will be inclined to cooperate with you sorting this out. People sometimes make mistakes, and are firmly and honestly convinced that they're right and the other guy is wrong even when they're not - as you very well know. Taking the most charitable interpretation is to be preferred until there's absolutely no alternative.
Ross isn't the sort to deceive. I wouldn't even make any concrete assertions that he's mistaken until I know exactly what he did and what his full arguments are, and we've gone through all the back-and-forth discussion. That's a dangerous game to play with partial information and partial understanding.
And I doubt that seeing the test would on its own prove the paper ill-founded. I expect the null was rejected, just as he says, and it's been accidentally left out of the online code because at some stage he's done some tidying up.
Just ask your questions, and leave out all the editorial asides. They're not helping you or your case.
Doug Keenan,
If Ross carried out the unit root tests on the monthly series rather than annual, yes, it could make a huge difference. There are known problems with using seasonally adjusted data in ADF. I am going to do some reading.
I agree with NiV regarding your tone. It's not helping anything.
For unit-root models of seasonal data, here are two references.
Dickey D.A. (2010), “Normally distributed seasonal unit root tests”, Economic Time Series (editors—W.R. Bell et al.) chap.16 (Taylor & Francis).
Ghysels E., Osborn D.R. (2001), Econometric Analysis of Seasonal Time Series (Cambridge).
"tone"
lol
time to get out more, into the real world, sissies !
[snip- manners]
ptw, if I wanted to read comments like yours, I'd go to CIF at the Guardian.
ossec
good for you!
please keep us update with your interesting surfing habits.
academics are a waste product of our society. they are digested through and through and waiting for an exit and to be dumped somewhere. The closest analogy to them is to be found in mammal digestion.
education? give me a break it is the least of their worries for the past 50years.
r&d? they are an abomination a skewering of the market place a taxparadise for multinationals an example of crony capitalism.
value for money? well bring them to account , they can presumably make more money if people VOLUNTARILY pay for their socalled services.
Dickey–Fuller tests on the global temperature series have been reported in several papers, including two cited (and linked) in my post: Woodward & Gray [1995] and Breusch & Vahid [2011]. The reported tests that I have seen do not reject a unit root. This is the opposite of McKitrick [2014]. Hence, I e-mailed Ross McKitrick to ask for the R code that he used to do the tests. He replied as follows.
I pointed out to him that the data series (HadCRUT etc.) are monthly and that the R functions he was using did not work with seasonal data. He had little to say in reply.
The problem of seasonality also presumably applies to McKitrick’s claim, in a comment above, to have done Zivot–Andrews tests.
Doug Keenan,
Thanks for this. Yes, Ross should have applied a t-12 model to transform the unadjusted monthly data before applying any test for a unit root. If he had done so, and if he could still show rejection of a unit root, it would have offered some plausible support for trend-stationarity. Even if he had done so, however, I would remain unconvinced in this instance that he had confirmed the validity of his statistical model. There are numerous papers which compare tests of integration order on monthly, quarterly and annual datasets. On real data series, tests on monthly data will always have a higher likelihood of rejecting a unit root. Using synthetic data, if the annual data is a simple aggregation of monthly data generated by a unit root process, then a test on the monthly series, carried out with the correct level of augmentation, should have a higher statistical power than a test on the annual data. Conversely, however, if there is structure in the interannual data which cannot be captured by a reasonable number of augmentation lags in the monthly data, then a test on the monthly data is likely to lead to false rejection of a unit root. For this reason, several authors recommend carrying out tests with varied data spans (and augmentation lag) to test integration order.
The specific problem here is that the temperature series has multidecadal structure which cannot be reproduced from a monthly-based generating function or process (in fact it is generated by a physical process which common sense tells us must be quite distinct from the controls on intra-annual variation) , and which cannot possibly be captured in any reasonable noise model derived from monthly data. In practice, the noise level from the intra-annual monthly data just masks and downgrades the importance of the multidecadal structural variation, leading to a test of dubious validity.
The structure CAN be captured quite well by a difference model. I have checked some realizations of a fitted ARIMA(0,1,2) model and interestingly, they show a consistent multidecade flatspot from the turn of the century.
My somewhat paradoxical position is that I do not believe that the temperature series has a unit root, but I do believe that any valid test on the series that does not deal explicitly with the multidecadal structure should fail to reject a unit root, since such test cannot distinguish between a unit root process and a regular superposed oscillatory cycle. The series is NOT trend-stationary in the narrow sense that Ross needs to justify his trend-plus-noise model, but it can be made trend-stationary in the broadest sense in at least two ways.
We can fit a low order (secular) model plus two fitted sine cycles of periodicity ca 65 and ca 22 years. This leaves a red noise model (with no unit root), which can be whitened with 2 or 3 AR terms. It is a trend-stationary model (but only) in the broadest sense.
We can consider an alternative model which makes use of the strong negative correlation in the series at times (t-32.5 years) and (t-11 years) - the half-cycles of the quasi-sinusoidal oscillations. The original series can be transformed by replacing each term by itself PLUS a weighted average of a few values centred around t-32.5 and t-11 such that the weights add up to 1 for each period. The transformed series lends itself to a low order (approximately quadratic) fit plus red noise terms. Importantly, the full composite model does not integrate error terms (i.e. there is no unit root introduced by the methodology).
Both of the above "trend-stationary" models predict a flat spot starting round the turn of the century, and continuing with some variation for about 30 years. Neither of these models allows Ross to assume a noise model appropriate to narrow-sense trend-stationarity, IMO.
@ Paul_K, Oct 13 at 4:34 PM
Yes, McKitrick [2014] botched the Dickey–Feller tests. Dickey–Feller tests have been done by many other authors: McKitrick should have known that, realized that he was getting results that were the opposite of those other authors, and then tried to find out why.
About a cycle of 65 years, the evidence is far from sufficient. The apparent cycle might just be due to slowly-decaying autocorrelations. A cycle adds three parameters to the model (frequency, amplitude, offset), and thus should be avoided unless there is a strong case. With so many parameters, we could model almost anything.
Doug Keenan,
The evidence for what exactly? Predictable recurrence?
Long term local temperature records show a repeat cycle - testable by goodness-of-fit.
MSL measurements going back to 1700 show a repeat cycle (See Jevrejeva et al 2008).
High resolution proxy records show a repeat cycle.
http://www.nature.com/ncomms/journal/v2/n2/full/ncomms1186.html
Slowly decaying autocorrelation?
Doug Keenan:
To summarize Doug, we should accept unit-root as the preferred model, which has absolutely no basis in physical reality, whilst rejecting oscillatory behavior, which we know to exist.
Paul_K thanks for the references. 8,000 years of quasi-oscillatory behavior represents "slowly decaying autocorrelation" indeed.
@ Paul_K, 5:27 AM
I did not know about the paper by Knudsen et al. [Nature, 2011]. Knudsen et al., however, provide only weak evidence for a cycle: a spectrogram. The difficulty here is that autocorrelated processes naturally give rise to apparent cycles. Economists call that the “Slutsky–Yule effect”.
Mistaking the Slutsky–Yule effect for a cycle is what led to the term “Pacific Decadal Oscillation”. For a really good discussion of this, see Roe [Annu.Rev.EarthPlanet.Sci., 2009]. As Roe puts it, “the PDO should be characterized neither as decadal nor as an oscillation”.
(Perhaps interesting is that the length of the cycle claimed by Knudsen et al. is about 2π times the length of the solar cycle.)
@ Paul_K, Carrick
By “slowly decaying autocorrelations”, I was referring to processes such as fractional Gaussian noise. With fGn, the autocorrelations decay polynomially fast; in contrast, with AR(p), the autocorrelations decay exponentially fast.
An fGn model has been proposed for many climatic time series, largely on the basis of statistical analyses: there is substantial literature on this. An fGn model has also been argued to be induced by thermodynamic considerations: see Koutsoyiannis [PhysicaA, 2011].
@Doug Keenan
Your comment represents a fairly serious understatement of what Knudsen et al actually did, and I would politely suggest that you read the paper with a little more care.
They found significance at p<.01 against a null that these cycles were generated by an AR(1) process. (See here for a description of REDFIT:- http://www.manfredmudelsee.com/publ/pdf/redfit.pdf )
While this does not entirely rule out a more exotic stochastic process being in play, such as FARIMA, the argument becomes strictly academic. If there is something that produces a quasi-sinusoidal cycle for 8000 years, then, in context, does it matter if it is modeled as a deterministic process with some stochastic variation or as a fully stochastic process? The latter for credibility must in any event be able to reproduce the recurrent quasi-65 year cycle in terms of phasing, amplitude and periodicity to a reasonable approximation, and the space of acceptable models is bounded by the length of the series and the strength of its signal in the power spectrum. We know that these cycles did not stop in 1850, because we have the modern instrumental temperature series to demonstrate their continuation - after thousands of years. It is a pretty safe bet that these cycles didn't suddenly stop recently to be replaced by something that still quacks like exactly the same duck.
To go beyond what Paul_K said, there is a physical basis for the presence of quasiperiodic motion in climate systems.
Given a choice between a physically unmotivated statistical model and an equivalent model based on actual physical processes, obviously the latter is preferred as having explanatory value (whereas the purely statistical model, especially those that are physically unrealizable, have none).
A few points to clarify….
I have never advocated adopting any particular statistical model. Rather, I have repeatedly stated that I do not know what model, or class of models, to select.
There might be a physical basis for an integrated model. This is mentioned in my post, citing Kaufmann et al. [2013]. (I think that I recall there is another plausible basis as well, but I am not certain.)
A statistical model does not need to be physically realistic. This is discussed in my post: see the Excursus.
Please consider this quote from Roe [2009]: “half of the variance in the spectrum occurs at periods ... that are 2π times longer than the physical response time”. Could there be something in the system with a physical response time of 11 years? If so, then that gives an apparent cycle with a period of 69 years—which is consistent with what Knudsen et al. [2011] found. I find this intriguing, because the solar cycle is 11 years. I have searched for some way to connect all this, but did not get anywhere; perhaps this is a red herring.
Note that I am not saying that the 65-year cycle is not real. I am only saying that the evidence for such a cycle does not seem to be conclusive—especially without a direct physical basis.
The AR(1) simulations done by Knudsen et al. are surely inadequate. I would like to see some ARFIMA simulations. I do not, though, know of a reliable program to do ARFIMA simulations. I do have a program for fGn simulations. I have now run some fGn simulations mixed with an 11-year cyclical input. There were no longer-period cycles present. This strengthens the argument of Knudsen et al.
+1
another Keenan post bites the dust
Above, we discussed how McKitrick [2014] botched the Dickey–Fuller tests, due to not considering seasonality. That lead me to realize something.
McKitrick [2014] just applies the method of Vogelsang & Fanses [2005] to temperature series—that is essentially all the paper does. The temperature series used are monthly. The method of VF, however, does not seem to work with seasonal data. I have now confirmed that with Tim Vogelsang: Vogelsang said (via e-mail), “Franses and I have not discussed extending the VF approach for seasonal data”.
This is another, independent, reason to reject the calculations of McKitrick [2014].
cmon guys your not gonna tell me McCritkric did not do a fuller test but seasoned his data instead??
statistics 101 my granny does better
chuckinthebin the paper
paulK,diogenes,carriage and co to aplogize , then go to prison ..worse: off with the long indefinite entitlements phd professorships whatsoforth
Thethird commenter above states:
"In any mathematical model, you have to make assumptions."
Perhaps in a statistical model, but if you are making assumptions you don't have a mathematical model you have a hypothesis.