Monday
Mar292010
by 
Bishop Hill
Bishop Hill Visser et al on the divergence problem
Mar 29, 2010 Climate: MWP Climate: Statistics There is a new paper up for open review at Climate of the Past. Visser et al look at the divergence problem and propose a way for dealing with it when calibrating proxies.
One for the statisticians.
Reader Comments (13)
Are any of the authors statisticians?
Rather like declaring a respectable form of corruption, as distinguishing between a 'bribe' and a 'facilitating payment', the former being a criminal offence and the latter an allowable tax deduction?
http://www.ato.gov.au/corporate/content.asp?doc=/content/00104079.htm&page=4
Are any of the authors statisticians?
Somehow I doubt it. Phillip. And while I haven't done statistics for 40 years, and I do not know time series analysis well -- certainly no where as well as VS, I would like to see the data, as well as basic statistics like the variance of the population, or SD. What I found particularly interesting was the following statement:
Here, the “intercept” μ, which is traditionally a constant, is replaced by a slowly bending
trend model μt, the integrated random walk (IRW) model. (top of page 229)
Hey, guys, we use to call that a polynomial. There is MORE than just one factor in effect here. It is also probably not a linear model. They may well have have something here, but a variable is not a constant unless it is constant.
I would like to see the result of any of the Unit Root tests we have all been taught by Professsor VS. My guess just by eyeballing the data on figure 2 that there may be a problem.
As you said, Bishop, this is certainly one for real statisticians to deal with, but I have my gut feeling suspicions.
Maybe someone who knows who VS is will drop him a pointer to this. I would like to see it discussed on Bart's Statistics Class.
Uniformitarianism, the idea that past presages future, reflects the statistical "persistence fallacy" which essentially seeks to limit growth-and-change. This occurs in context of the Principle of Mediocrity, which asserts that any given point in time will probably be near a long-term mean, for in "normal" circumstances regression-to-the-mean is near-definitive.
Despite long periods of relative stability, however, biological, climatic, geophysical processes tend to "punctuated equilibrium" whereby of a sudden regimes flip to radically different modes in s-curve spurts. Dinosaurs' extinction at the Cretaceous/Tertiary (K/T) Boundary is a good example, as are the Pleistocene Era's recurrent periods of 102,000-year continental glaciations interspersed with median 12,250-year remissions such as our current Holocene Interglacial Epoch. Geophysical processes include the Rocky Mountain orogeny, which converted shallow inland seas to towering massifs, perhaps Yellowstone super-volcano episodes at intervals of some 630,000 years.
The point is that modeling Amazon rainforest conditions absent context and perspective ignores not only the long-term adaptive capacity of large-scale ecosystems, which "persist" indefinitely until circumstances change, but cannot in principle define looming "tipping points" any more than geologists can predict earthquakes or volcanic eruptions-- not even the Rocky Mountains, for that matter. General environmental principles endorsing clean air and water, conservation of resources, species preservation are worthy ends; but only insofar as one realizes that Nature cares nothing for such preconceptions. Chixculub objects will set Earth's atmosphere aflame, radically pollute planetary environments for millennia (see India's Deccan Traps), extinguish up to 98% of land- and sea-life as in the Permian die-off.
Though someone always wins this devastating lottery, existing at the wrong place and time, chances are that any given generation is exempt. Are we?-- as Earth enters on a 20-year "dead sun" Dalton if not a 70-year Maunder Minimum, the Holocene Interglacial is statistically past-due to end by up to 1,500 years. All one can say is that treating Amazon jungles as fragile ecosystems subject to human whim reflects a profound ignorance of abstract principle, descriptive but not by any means prescriptive of the natural world.
Also notice how they do not compare their results with those one can obtain using simpler methods that have been proposed to look at instabilities in statistical relationships between climate and tree-ring chronologies. Typical academic strategy: make it sound like your method is great, without really comparing it in practice to the existing ones.
Reads to me like an academic potboiler. They got some new software to do even more adjustable fits (by allowing time-dependence in coefficients linking tree ring measurements to environmental variables) - with a commercial placement in the piece by way of thanks - and used it a bit before decorating their tinkerings with the usual torrent of references. Inconsequential I'd say.
The interesting stuff is their choice of language such as 'the so-called medieval warm period', '..DP ...co-occurring with global warming as well as anthropogenic-induced changes in atmospheric composition' ( I wonder how this would have been spun for a summary for policy-makers by the IPCC core-team? Maybe: 'man-made global warming could well be leading to erratic growth patterns in trees due to our contamination of the atmosphere')
The very phrase 'divergence problem' (DP) is evocative. To whom is it a problem? To whom is it an interesting feature? It is only a problem for those desperate to create hockey-sticks!
Here's another interesting extract: 'given the high relevance of climate reconstructions for policy-related studies ... it is important... to address this issue of potential model uncertainties associated with the DP..' I like the 'potential' bit! I think I would have preferred to have seen something like this: 'foolish people are ignoring the primitive state of climatology and using it to make policy recommendations which have been taken seriously by legislators'.
My conclusions? The data are messy, the science is thin, the speculations are rife, and multivariate statistical methods provide an almost unconstrained playground for mischief-makers.
Looks like a form of voodoo statistics designed to turn bad proxies into good proxies. No doubt useful for the production of hockey sticks.
The basic statistical model proposed here (drift in regression coefficients) is sound and represents one of the three possible solutions which i have considered as possible resolutions to the divergence problem. My primary concerns are
(1) it is not clear to me how they choose the hyperparameters which govern the random walk behaviour of the alpha parameters. The do state that they obtained noise variances using maximum likelihood estimation, but this makes it sound like their model assumed no underlying dynamics on the alpha parameters other than successively increasing noise addition. Such a model seems likely to rely overly heavily on data points which occur early on in the time series.
(2) Maximum a posteriori (MAP) estimation of the alpha and mu parameters is an inappropriate measure of model predictive power. In principle they don't just have error bars on their alpha and mu parameters, but full posterior distributions. When this is the case, you can average over your uncertainty to produce not just MAP estimates of your reconstructions but posterior distributions over your reconstructions. My guess is that given the size of the error bars on the alpha parameters, the error bars on the reconstructions (IRW plus trend in figs 3 and 4) will swamp the observed variability.
(3) Using drift diffusion models to model the lack of persistence while useful in some contexts does not really eliminate the assumption of stationarity. The random walk behaviour in the parameters which this model assumes constrains the parameters of alpha to change slowly. In at least one defence of the 'nature trick' it was suggested that there was an abrupt change in the relationship between the proxies and temperature around 1960. Drift models such as these are particularly bad at detecting such changes as they have the effect of smoothing them out. A point process change model is more appropriate.
I should add:
Conclusion (2) is incoherent. The authors state: 'Stochastic response functions can be used to test whether truncation of the calibration period might be a useful means of including a given proxy record in a reconstruction that did not pass a screening procedure otherwise.'
Data is data and should never be thrown out. A good statistical model which tell you certain data is more or less irrelevant for the purposes of some inferences and will 'ignore' it for you by, for example, making its regression weight zero where appropriate. This is the 'Let the Data speak' mantra of the Bayesian. If you find yourself rationalizing doing this by hand (even as a result of some statistical test) you will almost surely be artificially inflating the significance of your results. This is great for publishing but crap for science.
Conclusion 3 on the other hand is sound, but doesn't follow from the results they presented. The primary utility of this kind of approach is that (done properly) it gives you an estimate of the length of time for which a given proxy might be useful. But looking at these results it I would have concluded noise dominates and these proxies are rather useless for reconstructions on the relevant time-scales.
The basic statistical model proposed here (drift in regression coefficients) is sound and represents one of the three possible solutions which i have considered as possible resolutions to the divergence problem. My primary concerns are
My concern is they are pretending a non-linear phenomenon is linear and are "fitting" with voodoo statistics. Constants do NOT drift. The linear regression (to a linear model) is not appropriate if that is what they have to do.
I use to handle these sorts of data sets with Polynomial Regression. Been around for ages and will fit any sort of non-linear data. It still uses the assumptions of a linear regression and so a test for a unit root is appropriate find out if the data are non-stationary, as I suspect they are.
http://en.wikipedia.org/wiki/Polynomial_regression
The trouble with polynomial regression is that it basically says that there is more than one factor at play, and while you only have the value of x to calculate the coefficients, you do get more than one coefficient and that leads to the assumption of multiple factors, which would be AGW politically incorrect.
So why is nobody posting any comments on the journal web site? Are you just mouthing off?
http://www.clim-past-discuss.net/6/225/2010/cpd-6-225-2010-discussion.html
Doesn't appeal to me, maria. I had a look round the site, and the rather cumbersome process of recording comments as pdf files. Comments are only allowed from registered users, as in: 'Short Comments can be submitted by every registered member of the scientific community'. I note this one in particular:
'I had made the foolish assumption that this discussion board was an opportunity to discuss the paper with the authors. However, they seem to be curiously unwilling to respond to my questions.'
'If they are not willing to answer questions...what is the point of having a discussion forum at all?'
Source: http://www.clim-past-discuss.net/2/1001/2006/cpd-2-1001-2006-discussion.html
I suspect the point of having a discussion forum is to be able to say they have a discussion forum. The PR skills of the alarmist camp are not to be despised.
Can you tell us anything about their funding, maria?
@Don Pablo de la Sierra
it doesn't seem to me that what they are doing is 'pretending a non-linear phenomenon is linear'. The model is a bi-linear (a kind of second order polynomial non-linearity), latent variable model. It simply assumes that some 'hidden process' might be slowly altering the relationship between temperature and their proxies (like rainfall for example). This is not a bad idea, nor is it 'voodoo'. Quite the opposite, the assumption that regression coefficients are constants (even the regression coefficients of a polynomial regression) is a particularly limiting assumption and one which is likely to be false. A good statistical model should take this into account.
True, inferring the effects of unobserved quantities increases the flexibility of your model, but comes at what some statistical modellers consider a cost: it also increases the error bars on your predictions. But to my mind this is a good thing. Assuming (incorrectly) that your regression coefficients are constants will result in a model which is overconfident in terms of its ability to predict. A statistical model which allows for them to change over time will be (i would argue appropriately) less confident about its predictions of past temperatures and can in principle tell you something about how far back you should trust this particular proxy. Better models of uncertainty are always a good thing.
Now this was my concern with the work presented here. It was not clear that they bothered to propagate their error bars on their hidden parameters (mu and alpha) into their predictions.