Statistical significance

I know I should just ignore the Oz, but faced with its continuous campaign to promote innumeracy, cheered on by the likes of Alan Jones and Andrew Bolt, I can’t help but try to set things straight. We’ve seen on many occasions that nearly all “sceptics” either misrepresent of misunderstand the concept of statistical significance, assuming it to correspond to the ordinary meaning of “significant”. The classic example is the Lindzen talking point, made in 2008 that “there has been no statistically significant warming since 1995”. As everyone who understands statistical signifance (notably including Phil Jones, who gave an accurate response and saw a distorted version of his words become a delusionist meme), that’s because statistical significance depends on sample size. Roughly speaking, to see a significant upward trend in a noisy time series, the trend, multiplied by the number of years of data, needs to be about twice the standard deviation of the random variation about trend. So, if you have an upward trend of 0.015 degrees per year, and a standard deviation of 0.1 (these are estimates, but feel free to check)) you typically need 14 or 15 years of data to see a statistically significant trend. Over shorter periods, it’s easy to eyeball a pause or decline, as this graph from Skeptical Science shows.

Lindzen obviously knew this, and it was easy to check that he could go back 13 years from 2008 (but no further) without finding a statistically significant trend. He also knew that, given a few more years of data, the trend for the period since 1995 would be statistically significant, but correctly assumed that no-one on the delusionist side would know or care. Now, the Oz has this, from Michael Asten, professor of geophysics at Monash University. It’s worded carefully enough for me to think he knows he’s pulling the same swifty as Lindzen, but it’s hard to tell for sure[1]

Global temperatures have not increased in a statistically significant sense in the past 15 years. A pause of 10 years in the upward trend of the past 40 years would be unsurprising from existing models. A pause of 20 years would definitely surprise. Changes across the next five years will be watched closely.

As you would expect, Asten has to move Lindzen’s goalposts forward by a couple of years, to an implied starting date of 1997. Note also that he slides from “no statistically significant trend” to “a pause”. What can we say about this? In one sense he is right. As I’ve said, we need about 15 years of data to get a statistically significant trend, so we wouldn’t expect to find one with 10 years, and we would usually expect to find one with 20 years. But, of course, that number itself is variable. Asten is repeating basic facts about time series, in a way that would lead unwary or gullible readers (the vast majority, given the outlet) to suppose that recent evidence casts doubt on the observed warming trend. The only thing that’s hard to figure here is whether he is fooling himself as well as his readers.

fn1. (Lindzen himself often slipped from “no statistically significant warming” to “no warming” either out of sloppiness or because he thought no one was looking.

27 thoughts on “Statistical significance

  1. A simple statistical analysis of the data illustrated on the graph undoubtedly shows an uptrend over approximately 38 years. However, this mere fact does nothing to either prove or disprove the hypothesis that the earth is warming. First, in the grand scheme of things, 38 years is a mere blip. Secondly, unstated but assumed, is that the measurements from one time period to another are completely comparable. Knowledgeable people have postulated that many existing measuring stations have become greatly urbanized in the past thereby increasing the average measured temperature via the “heat island” effect and that the overall distribution of measuring stations has varied. Lastly, this article does nothing to address the biggest unknown of all, specifically what portion of climate change is due to human activity and what portion is due to natural cyclical phenomena beyond our control.

  2. I couldn’t help but notice the fallacy here. The same critique of the 6 year trend lines could be made of the 30 year trend line.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s