Boneheaded stupidity / Environment

Statistical significance

John Quiggin27 Comments

I know I should just ignore the Oz, but faced with its continuous campaign to promote innumeracy, cheered on by the likes of Alan Jones and Andrew Bolt, I can’t help but try to set things straight. We’ve seen on many occasions that nearly all “sceptics” either misrepresent of misunderstand the concept of statistical significance, assuming it to correspond to the ordinary meaning of “significant”. The classic example is the Lindzen talking point, made in 2008 that “there has been no statistically significant warming since 1995”. As everyone who understands statistical signifance (notably including Phil Jones, who gave an accurate response and saw a distorted version of his words become a delusionist meme), that’s because statistical significance depends on sample size. Roughly speaking, to see a significant upward trend in a noisy time series, the trend, multiplied by the number of years of data, needs to be about twice the standard deviation of the random variation about trend. So, if you have an upward trend of 0.015 degrees per year, and a standard deviation of 0.1 (these are estimates, but feel free to check)) you typically need 14 or 15 years of data to see a statistically significant trend. Over shorter periods, it’s easy to eyeball a pause or decline, as this graph from Skeptical Science shows.

Lindzen obviously knew this, and it was easy to check that he could go back 13 years from 2008 (but no further) without finding a statistically significant trend. He also knew that, given a few more years of data, the trend for the period since 1995 would be statistically significant, but correctly assumed that no-one on the delusionist side would know or care. Now, the Oz has this, from Michael Asten, professor of geophysics at Monash University. It’s worded carefully enough for me to think he knows he’s pulling the same swifty as Lindzen, but it’s hard to tell for sure[1]

Global temperatures have not increased in a statistically significant sense in the past 15 years. A pause of 10 years in the upward trend of the past 40 years would be unsurprising from existing models. A pause of 20 years would definitely surprise. Changes across the next five years will be watched closely.

As you would expect, Asten has to move Lindzen’s goalposts forward by a couple of years, to an implied starting date of 1997. Note also that he slides from “no statistically significant trend” to “a pause”. What can we say about this? In one sense he is right. As I’ve said, we need about 15 years of data to get a statistically significant trend, so we wouldn’t expect to find one with 10 years, and we would usually expect to find one with 20 years. But, of course, that number itself is variable. Asten is repeating basic facts about time series, in a way that would lead unwary or gullible readers (the vast majority, given the outlet) to suppose that recent evidence casts doubt on the observed warming trend. The only thing that’s hard to figure here is whether he is fooling himself as well as his readers.

fn1. (Lindzen himself often slipped from “no statistically significant warming” to “no warming” either out of sloppiness or because he thought no one was looking.

27 thoughts on “Statistical significance”

  1. John

    Do you think this is the same Michael Asten with a different hat?

    http://www.flagstaff-geoconsultants.com.au/mikeCV.htm whose principals include one H. Rutter

    compare

    http://www.monash.edu.au/research/profiles/profile.html?sid=1645&pid=3001 where a coauthor is one H. Rutter

    If so it doesnt look like he has any expertise in climatology to judge by the publication record but does have plenty of potential interest conflicts (e.g. Coal mining) and some history of public climate skepticism to judge by the article titles – reminds me a bit of Ian Plimer.

  2. The additional problem is that journalists tend either not to understand or willfully misunderstand statistics. A classic example is the way they misuse opinion polling either by using loaded questions or calling significant trend that is within the sampling error.

  3. Grrrr! Why do geologists keep coming out with this stuff. As a geologist, it is frustrating. I have cancelled my Australian Institute of Geology membership because of the continual representation by its members for this shoddy climate skepticism.

  7. @el gordo

    He used to label that curve “for entertainment purposes”, and now presents it without any qualification. For some real entertainment, look at the fit back to, say, 1800.

    Sad to say, Spencer is evidence that crazy ideas wrt one part of science (he’s a creationist) are likely to be associated with bad practice in another.

  9. It takes a lot of hard work to do the science or follow the science in any given field. Denialists and myth makers range at will over complex subjects and invent any nonsense they like. Modern denialists live, eat, drink and spread ignorance by free-riding on science and all its advances.

  11. This seems to be based on a scurrilous article from the UK that did the “no warming since 1997” story a couple of days ago. In that article they had a graph, and had conveniently truncated the first part of 1997, as that would have made the overall temperature trend look positive…

  15. WHen I studied statistics as an undergraduate the basic rule of thumb I learned was to be cautious about drawing conclusions from any sample smaller than 30 cases. Also, any undergraduate statistics textbook will include a table showing the relationship between the size of the t statistic required to infer statistical significance and the sample size (as the sample size increases the required magnitude of the t statistic decreases, although not in a linear way). The likes of Lindzen and Asten could not have got to their current positions without encountering such an undergraduate textbook and achieving at least a pass when examined on its contents, which makes their reported statements much more egregious than their uncritical repetition by (frequently) innumerate journalists and commentators.

  17. nice summary at http://en.wikipedia.org/wiki/Misuse_of_statistics

    1.1 Discarding unfavorable data
    1.2 Loaded questions
    1.3 Overgeneralization
    1.4 Biased samples
    1.5 Misreporting or misunderstanding of estimated error
    1.6 False causality
    1.7 Proof of the null hypothesis
    1.8 Data dredging
    1.9 Data manipulation
    1.10 Non-enduring class fallacies
    1.11 Other fallacies

  18. @John Quiggin
    Thanks for that John, I had a chat with others since then and it was generally agreed we didn’t understand the poly fit science.

    Creationism aside, I’m more sceptical but will keep a watchful eye open.

  20. fn1. (Lindzen himself often slipped from “no statistically significant warming” to “no warming” either out of sloppiness or because he thought no one was looking.

    Still plenty saying “no warming” at delusion central.

  21. @el gordo

    Actually I may have been a little rushed reducing the link between creation science and climate change denial to simple irrationality.

    Jeremy Leggett a geologist of sorts by original training and now a noted alternative energy promoter, in his higher profile days commented (in his book Carbon War around 2000 or in a lecture here – I cant remember which) when meeting some of these bible literalist oil executives observed that they werent actually climate change denialist but more of the belief it didnt matter because we were heading toward the end of days.

    Amusingly in the same lecture Leggett outlined his thoughts on the true cost of oil and how if you took military support costs (this is pre 11 Sept 01 developments) the real cost of oil would be a crippling $50 per barrel??!!

    I think I remember this as it is a good example of progressives like Leggett fearing the worst and explaining what simple modelling was telling them – and then arguments over statistics and modelling getting overtaken by reality – at which point rather than admiting that the arguments of progressives back to Ehrlich and the LTG MIT bunch in the modern era and others like Malthus in the more distant past were right – a collective amnesia sets in – partly promoted by propagandists – and as well, as annoying as Lindzen is, something else.

    For many of us older folk the present is a little weird because we remember the predictions and statistics from 30 to 40 years ago and, though civilization hasnt collapsed, all the predicted resource, and environmental impact trends that are now being borne out with a little delay arising from technology improvements/adaptation. Even the denialism is nothing new – especially for those who remember a quasi documentary satire on climate change- Alternative 3 – http://en.wikipedia.org/wiki/Alternative_3 timed for April Fools day.

    My worst fear is that in 10 years climate change will have been normalised as just one of those things that happen to poor people in minor places like Coral Atolls and the argument will have shifted from whether climate change is real (by then it will be rationalised as a necessity so that the larger bulk of humanity can prosper) to whether the increasing methane and CO2 emmisions will really produce a second Venus on Earth and the statistical validity of the new warning signs will be coming under question from a new bunch of scientists and journalist with questionable expertise.

    I wonder if prawns similiarly argue about statistical significance of that warm spot at the bottom of their cooking pot?

  22. @Newtownian I am sorry to sat that global warming is part of political theatre made up of the symbols we boo and cheer. People gain pleasure, excitement and self-definition for cheering for particular parties and worthy causes in the same way as they cheer and boo for sports teams.

    Geoffrey Brennan in ‘Climate Change: A Rational Choice Politics View’, Australian Journal of Agricultural and Resource Economics, July 2009 argues that we see many countries acting unilaterally to introduce carbon emission policies because expressive voters cheer for such policies.

    Brennan argues that the nature of expressive concerns is such that significant reductions in real GDP are probably not politically sustainable in the long term. This suggests that much of the CO reduction action will be limited to modest reductions of a largely token character.

    There are many expressive voting concerns that politicians must balance to stay in office and the environment is but one of these. Once climate change policies start to actually become costly, expressive voting support for these policies will fall away.

    Abbott’s big bad new tax rhetoric split away the working class and lower-middle class labour voters who worry more about bread and butter issues.

    The inner city Left’s high incomes allow them to be more indulgent as to what they cheer and boo for at the ballot box. But if you scratch an inner city Left’s voter’s superannuation entitlements, you will find a rather raw hip-pocket voter.

  23. Jim Rose, there are a few fallacies that need to be unpacked in your statements.

    Firstly, while there is obviously a degree of tribalism in support for the science (which is barking mad, and something that started from the far right and wormed it’s way back), there remains the underlying fact that the science itself is apolitical and undeniable.

    Secondly you can talk about the costs of action, but not intelligibly without considering the costs of inaction.

  24. @Jim Rose
    “Brennan argues that the nature of expressive concerns is such that significant reductions in real GDP are probably not politically sustainable in the long term. This suggests that much of the CO reduction action will be limited to modest reductions of a largely token character.”

    Jim

    I agree with wilful. There is the chattering on one hand which is problematic. But on the other hand there is the physical reality of decreasing ecological sustainability of which climate change is just one manifestation – an elephant in a jungle of other large beasts. Limits to Growth is the bigger game here and it is worryingly on track – see TURNER., G. 2008. A comparison of the limits to growth with thirty years of reality [Online].

    As to whether people will never put up with a reduced GDP/superannuation – I thought they already had? -In Australia since 2001/2008 (oil hikes start/GFC gets going) and in the US since Reagan’s ascendancy, hidden by debt (see Robert Reich’s Supercapitalism) and illusory GDP increases (the financial industry and tech bubbles). Maybe all will be fine when we move to Gross National Happiness.

    Nevertheless I do share your concern about the inconsistencies in the behaviour of the wealthy of whatever political shade – in this regard you might enjoy FOX, H. E., KAREIVA, P., SILLIMAN, B., HITT, J., LYTLE, D. A., HALPERN, B. S., HAWKES, C. V., LAWLER, J., NEEL, M. & OLDEN, J. D. 2009. Why do we fly? Ecologists’ sins of emission. Frontiers in Ecology and the Environment, 7, 294-296. It has interesting statistics so it does fit the thread.

  26. A simple statistical analysis of the data illustrated on the graph undoubtedly shows an uptrend over approximately 38 years. However, this mere fact does nothing to either prove or disprove the hypothesis that the earth is warming. First, in the grand scheme of things, 38 years is a mere blip. Secondly, unstated but assumed, is that the measurements from one time period to another are completely comparable. Knowledgeable people have postulated that many existing measuring stations have become greatly urbanized in the past thereby increasing the average measured temperature via the “heat island” effect and that the overall distribution of measuring stations has varied. Lastly, this article does nothing to address the biggest unknown of all, specifically what portion of climate change is due to human activity and what portion is due to natural cyclical phenomena beyond our control.

  27. I couldn’t help but notice the fallacy here. The same critique of the 6 year trend lines could be made of the 30 year trend line.

Leave a comment