The statistical significance of focus groups

I’ve generally taken a pretty dim view of focus groups, which seem to be used mostly to detect and amplify unthinking prejudices. But, I have to admit, that probably is due at least in part to the fact that my prejudices aren’t very close to those of the median focus group participant. So, it wasn’t until I saw focus group results matching my own thoughts that I paid any attention to the question of whether the results actually meant anything.

According to the SMH report, participants in two Ipsos focus groups almost uniformly saw Morrison’s response to the bushfires as “pathetic” and “lacking empathy”. In each case, eight of nine respondents gave such negative views. Ipsos also concluded there was “little confidence” Labor leader Anthony Albanese would have provided better leadership, with descriptions like “weak” and “bland” being offered. Regular readers won’t be surprised to learn that I agree with both judgements.

But how much can we learn from an exercise involving only eighteen participants? Surprisingly, the answer is, quite a lot. Use q to denote the proportion of the population who approve of Morrison

If the population from which the group was drawn was evenly divided between approvers and disapprovers, the chance of getting results like this would be tiny. This table (look at N=18, x = 16, p 0.5) gives it at 0.000584. A classical hypothesis test would reject the null hypothesis (or, in the usual jargon, find a statistically significant effect) for any chance below 0.05. Using a more sensible, Bayesian approach, with just about any prior distribution, the updated estimate of for the distribution of q would lie entirely below 0.25 (25 per cent approval). We could do something similar for Albanese, but the exact numbers aren’t given.

These numbers are substantially worse than the approval/disapproval numbers given by Newspoll. So what is going on?

Read More »

More news from the apocalypse (crosspost from Crooked Timber)

I’m still writing furiously (in both senses of the word) about climate change, the fire disaster in Australia and the responsibility the entire political right bears for this catastrophe, along with those of the centre and left who have shirked the struggle. Australian writer Richard Flanagan, in the New York Times, has compared our leaders to famous traitors like Benedict Arnold, Vidkun Quisling and Mir Jafar, and that’s a pretty good summary of how large numbers of Australians feel.

Over the fold, links to some of my latest commentary

An Open Letter on Australian Bushfires and Climate: Urgent Need for Deep Cuts in Carbon Emissions from 80 current and former Australian Laureate Fellows (our most prestigious research award, across natural and social sciences and humanities).

Humans are good at thinking their way out of problems – but climate change is outfoxing us (The Conversation)

Invest with the best, Inside Story (the case for divestment)

Neoliberalism is declining, but the Right wing refuses to die

Libertarians Can’t Save the Planet

As promised, my article on climate change and the death of libertarianism/propertarianism, in Jacobin.


Global warming is the ultimate refutation of Lockean propertarianism. No one can pump greenhouse gases into the atmosphere while leaving “enough and as good” for everyone else. It has taken thirty years, but this undeniable fact has finally killed the propertarian movement in the United States.

Australia Day

There have been Australia Days more loaded with fear and foreboding than yesterday’s (in 1942, for example), but not many. Richard Flanagan’s jeremiad in the New York Times might be overstated, but it’s a lot closer to the mark than our appalling Prime Minister’s suggestion that we should take the time to celebrate our greatness as a nation.