I’ve generally taken a pretty dim view of focus groups, which seem to be used mostly to detect and amplify unthinking prejudices. But, I have to admit, that probably is due at least in part to the fact that my prejudices aren’t very close to those of the median focus group participant. So, it wasn’t until I saw focus group results matching my own thoughts that I paid any attention to the question of whether the results actually meant anything.
According to the SMH report, participants in two Ipsos focus groups almost uniformly saw Morrison’s response to the bushfires as “pathetic” and “lacking empathy”. In each case, eight of nine respondents gave such negative views. Ipsos also concluded there was “little confidence” Labor leader Anthony Albanese would have provided better leadership, with descriptions like “weak” and “bland” being offered. Regular readers won’t be surprised to learn that I agree with both judgements.
But how much can we learn from an exercise involving only eighteen participants? Surprisingly, the answer is, quite a lot. Use q to denote the proportion of the population who approve of Morrison
If the population from which the group was drawn was evenly divided between approvers and disapprovers, the chance of getting results like this would be tiny. This table (look at N=18, x = 16, p 0.5) gives it at 0.000584. A classical hypothesis test would reject the null hypothesis (or, in the usual jargon, find a statistically significant effect) for any chance below 0.05. Using a more sensible, Bayesian approach, with just about any prior distribution, the updated estimate of for the distribution of q would lie entirely below 0.25 (25 per cent approval). We could do something similar for Albanese, but the exact numbers aren’t given.
These numbers are substantially worse than the approval/disapproval numbers given by Newspoll. So what is going on?
First, the focus groups were drawn from Sydney and Melbourne, where Morrison is less popular than in Australia as a whole (though that makes things look even worse for Albanese).
Second, the focus group was not a random sample, though the members were chosen to be representative of the population. With high refusal rates this is also a problem for standard polls, but maybe not as severe.
Third, for both focus groups and standard polls, the pollsters may ask questions in ways that encourage particular ways of framing the issues, and therefore encourage particular respondents. My guess is that the poll setting encourages people to answer more in line with their party preference, which limits extreme results like those in the focus group. To spell this out, Liberal voters might be critical of Morrison in a focus group discussion, but still tell a pollster they approve of his performance.
Finally, the focus group answers emerge from discussion and may therefore be subject to ‘groupthink’ effects. With an election two years away, it’s hard to know whether this is a plus or a minus. If a negative view of Morrison is widespread, people who were initially on the fence, and not closely interested in politics, are likely to be exposed to it over time.
That’s a lot of analysis for one little focus group. But it’s better than trying to read the tealeaves from my Twitter feed which is, of course, uniformly hostile to Morrison and mostly unenthused by Albo.
John Quiggin 
But I think a frequentist would compute the p-value as P(X >= 16 | n=18, p = 0.5) = 0.000656. Very little difference I admit.
Even if focus groups statistics are dubious, the results if widely reported can frame the narrative. The reporting of the Morrison bushfire group was about how unpopular he is except among hard core Liberal supporters. This might start a dynamic where people think he must be unpopular or a reason, and then he’ll actually become unpopular.
John, on an unrelated but nonetheless important topic, I am disappointed to learn that Japan is pushing ahead with building 22 new coal fired plants. I hoped that Japan would be at the forefront of decarbonising its economy. It’s one of the wealthiest and most advanced countries in the world and yet it continues to rely on coal. Does Japan have an ETS? I know Japanese politics is notoriously beholden to vested interests. I do hope that this galvanises the local environmental movement, which, for very good reasons, has been focussed on closing nuclear power plants. Surely the disaster at Fukushima presents Japan with an opportunity to rebuild its energy along more sustainable lines. Japan has always been short of fuel and this was one of the reasons it launched a disastrous war in the Pacific. Surely renewables present Japan with the opportunity to become energy independent?
Groupthink is fascinating. In the last election there was (on social media) as consistent theme of “I don’t like Bill Shorten”. Amongst Fremantle Football Club supporters last year there was a consistent theme of “I don’t like Ross Lyon, and he’s lost the players”. Now after Ross had been sacked and there was no reason to be nice about him, the players broke into spontaneous applause when Ross was mentioned at a club function. So, no, I don’t think Ross lost the players. And I never understood the supposed dislike for “that grub Shorten”. I never met him but based on his public appearances always thought he seemed like a nice bloke. His performance in the Beaconsfield mine rescue was epic, and I always admired his “Well, what she said then” retort when he was told that he’d contradicted his then leader, Julia Gillard.
So there seems to be a bit of magic involved in creating groupthink, and it seems the plitical neanderthals are better at it.
Trump’ s approval rating has improved and Republicans now have substantially more support than the Democrats, according to Gallup.
The Democrat contenders to take on Trump are about as inspiring as soggy rice crackers, so I’m currently looking for the silver lining in 4 more Trump years. I haven’t spotted it yet but it must be there somewhere.
Focus groups are paid. There is an obligation to take the work seriously, with the risk of not being hired again if you don’t.
“Focus groups are paid. There is awith n obligation to take the work seriously…”. Taking the work seriously means, for many people, being taken seriously, which means fitting in with any group consensus.
The strong tendency to groupthink makes statistical testing, Bayesian or frequentist, invalid because the experience of the focus group itself is changing people’s expressed preferences.You might in fact test this by doing conventional polling before their particpation and comparing it with after.
I agree with Derrida. Besides, in a group setting people may self censor and put on their social mask.