Signal and noise

Not surprisingly, given the mini-crisis surrounding Howard’s leadership, the Newspoll released yesterday, with the two party preferred position for Labor shifting from 59-41 to 55-45 was big news. But was there really any news, or just the usual random fluctuation. The ABC News on Monday night made it sound huge, describing it as the government cutting Labor’s lead by 8 per cent. On the other hand, if you started with the view that the true position was 57-43, neither this poll nor the last one would lead you to change this view. Two percentage points is well inside the usual allowance for sampling error in polls with a typical sample size of a bit over 1000. So, you might say, it’s all just a beatup.

Comparing the two polls gives a somewhat different answer, which my son Daniel kindly computed for me. Given two polls with 1000 respondents, and the stated results, you can reject, at the 5 per cent confidence level but not at the 1 per cent level, the null hypothesis that they are from the same population. That’s because, given no underlying change in the population, the chance of two variations in opposite directions is smaller than the chance of each variation considered separately.

But that doesn’t end the problems. If you run polls every fortnight, you’re bound, sooner or later to get two samples in a row that deviate in opposite directions, producing a big apparent swing.

FWIW, looking at the Newspoll results since July, I see no reason to think there’s been any real movement in either direction. Aggregating all the polls, and reducing the sampling error accordingly, I’d put the Newspoll 2PP vote somewhere in the range 56-57. The other polls seem to be much the same.

The real problem, though, is non-sampling error. If the question being asked doesn’t match up to the way people are actually going to vote, all of these statistical considerations count for nothing.

10 thoughts on “Signal and noise

  1. It’s remarkable really, how little the above issue is covered in the MSM. Of course, journos like easy things to write, and non-stories aren’t nearly as much fun. At least Crikey has devoted some time to this angle.

    Of course, all that never excuses Dennis Shanahan.

  2. Yes fooled by randomness into concocting a ‘story’.

    This pre-campaign campaign is crazy – journos are on the lookout for any quirk. A random walk helps them.

  3. I think Pr Q has put his finger on it. The polling question is to abstract, givent the attendant political circumstances.

    The polling questions do not get to the heart of the matter which is: does the average voter want to have Rudd + House-to-House bi-cameral federal ALP govt + Gillard-type front bench + Union Hack-type back bench + Coast to Coast state ALP govts?

    My guess is that a fair amount of the soft ALP supporters will flip their votes come polling booth time. The prospect of all that ALP deadweight in political bed together gives me the shivers.

    I am betting a 53-47 ALP victory. Howard to retain his seat.

  4. The way that the poll was written up is such that it will have determined Mr Howard to stretch out his PMship for as long as he can – although Janette probably isn’t keen to move out of Kirribilli early either.

    The plan is to continue to bombard the electorate with ads at no cost to the Liberal coalition until voters are convinced to re-elect the Liberals. The government doesn’t care if this behaviour is unconscionable, as it is really only working on brainwashing those who don’t pay much attention to the political sphere, and are suggestible to reassuring blandishments on Industrial relations, internet nasties, terrorists etc.

    However having watched two ads in succession this evening – one an aged care nurse giving up her job because of poor pay because of Workchoices compared to two men in the pub where one is pleased because the Workplace Authority made his employer pay more in wages, shows the government ad as unconvincing whereas the aged care nurse quitting is highly believable and also speaks to those who have family members in aged care facilities.

    JStroch is suggesting the line that will be taken by the government members – union bosses (don’t mention Workchoices); inexperience (don’t mention age); the states governed by Labor( don’t mention a cooperative approach); team (don’t mention the possibility of a leader other than Costello); big spending decisions to key demographics and any other desperate tactics to change the situation over the next three months. Some pretend children overboard or a Tampa would suit as well.

  5. Hey jstrocch, the content of the questions is not that important as long as the same questions are asked for each and every poll. It is really just a thermometer looking at the change rather than the absolute temperature. Of course if you were measuring pressure with a thermometer then the results could be misleading, but if the same method has been used for many elections and the polls reflected the end result then the process is meaningful, even if the wrong instrument is used. The fluctiations are more likely to do with mixing up the Centigrade with the Farenheit.

    Wall to wall ALP sounds like the harmony that we desperately need after the Howard devisiveness. What I see in the Coalition is a loud mouth, a pack of hyenas, and an awful lot of useless packing,…and Barnaby Joyce. I can ignore a few party hacks after that lot.

  6. What’s really funny is Gary Morgan’s weekly commentary on what caused the swing between polls. I mean, you expect journos to invent reasons for statistically insignificant movements in the polls, but the pollsters themselves should know better.

    IMO the pollsters should include multi-poll average with every new poll and only comment on statistically significant movements, but that wouldn’t make for near as exciting headlines.

  7. After what Howard has done – Iraq war, AWB scandal, “WorkChoices”, $800million to $1billion of taxpayers’ money spent on Liberal Party propaganda, in this term alone, inaction on global warming. privatisation, education cutbacks, emasculation of Medicare etc – he should count himself very lucky to have more than 10% support in the polls.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s