Ever since Bush’s turn to the UN, I’ve been debating the significance of UN resolution 1441 on Iraq with other bloggers, notably including Steven Den Beste, who initially thought Bush was knuckling under, then changed his mind. When the first draft was announced, that it was, in effect, a one-and-a-half resolution I observed that it effectively required a second UNSC vote

All that’s left in the reported draft resolution, and in the statements of the UK and US governments, is that, if inspectors report obstruction to the UNSC, the US and UK will not necessarily accept a veto on military action cast by, say, France.

Although, the US Administration is still resisting this interpretation, Tony Blair has now accepted it almost word for word

Mr. Blair said, “Of course we all want a second U.N. resolution. I believe we will get one.

But he added, “Where there is an unreasonable veto put down, we will not rule out action.

On this issue, Blair’s interpretation is more important than Bush’s. The US Administration has already made up its mind that it wants a war, whereas the British government has not.

Of course, all of this may turn out to be academic (in the pejorative sense) if the latest reported discovery of shells for chemical weapons turns out to provide the much-sought-after ‘smoking gun’. On the other hand, the precise interpretation of 1441 will be very important if the inspectors turn up incriminating, but not conclusive, evidence against Saddam. It’s still too early to tell on this one.

My Op-Ed piece in today’s Financial Review (subscription required) is on unemployment. A short grab:

At no time since the election of the current government has unemployment been an issue of real concern. Second-order trivia like the GST and waterfront reform have had far more attention. And, sadly, the Australian public has become inured to chronic mass unemployment. In the absence of a severe economic downturn, the government will pay no real political price for its worst policy failure.

Also, today the Committee for the Economic Development of Australia released a review of Australian experience with privatisation. You can download the <a Introduction here (PDF). Most of the contributors are broadly sympathetic to privatisation, though less so in the case of regulated monopolies. I argue, as I have done for some time, that we should be looking at renationalisation in many cases.

Most interesting to me was a study by Jonathan Kelly and Joanna Sikora showing that public opinion against privatisation has hardened steadily over time. In 1986, views on the privatisation of Telstra were about evenly divided. By 2002, 70 per cent were opposed and only 16 per cent in favour. Similar views apply even to firms like the Commonwealth Bank and Qantas that have been privatised for years, and opposition is even stronger in the case of Australia Post, the only business in the study still in full public ownership.

It’s difficult to attribute this to emotional attachments to Aussie icons, in view of the fact that people were quite willing to contemplate privatisation in the 1980s. Advocates of privatisation hoped that experience would lead people to accept it. In fact, the reverse has been the case, here as in Britain and New Zealand. People have experienced privatisation and they don’t like it.

In view of the rhetoric about elites we’ve heard so much about lately, it’s worth pointing out that the divide between elite and popular opinion is far sharper here than in the case of asylum-seekers.

‘Data mining’ is an interesting term. It’s used very positively in some academic circles, such as departments of marketing, and very negatively in others, most notably departments of economics. The term refers to the use of clever automatedsearch techniques to discover putatively significant relationships in large data sets.

The paradigm example, though a very old-fashioned one is ‘stepwise regression’. You take a variable of interest then set up a multivariate regression. The computer then tries out all the other variables in the data set one at a time. If the variable comes up significant, it stays in, otherwise it’s dropped. In the end you have what is, arguably, the best possible regression.

Economists were early and enthusiastic users of stepwise regression, but they rapidly became disillusioned. To see the problem, consider the simpler case of testing correlations. Suppose, in a given dataset you find that consumption of restaurant meals is positively correlated with education. This correlation might have arisen by chance or it might reflect a real causal relationship of some kind (not necessarily a direct or obvious one). The standard statistical test involves determining how likely it is that you would have seen the observed correlation if there was in fact no relationship. If this probability is lower than, say, 5 per cent, you say that the relationship is statistically significant.

Now suppose you have a data set with 10 variables. That makes 45 (=10*9/2) distinct pairs you can test. Just by chance you’d expect two or three correlations that appear statistically significant correlations. So if your only goal is to find a significant relationship that you can turn into a publication, this strategy works wonders.

But perhaps you have views about the ‘right’ sign of the correlation, perhaps based on some economic theory or political viewpoint. On average, half of all random correlations will have the ‘wrong’ sign, but you can at expect to find at least one ‘right-signed’ and statistically significant correlation in a set of 10 variables. So, if data mining is extensive enough, the usual statistical checks on spurious results become worthless.

In principle, there is a simple solution to this problem, reflecting Popper’s distinction between the context of discovery and the context of justification. There’s nothing wrong with using data mining as a method of discovery, to suggest testable hypotheses. Once you have a testable hypothesis, you can discard the data set you started with and test the hypothesis on new data untainted by the process of ‘pretesting’ that you applied to the original data set.

Unfortunately, at least for economists, it’s not that simple. Data is scarce and expensive. Moreover, no-one gets their specification right first time, as the simple testing model would require. Inevitably, therefore, there has to be some exploration (mining) of the data before hypotheses are tested. As a result, statistical tests of significance never mean precisely what they are supposed to.

In practice, there’s not much that can be done except to rely on the honesty of investigators in reporting the procedures they went through before settling on the model they estimate. If the results are interesting enough, someone will find another data set to check or will wait for new data to allow ‘out of sample’ testing. Some models survive this stringent testing, but many do not.

I don’t know how the marketing guys solve this problem. Perhaps their budgets are so large that they can discard used data sets like disposable syringes, never infecting their analysis with the virus of pretesting. Or perhaps they don’t know or don’t care.

Update Kevin Drum at CalPundit gives the perspective of a marketing guy, with lots of interesting points (for example, loyalty programs are there to collect data for mining). He doesn’t accept my main point and raises the dreaded “B” word – Bayesian.

For those familiar with debates among statisticians, this is the point at which things typically become both heated and incomprehensible (just like a lot of blogs, really). A real challenge, which I may tackle at some point, is to explain the Bayesian concept of statistical reasoning in ordinary language. For the moment, though, I’ll just agree that a debate like the one over data mining ultimately makes sense only if it’s cast in Bayesian terms, that is, with a discussion of the beliefs we hold before we begin the statistical analysis.

In two excellent articles, here and here,Thomas Friedman gets to the heart of the problem facing US Middle Eastern policy. To summarise, both Israelis and Palestinians are stuck with leaders who have repeatedly failed them, and led them into a cycle of attacks and reprisals. While the cycle goes on, neither side is prepared to gratify the other by dumping their failed leader and electing one who will make peace. By backing Sharon, the US ensures that it is hated by all those who support the Palestinians, which means virtually everybody in the region outside Israel. This in turn means that the idea of a democratic revolution in Iraq is doomed from Day One. Any democratically-elected government would be more fundamentally anti-American than Saddam (an American ally who merely miscalculated what he could get away with in Kuwait).

I’ll add my own analysis to this. What this means is that the Administration got its policy in the wrong order. It should first have leant hard on both sides until they agreed on peace terms essentially on the lines of the Clinton plan (though the paternity would obviously have to be denied). Once people in the rest of the region saw Israeli settlements actually being dismantled, Bush would have had a free pass as regards Iraq, and democratic reconstruction would actually have been possible.

It might still be possible to resurrect this strategy, by dragging out inspections and keeping the pressure on Saddam as long as possible. If the Israeli elections produce some swing to the left, and the US starts applying pressure, it might be possible for the Palestinian leadership to curb terror attacks long enough to reach an agreement. Then we might see a Palestinian state with democratic elections – Arafat might win the first of these, but not the second. And as I’ve said, success on this front would give Bush plenty of political capital to use in justifying an attack on Saddam.,

That’s a lot of ‘mights’ and ‘ifs’ but I think this offers a better chance than occupying Baghdad and then working out what to do next.

In today’s Age, Thomas Frank writes:

One of the most tenacious myths of the “culture wars” that have been going on in America for more than 30 years is that youth counterculture possesses some sort of innate transgressive power – that the eternal battle between hippie and hard-hat, disco-dweller and churchgoer or individualist and conformist is every bit as important as the struggle between classes used to be.

In a review of Frank’s last book, One Market Under God, I wrote

The most interesting part of Frank’s story relates to the way in which the entrepreneurs of the Internet boom appropriated the rhetoric of the Vietnam-era left, to the extent that venture capitalists refer to themselves as VC and the employees of Internet firms call themselves ‘dotcommunists’. Simultaneously claiming victory in the cold war, they denounced both governments and Old Economy corporations as little better than Soviet commissars. In this story, the entrepreneurs stand for the liberation, not of the workers, but of ‘new money’.

A cameo appearance is made by the postmodernists, whose alleged drive to destroy Western civilisation formed the basis of numerous denunciations of ‘political correctness’ in the 1990s. The postmodernist beliefs that all truth is relative, and that ‘there is nothing outside the text’ find their natural 1990s expression in work for the burgeoning advertising industry. The tools of critical theory and radical anthropology are pressed into service in the creation of a mythical reality for, say, a brand of toothpaste. The ultimate outcome of this radical theorising is the view that, not only nations, but individuals are, ultimately, brands, and that most are in dire need or rebranding.

I did another version of this review here. The para I quoted above is from Frank’s latest The Conquest of Cool, which sounds great.

Steven Den Beste sets out his scenario for a unilateral US invasion of Iraq

The way that war will begin is that President Bush will pick up his telephone, call General Franks, and say, “OK, Tommie, go for it.” And then American (and British) jets will begin major bombing, and shortly thereafter American (and British) troops will move into Iraq from multiple directions. The war will be brief, and we’ll win.

The fatal flaw in this scenario is in the parentheses. President Bush can’t pick up his phone and order British troops to invade Iraq. Any British participation will require, at a minimum, a Cabinet decision, which in turn will certainly produce demands for a public debate, which will result in a demand for a UNSC resolution. Blair knows this, as does the US Department of State, which explains the recent backing away from Jan 27 as a deadline.

And at this point, the whole thing starts to unravel. In email, Den Beste argues that British participation is an optional extra (which would justify the parentheses). Even in strategic terms, this is unclear – some of the things the British are providing, such as in-flight refuelling, are apparently indispensable in the short run. And in tactical and political terms it’s silly. The British and Americans are co-operating closely on the premise of a joint invasion. A call from Bush saying that American forces only should invade would produce chaos. More importantly it would trash the alliance between the US and Britain which, from the British side, is entirely premised on the idea of a ‘special relationship’ which gives the British government leverage over American policy in return for fairly reliable, and tangible, support.

Admittedly, there’s an element of ‘peace by timetable’ about this argument, but I don’t have a problem with this – war and peace are not symmetric in this respect.

The other interesting thing that became clear in my email discussion with Steven Den Beste is that the rift between the US departments of State and Defense has re-opened. State, which is, as I’ve noted responding to the concerns of the British is trying to slow things down, while Defense is trying to stick to the timetable. Den Beste, who seems to be pretty well attuned to the thinking of the hawks within the US Administration, writes

What has become increasingly clear over the last year is that the State Department as a whole is in serious disrepute with the White House, in part because it’s coming to be seen not so much as America’s foreign policy agency towards the world, but rather as the world’s foreign policy agency toward the US.

Many now suspect that the career bureaucrats in State identify more strongly with the nations they’re assigned to study than with their own. In some cases it appears that they’ve actually been bought.

I think it’s useful to observe that, however much State and Colin Powell are distrusted by some, they won the debate in 2002 about the need to take the UN route and focus on weapons of mass destruction. I suspect they will win again in arguing for delay.

This report that, following the expansion in the use of GM-based ‘Roundup Ready’ crops, the widely used crop herbicide is losing weed resistance doesn’t have any obvious implications for the GM debate other than to remind us that biological systems are complex and hard to manage.

It’s a useful rule, though that, if a policy piece discussing the use of herbicides and pesticides doesn’t discuss or refer to the implications of resistance, it should be treated with caution. I’ll have more to say on this later.

Update As Jim Birch points out, the NYT headline should have read “Weeds are gaining resistance to widely used herbicides”. Embarrassing! This will teach me to rely on the “Blog This!’ default instead of writing in my own link!

The blogosphere prides itself on beating print pundits to the punch. But in today’s Age, Gerard Henderson has a piece bagging the Howard government’s proposed appointment of John Carroll to vet the National Museum. A few blog-eons (months) ago we were treated to a full-length fisking of this guy from Tim Blair for his bizarre book about September 11. I thought about blogging on this a week or so ago, but couldn’t really be bothered. Irony alert on: I’m surprised, though, that Tim missed a chance to put the boot into the government for such a crazy appointment. Irony alert off

While we’re on the subject of the National Museum, I’m surprised by the hostility it’s aroused. Having produced a fair bit of it, I think my antennae are pretty well-tuned to detect material that will infuriate the average Quadrant reader and I found very little of it at the Museum. In its general feel, it reminded me of nothing so much as the opening ceremony of the Sydney Olympics – celebrating ordinary Australians of all kinds. While not ignoring conflict, I don’t think the Museum plays it up, either. The fact that it’s always packed with apparently ordinary Australians supports my view. Of course, most of the hostile reviews had been written before opening day and the Quadrant crew, like the Bourbons, forgets nothing and learns nothing.

With the resignation of Steve Case as chairman of AOL Time Warner (soon to be just plain Time Warner again, it seems) we can pronounce the “New Economy” officially dead. When the biggest merger in history was announced almost exactly three years ago (it seems like a century), I wrote

If the accounting numbers are taken at face value, AOL Time Warner will have a price-earnings ratio of about 350 to 1, modest by Internet standards. When options are taken into account, the ratio is more like 1000 to 1. It is difficult to see how an economy in which investment decisions are based on numbers like these can avoid some sort of financial catastrophe.

So is AOL Time Warner a super-profitable monopolist in the making or a jerry-built piece of financial engineering ? It can scarcely be both. But in the miraculous world of the New Economy, anything is possible.

Of course, as the blogworld and other phenomena show, the death of the New Economy has had almost no impact on the development of the Internet. For that matter, neither did its rise. In economic terms, the Internet has provided a source of modest but significant productivity gains in the market sector. However, the trillion dollars or so dissipated in the bubble has probably negated the impact of several years worth of Internet productivity growth.

Coming back to blogs, the big impact of the Internet has been in non-market services, of which email has been the paradigm example, but for which blogs are an even better illustration. Apart from the commodity service of ISPs providing the connection, no-one has ever made any significant money out of these things and, probably, no-one ever will. That doesn’t mean that they don’t count towards economic welfare, but the New Economy was never about economics.

I’ve been receiving a lot of interesting material on both Lomborg and global warming, but this time I’m going to stick to my plan of putting aside, for a while, the academic scandals that have been dominating the blog lately. In the meantime, those who want more can visit Ken Parish for what looks like a mirror-image of the Bellesiles story, involving pro-gun economist John Lott.

I’ll just offer the following thought. In the context of political debate, if a recently-reported scientific result seems too good to be true, it probably is.