The comments thread lately has been full of what might be called the “”ad hominem fallacy” fallacy”. This is the fallacy that, because a logical syllogism is equally valid or invalid no matter who propounds it, evidence in favour of a judgement about a matter of fact should be treated the same no matter who puts it forward. But classical syllogistic logic has essentially nothing to say in relation to reasoning about the plausibilty of judgements based on evidence.

No one sensible takes this idea seriously when, for example, money is at stake. A member of a board of directors who has a financial interest in a proposal is expected to declare it and withdraw from the discussion for example. By contrast, believers in the “ad hominem fallacy” fallacy would suggest that the director’s arguments were just as valid as anyone else’s, and they do not need to declare their interest before taking part in the discussion (though they should not vote).

The problems with conflict of interest are twofold. First, it is usually impossible to check every factual claim made by someone putting an argument. Second, even if all the facts asserted in support of some position are verifiable, they may have been selected (cherry-picked) to favour a case, while facts pointing the other way have been ignored. If you’re willing to go to the trouble of fully informing yourself about the topic using independent sources evidence from interested sources is redundant, and if not, it’s unreliable.

I had a lengthy go at this here, and for convenience I’ve reposted it over the fold.

There’s more from Don Arthur , Tim Lambert and Cathy Young

A question that’s often raised in relation to public policy issues involving science is whether conflicts of interest matter. For example, does it matter if scientists who publish reports suggesting that the dangers of smoking are overstated turn out to be funded by tobacco companies? Common sense suggests that it matters, but a lot of commentators, often with a vague recollection of classes in elementary logic, suggest that this is an ad hominem criticism and that the only thing that is relevant is the argument, not who makes it. You can see a defence of this position from Elizabeth Whelan at Spiked here[1] (hat tip, Jennifer Marohasy in the comments to this interesitng Catallaxy post on values and science.

I’ll argue that common sense is right here.

As an illustration, suppose you are considering buying a new car, and you come across an “independent non-profit” site called “Car Buyers Guide”, which gives advice on models A and B. Here are some possible reasons the site might advance for buying A rather than B. Assume that you can confirm that all factual claims made are correct, but you don’t know anything about cars yourself.

1. The fuel required for model B is not available in Australia, so it cannot be driven here, unlike A. Therefore you should choose A

2. We consulted ten leading experts. All recommended A

3. We looked at ten different criteria and A was superior on each of them

If you rely exclusively on syllogistic logic you ought to find argument 1 convincing (with the auxiliary premise that a car that can be driven is always better than one that cannot). On the other had, reason 2 is a standard fallacy: an argument from authority. Reason three is also logically invalid; the fact that A is superior on some grounds does not mean that it is superior on most or all grounds.

In practice, though, syllogistic logic is not very helpful. Very few decisions can be supported by watertight logical arguments like 1. In practice, we ought to find reasons like 2 and 3 pretty convincing. Assuming that the 10 experts are selected at random from a suitable population, the probability that most experts actually favour B is less than 1 in 1000. And if the 10 criteria are selected sensibly, it’s highly unlikely that consideration of omitted criteria will change the balance.

You can either accept this kind of reasoning or become an expert on the subject yourself. Since the latter course is feasible in only a few cases, inevitably you have to rely on the former most of the time.

Now suppose you find that the “Car Buyers Guide” is actually funded by the makers of Model A. Reason 1 is still logically valid and compelling. But reasons 2 and 3 now have very little force. Unless experts unanimously favour B, it shouldn’t be hard to line up 10 who favor A (or even to induce some who are neutral to endorse A). And similarly, it’s nearly always possible to find some criteria on which one option is better.

Exactly the same issues arise in relation to the dangers (or safety) of smoking. The evidence here is statistical, so if you’re looking for logical certainty you won’t find it. And it’s always possible to find some benefits from smoking and some qualified people willing to give a low estimate of risks. But if you rely on the general judgement of independent experts, you’ll reach the conclusion that smoking (direct or passive) is likely to shorten your life and damage your health.

The counterargument, from Whelan and others is that “All scientists have personal ideologies”, and therefore that scientific work should be evaluated on its merits, without regard to source. This sounds appealing until we ask the question “evaluated by whom?” The only people who can usefully do the evaluation are qualified scientists and the only way we can rely on their evaluation is if we believe them to be free of conflicts of interests.

If you accept Whelan’s argument, you end up in a position of complete agnosticism about anything you can’t know from direct experience. She denies this, saying that “If the Tobacco Institute had been funded by the Easter Bunny, its pronouncements would still have been scientifically outrageous, because the controversy had long since ended over whether cigarettes are the primary cause of premature, preventable death ” but, by definition, controversy only ends when one side gives up. (The exposure of the fact that most of the defenders of the safety of smoking were recipients of tobacco money was one of the things that helped induce them to give up.)

As far as the relevant scientific communities are concerned the controversy over evolution has ended and the controversy about climate change has resolved most of the key issues (for example, that warming is taking place and that human activity is a contributor), as has the controversy about the safety of consuming GM foods, but that doesn’t stop people claiming otherwise. And the tobacco lobby only retreated from the glaringly false claim that smoking is harmless to the claim (absurd if you accept that direct smoking causes cancer, but harder to disprove) that passive smoking is harmless. Unless you want to become an expert in biology, geology, climate science, clinical medicine and statisics, among other disciplines, you’ll never be able to resolve these disputes without relying, at some point, on expert judgement.

Obviously, there’s an element of circularity here. We not only have to trust scientists to give us the best advice, but we also have to trust them to tell us who the relevant scientists are. The big argument for accepting this is the undeniable success of the scientific enterprise as a whole, and its demonstrated capacity for correcting error. This can be contrasted with the demonstrated capacity of interest groups to maintain propositions that suit their interests in the face of strong, indeed overwhelming, evidence to the contrary.

fn1. For the fascinating history of Spiked see this Jason Soon post. For Whelan’s own background see Sourcewatch.