Cognitive biases

Ross Gittins cites some interesting questions used by some of my QUT colleagues to assess cognitive biases before undertaking a study of investment behavior. Here you go: Try to answer before reading on or checking comments:

Give me high and low estimates for the average weight of an adult male sperm whale (the largest of the toothed whales) in tonnes. Choose numbers far enough apart to be 90 per cent certain that the true answer lies somewhere between.

Don’t like that one? Try this: give me high and low estimates of the distance to the moon in kilometres. Choose numbers far enough apart to be certain that the true answer lies somewhere between.

Now something more personal. When you buy a Lotto ticket do you feel more encouraged regarding your chances if you choose the number yourself rather than using a computer-generated number?

Now, my answers
For the first question, I wanted a range that ought to give the right answer, but had some probability (ideally 10 per cent) of being wrong. I went for 5 to 50 tonnes (I suspect the frame encouraged me to pick a range of one order of magnitude.

Having taken this approach to the first question, the second was trivial. The range of 0 to 100 light years meets the criteria and I’m absolute certain that it’s right. (I had a fair idea of the actual distance, but that’s not what was asked for.

On the third question, last time I looked the advice was that numbers above 30 are less likely to be picked because a lot of people use birthdays. If that’s still true, I can do better in terms of expected return (but not in terms of chance of winning) by picking 31 to 36. However, if any significant number of people know this, then 31 to 36 is a bad choice. So, I’m left with the view that, if I had good access to the stats on other people’s choices, I could beat the machine, but in practice I can’t.

28 thoughts on “Cognitive biases

  1. Am I dense? I don’t see how the answers to the first two questions reveal anything about cognitive biases. Anyone reasonably intelligent who thought about them a bit would surely answer ‘between 1 and 1000 tonnes’ for the first, and much as you did for the second. Even if the question was reworded to instruct ‘far enough to be 90% certain but definitely not 100%’, I still don’t see the point. Quite a lot of people will already know the correct answer as a matter of general knowledge, and others won’t have a clue.

  2. I don’t get the “cognitive bias” angle either, Ken.

    I guessed 5 to 50 tonnes – because I thought that would very probably include the ‘correct’ weight, but I guessed it might also be around the requested 90% certainty.

    I also guessed 50,000km to 50,000,000km – thinking that would get me into the “100%” certainty, but I agree 0 to 100 light years would definitely get you there, and 0 to infinity is a guaranteed certainty.

    On the lotto, I answered: “No.” – Because it’s true (i.e. I don’t “feel more encouraged”).

    Professor Google tells me I’m probably right on the first 2 – but does that tell me anything about my cognitive biases?

  3. I read the Gittins article, but I’m still unclear. “Overconfidence” in oneself seems to be part of the point.

    However I did learn something: Gittins needs a sub-editor, the thing is full of typos.

  4. After reading this post, I also went and read the Gittins article – just to be sure I hadn’t missed something. Along with other posters above, I thought I might be mistaken.
    However, my confusion persists. Let me repeat at the risk of being considered stupid:

    There exist persons who will give an answer to questions (a) and (b) that are “wrong”. i.e. these answers give an interval which does NOT include the correct answer. This then shows some “cognitive bias”. Is this “bias” any different from the (in)ability to reason logically? Or are these “biases” just finer categorisations of the ways in which people fail to reason logically?
    If the answer to the last question is “yes”, then my confusion has been cleared.

    Can anyone can confirm this answer? Please assist. Thanks.

  5. With regards to the lotto question, the actual question used, as contained in the paper that Gittins links to, is:

    “When and if you purchase a lottery ticket, do you feel more encouraged, regarding your odds of winning, if you choose the number yourself rather than using a computer-generated number?”

    Thus it specifically addresses the odds of winning only, and side-steps the trickier issue of reducing the likelihood of sharing the winnings, but this is lost in the paraphrasing in Gittins’s article. This is crucial, since the question is designed to see whether people think they can control things they have no control over.

    As far as I can tell, suggestions to use consecutive numbers and numbers above 31 are based on this research from 1998: http://news.bbc.co.uk/2/hi/science/nature/240734.stm.

    But there have been plenty of newspaper articles publicising the results since then, so who knows how players’ behaviour has changed since then.

    There are so few runs starting above 31 that it would be surprising to me if they weren’t now being over-exploited by those trying to play strategically.

  6. Good one, totaram.

    Mind you, you’d want to know it was a blue sperm whale or green one first.

    Why am I thinking of Pascal’s wager and risk reduction?

    Megan, for example. Unless you knew Fairfax had handed over its sub editing to Kiwis, you’d put your money of Gittins getting decrepit, but because you an intelligent woman who goes to thinking peson’s blogsites, you know now what a rash mistake you could have made, but have now avoided and this backs up torams remarks involving the nature and origins of cognitive bias.

  7. There’s another paraphrasing (sub-editing?) fail with the second question, which is actually also about 90 percent, rather than total, certainty:

    “Give high and low estimates for the distance to the moon in kilometres. Choose numbers far enough apart to be 90 percent certain that the true answer lies somewhere between.”

  8. More pedantics. How can I be “90% certain that the true answer lies somewhere in between”. Firstly there is no “true” answer, only a correct answer. Secondly, I may be 90% certain, but that does not make it “true” or “correct” (I am 90% certain that Adam was the first man). Third, how can you measure a percentage on “certainty”. I am either 100% certain or I am uncertain. I cannot be 10% uncertain.

  9. Piling on, does Gittins know what a cognitive bias is? In the famous Kahneman and Tversky “Laura” example, the ranking of subjective probability that most people come up with can be proved wrong by logic. The question is one that is close to ones we face in real life. The propensity to jump to conclusions on such matters that are objectively and necessarily wrong is a cognitive bias. If missing cheap “gotcha!” answers requiring a close reading of trap questions is a cognitive bias, then I’m happy to plead guilty. The real-life application is I suppose seeing through weasel language in contracts; but we are on the alert there, or should be.

  10. I have a cognitive bias that this might not have been a well-designed survey. However, Ross Gittens’ poor article and inaccurate paraphrasing might be biasing me.

  11. Pr Q said:

    Ross Gittins cites some interesting questions used by some of my QUT colleagues to assess cognitive biases before undertaking a study of investment behavior. Here you go: Try to answer before reading on or checking comments:

    Give me high and low estimates for the average weight of an adult male sperm whale (the largest of the toothed whales) in tonnes.: Sperm whales have a solid (?) mass of blubber maybe 20 m (?) x 3 m x 3 m = 20 t – 200 t, prob. latter. A jumbo jet fully laden with fuel and cargo ways 400 tons. I can’t see a Sperm Whale weighing more than 1/2 of that. I’d probably be more comfortable with a mid–range estimate of 100 tons as I’m not sure how “hollow” a whales digestive system is.

    give me high and low estimates of the distance to the moon in kilometres.: The moon is 400,000 km, old enough to remember Apollo missions, so no range required. From memory there was a three second pause after asking a question to Eagle, equivalent to the time radio waves take to make the 800,000 km round trip.

    When you buy a Lotto ticket do you feel more encouraged regarding your chances if you choose the number yourself rather than using a computer-generated number?: Obviously one feels more optimistic if one chooses the numbers oneself, operating under the illusion that the numbers have memory and I can predict that pattern. But a true (unbiased) lotto result will generate a random sequence of numbers of the same form as a random series of numbers generated by a computer, at least a quantum computer. So I have no good mathematical reason to favour my own choice.

    The guy who built MOMA won alot of money on using fancy algorithims to win lotto compeititions. But he’s not telling anyone how he did it.

  12. Adult male Sperm whales can grow as long as 20 m and weigh as much as 56 tonnes.
    The (average?) distance to the Moon is 384,000 km,

    Quick Pick vs. Lucky Numbers

    Think your lucky numbers will help you win big? Actually, a greater percentage of lottery winners use a quick pick option — where a machine selects a list of random numbers — than winners who choose their own numbers. However, this can be misleading because more lottery players use the quick pick option than they do lucky numbers.

    In reality, the percentage of winners using the quick pick option is about the same percentage of players who use their lucky numbers. In other words, it doesn’t really matter how you pick your numbers.

    Statistically, the odds are the same that any combination of numbers will be picked. So, it doesn’t matter if you choose lucky numbers or let the machine choose for you. There is no predictable method to picking the winning numbers of the lottery.

  13. Well it wasn’t Gittins who was asking the questions, it was a gullibility study on behalf of the ASX to identify vulnerable groups on the one hand and then to find a way to encourage directors be more honest about their company’s debt position. At least that is the way I read it. The hybrid shares have the property of masking a company’s true debt level for a time providing a tempting opportunity for unscrupulous company directors pre-selling their shares without affecting their holding position ahead of a possible collapse. So for an investment product that might look tempting to self investors the question was what type of people are likely to fall prey to bad investment choices, and how big is the exposure. So the cognitive part is about where the true nature of the product is veiled, how good are people at assessing their risk exposure by examining the offered product from a variety of other perspectives.

    I think that is what it was about.

  14. @BilB

    Yes but what do the first two questions have to do with gullibility, or any kind of cognitive bias for that matter? That’s what a lot of people don’t understand. They seem to be crude tests of general knowledge and intelligence.

  15. I’d only be guessing, Ken_L.

    The best clues are in the definitions of cognition

    the mental action or process of acquiring knowledge and understanding through thought, experience, and the senses.

    a perception, sensation, idea, or intuition resulting from the process of cognition.

  16. This is not the first time that Gittins has misunderstood or misrepresented behavioural economics. To be fair, journos have a pretty tough gig, having to get across a range of material and generate numerous articles that are interesting and comprehensible to a lay audience. Unfortunately, too frequently Gittin’s short-cuts mislead or befuddle.

  17. @Ken_L
    I think the idea in this literature (on “overconfidence”) is that the intervals given by people in practice tend to trend in the opposite direction from those given in the replies on this blog—that is, participants tend to give intervals for which they feel 90% confident, but which are far more narrow than the actual empirical variation in the quantity asked about. That’s why it’s referred to as an ‘overconfidence’ effect; people report feeling 90% confident with intervals that only capture 30% of the true population’s variation, or what-have-you.

  18. I think the cognitive bias is about expectations, and expectations of outcomes. These can be manipulated by marketers so that consumers feel ripped off if they don’t make more than the average (the index).

    Hybrid securities is one example of clever marketing; by offering slightly better rates than term deposits banks can raise capital without having to dilute the the share base. They say that you can’t trade these securities but with falling interest rates their value falls. Far better to buy bank shares.

  19. Philip is on the money.

    I’ve seen (and taken) this ‘test’ where there are 10 questions (or 50) – and your objective is to get exactly 90% of the questions right. Questions like what is the length of the Nile, dry weight of an empty Boeing 747, how many beats per second for hummingbird’s wings, etc. If you got 10 out of 10 by giving excessive ranges, you ‘failed’.

    Given this setting, people routinely get around 4 or 5 right, not 9 – in other words, most people are systematically overconfident. I got 7/10 – still overconfident; there is a desire to be more precise than warranted by one’s actual knowledge.

  20. EconoMan that makes a lot more sense, but the results are still going to be contaminated by general knowledge questions to which quite a lot of people will know the correct answer. I can’t help thinking alternative tests would provide more reliable and valid data.

  21. @Ken_L
    ‘Knowledge’ is I think usually thought of in this area as a predictor, not a confound; i.e., experts (people with knowledge!) actually do *worse* in their subject areas (in the sense of picking way more precise intervals than are justified relative to empirical variation) than ‘naive people’. Or something like that.

    I do some behavioural economic work, but in this area specifically; these are just my vague impressions.

  22. @BilB
    I thought the point of the first question was that the numbers had to be far enough apart to allow a 90% range? (ie the first point in the range has to be 10% of the last point)
    For the record I said 3-30 tonnes, which allows a 90% range but is wrong in terms of what whales actually weigh! If you assume that most people don’t know how much a whale weighs, to choose a narrower range would show “cognitive bias’, in that you are excessively confident in your own knowledge (but wouldn’t be very useful test if you actually happen to know something about whales, as Phillip says)

    I guess the overall trend of these questions is to identify people who have excessive confidence in their own knowledge and/or predictive ability and thus are good targets for financial cons?

  23. @Val

    For the record I said 3-30 tonnes, which allows a 90% range but is wrong in terms of what whales actually weigh! If you assume that most people don’t know how much a whale weighs…

    I guess that is the point; on most things nobody knows much if anything but many are willing to give it a go, whatever it is.

  24. I can confirm the overconfidence thing. I thought 10 – 40 tonnes would cover the sperm whale, but I was wrong.

    Incidentally, when at quiz nights, if I don’t know an answer, my guess is almost always wrong. It took time to learn that when you don’t know, you don’t know. And I’m still biased towards thinking that maybe I do know.

  25. “… I’m left with the view that, if I had good access to the stats on other people’s choices, I could beat the machine, but in practice I can’t.”

    Here you go:
    auslotteries.com.au/lottery-statistics/
    and
    http://www.tattersalls.com.au/Number-frequencies.aspx

    The house’s vig is pretty big, though, so I’m not sure whether you could ever raise the expected return enough so it exceeds 1. Better to gamble on something like stock market futures, where the vig (in the form of brokerage) is small.

  26. @Jack Strocchi
    You’ve missed the point, Jack.

    Your choice of numbers has zero effect on your chances of winning a First Division. But it has a big effect on your chances of having to share that win, because many of your fellow gamblers do not pick their numbers randomly but based on such things as birthdays. That is, you can raise your expected payoff if you know common selection patterns and avoid them.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s