Polls vs prediction markets

With the advent of iPredict, New Zealand has jumped on the prediction market bandwagon. But now a paper suggests that, for election results at least, polling data is more accurate than at least one popular trading market:

The market price is superior to a naïve reading of the polls. For instance, if the
incumbent leads 60-40 in the polls in May while the market says the incumbent will win with
55 percent, the market price is likely to be closer to the Election Day vote division. But this is
not the appropriate test.
…
We could ask … what an analysis of polling history would show to be the odds of the incumbent winning in November given a 60-40 lead in May, and whether this prediction based on polls offers greater certainty than the May … price.
…
Based on our analysis, an investor with a modest knowledge of how … polls translate into Election Day outcomes would reap handsome profits from the … presidential market. The implication is that where candidate market prices depart from where the polls project that they should be, these deviations contain more noise than signal.

The authors offer a couple of reasons why markets tend to be less reliable than the polling:

The histories of [vote-share] market prices show that traders tend to hold persistent beliefs about the vote division that contradict the polls and that these persistent beliefs are often wrong. Incorrect
beliefs get corrected only in the last days before the election, when the polls are difficult to
ignore. The winner-take-all market … compounds its errors by overvaluing long-shot candidates’ chances of victory, as if the market expects more campaign surprises than occur in reality.

Essentially the authors are saying that we’re really bad at predicting what other people will choose to do. We hold priors that we refuse to rationally update and we invest on the basis of those priors. We also overestimate the liklihood of low probability events, which has been known for some time from experimental results.

Of course, figuring out what the polls REALLY mean — as opposed to a naive reading — isn’t a trivial matter, so beating the market isn’t as easy as all that. Still, one would hope that markets will become more efficient as they now seek to incorporate ‘sophistocated’ readings of poll results into their investments.

Note also that the trading in the Iowa Electronic Market that the authors use is a lot thinner than on major commercial markets such as InTrade. Those thicker markets may well be more efficient than the one studied.