Charts from the inTrade online predictions markets seem to pop up all over the blogosphere. My question: what do they mean?
People tell me that these market prediction models 'accurately predict' outcomes. As with the claims for the accuracy of IQ tests it is one of those claims that is universally asserted despite any evidence to suggest the claims are meaningful, much less true. Note that the axis on the chart is labeled 'chance of victory', an explicit claim that the markets are a prediction.
In particular, what does a claim to have predicted the future accurately amount to? It is quite easy to see that the likes of Mystic Mogg who get every prediction consistently wrong are wrong. But inTrade does not predict outcomes, it predicts the probabilities of certain outcomes. What does it mean when inTrade 'predicts' that McCain has a 40% chance of winning the 2008 election? Does it mean that if the election is run ten times that McCain will win four of them?
If we had ten parallel universes and could run the election four times we could test such an outcome, but even then the outcome of an election is not exactly random chance. Its not like a sports match where there are a small number of variables that can have a huge effect on the outcome.
Election outcomes are in fact fairly predictable, opinion pollsters usually provide a prediction that is within a few points of the final result. But what does today's inTrade 'prediction' that McCain has a 40% chance of winning mean?
As with IQ tests, the reasoning behind the claim of 'accurate predictions' is essentially circular and hides a political agenda: Markets work, therefore predictions markets must work, therefore the predictions must be accurate - thus providing more 'proof' that markets work.
Friday, March 21, 2008
Predicting from the predictions markets
Linkworks:
FARK
del.icio.us
StumbleUpon
reddit
Subscribe to:
Post Comments (Atom)
1 comment:
From what I've read there actually is some value in prediction markets. It is an area that is discussed somewhat in Freakonomics (and more particularly on their blog, where they also interviewed the founder of inTrade).
But the more rigorous info I've read about prediction markets was in 'The Wisdom of Crowds' by James Surowiecki. Several different prediction markets are discussed in the first chapter; he also references (and criticises!) some papers that have done analysis on their effectiveness.
My understanding - based primarily on these sources and the studies they quote - is that these type of prediction markets generally *do* produce better results than polls. The press have clearly picked this up; in the last Aust election I heard several reporters discuss the bookmaker odds and mention that these were more accurate than the polls.
Surowiecki's discussion of one market indicates that their futures contracts returned $1 if the outcome happened, and so the quoted price at a point in time (e.g. 40c) was a useful approximation of the market's current estimated likelihood that the outcome will take place.
On IQ tests - you may have already read Malcolm Gladwell's essays on the topic. IQ tests are a peculiarly American thing, and he's done a good job of challenging their value.
Post a Comment