Macroware Technology

Welcome Portlet


Welcome to Macroware Technology Blog.

The purposes of this engineering blog:

- Share knowledge
- Learn
- Have fun
- Document what I was doing / thinking

Search Box

 

Private Area

Mailing List

Engineering Education

Engineering Organizations

Professional Career Networking Groups

Electronics Hobby

Trade Publications

Some early Hubdub Results

posted Wednesday, 13 February 2008

The predictive market at hubdub.com is now a few weeks old and I've participated in several settled questions.  Basically the way the market works is someone posts a question and sets some initial odds for the answers.  People can then bet pretend money on the outcome and the payoff is based on the odds.  The odds are adjusted based on the betting.  The questions are settled by administrators after the outcome or answer is published.  You can cash in your bets before the settlement and the cash in value is determined by the odds at the time of cash in.  So it is possible to place a bet and then cash it in for more before the bet is settled thus making a profit.  Based on how much money some people have made in the short amount of time it is open, it appears that there is a lot of active trading.  I don't really intend to do any active trading, I'm more interested in how often the crowd is correct and am making that my experiment.  According to the book The Wisdom of Crowds by James Surowiecki, I'm not sure if the predictive market in this setup will work or not [1].  One of the things Surowiecki identifies in the book is the cascade effect.  In order for predictive markets to work people must be acting independently, where each person brings a little bit of their own knowledge to the collective wisdom.  In the cascade effect, people know how other people feel, and their own opinion is influenced by this [1].

Since this is a market where the bets change with time as more information is revealed, the end resulting percentages almost always match the outcome.  For example there was a question on who would be the MVP of the Superbowl this year.  Before the game, Tom Brady was quite a bit ahead.  As the game progressed, Wes Welker of the Patriots surged into the lead briefly, but then as the game ended Eli Manning was in the lead.  So even though the end result matched what happened (Eli was named the MVP), it was pretty obvious toward the end of the game that he would be so named, and so people changed their bets accordingly.  So as I tally results for whether the crowd made the right prediction, I will always assess what the crowd thought before an event starts.  In this example, the crowd predicted that Tom Brady would be the MVP and was wrong.  Here are the outcomes I wil keep a tally of:

The crowd is correct: More than 60% of the crowd predicts the correct outcome
The crowd is wrong: Less than 40% of the crowd predicts the correct outcome
It's a draw:  Between 40% and 60% of the crowd predicts the correct outcome

Here are my tallies so far:

  • Who will be named Superbowl MVP?
    • Wrong
  • Will Space Shuttle Atlantis actually launch on 7 February?
    • Wrong
  • Will the Writers Guild of America Strike end by February 29, 2008?
    • Correct
  • Will Hillary Clinton cry again after Super Tuesday before February 12th?
    • Draw
  • Grammys: Who will win Record of the Year
    • Wrong

So far the crowd's won-loss-draw record is 1-3-1.

 

[1] Surowiecki, James. 2004. The Wisdom of Crowds. Anchor Books: New York.

 




1. Nigel Eccles left...
Thursday, 14 February 2008 3:07 am

Doesn't look like a good start for our forecasting ability! I expect over time that will improve. One of the best ways to assess accuracy is to compare the % outcome of events with similar forecast probabilities. Google did a very nice analysis of the accuracy of their internal prediction markets. Their analysis is here: http://googleblog.blogspot.com/2005/09/putting-crowd-wisdom-to-work.html