This is also one of my criticisms of the Servan-Schreiber paper. While I believe there are probably markets in which the availability of serious money to be won could attract people who’d be willing to spend research in order to get a better answer, NFL sports isn’t an arena where spending thousands of dollars will help you uncover facts that aren’t already in the mainstream media.

When we talk about CEO markets, or product release dates, or market penetration numbers, we’re talking about markets in which the information isn’t already out there, and some people will spend time and effort to ferret out relevant facts for reputation (we see this often on Foresight Exchange) or money.

I think the dynamics of elections – which as a rule take place every four years or so – are much harder to capture by using statistical models than sports events are. They occur less frequently, and in very different contexts. What can the outcome of a presidential election in 1948 tell us about the outcome in 2012? A little bit, perhaps, but not much. Traders in prediction markets pay much more attention to the particular context every election takes place, and I believe this is one of the strengths of this method.

Understanding why voters vote the way they do is very hard to capture, especially in advance(!). Wlezien and Erikson claim that polls adjusted with some structural variables are equally good or better than market predictions. That may be true, but if I remember correctly that analysis was made after the result was known, which is another playing field.

I will gladly endorse another method if it outperforms prediciton markets in accuracy, but so far they have been the best tool in predicting the vote ex-ante. Once the market is up an running it is also cost-efficient for the organizer, and produces continuously updated predictions.

Prediction markets for baseball, we predicted, would perform better. Many more statistics are gathered for baseball and conventional wisdom has it that many variables like pitching rotation and recent batting performance of individual players need to be accounted for in predicting game outcomes. But again, simple statistical models that ignore all that do essentially as well as the market.

So maybe sports in general are an exception? That’s why we decided to try another domain that prediction market advocates have pointed to to showcase their efficacy: predicting box office revenue for movies. Same story.

And as we’ve been discussing in the comments above, for political elections it’s deja vu all over again.

So I think you’re making a God of the Gaps argument here.

As for the plots, we evaluate the prediction methods on RMSE and discrimination as well as calibration in the paper.