It’s true. Andrew Gelman made that point here in the comments as well. I guess for this version you’re on your honor to not do that. You could mitigate that problem by asking the questions one at a time and not revealing how many will be coming. It’s actually a really interesting question how to make the incentives exactly right. I’ll leave that as an exercise for the reader!

]]>sorry i am not a numbers person but have this as a homework assignment for behavioral finance what did you put for best guess? The correct answer?

]]>I’m trying to think about this in the context of the Delphi forecasting technique, which asks people for their forecast and their 90% confidence interval. One method of combining forecasts is to combine by the inverse of the expected error variance; but that’s completely wrong if those with a narrow confidence interval aren’t the most knowledgeable, just the most overconfident.

]]>As an engineer, I’m embarrassed I didn’t do the same. Ugh.

]]>I cheated. I gave nine intervals as (-infinity, +infinity) and one interval as (0,0). After all, the request was to be _perfectly calibrated_, not merely to be calibrated in expectation! This was the only way I could be sure of getting “90% of my intervals (no more, no less)” to contain the true values.

]]>