It’s true. Andrew Gelman made that point here in the comments as well. I guess for this version you’re on your honor to not do that. You could mitigate that problem by asking the questions one at a time and not revealing how many will be coming. It’s actually a really interesting question how to make the incentives exactly right. I’ll leave that as an exercise for the reader!

sorry i am not a numbers person but have this as a homework assignment for behavioral finance what did you put for best guess? The correct answer?

I’m trying to think about this in the context of the Delphi forecasting technique, which asks people for their forecast and their 90% confidence interval. One method of combining forecasts is to combine by the inverse of the expected error variance; but that’s completely wrong if those with a narrow confidence interval aren’t the most knowledgeable, just the most overconfident.

As an engineer, I’m embarrassed I didn’t do the same. Ugh.

I cheated. I gave nine intervals as (-infinity, +infinity) and one interval as (0,0). After all, the request was to be _perfectly calibrated_, not merely to be calibrated in expectation! This was the only way I could be sure of getting “90% of my intervals (no more, no less)” to contain the true values.