1000 Days of User-Visible Improvements
A version of this article appeared on the Beeminder Blog. This is the rest of the story, about how Beeminder bet $1000 on bettering Beeminder every day, then failed and had to actually cough up the $1000 to a lucky user. But only after making 1000 improvements in 1000 days — something that unambiguously would not have happened without said $1000 bet — so it was totally worth it.
When I’m not writing Messy Matters articles I keep busy being the cofounder/CEO of Beeminder. [1] You may remember Beeminder from such previous Messy Matters articles as “How To Do What You Want” and “Flexible Self-Control”. You may also know, from those and other Messy Matters articles, that I take the concept of dogfooding to literally ridiculous extremes. [2] I can’t seem to work on anything without living and breathing it.
So of course I beemind the bajeezus out of Beeminder. In particular, we have a Beeminder goal to average one User-Visible Improvement (UVI) to Beeminder per day. We started it almost three years ago, before we even publicly launched, and put $1000 on the line. Last month we hit 1000 UVIs in 1000 days. [3] More on how that turned out shortly.
Paul Graham, Joel Spolsky, and Immortality
Our inspiration for this was a crazy-sounding claim by Paul Graham in his 2007 essay, How Not to Die. He points out that if one of his Y Combinator startups is regularly doing new deals and releases and, importantly, sending updates to Graham and colleagues, they’re probably going to live.
Then, admitting how naive it sounds, he gets to the crazy part: “Maybe the linkage works in both directions. Maybe if you can arrange that we keep hearing from you, you won’t die.” He defends the claim by pointing out that every YC dinner is a kind of deadline. “Staying in regular contact with us will push you to make things happen, because otherwise you’ll be embarrassed to tell us that you haven’t done anything new since the last time we talked.”
Paul Graham: If every startup had such a commitment device the success rate would be 90%.
He even quantifies the power of a commitment device like this, using the example of two startup founders who appeared in Newsweek with ‘the next billionaires’ printed across their chests. It would be unthinkably humiliating [4] for them to fail after that, Graham points out. If every startup had such a commitment device the success rate would be 90%, about which Paul Graham emphasizes he’s not kidding.
Joel Spolsky suggests the same probability that an organic growth (as opposed to VC-funded) company that moves forward every day will eventually win, i.e., become a 10 million dollar a year business in 5-10 years. We’re right on schedule to do that.
Lifehack: Create Your Own Commitment Device
If you’re amenable to a bit of lifehackery, here’s what you should do to give yourself that same 90% chance of success without Newsweek writing an article about you. First, create a public graph [5] like this:
Of course it will take over 1000 days for yours to look like that. [6] The “eep!” and “$1000” on our graph mean that we’re skating the edge (as usual) and will owe someone $1000 if we don’t improve Beeminder in some visible way by midnight tonight!
Once you have a graph like that, let your users know (even if you only have a handful of beta users initially) that you’re on the hook to announce a User-Visible Improvement (UVI) once a day on average. We tweet ours to a special Twitter account — @beemuvi — but any public changelog will do.
Ours have ranged from trivial, e.g.,
“Added some padding so the Feedback button doesn’t overlap the text if you make your browser too skinny. HT Judy Soule.”
to epic, like recently:
“Android App v2.0! Vastly faster & includes a built in timer app for beeminding how much time you spend on things…”
You can start with a very unambitious definition of “improvement”. When we originally announced our commitment to do this we listed all of the following as things that count:
- New blog posts
- Fixing typos
- Tweaking the layout or the logo or whatever
- Improvements to how the bot responds to emails
- Tweaks to the algorithm for generating the yellow brick road
- Tweets from our main twitter account (@bmndr)
- Improvements to the log-in procedure or payment processing
- Any new feature or tweak to any feature
- New tips-of-the-day in the email responses from the bot
Basically, anything that makes Beeminder better, even in the most
tiny, trivial way. Over the last 1044 days now, we’ve
gradually gotten more ambitious. Fixing typos and adding tips of
the day and whatnot don’t count anymore. Bug fixes and CSS
tweaks still do. And we have a separate graph for blog posts,
which you can see in the sidebar (of both the Beeminder blog and
Messy Matters), so the original version of this article
didn’t count as a UVI.
The real criterion is simply anything we’re not ashamed to publicly announce as our UVI of the day. We have (just) enough pride that we’d rather cough up the $1000 than tweet anything that clearly violates the spirit of the commitment.
Signaling Value
This has been immensely important for us as a commitment device to stay focused on forward progress, but we’re extreme in our need for lifehacky tricks like that to keep ourselves on track — which is why we made Beeminder in the first place, i.e., itch-scratching. We’re entirely serious when we say that Beeminder would not exist if it weren’t for beeminding Beeminder this way, particularly our UVI goal forcing inexorable forward progress even when we felt demoralized.
If you think you don’t need to do this to survive, then, first of all, reread Paul Graham’s How Not to Die. Note the part about how demoralizing startups can be. “The low points in a startup are just unbelievably low,” Graham says. “I bet even Google had moments where things seemed hopeless.”
But even if you don’t need to play this kind of trick on yourself, there’s huge value in signaling your commitment to steady progress and improvement. Prospective users place a surprising amount of weight on things like “how long ago was the latest blog post” and other such heuristics for whether you’re still in active development. Probably because it’s so common for startups to peter out despite the front page still being all “everything’s great! sign up now!” Watching your public changelog will mean more to your users than you can imagine, and will be persuasive to new users, not to mention making them more tolerant of your foibles. They know you’re getting better every day! [7]
Even aside from the commitment device (about which more shortly) and the signaling value, our changelog is super handy for looking up what happened when, like when we compose monthly update emails and whatnot. Or even for debugging sometimes. In theory we have the git history but that’s too cumbersome and noisy.
Close Calls
On our way to 1000 UVIs there were two occasions where we were ready to cough up the $1000, but didn’t. Aside from UVIs, we have in fact coughed up hundreds of dollars to our users on other meta graphs — most recently $270 for not getting a blog post out on time, and $90 each for our “40 hours/week on Beeminder” goals. And I continue to pay up on the Messy Matters typo bounty. But that’s all been well under $1000.
The first close call on our UVI goal was two years ago when we first visited Portland to interview for the Portland Seed Fund. The time zone change threw us off and, although we had deployed an improvement earlier in the day, we forgot to publicly tweet it by official midnight. At that point we had few enough salivating users watching like hawks that no one called us out in the 20 minutes or so before we remembered and got the tweet out. We would definitely have paid if someone had. [8]
More recently, we deployed our daily UVI and even publicly tweeted it, but forgot to update the actual graph (which we still do manually, but we recommend you set up an IFTTT recipe to auto-update yours!). We actually convened a little tribunal to rule on whether we should cough up the $1000 in that case. The verdict from our users was no, we’re subject to the standard grace period for getting data manually entered. Phew!
Judgment Day
In the original version of this article, celebrating the milestone of 1000 UVIs, we said “at this point, if we do cough up this $1000 it will have been worth it about 100 times over.” Well apparently we jinxed ourselves because guess what happened two weeks later? Yes, you guessed correctly. But of course, at 98 cents per UVI, it really was a steal! Not that we purposefully derailed, or even got especially lackadaisical about it after the original version of this article gushing about how great our UVI commitment is.
The reason for the derailment was just the confluence of multiple beemergencies, as we call them. Most notably another blog post (both our Beeminder blogging commitment and our Messy Matters blogging commitment — see the sidebar — are themselves up to $800 at stake). We had some candidate UVIs written down and I was meaning to discuss them with Bethany (aka Queen Bee, aka our CTO) before we tweeted one. But we got wholly distracted by the emergency blog post, coming right down to wire and hitting publish at 23:59:30. We even spent another hour improving the post, forgetting all about the UVI beemergency until I saw the fateful blog comment come in, outing us as off track.
Heightening the sting of this, we had actually deployed one of the UVIs and merely failed to tweet it in time. It was tempting to call that close enough. But after the previous close call (where we did tweet it in time and just forgot to update the graph in time) we had decided on “tweeted by midnight” as the defining criterion for fulfilling the commitment. So there was simply no weasel room, and we paid up.
We also said, when we wrote this article, “we’d immediately re-up and keep ourselves on the hook,” which is what we’ve done. Another $1000 is on the line with nary a day’s respite! Which, we should emphasize, is all far harsher than is the norm for Beeminder, but for this flagship Beeminder goal we like the extreme stringency.
In any case, we’ve now paid the $1000 on this, to Henrik Wist, who was the first one to spot it, per our fine print.
Even better is what Henrik did with the $1000. But I’ll let Henrik tell that story.
Epilog
So here I am, getting this article out the door on New Year’s Eve, lest one of you claim the $800 that’s riding on timely Messy Matters posts. Maybe I should have a New Year’s resolution not to come down to the wire like this on blog posts. But of course that’s pure delusion. What I’ll do instead is keep following my yellow brick roads.
And what I will resolve to do to avoid spoiling next New Year’s Eve is, as one of the next 365 User-Visible Improvements to Beeminder, add a feature to customize the midnight deadline. Then I can scramble to get my beemergencies dispatched by some reasonable time during the day and party the rest of the night. Happy New Year!
Related Reading and Links
- Paul Graham’s How Not to Die
- Joel Spolsky’s Fire and Motion, Strategy Letter I: Ben and Jerry’s vs. Amazon, and his more recent Startup School talk
- Article from the CTO of Parse.ly, Why Startups Die (“to survive, you must continue moving forward”)
- Scott Adams (of Dilbert fame) on why goals are for losers
- Shout out to our friends at SERPs who are beeminding their UVIs
- Hacker News discussion
Footnotes
[1] Fun fact: I’ve spent 605 hours working on Messy Matters and 7382 hours working on Beeminder. I know this because of TagTime, on which I’ve spent 486 hours.
[2] Lest you think I’m exaggerating about taking dogfooding to perhaps laughable extremes, check out the Messy Matters guest post (“For Love and/or Money”) by my spouse and Beeminder cofounder, Bethany Soule. I was doing research on principles of fair mechanism design and wanted to incorporate said mechanisms into my daily life. Bethany and I literally still conduct auctions between the two of us for putting our kids to bed and whatnot.
[3] What Beeminder looked like 1000 UVIs ago, if you promise not to laugh:
[4] We’re not actually a fan of this style of commitment device, though we do have a competitor that takes that approach. Hollywood teaches us to make grand commitments; real life teaches us that little-and-often wins. We find that the motivation of a single event will wear off, that goals change, and that you need just the right amount of inflexibility.
[5] The graph itself isn’t the most important part in this case — I’d like to convince you first of all to create a public changelog and commit to changing it, publicly. That alone is a commitment device if you have users watching, though surprisingly easy to get behind on, deluding yourself that you’ll catch back up soon, until it’s hopeless. If you want to really get yourself on the hook and follow our example, sign up for Beeminder and create a new goal — a “Do More” goal — committing to one user-visible improvement per day (7 per week, or maybe start with 5 per week, or even just 3 per week to start out — you can always dial it up later).
[6] Two clarifications: We don’t yet have annotations like that. Those were added in post-production. But we’ll surely have them within 1000 days/UVIs from now! And, two, you don’t have to put $1000 at stake like that off the bat. In fact, you don’t put any money at stake at first, nor do you ever have to as long as you keep all your datapoints on your Yellow Brick Road.
[7] Trevor Blackwell comments: “making the public commitment to your users addresses the #1 (rational) fear of using a startup’s web app: that it will someday be abandoned and there’ll be cobwebs everywhere.”
[8] We hold ourselves to a stricter standard for our own meta Beeminder contracts. For our users, you can always just tell us that a derailment was due to a technicality and we’ll believe you and not charge you. For ourselves we have some stringent fine print. But we should also point out that Beeminder contracts are much less scary than they sound. Certainly less so than our esteemed competitor, StickK.com. Part of the beauty of Beeminder is that you can reassess your commitment or end it altogether at any time, with a one week delay. So you can adjust your goal, but you can’t change it out of laziness, unless you’re particularly forward-thinking about your laziness!
