Here’s why:

Because we measure the wrong thing.

Talk show bookers, business plan competitions, acquiring book editors, movie critics, tech entrepreneurs who run trade shows that try to predict the future, tech bloggers, marketing bloggers… when we’re trying to predict whether a new technology or web site or book or song is going to hit, we’re almost always wrong.

Take a look at some of the picks for past web2 shows, or see who got hyped on various morning TV programs or see which authors were turned down by five or ten or fifteen publishing houses… "surprise hits" they call them.

[Uncharacteristically, I’m leaving out the names of the clueless and the misinformed (other than me). I’m not sure why, I guess I just don’t want to have a fight with people, who, unlike me, are unwilling to admit they’re wrong all the time.]

The astonishing thing isn’t that we’re wrong so often (see below) but that given the amplifying power of our platforms, we’re unable to yell loud enough to make our predictions self-fulfilling prophecies. In English: You’d think that being featured by a big publisher or at a big conference would be enough in and of itself to make something undeserving a hit. Alas, only Oprah can do that.

So, why are we wrong? Why does your boss/in-law/friend/VC/editor/pundit always get it wrong?

Because they measure ‘presentation.’ Not just the PPT presentation, but the way an idea feels. How does it present. Is it catchy? Clever? Familiar? We measure whether or not it agrees with our worldview and our sense of the way the world is.

The problem is that hits change worldviews. Hits change our senses. Hits appeal to people other than the gatekeepers and then the word spreads.

How? Through persistence and hard work and constant revision. By getting through the Dip.

If I have a skill in developing stuff, it’s in ignoring these people. Purple Cow was turned down by my old publisher and a few others. Squidoo was dissed by some of the best in the business (the site is about to hit 8 mm monthly page views). The Dip was a hard sell to my agent and my publisher.

No one ‘pre-predicted’ the astonishing success of Flickr or Google or Twitter or Bill Clinton’s first run for President. Sure, it was easy to connect the dots after the fact, but that doesn’t count.

Of course, there are plenty of failures to go around (I know that I’ve got more than plenty). Just because everyone hates it doesn’t mean it’s good. Execution is everything. Execution and persistence and the ability to respond to the market far outweigh a pundit’s gut instinct. But, the thing to remember is this: if everyone loves it, it is almost certain to have troubles.

In fact, my rule of thumb is this: if the right people like it, I’m not trying hard enough.