Learning from bad graphs and weak analysis
Yesterday's Times features a blog post about the Kindle. There's a lot wrong with the post (which hopefully has been corrected by the time you read this) and I thought I'd point out two useful lessons. Nick Bilton, the author of the post, also did the graphs, and as a former newspaper art director, he has no one else to blame for the way the graphs appear or are interpreted. [Nick changed his post before my post went live this morning, and he dropped me a line indicating that his graphs weren't supposed to be deceptive, they were merely mislabeled. I think the points in my post below still stand.]
As you can see from the graph to the right, [it appears that] he's trying to make the case that lots and lots of Kindle owners are really unhappy (the large gold wedges).
Problem 1: The [original pie charts Nick used, at right, are incorrect]. The corrected one is below. 7% is a much smaller number than you see to the right.
Problem 2: Many of the reviews are from people who don't own the device.
Problem 3: Amazon reviews never reflect the product, they reflect the passion people have for the product. As Jeff Bezos has pointed out again and again, most great products get 5 star and 1 star reviews. That makes sense… why would you be passionate enough about something that's sort of 'meh' to bother writing a three star review?
Problem 4: This is a useful insight for anyone who markets anything–the people who buy the first generation of a product are more likely to be enthusiasts. They are more forgiving. They like new things. Bilton has tried to invent a trend by lining the items up in chronological order, but this is deceptive, both because of the number of reviews, but mostly because the people reviewing the new ones have a different agenda.
The Kindle has managed to offend exactly the right people in exactly the right ways. It's not as boring as it could be, it excites passions and it has created a cadre of insanely loyal evangelists who are buying them by the handful to give as gifts.
I think the lessons here are: Ignore graphs intended to deceive and understand the value of the negative review.
PS, as a bonus, here's the same analysis of the reviews of Catcher in the Rye, a book that has sold more than 20,000,000 copies (and changed many lives)–and the author doesn't even have a blog.