« JotSpot's suite dreams | Main | Is video the new fiber? »

How large is the long tail?

July 26, 2006

In his column in the Wall Street Journal today, Lee Gomes tries to debunk Chris Anderson's Long Tail theory, and on his Long Tail blog today, Anderson tries to debunk Gomes's debunking. It's an interesting - and important - debate, and I find myself agreeing with both gentlemen.

Gomes's main point is that the Long Tail has been oversold - that it's not as long or as important as it's been made out to be. He writes:

In the book's main sections, Mr. Anderson writes that as things move online, sales of misses will increase - so much so that they can equal or exceed the sales of hits. The latter is the book's showstopper proposition; it's mentioned twice on the book's jacket.

I was thus a little surprised when Mr. Anderson told me that he didn't have any examples of this actually occurring. At Netflix and Amazon, two of his biggest case studies, misses won't outsell hits for at least another decade, he said. None of these qualifications are in the book.

Anderson responds:

First, the book doesn't claim that there are any cases where sales of products not available in the dominant bricks-and-mortar retailer in a sector (my definition of "tail") are larger than the sales of products that are available in that retailer ("head").

What it does say is that the current data at Rhapsody, Netflix and Amazon show that the tail amounts to between 21% and 40% of the market, with the head accounting for the rest. Although I don't discuss this in detail in the book, in the case of Rhapsody, the trend data suggests that the tail (as defined above) actually will equal the head within five years.

I have no doubt that the Internet has created a Long Tail effect, making it easier for customers to find and buy rare or specialized products. Anderson's book provides pretty compelling evidence that that's true. And it's important. But I'm still not quite sure if it's really important or just mildly important. Some of my doubts stem from a crucial statistical change, relating to Amazon's sales, that's happened between the publication of Anderson's original Long Tail article in Wired in October 2004 and the publication of the book. In the Wired article, Anderson wrote:

What's really amazing about the Long Tail is the sheer size of it. Combine enough nonhits on the Long Tail and you've got a market bigger than the hits. Take books: The average Barnes & Noble carries 130,000 titles. Yet more than half of Amazon's book sales come from outside its top 130,000 titles. Consider the implication: If the Amazon statistics are any guide, the market for books that are not even sold in the average bookstore is larger than the market for those that are. In other words, the potential book market may be twice as big as it appears to be, if only we can get over the economics of scarcity. Venture capitalist and former music industry consultant Kevin Laws puts it this way: "The biggest money is in the smallest sales."

That claim becomes much more modest in the book:

What's truly amazing about the Long Tail is the sheer size of it. Again, if you combine enough of the non-hits, you've actually established a market that rivals the hits. Take books: The average Borders carries around 100,000 titles. Yet about a quarter of Amazon's book sales come from outside its top 100,000 titles. Consider the implication: If the Amazon statistics are any guide, the market for books that are not even sold in the average bookstore is already a third the size of the existing market - and what's more, it's growing quickly. If these growth trends continue, the potential book market may be half again as big as it appears to be, if only we can get over the economics of scarcity. Venture capitalist and former music industry consultant Kevin Laws puts it this way: "The biggest money is in the smallest sales." [pp. 22-23]

There's a very big difference between "more than half" of sales lying outside the top 130,000 sellers and "about a quarter" of sales lying outside the top 100,000 sellers. (Kevin Laws was right in the context of the article, but he's wrong in the context of the book.) Anyone who's followed Anderson's blog knows the story of this change. To his credit, he carefully documented the statistical issues in posts like this one. The overestimation of the size of the Long Tail in the article was due to an error in an academic study of Amazon's sales that Anderson relied on. (Amazon does not disclose detailed breakdowns of its sales.) The error's been corrected in the book - with the estimated size of the current Long Tail scaled back - but it still influences perceptions of the Long Tail's impact, and is even, as Gomes points out, echoed in the book jacket copy. (Please note that I don't hold authors responsible for what's on a book jacket.)

But the 25% is not the whole story, either. To get a clear sense of the impact of the Net on the Long Tail, you'd need another statistic: Before the Internet came along, what percentage of total book sales lay outside the 100,000 titles stocked in a typical large bookstore? There have always been specialized bookstores, selling everything from religious and spiritual books to textbooks to foreign-language books to used and out-of-print books to poetry books (though their ranks have been pruned by Amazon and other online sellers). And there have always been small presses - literary, academic and technical - selling books directly, through the mail. And you've always been able to go to a bookstore and order a book that it didn't carry on its shelves. How much of the Long Tail of books represents old demand moving through a new channel, and how much represents new demand? Only by knowing how big the old Long Tail was can you understand how much larger it's grown with the Internet.

My guess - and it's only a guess - is that the Internet Long Tail is substantially larger than the pre-Internet Long Tail, but that, in its current form, it amounts to something less than a monumental change in the market. The important question, then, is this: Is the Long Tail going to get a lot bigger, or has most of the growth already happened? Although Anderson and Gomes probably have very different views on that question - Anderson sees evidence that the tail is expanding, while Gomes sees evidence that it isn't - they would both, I think, agree on its fundamental importance. Anderson's Long Tail theory explains a lot about how the Internet has influenced markets, but the true extent of the Long Tail and its impact remains to be seen.

Comments

I'd say it's an effect comparable to WalMart's sales-chain innovations. Yes, it's real and somewhat important to people in the retail business - but it's also waaaay overhyped by professional marketers as some sort of democratizing revolution.

People have also pointed out that big chains being able to shift around more marginal inventory profitably, is not at all the same as the oft-implied little cottage industries. Instead, you can end up with rich getting richer, and sweatshops! (the implications for blog evangelism are then left as an exercise for the reader ...)

Posted by: Seth Finkelstein [TypeKey Profile Page] at July 26, 2006 11:05 AM

Seems to me that the effect of the Internet on the Long Tail is just beginning. We're still at the point where the Long Tail is being driven by simple availability of product. Ultimately, when consumers create a product just-for-me, you have a single customer for each ultra-customizable item.

Posted by: Cartwright Reed [TypeKey Profile Page] at July 26, 2006 01:01 PM

Most of the discussion of the Long Tail effect have been about consumers and retailers...not producers. Here's the Long Tail decision I have to make this week. We make a low volume electronic gizmo. We have to decide whether to make more gizmos using the old mold (in time for Xmas) or put our eggs in the next model for next year. We know we make a long tail product. And we are pretty sure which tail we are on. But we don't know how fat the tail is where we are right now. Or how much marketing it will require to move up the tail. Or how much new competition will be created by this new Long Tail thinking. I suspect some mfgrs. will conclude after reading the book or more likely a summary in an airline magazine that "What the hell, the Long Tail will bail me out even if I make a poor marketing or product design decision." That means more product clogging up our part of the Long Tail. We're going to wait.

Posted by: wren [TypeKey Profile Page] at July 26, 2006 01:54 PM

Great post, Nick.

(And not even a trace of bullying/thuggery for which, according to Michael Arrington, you're known.)

:-)

Posted by: John Koetsier [TypeKey Profile Page] at July 26, 2006 03:55 PM

Nick, you're right to point out that the real effect of the long tail would likely be felt not at the large mega-retailers, but in the booksellers that specialized before the Internet in books that following Anderson we call "long tail" or following Gomes we call "misses."

And in fact, if you look at the bookselling industry, it is precisely these specialty booksellers, rather than the mass retailers (e.g. Barnes and Noble), which have suffered most in the last ten years. Here in the Bay Area, the most visible example of this Long Tail market impact, was the recent shuttering of Cody's Books -- a Berkeley landmark for nearly half a century that was practically a library of long-tail books for sale. More than anything else, it is the collapse of the specialty bookseller that indicates the reality and power of the Long Tail.

Posted by: Nils Gilman [TypeKey Profile Page] at July 26, 2006 04:22 PM

Sales is one thing but it is profits that are the most important metric when talking about the business of the long tail. The long tail can do double duty in describing the fall in profits as you market to ever smaller niches. In fact, profits probably sink at a faster rate than the long tail because marketing costs become ever more massive (like approaching the speed of light mass increases massively... :-)

Posted by: Tom Foremski [TypeKey Profile Page] at July 26, 2006 08:46 PM

More than anything else, it is the collapse of the specialty bookseller that indicates the reality and power of the Long Tail.

Whoa. So the Long Tail's a bad thing?

More and more, Long Tail boosters (and Web 2.0 boosters more generally) remind me of the chapter in The Iron Heel when our socialist hero meets a lot of small businessmen who are being put out of business by 'trusts' and want the government to intervene ('Bust the trusts!'). Our revolutionary hero is having none of it. They're on the wrong side of history: concentration of capital into monopolies and oligopolies is all part of the process which will inevitably lead to the revolutionary overthrow of capitalism.

Good luck with that.

Posted by: Phil [TypeKey Profile Page] at July 27, 2006 05:56 AM

Nils -- First off, you're promulgating a half-truth, wittingly or not. It is true that Cody's Books' Telegraph Ave location has closed (press release here). However, Cody's still has a thriving business at two other stores, and also an online presence at http://www.codysbooks.com/ So, the location closed, but the institution is very much alive and well. The whole affair says vastly more about the woes of Telegraph specifically than it does about books in general.

Secondly, I used to be lucky enough to have a job that let me travel all over the country. I would take advantage of that travel to check out used book stores from coast-to-coast, if for no other reason than one would see new stock. And book store owners told me, again and again, that the Internet and what we would now call the long tail has helped them enormously. The reason is related to my reason for visiting them in the first place -- say you live in a town of 100,000. There are perhaps 3 or 4 used book stores in town. If you're a book addict, as I am, you get to know their stock very well -- as does everyone else with similar tastes in town. With the Internet, though, finally that one person in South Dakota who desperately wants that one book that's been sitting on your shelves for ten years can finally find out you have the book -- and thus you finally make the sale. In other words, the Internet is "nationalizing" the book market, in that rather than very fragmented markets dependent on actual physical presence, it's all becoming one big market.

And, yes, some bricks and mortar stores have closed. Whether at a greater rate than one would expect from regular business cluelessness, I cannot say. But I know the barrier to entry -- putting listings on Amazon, eBay, ABEBooks, others , as opposed to leasing a physical shop -- is vastly lower than it used to be. Perhaps ABA membership is down. I'd suspect that's because online booksellers aren't joing such older groups. That makes it very, very difficult to make genuine apples-to-apples comparisons to the state of bookselling. I can only say my contacts among book publishers say that business is better than ever -- and much ahead the growth rates of the Usual Two (ie, Borders and B&N).

Posted by: Hal O'Brien [TypeKey Profile Page] at July 28, 2006 06:06 AM

Nick, one unsung impact of "the net" is its centralization and structuring of previously unavailable data (because it was distributed and/or unstructured). Having the data on the "long tail" at all is a huge change and will - simply because of that visibility - probably grow its impact. It will change how we market to the tail, because now we know about it and can even find it, to some degree. In the past, it could be sold to (the specialty stores) but really was never marketed to. Some of the mom and pop stores have been disintermediated and that's a shame, but the result is that I found an out-of-print Steve Young CD on Amazon last week (sold to me by a new mom and pop doing business in their marketplace)... and now Amazon has some pretty interesting data about Steve's and my effect on the long tail...

Posted by: Phil Gilbert [TypeKey Profile Page] at July 29, 2006 03:34 PM

Post a comment

Thanks for signing in, . Now you can comment. (sign out)

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)


Remember me?


carrshot5.jpg Subscribe to Rough Type

Now in paperback:
shallowspbk2.jpg Pulitzer Prize Finalist

"Riveting" -San Francisco Chronicle

"Rewarding" -Financial Times

"Revelatory" -Booklist

Order from Amazon

Visit The Shallows site

The Cloud, demystified: bigswitchcover2thumb.jpg "Future Shock for the web-apps era" -Fast Company

"Ominously prescient" -Kirkus Reviews

"Riveting stuff" -New York Post

Order from Amazon

Visit Big Switch site

Greatest hits

The amorality of Web 2.0

Twitter dot dash

The engine of serendipity

The editor and the crowd

Avatars consume as much electricity as Brazilians

The great unread

The love song of J. Alfred Prufrock's avatar

Flight of the wingless coffin fly

Sharecropping the long tail

The social graft

Steve's devices

MySpace's vacancy

The dingo stole my avatar

Excuse me while I blog

Other writing

Is Google Making Us Stupid?

The ignorance of crowds

The recorded life

The end of corporate computing

IT doesn't matter

The parasitic blogger

The sixth force

Hypermediation

More

The limits of computers: Order from Amazon

Visit book site

Rough Type is:

Written and published by
Nicholas Carr

Designed by

JavaScript must be enabled to display this email address.

What?