This week’s pissing match – I mean, spirited conversation – between Tim O’Reilly and me regarding the influence of the network effect on online businesses may have at times seemed like a full-of-sound-and-fury-signifying-nothing academic to-and-fro. (Next topic: How many avatars can dance on the head of a pin?) But, beyond the semantics, I think the discussion has substantial practical importance. O’Reilly is absolutely right to push entrepreneurs, managers, and investors to think clearly about the underlying forces that are shaping the structure of online industries and influencing the revenue and profit potential of the companies competing in those industries. But clarity demands definitional precision: the more precise we are in distinguishing among the forces at work in online markets, the more valuable the analysis of those forces becomes. And my problem with O’Reilly’s argument is that I think he tries to cram a lot of very different forces into the category “network effect,” thereby sowing as much confusion as clarity.
Ten years ago, we saw a lot of fast-and-loose discussions of the network effect. Expectations of powerful network effects in online markets were used to justify outrageous valuations of dotcoms and other Internet companies. Disaster ensued, as the expectations were almost always faulty. Either they exaggerated the power of the network effect or they mistook other forces for the network effect. So defining the network effect and other related and unrelated market-shaping forces clearly does matter – for the people running online businesses and the people investing in them.
With that in mind, I’ve taken a crack at creating a typology of what I’ll call “network strategies.” By that, I mean the various ways a company may seek to benefit from the expanded use of a network, in particular on the Internet. The network may be its own network of users or buyers, or it may be a broader network, of which its users form a subset, or even the entire Net. I don’t pretend that this list is either definitive or comprehensive. I offer it as a starting point for discussion.
Network effect. The network effect is a consumption-side phenomenon. It exists when the value of a product or service to an individual user increases as the overall number of users increases. (That’s a very general definition; there has been much debate about the rate of increase in value as the network of users grows, which, while interesting, is peripheral to my purpose.) The Internet as a whole displays the network effect, as do many sites and services supplied through the Net, both generic (email) and proprietary (Twitter, Facebook, Skype, Salesforce.com). The effect has also heavily shaped the software business in general, since the ability to share the files created by a program is often very important to the program’s usefulness.
When you look at a product or service subject to the network effect, you can typically divide the value it provides to consumers into two categories: the intrinsic value of the product or service (when consumed in isolation) and the network-effect value (the benefit derived from the other users of the product or service). The photo site Flickr has, for example, an intrinsic value (a person can store, categorize, and touch up his own photos) and a network-effect value (related to searching, tagging, and using other people’s photos stored at Flickr). Sometimes, there is only a network-effect value (a fax machine or an email account in isolation is pretty much useless), but usually there’s both an intrinsic value and a network-effect value. Because of its value to individual users, the network effect typically increases the switching costs a user would incur in moving to a competing product or service or to a substitute product or service, hence creating a “lock-in” effect of some degree. Standards can dampen or eliminate the network-effect switching costs, and resulting lock-in effect, by transforming a proprietary network into part of a larger, open network. The once-strong network effect that locked customers into the Microsoft Windows PC operating system, for instance, has diminished as file standards and other interopability protocols have spread, though the Windows network effect has by no means been eliminated.
Data mines. Many of the strategies that O’Reilly lumps under “network effect” are actually instances of data mining, which I’ll define (fairly narrowly) as “the automated collection and analysis of information stored in the network as a byproduct of people’s use of that network.” The network in question can be the network of a company’s customers or it can be the wider Internet. Google’s PageRank algorithm, which gauges the value of a web page through an analysis of the links to that page that exist throughout the Net, is an example of data mining. Most ad-distribution systems also rely on data mining (of people’s clickstreams, for instance). Obviously, as the use of a network increases, particularly a network like the Net that acts as a very sensitive recorder of behavior, the value of the data stored in that network grows as well, but the nature of that value is very different from the nature of the value provided by the network effect.
Digital sharecropping, or “user-generated content.” A sharecropping strategy involves harvesting the creative work of Internet users (or a subset of users) and incorporating it into a product or service. In essence, users become a pool of free or discount labor for a company or other producer. The line between data-mining and sharecropping can be blurry, since it could be argued that, say, the formulation of links is a form of creative work and hence the PageRank system is a form of sharecropping. For this typology, though, I’m distinguishing between the deliberate products of users’ work (sharecropping) and the byproducts of users’ activities (data mining). Sharecropping can be seen in Amazon’s harvesting of users’ product reviews, YouTube’s harvesting of users’ videos, Wikipedia’s harvesting of users’ writings and edits, Digg’s harvesting of users’ votes about the value of news stories, and so forth. It should be noted that while sharecropping involves an element of economic exploitation (with a company substituting unpaid labor for paid labor), the users themselves may not experience any sense of exploitation, since they may receive nonmonetary rewards for their work (YouTube users get a free medium for broadcasting their work, Wikipedia volunteers enjoy the satisfaction of contributing to what they see as a noble cause, etc.). Here again, the benefits of the strategy tend to increase as the use of the network increases.
Complements. A complements strategy becomes possible when the use of one product or service increases as the use of another product or service increases. As more people store their photographs online, for instance, the use of online photo-editing services will also increase. As more blogs are published, the use of blog search engines and feed readers will tend to increase as well. The iPhone app store encourages purchases of the iPhone (and purchases of the iPhone increase purchases at the app store). While Google pursues many strategies (in fact, all of the ones I’ll list here), its uber-strategy, I’ve argued, is a complements strategy. Google makes more money as all forms of Internet use increase.
Two-sided markets. Ebay makes money by operating a two-sided market, serving both buyers and sellers and earning money through transactional fees imposed on the sellers. Amazon, in addition to its central business of running a traditional one-sided retail store (buying goods from producers and selling them to customers), runs a two-sided market, charging other companies to use its site to sell their goods to customers. Google’s ad auction is a two-sided market, serving both advertisers and web publishers. There are a lot of more subtle manifestations of two-sided markets online as well. A blog network like the Huffington Post, for instance, has some characteristics of a two-sided market, as it profits by connecting, on the one hand, independent bloggers and, on the other, readers. Google News and even Mint also have attributes of two-sided markets. (Note that the network effect applies on both sides of two-sided markets, but it seems to me useful to give this strategy its own category since it’s unique and well-defined.)
Economies of scale, economies of scope, and experience. These three strategies are also tied to usage. The more customers or users a company has, the bigger its opportunity to reap the benefits of scale, scope, and experience. Because these strategies are so well established (and because I’m getting tired), I won’t bother to go into them. But I will point out that, because they strengthen with increases in usage, they are sometimes confused for the network effect in online businesses.
None of these strategies is new. All of them are available offline as well as online. But because of the scale of the Net, they often take new or stronger forms when harnessed online. Although the success of the strategies will vary depending on the particular market in which they’re applied, and on the way they’re combined to form a broader strategy, it may be possible to make some generalizations about their relative power in producing competitive advantage or increasing revenues or widening profit margins in online businesses. I’ll leave those generalizations for others to propose. In any case, it’s important to realize that they are all different strategies with different requirements and different consequences. Whether an entrepreneur or a manager (or an investor) is running a Web 2.0 business (whatever that is) or a cloud computing business (whatever that is), or an old-fashioned dotcom (whatever that is), the more clearly he or she distinguishes among the strategies and their effects, the higher the odds that he or she will achieve success – or at least avoid a costly failure.
You say you want clarity? I’d like to try to help: You can’t see
what’s going on in the industry without some better sense of the
technology involved.
Let’s talk first just about utility computing (before we get to
“apps” or “devices” etc.).
There seems to be a naive model at work in the press and in many
parts of the industry. To build a computing utility (this naive view
goes) you build out rack space, load it up with carefully selected
commodity HW, install some virtualization software and on top of that
install an OS like GNU/Linux or Windows. Hook up some big pipes
(network bandwidth) and you’re good to go. Of course, people get
caught up in the minutiae of this: perhaps Google has a novel way to
cool the racks; perhaps Amazon is doing the best job creating
commodity products out of it; and so on… but the general perception
is that utility computing is nothing more than “really big clusters
(of more or less off the shelf stuff).
If you accept that “the cloud utility” is going to be that
but just “scaled up really, really big” then I think you’ll find
Nick’s argument takes the day. There’s a very good chance that the
specific case of utility economics comes into play: very well
capitalized entrants to the market can handle such a large and diverse
set of demand that they can average out demand and operate far more
efficiently than smaller players, shutting smaller players out of the
market.
We need to look more closely at the naive assumptions because both
Nick and Tim are confused:
An example of that naive perspective can be found from Tim when he
argues that open source sucked all the profit out of systems software
and thus created new profit opportunities adjacent to that — of which
his category “Web 2.0” is supposed to be the prime suspect. It’s a
simple idea he’s proposing: system software got to be all but gratis
so you could no longer hit mass markets “selling software [licenses]”
but you could suddenly afford to run a lot more systems
software (as much as you’ve got HW for) and so the action moved
downstream to applications. Not just any applications — for
individual users could themselves acquire any gratis software they
needed — but applications of some sort that users could not
“self-host”.
There are some “natural monopolies” in applications that many
people want but few can host — often around data. For example,
each stock exchange has a natural monopoly in its real-time ticker.
Or, the Geologic Survey has a natural monopoly in real-time reports of
seismic events detected by its equipment. Those kinds of natural
monopolies aren’t common, however. An investor can’t simply “conjure
up” a scarce data source like that.
There are natural monopolies in services, too. For example, the
City of Berkeley has a natural monopoly for the service of allowing me
to directly pay my parking ticket online. Again, those service
opportunities are scarce and investors can’t “conjure them up at
will” — not much of an investment arena.
Tim’s argument (I wonder if he would recognize it in this form)
continues by asking: Well, what if the data or service which is to be
monopolized is itself a byproduct of the activities of the
complete community of users. There is a natural monopoly in that case
because the complete community of users talks to just one application
provider. If from what they “say” or “do” the application provider
can carve out a range of products to sell back to those very same
users, then the provider has a natural monopoly in the aggregate.
(Aside: Nick you should think of everyone who’s site is crawled by
Google as a Google customer — there is a trade there. It’s an
involuntary trade that is a side-effect of the architecture of the net
but a trade nonetheless (else Google’s crawlers would be sued away or
blocked away). Thus, Google search really does enjoy the specialized
form of network effect you describe as a two-sided market. They also
enjoy “data mining” but that is really the part that lives behind
their walls — they mine data they’ve “bought” on the crawler
market.)
Tim points out that unlike the rare kinds of natural data and/or
service monopolies, all it takes to create these “user generated”
monopolies is a good idea, some compute clusters, and some talent to
assemble the network and software. Thus, it is an area begging for
investment. Let’s call it “Web 2.0”
Most importantly, make sure you design your “web 2.0” application
so that both power laws and 2-sided market laws apply: You need a
2-sided market so you can extract data or services from users and sell
it back to them; you need a power law effect so that you have a
barrier to entry against very well capitalized late entrants (e.g.,
Microsoft search).
All well and good except it isn’t entirely working out that way and
the problem is the technology. Fans of Rob Pike will like this:
Open source is irrelevant. Systems software research is
relevant. So is radical hardware research, although that is probably
not going to deeply pay off for 15 or 20 years.
Earlier I described how to launch a Web 2.0 app (idea + commodity
cluster + network design + software). And I mentioned that the naive
view is that utility computing looks rather like that, server side,
just much, much larger. That naive view is balderdash.
To see why, let’s work backwards from applications. Google’s
search engine is a fine example.
We’ve heard quite a bit about Google’s constant movement towards
every higher-level, domain-specific-language ways of programming the
search engine. “Map-Reduce” is probably the most famous example.
Google’s heavy use of Python is also famous. Oh, and, did anyone
notice they hired Rob Pike?
As those programming systems become increasingly perfected, less
and less of anything resembling a “traditional operating system” is
needed underneath. If you started today with the problem of designing
a system that was specialized to do nothing more than handle network
requests using python code and, say, map-reduce — you would
never begin work on the problem by saying “Ok, let’s start by
designing Unix.” It’s a complete mismatch.
Similarly, as high-level administrative controls are perfected for
large clusters and virtualization systems, the question of what kind
of OS is wanted underneath is re-opened.
And with a radical change in OS and with a radical shift to
higher-level programming languages, suddenly all the equations that
determine HW economics right down to the chip level go kablooey. For
example, if what I really want to do 100% of the time is run lisp,
there’s no way in heck I’ll start working on the problem by saying
“let’s design the x86”.
The enormous capitalization of a few “Web 2.0” players, plus the
innate smarts of a smaller few, mean that they are going to (already
are) investing very heavily (by comparitive terms) in systems
software research and eyeing ever lower-level areas of HW research.
They will, indeed “make do” with commodity HW and gratis foundations
to their software stacks for now but…
Eventually they will have a breakthrough new OS. I doubt it’s that
far off in the future. It won’t be all that long before they are
fabbing their own custom HW to go with it — and thus enjoying the
benefits of what Nick dubs the utility model, complete with very high
barriers to entry.
And so, back to applications:
A computing monopolist needs only to build out a critical mass of
applications to get the ball rolling — to start getting a very
diverse “audience share” — and then to permit third parties to add
new applications on the monopolists utility. The utility computing
monopolist can let third parties chase down the “long tail” of app
niches until eventually anyone wanting to write a new app makes the
default choice of writing the new app to run on the monopolist’s
utility platform.
So, take a “youtube” for example. It started out life as a third
party ISV building out a standard cluster and adding some code —
caught on and caught a bunch of users. The acquisition was a natural
consequence because Google thinks to themselves that any early
“cloudy” application like that needs to be ported asap to their
(future) utility platform rather than some other contenders. It
wouldn’t do if youtube had grown and grown on its own only to be later
moved to some other firms (future) utility platform because the game
is to capture future ISVs and get them to target your own (future)
utility platform.
There is a big social risk there.
The best extant solution I’m aware of is the regulation of utilities. We
do not want a private monopoly on secret hardware and secret systems
software, as a matter of social policy. Yet under the current
regulatory regime that’s just what we’ll get. It will be necessary, I
think, to bust up the computing utilities and to make open standards
out of them. Yes, we want a utility infrastruture but no, we don’t
want the power company to be the sole provider of compatible toasters:
to the extent commodity computing arises out of application
monopolies, we will have to bust up and separate the two realms of
business. One side can sell utility computing of an open sort on an
equal basis to all comers, the other side can provide competing
applications. That’s a messy route to go, however. That kind of
“bust up” and new regulation doesn’t ever get going until after
the problems have started happening.
The best speculative solution I’m aware of is to start investing in
open source systems software research — design the stacks needed, in
advance, for a “common carrier” environment of competing utility
computing providers. (Ahem: flower)
Otherwise, it won’t be long before things like a “Web 2.0”
conference start looking more like an Oracle developer conference: a
dog and pony show / party for ISVs locked-in to a monopolist platform
provider.
-t
Nick — I haven’t read all of your exchanges with O’Reilly — but kind of reminds me of Chinatown — “sister” slap “daughter” slap “sister and daughter” . . . cause here’s a thought experiment to consider . . .
Take your Flickr example — specifically the tagging part, which you indicate you would put under the network effect heading. Now abstract that to any user behavioral information, whether explicit like tagging or implicit (e.g., clickstreams). Then say that behavioral data is used to generate recommendations to Flickr users. Still under the network effect heading since it is just a more sophisticated “tagging,” no? But, slap, it’s clearly also data mining.
So it would seem from this brief thought experiment that data mining is an activity, not a business model, like two sided markets, nor an economic effect like increasing returns to scale (of which network effect is one particular type). Rather it is a means to enable certain types of business models, and may be influenced by various economic effects, i.e., increasing/decreasing economies of scale and/or scope.
So I would vote to drop data mining from your list that otherwise includes business model types and economic effects (and it would be better to be crisper in separating those concepts than lumping them together, IMO).
There’s an old computer networking aphorism, “Topology IS politics.”
Tom: Thanks. I’m still digesting your comment. (One passing thought: Maybe Google is running a three-sided market, encompassing information producers, information consumers, and advertisers, in which a single entity can play two or three of the roles simultaneously.)
Steve: You’re right that there are some apples and oranges problems in my list. I’ d like to think that at least some of the problem comes from sloppiness of expression, which should be fixable without necessarily changing the categories. (I’ve already changed the category “data mining” to “data mines” to focus on the source of the potential benefit rather than the action to exploit it.) While acknowledging that the boundaries are fuzzy (as are the boundaries between explicit and implicit information), I still think data mines need to be considerately separately from network effect because they have very different implications for a business seeking competitive advantage and superior profits. The network effect can create a very strong advantage because a customer becomes dependent not only on the product but also on the other users of the product. That’s not there, at least not in a proprietary way, with data mines. PageRank gave Google a strong product advantage but it did not create a dependence between users of Google’s search engine. And the same data mine was available to Yahoo, Microsoft, et al., so it was not in itself a barrier to entry or competition (in the way that the network effect can be). To lump a data mine in with network effects obscures the very different implications they have for strategy.
Still, though, you’re correct that the list needs refinement.
Nick
Re your passing thought about Google and a 3-sided market (advertisers being #3).
That’s close to how I would analyze it in the framework of your typology but we don’t need the concept of a 3-sided market — we can use a more general concept: that 2-sided markets are “composable”.
By “composition”, as a technical term, I mean a concept from math or CS but also intuitively familiar for the ordinary construction trade. A set of things is “composable” if each thing has inputs and outputs and the outputs of some are the inputs of others — so you can build “circuits”. Standard plumbing parts are “composable”. In CS, two programs are composable if their outputs and inputs align. For example, a word processor can be composed with a page layout system (the output of the word processor doubles as input to the page layout system). That’s the rough idea.
So, the business function “provide search results” has two sets of inputs: one from the data mining of crawler results and another from the advertising side. Similarly, publishing on an ad-carrying blog (say) has two sets of inputs: one from blog content generators and another from an ad broker.
The net effect is that users who search and then explore pages found are engaging in several separate two-sided markets at once trading unusual things (like trading a willingness to view ads for a skill at providing ads that don’t offend too much). And you can make little circuits taking a bunch of 2-sided markets with “weird” inputs and outputs and composing them in a single app so that the weird form of payment from one (user eyeballs, brokered ad placement, etc.) all get traded down in a single (complex) transaction to an actual cash transaction (so the advertiser owes google and google owes the blogger after all the various trades settle out). The user paid eyeballs, the broker paid the user tasteful placement. The blogger paid content and the broker paid the blogger cash. The broker paid pageranking and every web site in sight paid rights to crawl (at least so far as to read robots.txt). Etc. What’s produced for Google from a lot of those trades is then spent (in a single web interaction) to pay for some of those other trades — with things aligned so that among the various trades money flows from advertisers, though Google, to content providers (including Google itself for ads on search results pages). When I look at roughtype.com this morning, with a single click I’m doing several transactions with Google and with your hosting provider, etc. Google is turning around and doing several similar transactions. Snaking through those is a money flow.
Tim is sloppy with language (you’ve convinced me — and as if I were not) using terms like “network effect” but one thing that comes out of the pissing match is that he’s struggling to find the more abstract analytic framework that I think I did a better job than he of pointing to in my earlier reply.
-t