The Wall Street Journal reports today that digital music also-rans RealNetworks and SanDisk are teaming up to launch a proprietary music downloading system to compete with Apple’s dominant iPod-iTunes system. This news comes on the heels of Microsoft’s announcement of its proprietary Zune music system, launched in close collaboration with hardware maker Toshiba. Samsung is also preparing to enter the market with its own proprietary system, in partnership with MusicNet, and Sony, too, is peddling a proprietary system. As Om Malik points out, Apple’s “walled garden” strategy, wherein hardware, software and content are tightly integrated, has long been mocked by competitors, who have argued that an open system based on standardized, interchangeable hardware and software components would ultimately triumph, as it did with personal computers. Now, those same competitors are abandoning open systems, like Microsoft’s “plays for sure” system, and building their own walled gardens.

What’s particularly fascinating here is how the market’s development contradicts the received wisdom about the evolution of technology markets. As Clayton Christensen argues in his book The Innovator’s Solution, complex technologies tend to evolve from proprietary “integrated architectures” to open “modular architectures” as they mature. An integrated architecture, controlled by a single company, is necessary to make a new technology perform well enough to be embraced by customers. But when the performance of the technology becomes “good enough” for the general market, a modular architecture takes over, as it enables lower cost production and greater flexibility and also allows many companies to contribute innovations.

“When there is a performance gap,” Christensen explains, “companies must compete by making the best possible products. In the race to do this, firms that build their products around proprietary, interdependent architectures enjoy an important competitive advantage against competitors whose product architectures are modular, because the standardization inherent in modularity takes too many degrees of design freedom away from engineers, and they cannot optimize performance.” But “ultimately, companies that have excelled in the race to make the best possible products find themselves making products that are too good.” At that point, “nonintegrated competitors,” who are able to “introduce new products faster because they can upgrade individual subsytems without having to redesign everything,” begin to take over the market, “disrupt[ing] the integrated leader.”

In a Business Week interview in January, Christensen predicted that the digital music market would follow this same path, dooming Apple’s integrated iPod-iTunes system. When asked whether Apple could sustain its dominance, Christensen replied:

I don’t think so. Look at any industry – not just computers and MP3 players. You also see it in aircrafts and software, and medical devices, and over and over. During the early stages of an industry, when the functionality and reliability of a product isn’t yet adequate to meet customer’s needs, a proprietary solution is almost always the right solution – because it allows you to knit all the pieces together in an optimized way.

But once the technology matures and becomes good enough, industry standards emerge. That leads to the standardization of interfaces, which lets companies specialize on pieces of the overall system, and the product becomes modular. At that point, the competitive advantage of the early leader dissipates … the same thing will happen in the iPod world as well. Apple may think the proprietary iPod is their competitive advantage, but it’s temporary. [Apple’s current situation is] comparable to the fork they faced when they chose not to open up the Mac in the 1980s, when they let Microsoft become Microsoft.

But that prediction is not, at least so far, panning out. In fact, suppliers are actually abandoning attempts to standardize interfaces and instead building their own nonstandardized proprietary systems. As the market matures, it is getting less modular.

Now, this still may prove to be just a temporary aberration – modularity may still prevail. Or the digital music market itself may be an aberration. It certainly has some unusual qualities. For one thing, it hinges on copyrighted content (songs), and the owners of that content want to control, through digital-rights protection schemes, customers’ ability to copy it. At the same time, most of the systems support an unprotected version of the content – MP3 files – which remains, by far, the most popular version. MP3s, in other words, form the gap in the garden walls – a standard module shared by otherwise proprietary systems.

But even such unusual characteristics don’t explain away the strange path of evolution of the digital music market. What we’re seeing here is a pattern of technological progress that goes against the grain of what’s expected. I think what it shows is that the advantages of standardization may be overrated, at least in certain consumer markets. Given the existing standards in computer hardware combined with today’s flexible global supply chains, a single company, like Apple, may be able to design a proprietary system that is modular enough to counter most of the benefits of open systems while also maintaining the performance advantage that a closed system can provide. At the hardware level, after all, the iPod is constructed of standard modular components like flash drives and USB ports, produced by outside suppliers. It’s the software that makes the system proprietary.

And it surely doesn’t hurt that it’s easier to fine-tune the fashionability of a complex consumer product when you control all of its parts. Elegance matters.

The man who invented Wikipedia now wants to bury it. Larry Sanger, the controversial online encyclopedia’s cofounder and leading apostate, announced yesterday, at a conference in Berlin, that he is spearheading the launch of a competitor to Wikipedia called The Citizendium. Sanger describes it as “an experimental new wiki project that combines public participation with gentle expert guidance.”

The Citizendium will begin as a “fork” of Wikipedia, taking all of Wikipedia’s current articles and then editing them under a new model that differs substantially from the model used by what Sanger calls the “arguably dysfunctional” Wikipedia community. “First,” says Sanger, in explaining the primary differences, “the project will invite experts to serve as editors, who will be able to make content decisions in their areas of specialization, but otherwise working shoulder-to-shoulder with ordinary authors. Second, the project will require that contributors be logged in under their own real names, and work according to a community charter. Third, the project will halt and actually reverse some of the ‘feature creep’ that has developed in Wikipedia.”

UPDATE: The project is being discussed at Slashdot, with Sanger chiming in.

UPDATE: Wikipedia is a mirror, if a warped one, and whether noble or merely quixotic Sanger’s project is stirring some interesting reactions. At Techcrunch, Marshall Kirkpatrick scoffs at the plan: “Does the world need a Wikipedia for stick-in-the-muds?” W.A. Gerrard, of StrayPackets, fires back: “If an attempt to craft a wiki that strives for accuracy, even via a flawed model, is considered something for ‘stick-in-the-muds,’ then it’s apparent that many of Wikipedia’s supporters value the dynamics of its community more than the credibility of the product they deliver.”

This is the first in a series of occasional commentaries on the future of corporate IT.

In the wake of the popular embrace of the buzzword Web 2.0, the suffix “2.0” has become an all-purpose signifier of putatively revolutionary newness. It’s hard to pin down exactly what it means to be 2.0 – which is one of the suffix’s main attractions – but the term seems to denote a software program or a software-based service that has two qualities: (1) it is delivered over the internet rather than installed on local machines, and (2) it facilitates the collaborative creation of some sort of product. To get any more specific than that is probably unnecessary and almost definitely futile.

For businesses, one of the more intriguing of the recent coinages is “Office 2.0.” That term is being used, with increasing frequency (and, naturally, decreasing specificity), to describe a new generation of personal productivity applications – the would-be successors to the component applications of the ubiquitous Microsoft Office. Office 2.0 applications are delivered as services over the internet, running in most cases within the user’s web browser. Many such “web apps” are already available, ranging from Google’s Writely word processor to Dan Bricklin’s wikiCalc spreadsheet program to Zoho’s Show presentation creator. They are, by design and necessity, much simpler than traditional “desktop apps,” and because they run on the internet they are in many ways (though not in all ways) more conducive to collaboration among many users. They are also, in general, easier to integrate with other popular Web 2.0 formats and tools such as tags, wikis and blogs.

Because of the centrality to the white-collar world of Microsoft Office, which has expanded to encompass email and calendar programs as well as various other components, a major shift in the nature of personal productivity applications would have far-reaching implications for companies and their IT vendors. So it’s worth taking a hard look at the state of the Office market and its likely evolution. First, though, it’s necessary to debunk “Office 2.0.” Though not without a few important grains of truth, the term is misleading, a red herring in effect if not intent. First, by suggesting that we’re now seeing the first generational shift in personal productivity applications (PPAs), it oversimplifies the history of office software, obscuring past developments that can help illuminate the future. Second, it jumps the gun. We’re still a generation away from the adoption of purely web-based PPAs.

If I had to summarize the generations of office software, here’s how I’d do it:

Office 1.0 (1980s): a set of discrete and often incompatible applications for word processing, spreadsheets, presentation creation, and simple database management. Archetype: Lotus 1-2-3.

Office 2.0 (1990 – present): integrated suites of PPAs, with expanded, if still limited, collaboration capabilities. Archetype: Microsoft Office.

Office 3.0 (present – early 2010s): hybrid desktop/web suites incorporating internet-based tools and interfaces to facilitate collaboration and web publishing.

Office 4.0 (c. early 2010s): fully web-based suites.

It’s been widely assumed, among the tech-forward Web 2.0 crowd, that it will be the end users who will drive the adoption of purely web-based office apps – and that corporate IT departments will be the obstructionists. I think it will actually play out in the opposite way.

Whatever the flaws of Microsoft Office, most end users are comfortable with it – and they have little motivation to overturn the apple cart. What is absolutely unacceptable to them is to take a step backward in functionality – which is exactly what would be required to make the leap to web PPAs today. Web apps not only disappear when you lose an internet connection, they are also less responsive for many common tasks, don’t handle existing Office files very well, have deficiencies in printing (never underestimate the importance of hard copy in business), and have fewer features (Microsoft Office of course has way too many, but – here’s the rub – different people value different ones). Moreover, many of the current web apps are standalone apps and thus represent an unwelcome retreat to the fragmented world of Office 1.0. Finally, the apps are immature and may change dramatically or even disappear tomorrow – not a strong selling point for the corporate market.

What will be attractive to end users – at least a sizable number of them – is to extend the usefulness of the traditional office suite through the addition of web-based tools and interfaces. The key is to extend both functionality and interoperability without taking away any of the capabilities that users currently rely on or expect. Reducing interoperability or functionality is a non-starter, for the end user as well as the IT departments that want to avoid annoying the end user. You screw with PowerPoint at your own risk.

What we’re entering, then, is a transitional generation for office apps, involving a desktop/web hybrid. This generation will last for a number of years, with more and more application functionality moving onto the web as network capabilities, standards, and connectivity continue to advance. At some point, and almost seamlessly, from the user’s perspective, the apps will become more or less fully web-based and we’ll have reached the era of what I call Office 4.0 (and what others currently call Office 2.0). Driving the shift will be the desire of companies, filtered through their IT staffs, to dramatically simplify their IT infrastructure. Mature web-based apps don’t require local hardware, or local installation and maintenance, or local trouble-shooting, or local upgrading – they reduce costs and increase flexibility. These considerations are largely invisible to end users, but they’re very important to companies and will become increasingly important as the IT world shifts to what might be called utility-class computing.

Those who look to “Office 2.0” to destroy the dominance of Microsoft Office are likely to be sorely disappointed. In the absence of widespread user disgruntlement, Office isn’t going anywhere – and Microsoft’s fledgling Live services have a strong natural advantage over most contenders. That doesn’t mean that everything’s going to be rosy for Microsoft. Even during this transitional period, the company is going to find it harder and harder to maintain the traditional pricing and upgrade models that have made Office such an enormously lucrative franchise. Those models have been crumbling for some time, and as PPAs steadily become more like services and less like products, they’re going to collapse altogether. The collapse will begin with small and medium-sized businesses, which have the most to gain and the least to lose from moving quickly to a software-as-a-service model, even for PPAs, and it will roll upward to larger companies from there. Microsoft sees this coming, and one of its biggest challenges in the years ahead will be figuring out how to replace the revenues and profits that get sucked out of the Office market.

As for the many Office 2.0 companies that are popping up, most are doomed. Some, though, may find nice niche markets – consumers who need to do a specific task, inveterate Microsoft haters, tech tinkerers, tiny companies and nonprofits, educators – or carve out a role, at least temporarily, as a complement to the hybrid version of Office. But don’t expect today’s Office 2.0 contenders to make meaningful inroads in the mainstream business market, at least not anytime soon.

In the near term, the software-as-a-service companies with the greatest opportunities in the enterprise market are those looking to replace applications that traditionally run on servers rather than those looking to replace applications that traditionally run on personal computers. But that’s another story.

UPDATE: The Bb Gun offers an interesting, and even more conservative, take on the subject, pointing out that it’s a mistake to assume that office apps are fundamentally about collaboration: “There are tools for collaboration and there are tools for individual contributions. You mix the two, and you’re not necessarily working to people’s expectations.” It’s a good point – there’s a reason they’re called personal productivity apps – but it’s also true that “office work” is often collaborative, in one way or another, and there’s rarely a bright line between personal and group productivity. That said, it’s a mistake to think that online collaboration is always superior to, say, emailing a file back and forth or even distributing hard copies and gathering comments. There are a lot of subtleties in – and forms of – business collaboration, and in general a successful office app will support as many of those forms as possible (or at least not get in the way of any of them) while also, of course, supporting the considerable amount of work that people do alone.

At the heart of the commercial internet is a conflict between technological structure and economic interest. The net’s technological structure, developed without regard to the possible profit-making interest of any future commercial site owner, is one of openness, providing the individual user with unfettered freedom to go where he wants, see what he wants and do what he wants. The economic interest of the commercial site owner is usually in conflict with the openness of the technology. The dependence of much of the commercial internet on advertising means that the commercial site owner is rewarded for maintaining control over the user, keeping him within the bounds of the site in order to expose him to more ads.

This tension between technological structure and economic interest has been one of the main formative forces influencing the commercial internet since profit-making players first appeared. But it has become even more salient with the rise of Web 2.0, which amplifies openness at the technological level (by encouraging the free flow not only of the user’s attention but also of underlying data and applications) while at the same time increasing the economic rewards for keeping the user within a particular site (by making advertising even more important to profit-making).

The tension, which often lies hidden beneath the popular rhetoric of openness that characterizes much of the discussion of the internet in general and Web 2.0 in particular, erupted into view yesterday when News Corp’s COO Peter Chernin discussed MySpace’s relationship with other Web 2.0 companies like YouTube. As Multichannel News reported:

At the Merrill Lynch Media & Entertainment Conference in Pasadena, Calif., Tuesday, Chernin said that because most of YouTube’s traffic starts at MySpace, it may be time for the No. 1 social networking site to cut out the middle man. “If you look at virtually any Web 2.0 application, whether it’s YouTube, whether it’s Flicker, whether it’s Photobucket or any of the next-generation Web applications, almost all of them are really driven off the back of MySpace,” Chernin said at the conference. “There’s no reason why we can’t build a parallel business.”

While Chernin said MySpace’s video efforts are small at the moment, that could change. He estimated that 60%-70% of YouTube’s traffic comes from MySpace … “Given that most of their traffic comes from us, if we build adequate, if not superior, competitors, I think we ought to be able to match them, if not exceed them.” Chernin also added that News Corp. is experimenting with offering some of its studio content on the Web, also through MySpace. “You’re going to see us starting to play more aggressively on the entertainment side of that site,” he said.

Much has been made of the ease of launching Web 2.0 sites – the costs of the equipment are relatively low and most of the necessary coding is relatively simple. But for entrepreneurs, that cuts both ways. It’s easy to launch a site that, by capitalizing on the net’s open technological structure, “plugs into” other popular sites, drawing users that can then be fed ads, but it’s also easy for the popular sites to incorporate new services and tools, in effect blocking the exits. And because it’s in the economic interest of popular sites like News Corp’s MySpace to maintain control over the user, you can expect them to aggressively incorporate new tools and services into their sites, either through acquisitions or by building the components themselves. Profit-making companies are going to pursue their economic interests.

The same tension can be seen influencing Google’s evolution, though with a twist. On the one hand, Google’s very existence hinges on the technological openness of the net. On the other hand, as an advertising-driven operation, Google has strong economic incentives to keep users on its own property. Because it not only sells ads but also auctions them – that’s the twist – it’s been able to relieve some of the tension. Its AdSense program enables it to make money even when users explore other sites. But the tension remains. Because it gives the bulk of AdSense advertising revenues back to the site owner, Google makes much more money when it keeps the user on its own property. And so we see Google perform an exquisite balancing act, supporting openness while also extending its own tools and services, through acquisition and in-house development. Google would never say what Chernin said, but its strategy in the end is not so different from the one Chernin laid out.

The reaction to Chernin’s remarks, in the blogosphere anyway, has been reflexively negative. That’s to be expected. Openness is not only the technological structure of the net but also its reigning ideology. But it would be a naive mistake to assume that economic interest will inevitably lose out to technological structure in determining the future shape of the commercial internet. The tension is far from being resolved, and its effects are complex and hard to predict. But in pinning their hopes on the advertising model, the proponents of a diverse and open Web 2.0 may have also sealed their fate.

UPDATE: The full interview can be read here for a limited time.

The Wall Street Journal includes a special report on technology today, which features a fairly lengthy interview with me on “how to be a smart innovator” (requires subscription). I question the popular notion that “companies should be innovating everywhere,” arguing instead that companies should narrow their sights when it comes to innovation: “You need to bring the same kind of discipline to deciding where you innovate as you’d bring to any other kind of management question. You want to make sure that you innovate in those few areas where innovation can really pay off and create a competitive advantage and not innovate in other areas where it won’t pay off.” One of the dangers of placing too much emphasis on innovation, I suggest, is that a company can end up devaluing the work of the “merely competent.” In fact, having highly competent employees is usually every bit as important to a business’s success as having highly creative ones.

There have been some interesting comments on my post about the divide between deletionist and inclusionist wikipedians. I suggested that the split is a manifestation of the deeper divide between absolutists and relativists. Morgan Goeller sees it as a replay of “the battle between coherentism and foundationalism.” Dermot Casey says it’s “another round in the battle between the Big Endians and the Little Endians.” Kevin Kelly writes, “This grand dichotomy also resembles the ancient and huge gulf between ‘lumpers’ and ‘splitters’ in the biological taxonomic world (and somewhat in the library classification world). The lumpers tend to want to shoehorn a new-found organism (or subjects) into the existing categories (too many categories become a junkyard), while splitters tend to want to fork categories and create new species or subjects (to increase distinction and precision).”

All of these seem reasonable interpretations. I would suggest, though, that the “lumpers” are better represented, in the Wikipedia world, by the “mergists” than by the deletionists. The adherents of mergism – the Association of Mergist Wikipedians (see logo) currently has 109 members – believe “that while much information may warrant inclusion somewhere, very little of it probably warrants its own article.” Like lumpers, they look to combine narrow subjects into broader categories. I do think, though, that the mergists are best viewed as a subsect of the deletionists, even though they see themselves as being “grounded firmly in the center of the Inclusionism-Deletionism spectrum.” (They are also, of course, a subset of the delusionists.)

Maybe the most interesting aspect of all this mumbo-jumbo is the way it reveals the rampant sectarianism of the wikipedians. In adddition to the deletionists, inclusionists, mergists, and delusionists, there are (in alphabetical order) the antistatusquoists, authorists, communalists, communityists, darwikinists, encyclopedists, essentialists, eventualists, exclusionists, exopedianists, immediatists, incrementalists, metapedianists, politicists, rehabilists, statusquoists, sysopists, and wikipacifists. Many of these sects also have subsects. For instance, there are extreme statusquoists and moderate statusquoists as well as extreme antistatusquoists and moderate antistatusquoists.

Which leads me to an inevitable conclusion: The human urge to divide communities is just slightly stronger than the human urge to create them.

There’s something rotten in the commune of Diggland. First the peasants revolt, claiming they’re being starved of attention by a cabal of self-dealing aristocrats. Then Kevin, the boy-king, tries to placate the masses by blogging a mealy-mouthed missive. Insulted by the boy-king’s insinuation that they’re gaming the system, the aristocracy rebels. In high dudgeon, the #1 Digger of all, the mysterious X-P9, announces his self-exile:

As a direct result of your blog this evening. I will no longer [be] supporting Digg going forward. I bequeath my measly number one position to whoever wants to reign … I believe you to be a good man, Kevin. Well intentioned or not: your blog satisfied malcontents equipped with baseless allegations while you effectively urinated on your top diggers.

Didn’t anyone tell these guys that they’re “an egalitarian community of readers,” a model for our great new world of social production and citizen media? Live up to our ideals, dammit!

UPDATE: Donna Bogatin sees the bigger picture: “Digg, Google and Wikipedia are invested in maintaining their ‘as-is’ status-quo, no matter how flawed. Not one of the three powerhouses can risk diminishing public confidence in the grandiose vaunted missions each espouses. The leaders of each of the flawed systems publicly evangelize a revolutionary worthiness of their endeavors to rationalize away allegations of abuse, entrenchment, spam, falsehoods, libel, infingement … with a ‘net-positive’ argument.”