The editor and the crowd

Last weekend, two prominent technology bloggers, Dave Winer of the venerable Scripting News and Robert Scoble of the Microsoft-sponsored Scobleizer, expressed their frustration with Tech Memeorandum, a popular website that highlights the headlines of technology-related stories appearing in blogs, newspapers and other media. In Winer’s view, Memeorandum has turned into a tedious contest “with one blogger trying to top another for the most vacuous post.” Scoble, echoing Winer’s complaint, announced that he was going to avoid looking at Memeorandum “for at least a week” and instead rely on his self-selected RSS feeds to track technology news. Others have also been critical of Memeorandum, suggesting that its content is overly narrow or that it draws from too small a pool of sources.

But what exactly is being criticized here? How does Memeorandum choose what appears on its much-trafficked homepage? The answer is, it doesn’t choose – at least not in the way we typically think of “choosing.” Memeorandum doesn’t employ any editors to sift through the hundreds of technology stories that appear every day and select a handful to highlight. Rather, the site uses a software formula, or algorithm, to do the sifting and selecting. The exact nature of the algorithm, written by Memeorandum founder Gabe Rivera, remains confidential, but we know that it works by tapping into what’s come to be called “the wisdom of the crowd.” Like Google’s search algorithm, it tracks the actual choices people make while using the internet – what they look at, what links they follow, what links and words they choose to put into their own blogs or sites, and so on – and uses that information to calculate the crowd’s collective judgment about popularity, authority, timeliness and importance. Those calculations in turn determine which headlines appear on the Memeorandum homepage and the order in which they appear – just as similar sorts of calculations determine the results served up by Google, Yahoo and other search engines. The content of the Memeorandum homepage changes every five minutes, as the algorithm takes in more information and revises its calculations.

In a very real sense, the crowd takes the place of a human editor on a site like Memeorandum.

Collecting crumbs

Because it uses software to, in effect, model the mind of the crowd, Memeorandum is a good example of a second-generation internet company – a “Web 2.0″ business, as they say. In his influential essay What Is Web 2.0?, Tim O’Reilly identifies “harnessing collective intelligence” as one of the tenets of Web 2.0. In fact, he says, it’s “the central principle behind the success of the giants born in the Web 1.0 era who have survived to lead the Web 2.0 era.”

O’Reilly points out that the “intelligence” of the internet’s users is naturally (and automatically) embedded in the web’s hyperlinked structure, or architecture:

Hyperlinking is the foundation of the web. As users add new content, and new sites, it is bound in to the structure of the web by other users discovering the content and linking to it. Much as synapses form in the brain, with associations becoming stronger through repetition or intensity, the web of connections grows organically as an output of the collective activity of all web users.

What Memeorandum and companies like it do is use software to discern patterns in this “web of connections,” patterns that, they hope, can be turned into useful or otherwise desirable online products and services. They mine the wisdom of the crowd, and then sell that wisdom back to the individual members of the crowd (either directly or, more typically, indirectly through advertising). As O’Reilly takes pains to note, this process has less to do with encouraging the active, conscious participation of web users in creating content than with simply following behind users, collecting the crumbs of “intelligence” that they leave behind as they journey through the web’s, or a single site’s, hyperlinked architecture:

One of the key lessons of the Web 2.0 era is this: Users add value. But only a small percentage of users will go to the trouble of adding value to your application via explicit means. Therefore, Web 2.0 companies set inclusive defaults for aggregating user data and building value as a side-effect of ordinary use of the application … They build systems that get better the more people use them … The architecture of the internet, and the World Wide Web, as well as of open source software projects like Linux, Apache, and Perl, is such that users pursuing their own “selfish” interests build collective value as an automatic byproduct.

In his book on Google, The Search, John Battelle makes a similar point using different terms. He says that the internet contains a “database of intentions.” Every search we make, every link we click, every word we write, every moment we spend looking at a page – each is a little piece of data about ourselves that we leave behind. In combination, all the billions and billions of bits of data left by millions and millions of web users turn the internet into a great database not just of intentions – “of desires, needs, wants, and likes” – but also of “collective intelligence,” which companies like Memeorandum are free to mine with software and forge into new products and services.

Mild or spicy?

When people criticize Memeorandum, therefore, they are not really criticizing Memeorandum. They are criticizing the crowd and the crowd’s “wisdom.” After all, in good Web 2.0 fashion, it is the crowd that is “choosing” what appears – and what does not appear – on the Memeorandum homepage.

It’s useful at this point to take a closer look at that homepage, and to compare it with the homepage of a similar site that uses a much different, and more familiar, method of “choosing”: Slashdot. Founded in 1997 by Rob Malda, Slashdot provides “news for nerds”; it is a forum where software developers, information technology professionals and other “geeks” can discuss various topics of interest to them. Slashdot has developed sophisticated software to mediate those discussions. But decisions about which stories to highlight on Slashdot’s most valuable real estate, its homepage, are not made by software algorithms. They’re made the old-fashioned way: by people. Usually, in fact, they’re made by just one person, Malda. He acts as Slashdot’s very human editor. “If you’ve been reading Slashdot,” Malda writes about the site’s editorial method, “you know what the subjects commonly are, but we might deviate occasionally. It’s just more fun that way. Variety Is The Spice Of Life and all that, right? We’ve been running Slashdot for a long time, and if we occasionally want to post something that someone doesn’t think is right for Slashdot, well, we’re the ones who get to make the call. It’s the mix of stories that makes Slashdot the fun place that it is.”

As I write this, there are 15 stories on Memeorandum’s home page. Twelve of them have to do with the development of internet-related products or services: new web sites, new features on existing web sites, new mobile services, new computing or communication devices or components. One is a list of the most memorable villains in video games. One is a list of “proverbs” for technology entrepreneurs. And one is about the invention of a new device to help paralyzed people communicate. If you visit Memeorandum regularly, you’ll recognize this as a fairly typical assortment. The great majority of the stories Memeorandum highlights tend to be about actual or rumored introductions of new web sites and services or computer or communications devices, supplemented by debates about blogging practices and passing controversies involving the media or internet technologies or companies. By any measure, the site presents a very , very narrow slice of the world of “technology.”

Three of the stories on Memeorandum are also among the 16 stories featured on the Slashdot home page right now. Slashdot also has six other stories on information technology topics, mainly involving software development or the operation of corporate IT departments. But nearly half of Slashdot’s stories don’t fit the mold. They range across a variety of technology and science subjects, and they’re often surprising: one’s on the use of bacteria to “eat” discarded styrofoam, one’s on a possible link between coffee consumption and heart attacks, one’s on evidence that human genes are still evolving, one’s on the discovery of a “hairy lobster,” one’s on nuclear power and climate change, and one’s on the breeding of “designer mice” for experiments. Although Slashdot’s target audience is far more limited than Memeorandum’s, its content is far more diverse. It’s also, to my eye, anyway, more engaging, interesting and, yes, fun.

One can see on Slashdot an active, interested, engaged mind at work – the mind of a skilled editor. In comparison, Memeorandum feels flat and wooden, like the output of a computer. Memeorandum is claustrophibic where Slashdot is expansive. Memeorandum is mired in the predictable while Slashdot revels in the unexpected. Memeorandum plays it safe; Slashdot takes chances.

I’m not trying to pick on Memeorandum – as I said, it’s a popular site that’s clearly delivering a valuable service to a lot of people. Its flaws are the same flaws you see on other sites that use algorithms to filter content. But I do think we can learn something important here, something about “the crowd” and “the editor” and their respective roles – and maybe, at least by implication, something about the evolution of media, too.

Mindfulness and mindlessness

As the comparison of Memeorandum and Slashdot shows, the software-mediated crowd is a poor replacement for a living, breathing, thinking editor. But there are other things that the crowd is quite good at. The crowd tends, for instance, to be much better than any of its members at predicting an uncertain future result that is influenced by many variables. That’s why stock market indexes beat individual money managers over the long run. It’s easy to understand why. First, there are limits to the ability of any single individual to understand the complexities in how a large number of variables change and influence one another over time. Second, every individual’s thinking is subject to idiosyncracies and biases – some conscious, some not. The crowd aggregates all individuals’ knowledge about variables while balancing out their personal biases and idiosyncracies. It’s not the “wisdom” of crowds that makes crowds useful, in other words; it’s their fundamental mindlessness. What crowds are good for is producing average results that are not subject to the biases and other quirks of human minds.

That’s also why search engines work pretty well with algorithms (until, at least, they begin to be gamed by individuals using their minds): They produce the result that best suits what the average searcher is looking for. You don’t want generally used search engines to reflect individual biases. Indeed, one of their main jobs is to filter out those biases – and revert to the average.

But that’s also why algorithms don’t work very well as editors. With an editor, you don’t want mindlessness; you want mindfulness. A good editor combines an understanding of what the audience wants with a healthy respect for the idiosyncracies of his own mind and the minds of others. A good editor doesn’t aim to provide a bland “average result”; he wants to wander widely around the average, at times even to strike out in the opposite direction altogether. The mindless crowd filters out personality along with idiosyncracy and bias. The mindful editor is all about personality. “It’s just more fun that way,” as Malda says.

Of course, such distinctions may not matter all that much in the future. Running an algorithm, after all, tends to be a lot cheaper than paying a staff of idiosyncratic editors, particularly when you’re trying to corral something with the vastness of the web. As Memeorandum’s popularity shows, moreover, an algorithm may often be “good enough.” For distracted people looking for a quick fix of information, a mindless average may be just what the doctor ordered. And if we can give that mindless average a sexy name like “collective intelligence,” it can start to look downright attractive. As we adapt to the internet, we may just learn to forget that an algorithm, no matter how elegantly conceived, is no substitute for a person, and that a crowd, no matter how full of “wisdom,” is no substitute for an editor.

But I hope we don’t.

15 Comments

Filed under Uncategorized

15 Responses to The editor and the crowd

  1. This post is Memeorandum-bait, right? :-)

    Just a few notes:

    This is a bad example – “That’s why stock market indexes beat individual money managers over the long run”. As you know, indexes beat almost all managers over the long run, for the simple reason that everyone can’t be above average. But I think it’s misleading to call it any sort of group “ability”. What’s more interesting is that *dartboards* are competitive with individual money managers – but nobody talks about the “wisdom of darts” (because there are no DartBoard 2.0 salesmen …).

    I don’t think this is correct: “Usually, in fact, they’re made by just one person, Malda” – that is, I think the percent of stories he actually selects these days, is very low (though I haven’t been following the site much for a while). Doesn’t affect your main point, but I believe it’s mostly staff. Plus he hired at least one person who had formal, professional, journalistic editing experience (to baby-sit everyone else, wags claimed …). And there were problems with one abusive “editor” who seemed to regard the site as a personal ranting-platform, as well as trashing subsequent criticism.

    Basically, I agree with the idea that data-mining is hardly as great as its mystification. On the other hand, the comparison you use here is a bit apples-and-oranges. Better would be Slashdot vs. Digg.

  2. To complement you analysis, I deeply recommend this presentation by the author of Wisdom of Crowds about the danger of “information cascade” that can ruin the positive effect of wisdom of crowds

    http://www.itconversations.com/shows/detail468.html

    I believe that what D. Winer & R. Scoble express is another way of living this “information cascade”

    This presentation is summarized this way:

    In technophile circles, the idea that networks and network effects will inherently provide for better decision making is an understood, a truism widely agreed. Author and New Yorker columnist James Surowiecki, argues that while there are many benefits to aggregate decision making, there are several perils and misbehavior that individuals and observers would be wise to take into account.

    Drawing on research for his recent book The Wisdom of Crowds, Surowiecki explores several areas in which a group’s process can result in improper decisions. These failures are traced to the problem of individual humans acting when aware of their membership – the irony of group wisdom is that it is only when a group is unaware of its intelligence that it can be effective. In aggregate, individuals in groups can fall into one of two behavioral traps – either herd behavior (where the group will inherently move in one direction), or imitative behavior, where each member of the group will, for rational reasons, become like any other member of the group.

    Upon exploring these contradictions, Surowiecki provides several ‘food for thought’ points by which actors can make better decisions by maintaining weak rather than strong ties with other group members and by tuning into a cacophony of contrary opinions rather than the self-reinforcing common opinions of a small group.

  3. The Wisdom of the Crowd

    Yesterday, when I wrote why I don’t use Google News and alike, I wrote: While in newspaper I entrusted the editorial decisions to someone else on the internet I make my own editorial decision. The decisions, of what important or interesting, …

  4. why do local TV stations and newspapers focus more on local issues v/s national or global (more on accidents, local crime etc). They have distribution limitations. Blog world has a global distribution opportunity.

    The blog audience today is a consumer oriented and web 2.0 one. But it we are to make it much more inteersting to corporate IT and other execs, policy makers you and I and others have to focus more on their issues. Trust me a CIO does not spend more a few minutes a week focused on Google calendar or what Dave Winer thinks of you. On the other hand he racks his brains on how to get more payback or discoutns on his piece of IBM’s $ 90 b revenues and Microsoft’s 40b.

    The mistake we may make is to let an initial segment of blog market define other segments which are yet to participate. That’s where the bigger economics are and also the more thorny issues. Let’s get past our gossip, local accident coverage and keep focused on the big issues.

  5. But isn’t content on memeorandum influenced by the choice of blogs Gabe Rivera has made? This is taken from blog.memeorandum.com:

    “The source-picking algorithm is based on this philosophy and works roughly as follows: I feed it a number of sites representative of the topic area I want coverage. It then scans text and follows links to discover a much larger corps of writers within that area.”

    So memeorandum does represent wisdom of the crowds, yes, but from a preselected pool, reflecting inevitably the views of the person feeding the system.

    Not that it’s necessarily bad, but I’d like to be able to use Gabe’s system with my own choice of blogs, just to see what happens: I’m sure we’d see less RSS wars, less Google calendar, and definitely more Lego! ;)

  6. Nick, I agree on many of the limits of automation, and have felt that way for some time. I wouldn’t recomment tech.memeorandum as a wholesale replacement for human editors. Scoble neglected his favorite (human-edited) feeds, and realized the problem in doing so last week.

    But if Scoble read only Slashdot instead of his favorite feeds, I bet the results would be just as ugly.

    Now I think you’re suggesting a false alternative: either-or, automated vs. human edited, when there are ways in which automation beats humans, creating much value in the process. A human editor can’t perform the kind of real time clustering you see on tech.memeorandum. The result of this clustering is a very useful visual representation of the importance of the story to a certain group of writers, and a helpful, timely list of links to other points of view.

  7. Makio Yamazaki

    Nick-san,

    From the viewpoint of “Enterprise use” of Web 2.0,MIT Thomas Malone professor said Organization would be

    changing into the adjustment and the upbringing-movement from the top-down management.

    In other words, it looks like the dicision-making of bottom-up in the organization.

    There would be possibilities of “the distributed organization” which is different from the conventional organization. The software tool of web 2.0 would have new business styles that would be similar to “Kaizen”(TOYOTA),I mean.

  8. Nick:

    Gabe confirms there is an element of human selection. Given the content, that makes sense. In one sense he’s done a great job of aggregating the collective wisdom of a relative few in a narrow space.

    But as Vinnie implies, it’s time to move on. There are huge issues out there that could do with reaching a wider audience. Tech.memorandum is one way to do that. Others will emerge.

  9. Memeorandum: Does a Diggbot Trump Manual Digg-ing?

    From Nick Carr’s Rough Type blog: this review of Memeorandum. Very relevant for online publishers looking for ways to embrace the blogosphere and build valuable services over it (like Edgeio, which we commented on earlier this week.) Carr’s critiqu…

  10. Internalizing, Interpreting, and Identity: What do Attention Trackers Do in a Social Sense?

    David Smith has an amusing and intriguing post about the surveillant potential of attention trackers. I briefly covered this in my last post on the new control society but it was a little buried in the entry. I think it…

  11. Marcelo Lopez

    Nick,

    Good post, sorry I didn’t pick up on it sooner, but just the same. But just the same, I want to point out what has been a great issue with Slashdot ( /. as we regulars call it ). Over the past 7 or so years that I’ve been on /., I’ve noticed that progressively the pace of which it has been able to process and present ( and this is for you Didier Durand ) what Dave W. calls a “River of News”. /. has progressively been less and less at the forefront of providing TIMELY news and information to the reader. Whereas a year ago, I would read it almost religiously, now I scan it perhaps daily, if I’m so moved.

    Why ? I’m glad you asked. It’s because the “River of News” coming from other avenues. Digg.com ( user-moderated, and therefore closer to Meme’s way of doing things, WITHOUT the fancy shmancy algorithm ), or a conjunction of some other RSS feeds will give me my news these days. The traditional model of Editor Moderated news simply cannot keep up in today’s web with group think.

    Unless rather narrowly focused ( like Engadget, Gizmodo and others like them ) to certain boundaried realms, news sites like /. are at risk of being “washed over” by a river of other news providers that can more quickly adapt.

    I think that that can also be said of most print newspapers today. The “River of News” could be the “saving grace” of Web 2.0. But I’ve still yet to see someone “get it right”. Close, but no cigar.

  12. Steely gal

    Tech memeorandum is a great site and very workabl;e. This is the first time i came across it thanks to his blog. ITs like an everyting under 1 roof kind of a place.

    Steely gal

  13. why do local TV stations and newspapers focus more on local issues v/s national or global (more on accidents, local crime etc). They have distribution limitations. Blog world has a global distribution opportunity.

    This is a bit misleading. While it’s true they have distribution limitations it’s also true that people care more about their immediate community, things that may have a more direct impact on their lives, than many more global events. As important as some global news is, it often doesn’t have an immediate or obvious impact on everyone. However, the city increasing my property taxes does.

  14. Sorry … I should have noted that quote was from Vinnie’s comment not the post.

  15. Like others commented, Apple has been consistently providing the best tools to create and manage digital content in the last years, it’s the very card they are playing in their commercials to say that MacOS X is different from Windows.

    Saying that Apple is against UGC seems quite stretched. They do want full control of their platorm, it’s the way the choose to keep users experience clean and rich, but changing batteries is not UGC, imho.saç ekimi saç ekimi evden eve nakliyat

    evden eve nakliyat