The law of the wiki
October 18, 2005
The Register's Andrew Orlowski finds "encouraging signs from the Wikipedia project, where co-founder and überpedian Jimmy Wales has acknowledged there are real quality problems with the online work." Noting that up to now "criticism from outside the Wikipedia camp has been rebuffed with a ferocious blend of irrationality and vigor," Orlowski quotes from an October 6 post that Wales made in response to my criticism of Wikipedia's quality: "I don't agree with much of this critique ... but the two examples he puts forward are, quite frankly, a horrific embarassment. [[Bill Gates]] and [[Jane Fonda]] are nearly unreadable crap." Although the Internet encyclopedia "appears ill-equipped to respond to the new challenge," writes Orlowski, "at least Wikipedia is officially out of QD, or the 'Quality Denial' stage."
Other responses to my critique from wikipedians (as I write that word, I am overwhelmed by a vision of very tiny, very cheerful people dressed in green velvet dancing in a dell) point out that some of the encyclopedia's entries, particularly those on technical or otherwise esoteric subjects, are quite good. That's true, and it's worth thinking about. Why are entries on arcane topics generally better than those on more broadly understood topics? I would guess that the reason's pretty simple: Fewer people contribute to the arcane entries, and the contributors all tend to have some degree of specialized expertise in the field. For general-interest subjects, on the other hand, it's a free-for-all. Everyone feels qualified to add his two cents, and the entries decay toward, as Wales puts it, "unreadable crap."
In other words, the example of Wikipedia actually undercuts, rather than supports, the Web 2.0 tenet of "collective intelligence." It reveals that collectivism and intelligence are actually inversely correlated. Here, then, is what I'll propose as the Law of the Wiki: Output quality declines as the number of contributors increases. Making matters worse, the best contributors will tend to become more and more alienated as they watch their work get mucked up by the knuckleheads, and they'll eventually stop contributing altogether, leading to a further fall in quality. As Dave Winer puts it, "No matter how good something is, there are always more idiots and morons to take it down."
Now, there's a way around this "collective mediocrity" trap. You can abandon democracy and impose centralized control over the output. That's one of the things that separates open-source software projects from wikis; they incorporate a rigorous quality-control filter to weed out the crap before it pollutes the product. If Wikipedia wants to achieve it's goal of being "authoritative," I think it will have to abandon its current structure, admit that "collective intelligence" makes a pretty buzzphrase but a poor organizational model, and define and impose some kind of hierarchical power structure. But that, of course, would raise a whole other dilemma: Is a wiki still a wiki if it isn't a pure democracy? Can some wikipedians be more equal than others?
I don't think it's necessarily true that the number of participants decreases the quality of the output. It depends on the subject and the people involved. Two physicists can write an entry on Quantum Mechanics, but twenty of them can fill it out will all kinds of specialized information that only two wouldn't have to offer. But that's because it's a circumscribed topic relevant to a somewhat narrow community of practice (which is actually what wiki's were sort of created to support to begin with).
Once you start opening things up to *anybody* about *anything* -- i.e. "vulgar" interests -- you will end up with mediocre writing and information about topics that appeal to the lowest common denominator. That's just how human systems structure themselves.
So yes, once you take the "pure wiki" out of the rarified environment of specialists or small groups, you definitely have to impose some top-down structure and peer review. I don't think anyone but the most absurdly wide-eyed idealists would say differently.
Posted by: Andrew Hinton at October 18, 2005 03:40 PM
Good point, Andrew. I was talking about truly democratic wikis. Specialized wikis, with expert participants, would certainly get better as the membership grows. I wonder, though, whether even these would reach a breaking point - beyond which additional members begin to erode quality rather than bolster it.
Posted by: Nick at October 18, 2005 04:52 PM
Your Law of the Wiki that output quality declines as the number of contributors increases is an interesting observation and is more than likely true. However there is one more related observation of an information source - as the level of general knowledge on a particular topic decreases, the value of an article of that topic increases for the majority. For example, most people would claim reasonable knowledge of what tap water is, where it comes from and it's general properties. As you might expect from your "law", the article isn't of a hugely high standard. However, the value to be gained by most from such an article is limited as there is less new information. In contrast, a lot of people might not know so much about heavy water and therefore the (higher-quality) article is of more value to them. Orlowski misses this point in the original article: The value of wikipedia's content lies in the thousands of well-written articles of low knowledge base, rather than in those of common knowledge to which everyone feels they can contribute.
Posted by: Douglas Stokes at October 18, 2005 06:06 PM
In other words, wikipedia has a long tail (of sorts).
Posted by: rd_cc_mtnview at October 18, 2005 08:54 PM
In his book, The Wisdom of Crowds, James Surowiecki noted 3 requirements that needed to be met for a crowd to be smarter than the individuals in it. One of the requirements was the ability to aggregate results, which is the one that Wikipedia needs to address.
In an open source project there are two ways to aggregate contributions: (1) by the owner of a current module incorporating the "good" code contributions & discarding the "bad", and (2) by a new contributor setting up a new module if his contribution is rejected (i.e. forking). The users and other developers also vote on which code base they use.
For Wikipedia an alternative to the "hierarchical power structure" proposed by this blog might be to give ownership of an article to whoever wrote it. People who want to improve it need to submit their changes to be incorporated by the page owner. Like open source, some of these contributors will be granted the right to edit the page directly & to approve other suggested changes. If a person's suggestions are rejected then he or she has the option of creating a new page on the same topic.
Anyone trying to visit that topic would then see a special page with pointers to all the different articles various people had written---hopefully ranked in order of usefullness by reader votes. Such a process rewards quality without the need for a centralized authority structure.
Posted by: Ron Goldman at October 18, 2005 09:53 PM
how to become a wiki critic and get rich quickly?:) not unlike in academia, ownership of an article in wikipedia could lead to what is known from domain grabbing, buzzwordism, and folksonomy-sloganomics, finding words and terms which are formerly unused and claim them related to an author name or brand. certainly the desire to become a "term-owner" can lead to more diversity, but not neccesarily to more substance. on the other hand people would become more inventive of establishing and extending existing terminologies. however wikipedia due to an architect (jimbo wales) it's rather a cathedral than a bazaar, thank god :)
Posted by: pitsch at October 19, 2005 12:02 AM
There's another dynamic at work here, too, and simply giving "ownership" of a subject to an individual may amplify Wikipedia's quality problems.
Former contributors to the project, such as Jason Scott (here and here) point out that the project rewards administration rather than content. This deters expert contributors, who simply give up fighting. The winner of a dispute is the last man standing.
It's the "Look busy, here's the boss" syndrome. I hear this so often it can't be attributed to sour grapes from someone who's lost a vote.
The main difference between this and other similar academic environments is the pure speed at which stuff can happen; you can literately have 30-40 little editing nibbles on a page within a single day. If people are feeling frisky, it can take place in a few hours. This means that you get all the politics and turf war of Ivory Tower Academia without the mitigating barrier of time to cool down or consider. That is, you get a nice big mess.
Wikipedia has a large contingency of users who play the Wikipedia Rules of Etiquette and Procedure like they were Role Playing Games and function within them causing havok and personal gratification at the expense of moving the project forward.
The result is that even less premium is placed on quality. Look how highly the issue figured in the candidates' election literature. One wrote of "... our core principles of neutrality, civility, freedom and openness". Creating a great reference work isn't even a goal.
Institutions can and do change, but only when they realize they must reform to stay relevant. Such is the determinism exhibited by Wikipedia's supporters - it reminds me of Marxists - this may not happen any time soon. But it will.
If we're honest, the only reason Wikipedia has generated such attention is through the absence of free online reference material, because of copyright holders' compensation issues.
But these are fixable: the quality of reference works available to me instantly online here is amazing - have a look. This is Wikipedia's real competition.
Libraries are already experts at collectively negotiating with rights holders. If I had to place a bet on the future of reference works, I'd bet on this.
Posted by: Andrew Orlowski at October 19, 2005 01:50 AM
Del.icio.us is another example where the quality declined as the number of contributors increased. In its initial run the social bookmarkers of del.icio.us were the technical and web-savy elite and inevitably they converged on interesting things. Now the bookmarks and quality of annotation ('tags') are mostly noise/
Posted by: Richard Frank Meraz at October 19, 2005 03:37 AM
After a similar discussion with students, I posted an entry to Wikipedia which, if not corrected, would be perfectly innocent but which should not have been allowed. In fact it was picked up. So some form of auto-regulation is in place. However, many of the articles are very incomplete and one-sided. If the specialised pieces seem better it may be because they are harder to understand. Maybe all entries should remain but readers should assign them ratings, which would automatically force the dross to the bottom.
Posted by: Peter Dean at October 19, 2005 05:25 AM
Another relevant point is that if you make a fool of yourself on wikipedia, you don't do it in front of an audience you care about. We're all tourists there. That's much less true of specialist subjects.
Posted by: Andrew Brown at October 19, 2005 06:45 AM
Though I don't mean to act as a hype-mouthpiece for an unfinished project, I did come across a project on Seth Godin's website (http://sethgodin.typepad.com/) yesterday that perhaps offers an approach to this problem. Read down for "Squidoo". From what I can gather (and I might have it wrong), they are seeking to keep individuals' inputs separate. Seems like this would go a long way towards solving this problem of mixing red, green and blue and only ending up with grey.
Posted by: Giles at October 19, 2005 07:47 AM
Nick's focus on QC at Wikipedia is laudable and necessary. The examples he cited were atrociously bad, indeed.
I do not believe the Gates & Fonda entries -- or any of the possibly thousands which are incomplete or distorted -- are enough to take Wikipedia down, nor to quell my own enthusiasm for some of the pricipals of Web 2.0 we've been talking about a lot lately.
The educated human can suss information quality for herself -- and Bill Gates or Jane Fonda could with little trouble have their maids emend the text.
Posted by: Sam Hiser at October 19, 2005 10:00 AM
You have some valid points but I have to disagree. The concept of Collective Intelligence promises a potential for great knowledge but ONLY if ALL participate and offer what they know (see Pierre Levy’s book Collective Intelligence for more). The real problem here isn’t that there are immature yahoos posting inaccurate information because those posts are rectified quickly enough. The problem is that not enough qualified, intelligent people are willing to share what they know for the greater good.
What’s the last thing YOU added to Wikipedia? Why critique it when we can improve it?
Wikipedia, and many other Web 2.0 applications, function on an evolving knowledge model that encourages valid information (don’t read that as accurate because I think that biased information can just as important) will be sustained and invalid information will be cut out. It’s a “Survival of the Fittest” model for knowledge collection and it can work.
Posted by: Sarah Robbins at October 19, 2005 11:48 AM
Actually, your "law of the wiki" has been formulated more generally in the past, and is accompanied by an equal and opposite formulation of why problems occur with critical but esoteric issues about which few have expertise:
I had just posted this link in a comment on your October 3rd blog entry; it seems as if it may be even more relevant to this one.
Posted by: Jud at October 19, 2005 12:15 PM
I'm not sure how a discussion of "collective intelligence" is at all relevant to Wikipedia.
By definition, Wikipedians are supposed to regurgitate unbiased articles based on existing knowledge and universally accepted meanings. Just the facts, with no room for independent thinking, creativity, knowledge creation, etc. Indeed "original research" is actively discouraged, and removed from the site.
This is not collective intelligence we're talking about, this is more just a collective unpaid internship.
As to your point that "Output quality declines as the number of contributors increases" - I'm not sure if I follow your jump. As long as there is sufficient fragmentation - experts in Britney Spears are not posting in the Quantum Physics articles, and vice versa - I'm not sure if I buy that simply increasing the number of unspecialized participants will infect the specialized articles.
Posted by: lawrence coburn at October 19, 2005 01:19 PM
If anything, *over* contribution seems to be the problem, as much incomplete contribution. Large chunks of our culture emphasises blagging, the act of doing your best to seem and act like you know what you're talking about and in most cases that's a useful skill. Armchair professionals and all that. Rarely are we taught to confess that "I don't know anything about that" and back off and wait for someone who knows better to come along. So you get a lot of people chipping in, feeling like they should add their 2 cents to a particular article, but actually they are only serving to dilute the quality of information rather than enhance it.
And um ... "What’s the last thing YOU added to Wikipedia? Why critique it when we can improve it?" ... by critiquing it and raising this debate, he's helping to improve it. He doesn't have to act directly to assert influence.
Posted by: Giles at October 19, 2005 01:47 PM
Those that can, do. Those that can't, teach. Those that have nothing better to do, maintain Wiki. The competent ones are too busy "doing" to babysit the Wikers' work.
Let's face reality. The ones who are most competent aren't going to be uber-contributers because they are busy doing, and wouldn't get paid to "contribute." Their contribution is focused elsewhere. Traditionally, their teaching comes through mentoring, leadership in management, or formal teaching at a school after they have established themselves. Their time would be wasted performing maintenance on a Wiki where their contribution could get overwritten by an eager beaver who did enough cleanup to become an administrator.
That's not to insult all Wiki contributers. But I've been exposed to quite a few situations (without being personally involved) in which competent people have their competent entries replaced by incompetent, biased, or politically-polerized fluff. It happens. Although Wiki's are useful, they are not a haven for dependably trustworthy information.
Posted by: Brian Westmoreland at October 19, 2005 03:48 PM
"By definition, Wikipedians are supposed to regurgitate unbiased articles based on existing knowledge and universally accepted meanings. Just the facts, with no room for independent thinking, creativity, knowledge creation, etc."
Supposed to...yet some of the better-quality entries will always reflect special knowledge.
Taking my example of the OpenDocument entry, there is a lot of knowledge imbedded in David Wheeler's recent updates, in the organization and selectivity as to what is relevant.
The facts aren't the half of it.
Posted by: Sam Hiser at October 19, 2005 09:25 PM
That was actually Lawrence Coburn's comment you were replying to (comments precede rather than follow sigs).
Posted by: Jud at October 19, 2005 11:30 PM
I've been following this as best as I can and I'm not sure that the problem of Wikipedia, or indeed collective intelligence, is restricted to the web in any of its incarnations. As a writer myself, I value the contributions that one or two people can make to a piece. any piece. They may be people with special knowledge but no writing ability, or vice versa, but either way they can often improve something I've done. Start to increase the number, though, and the piece rapidly falls apart if I accept all their suggestions.
And that's the point I think everyone is making. Let the writers and editors write and edit. Let the experts contribute expertise. Let those who can do both be praised throughout the land.
Thanks for a thoughtful blog.
Posted by: Jeremy Cherfas at October 22, 2005 02:31 AM
Often a single hand is worse. The entry on Franjo Tudjman, the former Croatian leader was, as someone else pointed out, taken verbatim from a Croatian government handbook. Unfortunately it still is since various insertions of fact, for example, as to his public commitment to the genocide of Orthodox Christians, on God's command, have, in turn been deleted.
Posted by: Neil Craig at October 24, 2005 06:36 PM
I think that Wiki represents a part of our common future. Hierarchy may or may not be needed, but not to the extent that it has been used in the past. The fact that it needs an editing "function", does not neccessarily imply a hierarchy. For an example of technology that performs judgement without a central heirarchy, look at any of the predictive markets or idea futures: http://en.wikipedia.org/wiki/Prediction_market .
I am wondering how you could use something like this as a place to "vote" on an article. The voters would be people viewing a Wiki entry and then making a "bet" on whether a given article is going to "survive". Granted this is just quantitative, but over time more measures can be found and applied.
So this would be using polling to try to replace individual editorship. Wonder if anyone cares to suggest other technologies?
Posted by: Bob Besaha at October 25, 2005 09:11 AM
The definition of "irony":
Posted by: John at October 26, 2005 03:52 PM
John: That's pretty funny. What's even funnier is that there's a wikipedia entry for myself and at the moment under "see also" there's a link to the entry for "Satan." I've arrived.
Posted by: Nick at October 26, 2005 04:15 PM
The problem of too many additions to the articles is compounded by the fact that in wiki it is much more difficult to delete something than to add something. It is somewhere deep in our culture that everyone is entitled to say his opinion on a fact, this is the freedom of speech and democracy. Deleting is like censorship, it is restricting the users right to express his opinion.
Posted by: Zbigniew Lukasiak at October 27, 2005 03:48 AM
Post a comment
Thanks for signing in, . Now you can comment. (sign out)(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)
"Riveting" -San Francisco Chronicle
"Rewarding" -Financial Times
"Ominously prescient" -Kirkus Reviews
"Riveting stuff" -New York Post