« An IT sea change for smaller companies | Main | Concerns grow over online advertising »

What will kill Citizendium

September 20, 2006

"An expert-focused Wikipedia," writes Cory Doctorow, in reference to Larry Sanger's proposed Citizendium, "would likely devolve into interminable pissing matches over who was and was not qualified to be called an expert, because expertise isn't a measurable quantity, but rather something that is socially constructed." What Doctorow is predicting in the first half of that sentence - without actually saying it - is that Citizendium will turn into Wikipedia. Wikipedia is nothing if not a forum for interminable pissing matches, over everything from the qualifications of contributors to the worthiness of subjects to the meaning of vandalism, among people with remarkably capacious bladders. As Angela Beesley, a Wikipedian steward and board member, recently said, "Creating fewer articles as time goes on seems fairly common as people get caught up in the politics and discussion rather than the editing." Wikipedia today has more layers of bureaucracy than the average Fortune 500 company and more factions than the Italian parliament.

But it's the second half of the sentence that delivers the payload: "expertise isn't a measurable quantity, but rather something that is socially constructed." I love the offhandedness of that, as if Doctorow were simply repeating a universally acknowledged truth. Expertise? It doesn't really exist. It's just something we imagine together, a vaporous communal conjuring that can shift its shape with opinion's wind. Expertise is no more solid than celebrity.

Doctorow here is glossing Clay Shirky's recent critique of the idea of Citizendium (it doesn't actually exist yet), in which he labeled as "false" the belief that "experts are a special category of people, who can be readily recognized within their domains of expertise." Continues Shirky, "experts are social facts - society typically recognizes experts through some process of credentialling, such as the granting of degrees, professional certifications, or institutional engagement. We have a sense of what it means that someone is a doctor, a judge, an architect, or a priest, but these facts are only facts because we agree they are."

This is top-shelf guff, which reveals, as if we needed to be reminded, that intellectuals make the very best anti-intellectuals. An architect does not achieve expertise through some arbitrary social process of "credentialling." He gains expertise through a program of study and apprenticeship in which he masters an array of facts and techniques drawn from such domains as mathematics, physics, and engineering. If you doubt that, grab someone off the street and have him design a skyscraper. The same goes for a doctor, an electrician, a jet pilot, a software engineer, a zoologist, a scholar of romantic poetry, or a manicurist. The degree or certification is no more the essence of expertise than a wedding ring is the essence of love. The degree or certification is a trivial adornment. The expertise is what's real - it's what abides - and to call it socially constructed is not only to denigrate the expert but to denigrate the very basis of culture.

I tend to agree with Shirky's belief that Sanger's concept of Citizendium is fatally flawed (though I wholeheartedly support Sanger's desire to counter the destructive force of anonymity). But Shirky's diagnosis of the flaw is all wrong. What "will kill Citizendium," he writes, is that, unlike Wikipedia, "it won't go far enough" in "its disrespect of experts." No, Citizendium's flaw does not lie in having too much faith in what Shirky dismissively calls "the rugged condition of expertise." Its flaw lies in not having enough faith in it. By creating a vague bureaucratic system in which experts gain their Citizendium credentials through community certification, Sanger is, in fact, reducing expertise to a social construct and thus rendering it meaningless, or at least turning it into a bone of endless and silly contention. He wants to have it both ways, and as a result will likely do no better than create another Wikipedia: a vast, labyrinthine garden of mediocrity.

Relativists like Doctorow and Shirky like to think of Wikipedia as a success. And they're right - as long as we agree that success, unlike expertise, is a social construct.

UPDATE: Seth Finkelstein points out that Clay Shirky adds a clarification in the comments to his post: "I should have been clearer that I'm talking about the distinction between having expertise and being an expert. The latter category is where issues of authority come in, and what seems to get Sanger worked up is his belief that the overlap of expertise and experts is large." More discussion follows in the comments to this post.

UPDATE: Larry Sanger has responded to Clay Shirky.

Comments

What is fascinating about Wikipedia is the founder's claim that it will deliver "the sum of human knowledge", or some such claptrap. The Greeks called this "hubris", and Wales seems to have it in spades.

What Larry Sanger should do is fork WP, then cut it to pieces.

There is no reson why groups of highly qualitied experts in a field cannot take the Wikipeda model and make it better, but NARROWER.

The encyclopedia model is as dead as a dodo.

Posted by: Norm Potter at September 20, 2006 05:54 PM

I would be charitable to Clay Shirky on that point - he did get taken to task for it in comments, and clarified:

"You're right; I should have been clearer that I'm talking about the distinction between having expertise and being an expert. The latter category is where issues of authority come in, and what seems to get Sanger worked up is his belief that the overlap of expertise and experts is large."

I think there's a deep point which got expressed badly:

BEING an expert may be a measurable quantity, in an abstract sense, but BEING RECOGNIZED as an expert is a social construct.

That is, in many situations, it doesn't help much to *be* an expert if you're not *recognized* as an expert.

(I am an expert in censorware, to be immodest, among the top people in the world on that topic. But I do not have a concomitant institutional backing, so I'm acutely aware of the distinction between expertise and recognition of such).

Posted by: Seth Finkelstein at September 20, 2006 06:04 PM

Nick, could you please elaborate this?--

"No, Citizendium's flaw does not lie in having too much faith in what Shirky dismissively calls 'the rugged condition of expertise.' Its flaw lies in not having enough faith in it. By creating a vague bureaucratic system in which experts gain their Citizendium credentials through community certification, Sanger is, in fact, reducing expertise to a social construct and thus rendering it meaningless, or at least turning it into a bone of endless and silly contention. He wants to have it both ways, and as a result will likely do no better than create another Wikipedia: a vast, labyrinthine garden of mediocrity."

Interesting. Please clarify. I sure want to avoid vast, labyrinthine gardens of mediocrity.

I agree with you, on the one hand, that expertise is not merely a social construct. Some people really are experts, and other people really ain't, and the difference is not constituted by any social institutions, agreements, and so forth, but epistemic states of individuals.

But insofar as expertise can be operationally defined, and thus made a practically usable notion, it does function as a social construct (as I said in reply to Clay). The question, as so many people have asked recently, "But what constitutes an expert?" is for them a deep theoretical question. But I have a reference project to start and thus am focusing on the practical question: "What process should be followed for identifying Citizendium editors?" I have made one proposal. Nothing is set in stone and I'm not wedded to the proposal I have made. What's your proposal?

Posted by: Larry Sanger at September 20, 2006 07:13 PM

Just wait until the experts start arguing about who's the real expert. Then the fun will really start :-)

Posted by: Mathew Ingram at September 20, 2006 10:08 PM

I absolutely adore watching Doctorow make a monkey of himself, he really is an expert at it.

Posted by: Chris_B [TypeKey Profile Page] at September 20, 2006 10:13 PM

It's worth pointing out that someone who has been taught and trained, even by himself, but has no degree or formal education, can be just as skilled and knowledgeable as an expert. However, there's no way anyone will hire him. (Unless, of course, he builds a reputation or joins a larger company which has a reputation.)

The degree and certification are definitely the social constructs used to judge expertise. Sanger talks about this in his response to Shirky; the (draft of the) determining criteria of expertise will be published in a few weeks, and will be on your CV. Which is a pretty bad sign.

Posted by: Michael Chui at September 20, 2006 10:43 PM

Why did you wait for other people to write about Citizendium before you offered your own thoughts, Nick? Why attack Shirky and Doctorow's understanding of the idea, why not directly address Sanger's arguments? Is it because you lack the courage to form your own opinions without them being framed as a response to others' opinions? I suppose if you did that then you'd open yourself up to attack, and that wouldn't do.

Posted by: Paul Montgomery at September 21, 2006 12:02 AM

Nice to see you and Clay agreeing on something for a change. Isn't anyone else in the "OK, Citizendium may fail for all sorts of reasons, but we don't know yet that it will, and it would be good if it succeeded" camp? Who's with me? (We may need a snappier name.)

I don't entirely agree with your argument about expert status, though. I think the fact that academic expertise is socially constructed actually works in favour of Citizendium. We know that there are such things as experts on crystallography, Elizabethan lute music, freshwater invertebrates and Italian terrorist groups of the 1970s; we also know that most of those experts are contributors to an ongoing conversation within a community of mutually-recognised and credentialled experts, and that those who aren't will almost certainly be known to the community. This kind of expertise is real - at least, it's hard to contrast it with anything equally valid and more real - and it's socially constructed. So, if you're writing an encyclopedia article about crystallography, Elizabethan lute music, freshwater invertebrates or Italian terrorist groups of the 1970s, why wouldn't you go to the (socially-constructed) experts? Only, I think, because of a prior ideological commitment to creating an encyclopedia anyone can write - and we've seen the problems that leads to.

Posted by: Phil at September 21, 2006 04:44 AM

Seth,

Thanks for pointing out Shirky's comment. It makes more sense to me then what he originally wrote, but I still find it hard to square the two. I'd suggest that what Shirky's talking about isn't really expertise at all but rather what I'll call, for lack of a better term, "intellectual authority." For an example of the difference, think about choosing, from among ten different plumbers, one to put the plumbing in your new house. You're looking for expertise. All of them can demonstrate, through their knowledge and referrals more than their credentials (does anyone ask to see a plumber's license?), that they have the necessary expertise - they've all successfully completed such jobs before. So you're comfortable that you could choose any one of the ten and have the job completed successfully. You know that each will use a slightly different technique, but you really don't care because it won't affect what you care about - which is simply that the system works. All you care about is expertise, and that's easy to establish because it's a pretty damn "rugged" quality, to use Shirky's term

Now think about choosing among those same ten plumbers one to write the definitive explanation of how plumbing is installed in a home. You're looking for intellectual authority, and that complicates things. All of a sudden, technique matters very much, as do a lot of other things, like breadth of knowledge and point of view. If you ask the 10 for their recommendation on which of them has the greatest intellectual authority, you may well get ten different answers, as each may believe his own technique and knowledge to be superior and thus believe himself to have the greatest intellectual authority (even while freely granting that all his colleagues have the necessary expertise to actually plumb a house).

It's the establishment of intellectual authority, not of expertise, that is the sticking point in the creation of Wikipedia - and of encyclopedias in general. And I think what Shirky argues is that, in the context of having a large number of volunteers writing an encyclopedia, you shouldn't bother trying to grant intellectual authority to individuals. Instead, you should ignore intellectual authority altogether - in fact, make everyone anonymous - and trust the process of communal editing to create a quality output. Now, Sanger comes along and watches that process and says, "But this is crazy. There are real experts on these subjects - not just socially constructed experts - and they should be granted greater intellectual authority." And Shirky responds, "No, that won't work. You'll just end up with the 10 plumber problem magnified many-fold, and it will impede the process. You've got to trust the process." And Sanger replies, "But look at the product. It's not very good. An expert would do better." And Shirky says, "Relax. It'll get better. It's still emerging. Trust the process."

And Sanger says, "To hell with that. I'm going to start a new project, Citizendium, where we'll know the actual identities and credentials of our contributors and we'll defer to those with real expertise." Now, I think that Sanger has a good chance - if (and it's a big if) he can enforce honesty in the way people present their identities and credentials - to establish expertise in the world of Citizendium. I don't think people have much problem recognizing expertise. We'll know, in other words, who the qualified plumbers are. But he'll still be left with the intellectual-authority problem. As long as he leaves the establishment of intellectual authority up to the crowd, as he seems to intend, he'll still have a thousand professional and amateur plumbers - along with another thousand smart teenagers who just read some web sites about plumbing - bickering about the true nature of plumbing. (And I think this is, more or less, the point that Phil made, though correct me if I'm wrong, Phil.)

So how do you solve the problem? You grant intellectual authority by some form of fiat - choose a widely respected expert and simply give him intellectual authority, or convene a small committee of respected experts and have them grant someone outside the committee intellectual authority, or hire an intelligent writer and give him the intellectual authority to synthesize the views of the experts. In other words, you keep the crowd out of it and, in essence, create a traditional encyclopedia.

So I guess I agree with Sanger's diagnosis of the flaws of Wikipedia - I see at this point no reason to "trust the process" - but at the same time I agree with Shirky's diagnosis of the flaws of Sanger's proposed solution, at least as it currently exists in outline.

And that, Paul Montgomery, is about as far as I feel comfortable in going in giving an opinion on Citizendium, given that it doesn't yet exist and that Sanger is still trying to figure out the details. I share Sanger's dissatisfaction with Wikipedia as product, I share his belief that anonymity is counterproductive, and I share his discomfort with the idea that one should maintain disrespect for experts and just trust the process. But I have doubts that his solution is going to solve the problems and, more broadly, I have doubts that the "problem" that things like Wikipedia and Citizendium set out to solve is in fact a problem. I fear that the cure is the disease.

Anyway: Establishing expertise is not a social process. Granting intellectual authority is. And writing an encyclopedia is, at a practical level, more about the latter than the former.

[By the way, I've been unable to get Sanger's response to Shirky to load on the Corante site. I suppose it's still getting BoingBoinged.]

Nick

Posted by: Nick Carr at September 21, 2006 08:50 AM

Maybe I'm being dense, but I can't see any connection between what I wrote and your thousand-plumber problem. Which I think is spurious in any case. There are just not that many credentialled experts out there for any given subject, once you define 'subject' narrowly enough to get an academic career out of it. The ten-plumber problem is real enough, but I think it would be a huge advance on the Wikipedia status quo - which is a thousand bodgers with varying degrees of expertise, all being told what to do by a foreman with no expertise at all.

Posted by: Phil at September 21, 2006 09:10 AM

Phil,

In your first comment you wrote, "So, if you're writing an encyclopedia article about crystallography, Elizabethan lute music, freshwater invertebrates or Italian terrorist groups of the 1970s, why wouldn't you go to the (socially-constructed) experts? Only, I think, because of a prior ideological commitment to creating an encyclopedia anyone can write - and we've seen the problems that leads to." The 1000 plumber problem is, I think, one of the problems you refer to. And my point is that I think it will endure in Citizendium because Sanger still appears to be maintaining the ideological commitment to an encyclopedia that anyone can edit. As Sanger writes, "It's open to virtually everyone. Virtually anyone can come to the website and, within a few minutes, be working on an article." I may just be confused: If Citizendium is going to be a product of experts, as you suggest, what exactly is the role of the crowd? Are the experts going to have to review, pass judgment on, and edit all the contributions from the crowd? That seems unworkable. Or is it that the experts will be in charge of the major entries while the crowd works on the periphery - Encyclopedia Britannica wrapped in Wikipedia?

Nick

Posted by: Nick Carr at September 21, 2006 09:26 AM

So in summary, you think that what will kill Citizendium is that it doesn't go far enough back to the EB model for your liking. Is there any point beyond the EB model at which you feel comfortable, Nick? Or do you just want EB Online and no competitors? I'd be fascinated to hear exactly how recidivist you want Sanger to be.

Posted by: Paul Montgomery at September 21, 2006 10:44 AM

Paul,

My problem with Wikipedia lies not in the fact of its existence but in the way it presents itself, is perceived, and is used and in the consequences of that positioning, perception and usage. If you go back through my writings on Wikipedia here, starting with "The Amorality of Web 2.0," I think you'll follow that theme pretty clearly, though I admit that at times I've expressed it well and at other times less well. One thing that I very much congratulate Sanger on is that he has gone out of his way not to refer to Citizendium as an encyclopedia. Wikipedia, in contrast, has gone out of its way to present itself as an encyclopedia - and I have tried to judge it on that basis.

And, by the way, I appreciate you pushing me on this. I should be pushed on it.

Nick

Posted by: Nick Carr at September 21, 2006 11:00 AM

I may just be confused: If Citizendium is going to be a product of experts, as you suggest, what exactly is the role of the crowd? Are the experts going to have to review, pass judgment on, and edit all the contributions from the crowd? That seems unworkable.

Well, the experience of Wikipedia seems to be that having nobody police anything is unworkable - which is interesting in itself, as it seems to me that nobody-polices-anything (or at worst anyone-polices-anything) was a big part of the original promise of Wikipedia. So what happens at the moment is that Wikipedia regulars review, pass judgment on, and edit the crowd's contributions following essentially formal criteria. The major change Larry's proposing (if I've got it right) is that actual subject-area experts would do that job instead, following criteria based on subject-area expertise.

In the Red Brigades article I keep banging on about, I deleted a false claim and immediately had the delete reverted. My guess is that the revert was done either by the original author sticking to his (inaccurate) source, or by a Wikipedian who objected to the lack of sourcing for my claim. But this is crazy - you'd never run a real-world encyclopedia that way. If that delete-and-revert happened in Citizendium - which is a worst-case scenario in itself - I'd be able to check out the credentials of the editor who'd done the revert, ask him what his sources were, ask him what he thought of my sources and generally talk academiese with him. You might say that I could do most of that with the Wikipedia editor; the difference is that there's no real-world motivation to do so. The Citizendium version, by contrast, fits right into the process of academic networking.

Posted by: Phil at September 21, 2006 11:56 AM

If Sanger avoids saying that Citizendium should be an encyclopedia, isn't that effectively abandoning the goals he had in helping start Nupedia and then Wikipedia?

With respect to positioning, it would be better to say that Wikipedia aims to become an encyclopedia, but frequently is not there yet, and how closely it approaches the goal is highly variable. The dilemma is that to keep the crowd focused on this goal, Wikipedia needs to declare that it is an encyclopedia already. The message is really for the crowd of editors, but due to the overlap with the crowd of readers, the audience also hears the same message. The confusion of being unable to neatly distinguish groups is one of the challenges for the more participatory models being tried these days.

Posted by: Michael Snow at September 21, 2006 12:16 PM

Nick, I think you've gotten to the core of the original issue with Clay's comments with your "intellectual authority" distinction. The real issue, in my opinion, is still a basic one: will this work?

What I find interesting is that Wikipedia's hierarchy somewhat implicitly identifies editors already; Sanger is merely saying that those editors should be explicitly qualified. It is hard to argue with this other than saying that people who could legitimately be referred to as "experts" will still often disagree with each other. Disagreement with one's peers is not inconsistent with being an expert! However, I would say that this is a non-issue as long as you don't state grandiose goals such as being a definitive source of all human knowledge. Acknowledge that there is, of necessity, some editorial effect, and the world can go on just fine using your resource if that editorial voice is found to be inoffensive to the masses, or ignoring your resource if it is offensive.

As for Wikipedia, I would say that the "editors" there are self-selecting in a very pragmatic way. From a cynical point of view, the experts at Wikipedia are experts in the Wikipedia system who, critically, have a desire to spend a fair amount of time working within that system. The experts in Sanger's system are instead declared experts by external societal mechanisms (e.g. academic credentials), but do they also have a desire to spend a lot of time in the Citizendium system? That, I think, is the most important issue here. Editors need at least two significant qualities in perhaps equal measure: qualification in the subject matter, and incentive to put in some work. Wikipedia addresses the latter by being more open (if you want to work, you're welcome here); Citizendium addresses the former (if you're qualified, you're welcome here). Wikipedia's approach at least guarantees the generation of content, and this may prove a crucial advantage.

Posted by: Anthony Cowley [TypeKey Profile Page] at September 21, 2006 02:01 PM

Anthony,

Well put. I agree.

Let me throw out another (not entirely baked) thought. If you really want to compete with Wikipedia, don't recruit experts to act as editors. Recruit good, smart editors to act as editors, and recruit experts to act as contributors. The editor's job should be to synthesize the knowledge of experts (whether those experts are professionals or amateurs) and distill it into good, consistent, readable (and preferably concise) prose. If the experts trust the judgment and skill of the editor, I think the likelihood that they'd contribute would be increased substantially - particularly if their contributions were acknowledged in some way. In this way, you'd not only address Wikipedia's content problem but you'd also address its equally large writing problem (which Sanger's Citizendium concept doesn't address, so far as I can see).

Of course, to recruit those editors, you'd probably have to pay them, which would mean either finding someone to donate a lot of money or using ads to generate money (while keeping the venture nonprofit).

That's probably not practical, but I think editors as gatekeepers makes more sense than experts as gatekeepers.

Nick

Posted by: Nick Carr at September 21, 2006 02:58 PM

As Delmar proclaimed in the movie O'Brother, Where Art Thou? - "Okay... I'm with you fellas."

Posted by: Mike O. at September 21, 2006 09:20 PM

Nick said above: "I may just be confused: If Citizendium is going to be a product of experts, as you suggest, what exactly is the role of the crowd? Are the experts going to have to review, pass judgment on, and edit all the contributions from the crowd? That seems unworkable." That would be unworkable, but it's not the proposal. Perhaps the word "editor" is tripping you up. These people, whatever you want to call them, will have at least two functions with respect to articles in their areas of expertise: to articulate content decisions on article "discussion" pages, and to place "category:approved" tags on articles. When there are disputes about how an article should read, editors are decisionmakers. It's obviously more complicated than this, but that's the basic idea. So editors are working right alongside authors. It really is a wiki system I have in mind (you'd probably hate it), except that editors have certain social rights and obligations that are understood by everyone. That, and constables will be able to rein in the difficult people more easily. Otherwise it will operate more or less the same way as Wikipedia. Or that's the notion, anyway.

Posted by: Larry Sanger at September 21, 2006 10:35 PM

Thanks, Larry. Good luck. Nick

Posted by: Nick Carr at September 22, 2006 08:58 AM

With all due respect to the august gentlemen who have all commented here, there is one aspect of Sanger's description of Citzendium that's been overlooked--a certain condescention in the language referring to "the people."

In fact, the whole idea of "the people" that Sanger speaks of reminds me of what a friend once said about her father in relation to his children: "no, he doesn't like us...he likes the idea of us."

While I do not know Sanger's work directly, his words speak of a man who likes the idea of the citizens, or "the people," but doesn't really like them.

What gives this attitude away? The phrase: "public participation with gentle expert guidance."

"gentle guidance"--as if the people are small children--or even sheep--who must be gently guided by Experts.

What makes experts so grand that they should offer their "gentle guidance"?

Will the experts be folks with uncredentialled expertise, or will they be certified, degree'd, Experts?

As I've discovered in life, you can be an expert at a lot of things, but if those who have become the ones who are constructing the society do not have something tangible to view as a credential of expertise, then one is not, in their eyes, an Expert.

And if it's all relying on the "gentle guidance" Experts, will there be any real difference between Citizendium and About.com??



Posted by: tish grier [TypeKey Profile Page] at September 25, 2006 11:15 AM

Who qualifies as an expert?

I was informed after signing on and contributing a number of articles (winning Draft of the Week twice), that can I never be an editor. I asked Mr Sanger why, considering I hold a university degree with first class honours. He told me that I have to be "published" first. This is absurd. Many people who graduate from university do not necessarily publish within universities. One would think that both practical work experience combined with the university degree and previous article contributions would be sufficient but apparently not.

If being an expert means you have to be "published" first then I cannot see a viable future for this project. There is no system within the Citizendium structure for an author to work his way up to a more meaningful level by virtue of merit and contribution alone. It reeks of elitism and is no incentive for the average internet user to sign up.

Posted by: Disappointed [TypeKey Profile Page] at June 21, 2008 10:38 PM

Post a comment

Thanks for signing in, . Now you can comment. (sign out)

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)


Remember me?


carrshot5.jpg Subscribe to Rough Type

Now in paperback:
shallowspbk2.jpg Pulitzer Prize Finalist

"Riveting" -San Francisco Chronicle

"Rewarding" -Financial Times

"Revelatory" -Booklist

Order from Amazon

Visit The Shallows site

The Cloud, demystified: bigswitchcover2thumb.jpg "Future Shock for the web-apps era" -Fast Company

"Ominously prescient" -Kirkus Reviews

"Riveting stuff" -New York Post

Order from Amazon

Visit Big Switch site

Greatest hits

The amorality of Web 2.0

Twitter dot dash

The engine of serendipity

The editor and the crowd

Avatars consume as much electricity as Brazilians

The great unread

The love song of J. Alfred Prufrock's avatar

Flight of the wingless coffin fly

Sharecropping the long tail

The social graft

Steve's devices

MySpace's vacancy

The dingo stole my avatar

Excuse me while I blog

Other writing

Is Google Making Us Stupid?

The ignorance of crowds

The recorded life

The end of corporate computing

IT doesn't matter

The parasitic blogger

The sixth force

Hypermediation

More

The limits of computers: Order from Amazon

Visit book site

Rough Type is:

Written and published by
Nicholas Carr

Designed by

JavaScript must be enabled to display this email address.

What?