If over the last decade you have read any of the many books and articles promoting the Net as a new world where people are able to form self-regulating, super-democratic communities, you have no doubt come across glowing descriptions of eBay’s feedback system. By providing buyers and sellers with a simple means for rating one another, eBay has been able, we’ve been told, to avoid lots of rules and regulations and other top-down controls. The community, built on trust and fellow-feeling, essentially manages itself. Tom Friedman, in his book The World Is Flat, voiced the common opinion when he called eBay a “self-governing nation-state.”
Nice story. Too bad it didn’t work out.
EBay has been struggling for some time with growing discontent among its members, and it has rolled out a series of new controls and regulations to try to stem the erosion of trust in its market. At the end of last month, it announced sweeping changes to its feedback system, setting up more “non-public” communication channels and, most dramatically, curtailing the ability of sellers to leave negative feedback on buyers. It turns out that feedback ratings were being used as weapons to deter buyers from leaving negative feedback about sellers.
When Bill Cobb, the president of the company’s North American operations, announced the changes, he underscored just how broken the feedback system had become:
To give you some background, the original intent of eBay’s public feedback system was to provide an honest, accurate record of member experiences. Over the years, we’ve adjusted the system to add non-public means of providing feedback to try to improve its accuracy. For example, we instituted Unpaid Item Reports in 2006, and that has helped us to hold buyers accountable.
But overall, the current feedback system isn’t where it should be. Today, the biggest issue with the system is that buyers are more afraid than ever to leave honest, accurate feedback because of the threat of retaliation. In fact, when buyers have a bad experience on eBay, the final straw for many of them is getting a negative feedback, especially of a retaliatory nature.
Now, we realize that feedback has been a two-way street, but our data shows a disturbing trend, which is that sellers leave retaliatory feedback eight times more frequently than buyers do … and this figure is up dramatically from only a few years ago.
So we have to put a stop to this and put trust back into the system.
But I think – and I’m sure you’ll agree – that the most compelling reason we need to change feedback is so that buyers will regain their confidence on eBay and they will bid and buy more often.
We explored a number of solutions, and talked to eBay’s founder Pierre Omidyar, who created the Feedback system. He agrees that bold changes are required to fix Feedback. And that’s exactly what we’re going to do … here’s the biggest change, starting in May:
Sellers may only leave positive feedback for buyers (at the seller’s option).
I know this is a huge change, but we’re also putting into place protections that sellers have wanted for years. In addition to holding buyers accountable via non-public seller reporting tools, such as Unpaid Item reports, we are planning a number of other Seller Protections against inaccurate feedback.
He goes on to list seven new “protections,” including more aggressive central monitoring of members’ behavior and various restrictions on buyers’ ability to leave feedback about sellers.
Patti Waldmeir, in a column in the Financial Times today titled “The death of self-rule on the internet,” writes, “For those who were there from the start of this experiment in digitising utopia, including me, this is very disillusioning.” By “radically rewriting the constitution of the democratic republic of Ebay,” she says, the company has closed the book on a certain brand of internet idealism:
For most of [its] 13 years, Ebay has been run largely as a self-policed island, a place where order was preserved less by real world laws than by norms and customs and expectations and reputations that were almost entirely virtual. Ebayers governed themselves by rating each transaction using the site’s “feedback” system, where they could report crooks, not to the state but to each other. The theory was that, as in a medieval souk in which everyone knew everyone, everyone on Ebay would know who the crooks were by reading their feedback. Now the company has basically admitted that the cybersouk model does not work: buyers did not tell the truth about sellers, and sellers did not tell the truth about buyers. And in a market where traders lie, the trust that is so central to online commerce cannot flourish.
This isn’t unusual. It follows a common pattern that we’ve seen play out in other “social production” sites like Digg and Wikipedia. (Disclosure: I’m on the editorial advisory board of the Encyclopaedia Britannica.) As these sites grow, keeping them in line requires more rules and regulations, greater exercise of central control. The digital world, it seems, is not so different from the real world.
In a new post about how “bottom-up” communities need “top-down” controls to work successfully, Kevin Kelly notes that “the supposed paragon of adhocracy – the Wikipedia itself – is itself far from strictly bottom-up. In fact a close inspection of Wikipedia’s process reveals that it has an elite at its center (and that it does have an elite center is news to most). Turns out there is far more deliberate top-down design management going on than first appears.”
Kelly argues that “the reason every bottom-up crowd-source hive-mind needs some top-down control is because of time. The bottom runs on a different time scale than our instant culture.” He’s implying that, if you gave them enough time, self-governing communities would eventually work out their problems and run just fine – like happy beehives. But that’s contradicted by experience. What we’ve seen happen with self-regulating communities, both real and virtual, is that they go through a brief initial period during which their performance improves – a kind of honeymoon period, when people are on their best behavior and rascals are quickly exposed and put to rout – but then, at some point, their performance turns downward. They begin, naturally, to decay. Leave them alone long enough, and they’re far more likely to collapse than to reach perfection.
Kelly confuses human with nonhuman systems. He writes: “The main drawback to pure unadulterated darwinism is that it takes place in biological time – eons. Who has eons to wait during internet time? Nobody.” But darwinism has little to do with the development of human systems like eBay or Wikipedia or Digg. People aren’t genes (or bees). You can build a good emergent system out of genes because genes are dumb – they don’t make their own decisions, they don’t consider what other genes are doing, they don’t think. People, in contrast, actually do think. Sometimes, we’re inspired by fellow-feeling. Other times, we act selfishly or with prejudice or we try to game whatever system we’re part of. And the more times we’re confronted with other people acting selfishly, or fraudulently, the more we retreat into self-interest ourselves. Trust, a fragile thing, breaks down.
And that’s why eBay’s feedback system decayed. Time was its enemy, not its friend.
I think the take away is that any system will have unintended consequences and any system can be gamed.
In a few years, I would expect another revamp of the system.
Nick, while I agree with your skepticism of purely bottom-up content production, I think your explanation of why emergent systems comprising humans don’t work is off. Many biological systems have been shown to function based on locally greedy decisions. That is, individual agents are thinking of their own self-interest. The trick is that the only such systems that are sustainable are those wherein individual concerns, when acted upon, yield a globally stable system.
So the trick, I think, is to concentrate on developing systems that directly appeal to users’ self-interest, not their sense of community. For instance, if I am rewarded whenever someone else’s review of a buyer/seller is consistent with my own, then I will have incentive not only to leave feedback, but also to leave feedback that is most likely to be echoed by those who come after me. While such a feedback mechanism may not be right for Ebay, the critical aspect of my role in it is that I am encouraged to do the right thing by an appeal to my greed.
Anthony, I see your point about the term “self-interested.” I probably could have drawn the distinction – between thinking and nonthinking agents – more precisely. But I’m not convinced that a human system based purely on self-interest, as you describe, would require less top-town direction and controls (and, as you point out, it would have little to do with the idea of community). Nick
I didn’t know about these “retaliation ratings” from sellers, but I’m not a very big ebay user, and maybe a too good buyer to have encountered it ! :-)
On the other hand, one thing I don’t quite understand on eBay is why auctions work at all !
Indeed, the only sensible buying strategy on eBay seems to me to be “sniping” (and you have software and sites to help you do it), why on earth would you raise the price like 2 or 3 days before the end on something ? (this is especially true for “standard” items)
This strict timing constraint is a major difference from true auctions, but it also completely changes the buying strategy, isn’t eBay planing to set up true auctions with an auctionneer and skype calls/links or something, and removing the strict end time ? This would probably make sense especially for rare/expensive items.
Note : maybe there could also be some automated auctionneer, that is once somebody raises the price, you have so many seconds or minutes to make a better offer. (this would probably not work very well on a pure html site, but with the fat client or some more dynamic web technology it would be fine)
This is the last straw for me, and I switched from eBay to Amazon for selling off my old electronics and photo gear (after I upgrade) last week. A much better solution would be to remove feedback from sellers who have too much negative feedback themselves, rather than throwing the baby with the bathwater as eBay has done.
The key point is that an efficient market needs to provide either an efficient contract enforcement mechanism (which is why the economy of the US with the rule of law works better than that of Pakistan or Nigeria), or a reliable trust metric. To paraphrase Beaumarchais, if you lack the ability to provide negative feedback, there is no such thing as positive feedback. Sellers cannot rely on feedback alone to trust potential buyers and either have to resort to draconian measures or reject clients who do not have a lot of feedback. This creates tremendous deadweight loss and destroys half the value proposition of eBay, which is both marketplace and trust assertor.
An691, you’re missing one of the core ebay features – if you set your max price it will bid for you automatically, until someone sets a higher maximum. Sniping is an attempt to game this system, but if you set your bid to your true max price it will do the right thing for you.
I was thrilled to hear about this revision. I had a recent bad experience when a seller, promising to send an item within two days, took twelve. When I mentioned this in my review, he blasted my 100% positive reputation with his negative and untrue response. There was no recourse with eBay for this.
The observations on top-down and bottom-up interaction might be well applied to capitalism in general. The vast difference in the percentage of interest both parties have in their reviews in the old system – a frequent seller is mostly unaffected, while a new buyer may be devastated – shows clearly the power of the seller over the buyer. This exists everywhere in our society, but it’s not a necessary condition.
Isn’t this a variation on the prisoner’s dilemma? Ethics aside, buyers who leave retaliatory feedback are being rational in the game theory sense. One wonders if eBay has any game theory specialists thinking about these problems.
There must be more clever ways to design a feedback system, possibly based on algorithms that detect behavior patterns and show more elaborate statistics. For example, a pattern of retaliatory feedback is easy to automatically detect and display to eBay users. One imagines Google nerds coming up with a smarter algorithm.
The fixed end time for auctions is definitely an oddity that’s exploited too much, and “automatic bidding” doesn’t quite solve this. Sniping probably results in lower final prices, which isn’t in sellers’ or eBay’s interest.
Why not remove the ratings for the buyer?
In the real world, we never rate a buyer. We always rate a seller.
Fazal – Because those sellers never accumulate negative feedback to begin with, since buyers are too afraid to give them that negative feedback
Don’t think you realize how many potential buyers good ebay sellers lose out on because of bad ebay sellers, since it’s become extraordinarily hard to tell the difference between the two. The ability to give buyers negative ratings isn’t going to help very much if there are no good buyers left.
Niyaz – of course buyers are rated in the real world. FICO scores, credit reports, blacklists, defaulters lists from collection agencies… it is just a bit more hidden, that’s all
The e-Bay commons is a victim of anarchy, not time. It has been off the tracks for a long time and its a bad idea for eBay to think it can fix it’s feedback flaws by amputation.
Great article Nick A++++++++++
Kevin, yes I know about the automatic bidding up to max offer, but I don’t think that it really changes sniping as being the only sensible buying strategy (maybe except for one of a kind items that you truly want). Releasing your “max offer” as late as possible can only lead to lower end price it seems to me.
the problem could have been eliminated by enforcing a feedback protocol that requires sellers feedback before the buyer CAN leave feedback.
The buyer wins an auction and pays (or doesn’t pay) and is finished with their required performance before the seller begins his (the shipping of the product). The seller should rate the buyer on THAT performance, and only that performance, before the buyer is capable of rating him. If the buyer performs that task satisfactorily, such that the seller deems it reasonable to ship the product, then he should also give feedback limited to that performance.
Once the seller’s feedback is complete, the buyer can now leave feedback on the seller’s performance.
This is how a “polite” feedback system should work… and when I’m a seller, I leave feedback as soon as the financials are complete. I have always been a little peeved when a seller waits on MY feedback to him before leaving any for me. “My performance is complete, you can go ahead and rate me now.”
Sellers can leave negative feedback for non-payment. Buyers can leave negative feedback for poor/slow/non- shipping, misrepresented items, etc without fear of reprisal. There should be a response field for sellers to explain negative feedback, but it wouldn’t count in the buyer’s score.
I have dealt with buyers/sellers with several negative entries. You can see their “feedback left for others” and easily tell that some folks are just complainers.
To the many ostriches who perpetuate the myth that eBay sellers have no justification for not posting positive feedback for buyers who have paid, I say “take your head out of the sand.” As clever display name posted above, “My performance is complete, you can go ahead and rate me now.” Perhaps your performance is complete, but unfortunately that is not the case for a small percentage of other buyers who pay for the item and then bounce a check, do a credit card chargeback, or claim the package arrived either damaged or not at all. We were able to reduce our so-called “lost package” customers by about 95% once we instituted tracking on every package. Most eBay sellers have learned the hard way that the only reliable method to prevent being held hostage by scam artists posing as buyers is to wait until the customer acknowledges receiving the goods and is satisfied with the purchase. That is when the transaction is truly over, and is signalled by the buyer leaving positive feedback first. You may be “a little peeved when a seller waits on MY feedback to him before leaving any for me,” but take my word for it – you’d be a lot more peeved if you were a seller who just shipped a valuable item and then became the victim of a PayPal chargeback.
It seems to me that the co-existence of eBay and PayPal is part of the problem. Their systems used to separate ones and that confused people.
I stopped buing from eBay over a year ago because…the the seller never responded my e-mails/eBay requests and later complained through PayPal channels that I was not responding. So, e-mail, eBay and PayPal were three different channels and the seller was using only one of them :-(
Nowadays I seek a seller with a credible web presence and reliable feedback channels ;-)
It seems to me that the co-existence of eBay and PayPal is part of the problem. Their systems used to separate ones and that confused people.
I stopped buing from eBay over a year ago because…the the seller never responded my e-mails/eBay requests and later complained through PayPal channels that I was not responding. So, e-mail, eBay and PayPal were three different channels and the seller was using only one of them :-(
Nowadays I seek a seller with a credible web presence and reliable feedback channels ;-)
If you want a bottom up system to acheive better equalibrium I suspect that you need a certain level of equality between the participants. Ebay has an inherent inequality built in that is probably very hard to get around – there are fewer sellers than buyers. I.e. the population from which sellers draw there feedback is larger and hence anomalies can be disregarded whereas buyers have a smaller population of feedback and hence one negative feedback is of more importance. This is a similar problem every power structure faces, and I suspect why in many democratic societies we are so concerned about accountability.
People will always try to misuse the system. Let say we leave money on the street. What can we expect? Will we find it there after one hour? We will not. And how we call the one who does not admit this? Hypocrite, liar, idealist, fool,… And eBay knows that all the time. eBay is responsible like car producers who produce cars without safety equipment. Such irresponsibility is still allowed on Internet and measures for people safety are not required by law like in car industry. Therefore we can produce cars without safety and let people create their own traffic rules, … With system of reporting eBay just wanted to transfer responsibility out of their hands. Am I willing to report my experience? How can I know I will not hurt people, take them only possible way of living? Am I responsible for offers on eBay? Why they are there if I’m not allow to buy? What kind of rules I need to obey when reporting? Anyhow this is just cheap trick. eBay is their system, not mine and they want to attract people to use it. When something goes wrong they do not want to accept responsibility. eBay is responsible for every fraud happening inside eBay system. eBay would have to invest a lot in system and even much more in identification of users. But of course eBay does not want to do this because this could be quite unpopular, more complicated and most important: expensive. Users have made eBay rich company and they have trusted eBay. I think it is time to start giving something to users in return or maybe just say bye bye, we made a lot of money, you did not loose it too much, game over. It was nice joke. If impossible would be possible eBay would choose first option. But they will not. Many Internet services like social networks were made only for money, without any feeling for social responsibility. And society allows that. It is completely normal, that does not work. It is just cheap trick people believe for some limited period of time. When organizing concerts, football, soccer matches the one who organizes this need to take care about security. You cannot transfer this responsibility to people. It is the same situation with MySpace and many others social networks, where users identity can be stolen with ease. This can have unpredicted and very painful consequences for victims. All this services were prepared without any social responsibility. This services were made just for money, not for you or me. We were elected to make them money. No, thank you. They should take care about users security much much more.
Was sent here (long after the post was submitted) by a friend of mine who knows my interest in using social science to deal with online issues. When eBay changed its feedback policy (and before this piece was posted), there was some discussion about both the perceived relevance of eBay and about alternatives to the feedback system. These are all very interesting, especially to those who frequently buy and sell things online. Here, the main point seems to be: eBay isn’t the only way to trade goods and money and eBay may even have failed, so we’ll look at what happened. There’s still room for selling and buying things online, and eBay can now be an example of what didn’t work.
Fair enough.
But the mention of Wikipedia and the “crowd control” angle bring about a broader point about social systems.
Social scientists in general (including economists, game theorists, etc.) are well-aware of many of the issues which are now being addressed with disillusion. And while several of those who were “illusioned” into thinking the ‘Net would change all the rules of social behavior are themselves social scientists, the disillusion depends less on any characteristic of social behaviour than on people’s expectations.
The same principle applies to both eBay and Wikipedia: people have been extremely optimistic and are now disappointed by what they perceive to be broken promises.
The notion that time may be a factor merits consideration. Not because of “biological time,” IMO that’s going a bit far. But maybe because of scale. The time needed to make decisions about a purchase, to write feedback about somebody else, to assess the value of an encyclopedia entry, to discuss things… That kind of time “doesn’t scale well.”
There’s a related issue, in terms of scale: the number of people involved. When some issues with Wikipedia’s management started surfacing, a year or two ago, and people started using Wikipedia as a proof that “crowds” aren’t self-regulated, some people (including those on CNET’s Buzz Out Loud) commented on scale. These comments were, IMHO, very insightful. Wikipedia was, in fact, a pretty decent example of self-regulation. But self-regulation doesn’t scale well and this dimension of the site “collapsed.” At least, that’s one way to put it.
The broad principle of scale is well-understood by technologists. What can work with some measure X (number of chips, number of visitors, etc.) may not work at 2X and is even more likely to fail at X^2. Seeing social systems in the same way may be a step in the right direction. It’s still misleading, because we’re talking about diverse human beings, but it’s a start.
In ethnography, we have quite a bit of experience with small groups of people which can be said to approximate the self-regulating ideal. Not really crowds. Loosely-joined group of maybe 30-40 people who get together through part of the year to accomplish some general tasks related to their survival. The technical term is “band” and foraging societies (of which very few remain, nowadays) could often be described using the “band” model. Some people (ethnographers and others, including Marx) have even been so taken by those “band” structures that they wanted to apply the same ideas to broader groups. One reason they failed is that these groups were not only self-regulating but also self-forming. Creating that kind of group from scratch is unlikely to result in the same type of self-regulation. Another reason is that the model doesn’t scale well. It’s easy to “grok”: a city neighbourhood is very different from a large village of the same population. They’re different models.
So, scale is an important factor to consider. eBay and Wikipedia have possibly been among the largest groups in human history to regulate themselves. People who were dependent on these sites for actual survival were few and far between. But the overall structures were still quite big. Maybe these sites just reached a break point. “Peak People,” if you will.
Or, maybe, other issues were at stake.
Size does matter, in a social system. But other things matter too. Including the internal structure of the group. “What kind of a group is it?”
Apart from being hierarchical or “fairly egalitarian,” groups can be goal-oriented, clustered, open, inward-looking, etc. All of these dimensions cut across one another: you can have a very hierarchical, closed, highly clustered, outward-looking, goal-oriented group with millions of member or with twenty.
Those group characteristics matter a lot, in terms of adaptation.
So, going back to Wikipedia (because it’s a better-known case of this than eBay). One reason the self-regulating dream didn’t come true might have to do with group structure. People interpreted the group to be extremely large, quite egalitarian, unclustered, open, outward-looking, and goal-oriented. It turns out that there was, within Wikipedia, a comparatively small, rather hierarchic, highly clustered, semi-closed, and inward-looking group acting as if they were survival-oriented (not oriented toward a specific task or goal).
Wikipedia worked for a while. More than many people seemed to realize, Wikipedia.org had succeeded at creating an actual “community”: a group of people who interact with one another on a fairly regular basis. This community included the “core” group of editors, those Wikipedians who seem to get a primary identity from their participation in Wikipedia. That community was quite large, but still fairly limited, in comparison with the number of people who use Wikipedia or who may casually edit some entries, on occasion.
From the outside, Wikipedia looked different from this. It looked like a fully-open model in which “anyone can edit,” out of their own interests, and nobody is “more equal” than the others. At that level, the difference with the reality wasn’t that significant. It was true that (just about) anyone could edit (just about) any entry. But the Wikipedia’s overall “social structure” included a “behind the scenes” group of people who took a stake in Wikipedia. Mostly through their time, passion, interest, instead of money or survival. But still, a group of people who were deeply invested in Wikipedia. Their behaviour was radically different from that of casual Wikipedians and, even more, from the Wikipedia-reading general public.
The difference was primarily one of social structure. The “in-group” wasn’t like other people associated with Wikipedia.
Probably a year or so ago, all those stories broke out about the apparent “cabal” which seems to regulate (and badmouth) some individual editors, and about the cultish aura surrounding “Jimbo.” Regardless of the factual accuracy of those stories (and I have little reason to doubt them), they showed how people perceived Wikipedia. Since then, people probably became disillusioned by Wikipedia. Utopia and dystopia might be just one step from one another.
One thing which would merit a lot more discussion is the assumption that the system itself can be self-regulating. It might be useful to look back at Adam Smith’s critique of the “invisible hand” concept, but there are many other ways to look at this.
From the outside, Wikipedia looked different from this. It looked like a fully-open model in which “anyone can edit,” out of their own interests, and nobody is “more equal” than the others.