The following is a reposting of one of Rough Type’s greatest hits. It originally appeared in these pages on March 7, 2011.

“It’s not information overload. It’s filter failure.” That was the main theme of a thoughtful and influential talk that Clay Shirky gave at a technology conference back in 2008. It’s an idea that’s easy to like both because it feels intuitively correct and because it’s reassuring: better filters will help reduce information overload, and better filters are things we can actually build. Information overload isn’t an inevitable side effect of information abundance. It’s a problem that has a solution. So let’s roll up our sleeves and start coding.

There was one thing that bugged me, though, about Shirky’s idea, and it was this paradox: The quality and speed of our information filters have been improving steadily for a few centuries, and have been improving extraordinarily quickly for the last two decades, and yet our sense of being overloaded with information is stronger than ever. If, as Shirky argues, improved filters will reduce overload, then why haven’t they done so up until now? Why don’t we feel that information overload is subsiding as a problem rather than getting worse? The reason, I’ve come to believe, is that Shirky’s formulation gets it precisely backwards. Better filters don’t mitigate information overload; they intensify it. It would be more accurate to say: “It’s not information overload. It’s filter success.”

But let me back up a little, because it’s actually more complicated than that. One of the traps we fall into when we talk about information overload is that we’re usually talking about two very different things as if they were one thing. Information overload actually takes two forms, which I’ll call situational overload and ambient overload, and they need to be treated separately.

Situational overload is the needle-in-the-haystack problem: You need a particular piece of information – in order to answer a question of one sort or another – and that piece of information is buried in a bunch of other pieces of information. The challenge is to pinpoint the required information, to extract the needle from the haystack, and to do it as quickly as possible. Filters have always been pretty effective at solving the problem of situational overload. The introduction of indexes and concordances – made possible by the earlier invention of alphabetization – helped solve the problem with books. Card catalogues and the Dewey decimal system helped solve the problem with libraries. Train and boat schedules helped solve the problem with transport. The Reader’s Guide to Periodicals helped solve the problem with magazines. And search engines and other computerized navigational and organizational tools have helped solve the problem with online databases.

Whenever a new information medium comes along, we tend to quickly develop good filtering tools that enable us to sort and search the contents of the medium. That’s as true today as it’s ever been. In general, I think you could make a strong case that, even though the amount of information available to us has exploded in recent years, the problem of situational overload has continued to abate. Yes, there are still frustrating moments when our filters give us the hay instead of the needle, but for most questions most of the time, search engines and other digital filters, or software-based, human-powered filters like email or Twitter, are able to serve up good answers in an eyeblink or two.

Situational overload is not the problem. When we complain about information overload, what we’re usually complaining about is ambient overload. This is an altogether different beast. Ambient overload doesn’t involve needles in haystacks. It involves haystack-sized piles of needles. We experience ambient overload when we’re surrounded by so much information that is of immediate interest to us that we feel overwhelmed by the neverending pressure of trying to keep up with it all. We keep clicking links, keep hitting the refresh key, keep opening new tabs, keep checking email in-boxes and RSS feeds and Facebook notifications, keep scanning Amazon and Netflix recommendations – and yet the pile of interesting information never shrinks.

The cause of situational overload is too much noise. The cause of ambient overload is too much signal.

The great power of modern digital filters lies in their ability to make information that is of inherent interest to us immediately visible to us. The information may take the form of personal messages or updates from friends or colleagues, broadcast messages from experts or celebrities whose opinions or observations we value, headlines and stories from writers or publications we like, alerts about the availability of various other sorts of content on favorite subjects, or suggestions from recommendation engines – but it all shares the quality of being tailored to our particular interests. It’s all needles. And modern filters don’t just organize that information for us; they push the information at us as alerts, updates, streams. We tend to point to spam as an example of information overload. But spam is just an annoyance. The real source of information overload, at least of the ambient sort, is the stuff we like, the stuff we want. And as filters get better, that’s exactly the stuff we get more of.

It’s a mistake, in short, to assume that as filters improve they have the effect of reducing the information we have to look at. As today’s filters improve, they expand the information we feel compelled to take notice of. Yes, they winnow out the uninteresting stuff (imperfectly), but they deliver a vastly greater supply of interesting stuff. And precisely because the information is of interest to us, we feel pressure to attend to it. As a result, our sense of overload increases. This is not an indictment of modern filters. They’re doing precisely what we want them to do: find interesting information and make it visible to us. But it does mean that if we believe that improving the workings of filters will save us from information overload, we’re going to be very disappointed. The technology that creates the problem is not going to make the problem go away. If you really want a respite from information overload, pray for filter failure.

Image: Randy Sears.