I will be in Cambridge, Mass., this afternoon to give a talk entitled “The World Is Not the Screen: How Computers Shape Our Sense of Place.” It is part of the ongoing Navigation Lecture Series presented by Harvard’s Radcliffe Institute for Advanced Study. The talk starts at five, and is free and open to the public. So if you’re in the neighborhood, please come by. Details are here.
We’ve been getting a little lesson in what human-factors boffins call “automation complacency” over the last couple of days. Google apparently made some change to the autosuggest algorithm in Gmail over the weekend, and the program started inserting unusual email addresses into the “To” field of messages. As Business Insider explained, “Instead of auto-completing to the most-used contact when people start typing a name into the ‘To’ field, it seems to be prioritizing contacts that they communicate with less frequently.”
Google quickly acknowledged the problem:
We’re aware of an issue with Gmail and auto-complete and are currently investigating. Apologies for any inconvenience.
— Gmail (@gmail) February 23, 2015
The glitch led to a flood of misdirected messages, as people pressed Send without bothering to check the computer’s work. “I got a bunch of emails yesterday that were clearly not meant for me,” blogged venture capitalist Fred Wilson on Monday. Gmail users flocked to Twitter to confess to shooting messages to the wrong people. “My mum just got my VP biz dev’s expense report,” tweeted Pingup CEO Mark Slater. “She was not happy.” Wrote CloudFlare founder Matthew Prince, “It’s become pathological.”
The bug may lie in the machine, but the pathology actually lies in the user. Automation complacency happens all the time when computers take over tasks from people. System operators place so much trust in the software that they start to zone out. They assume that the computer will perform flawlessly in all circumstances. When the computer fails or makes a mistake, the error goes unnoticed and uncorrected — until too late.
Researchers Raja Parasuraman and Dietrich Manzey described the phenomenon in a 2010 article in Human Factors:
Automation complacency — operationally defined as poorer detection of system malfunctions under automation compared with under manual control — is typically found under conditions of multiple-task load, when manual tasks compete with the automated task for the operator’s attention. … Experience and practice do not appear to mitigate automation complacency: Skilled pilots and controllers exhibit the effect, and additional task practice in naive operators does not eliminate complacency. It is possible that specific experience in automation failures may reduce the extent of the effect. Automation complacency can be understood in terms of an attention allocation strategy whereby the operator’s manual tasks are attended to at the expense of the automated task, a strategy that may be driven by initial high trust in the automation.
In the worst cases, automation complacency can result in planes crashing on runways, school buses smashing into overpasses, or cruise ships running aground on sandbars. Sending an email to your mom instead of a colleague seems pretty trivial by comparison. But it’s a symptom of the same ailment, an ailment that we’ll be seeing a lot more of as we rush to hand ever more jobs and chores over to computers.
A few highlights from Lee Gomes’s long, lucid interview with Facebook’s artificial-intelligence chief Yann LeCun in IEEE Spectrum:
Gomes: We read about Deep Learning in the news a lot these days. What’s your least favorite definition of the term that you see in these stories?
LeCun: My least favorite description is, “It works just like the brain.” I don’t like people saying this because, while Deep Learning gets an inspiration from biology, it’s very, very far from what the brain actually does. And describing it like the brain gives a bit of the aura of magic to it, which is dangerous. It leads to hype; people claim things that are not true. AI has gone through a number of AI winters because people claimed things they couldn’t deliver.
Gomes: You seem to take pains to distance your work from neuroscience and biology. For example, you talk about “convolutional nets,” and not “convolutional neural nets.” And you talk about “units” in your algorithms, and not “neurons.”
LeCun: That’s true. Some aspects of our models are inspired by neuroscience, but many components are not at all inspired by neuroscience, and instead come from theory, intuition, or empirical exploration. Our models do not aspire to be models of the brain, and we don’t make claims of neural relevance.
Gomes: You’ve already expressed your disagreement with many of the ideas associated with the Singularity movement. I’m interested in your thoughts about its sociology. How do you account for its popularity in Silicon Valley?
LeCun: It’s difficult to say. I’m kind of puzzled by that phenomenon. As Neil Gershenfeld has noted, the first part of a sigmoid looks a lot like an exponential. It’s another way of saying that what currently looks like exponential progress is very likely to hit some limit—physical, economical, societal—then go through an inflection point, and then saturate. I’m an optimist, but I’m also a realist.
There are people that you’d expect to hype the Singularity, like Ray Kurzweil. He’s a futurist. He likes to have this positivist view of the future. He sells a lot of books this way. But he has not contributed anything to the science of AI, as far as I can tell. He’s sold products based on technology, some of which were somewhat innovative, but nothing conceptually new. And certainly he has never written papers that taught the world anything on how to make progress in AI.
Gomes: You yourself have a very clear notion of where computers are going to go, and I don’t think you believe we will be downloading our consciousness into them in 30 years.
LeCun: Not anytime soon.
“Will human replacement — the production by ourselves of ever better substitutes for ourselves — deliver an economic utopia with smart machines satisfying our every material need? Or will our self-induced redundancy leave us earning too little to purchase the products our smart machines can make?” So ask three Boston University economists, Seth Benzell, Laurence Kotlikoff, and Guillermo LaGarda, and Columbia’s Jeffrey Sachs. In an attempt to answer the question, the researchers turned to — what else? — a computer. They programmed a “bare-bones” model of the economy, featuring high-tech workers (who produce software) and low-tech workers (who produce services), and let the simulation run under different sets of variables.
The results were, as the economists put it in a new paper on the experiment, “disturbing.” The simulation suggests that “technological progress can be immiserating” and that even talented software programmers may face tough times in an ever more automated economy. The reason lies in the durability and reusability of software. Code is not used up; it accumulates. As the cost of deploying software for productive work (ie, the cost of automation) goes down, demand for new code spikes, bringing lots of new programmers into the labor market. The generous compensation provided to the programmers leads at first to higher savings and capital formation, fueling the boom. But “over time,” the model reveals, “as the stock of legacy code grows, the demand for new code, and thus for high-tech workers, falls.”
As a simple illustration, the authors point to the development of a robotic chess player. Once you have a robot that can outperform all human players, the incentive for programming new robotic players drops sharply. This is something we’ve already seen, as the authors point out: “Take Junior – the reigning World Computer Chess Champion. Junior can beat every current and, possibly, every future human on the planet. Consequently, his old code has largely put new chess programmers out of business.” Once any program reaches a superhuman level of productivity in a task, the incentive to invest in further, marginal gains falls.
The authors lay out the resulting economic dynamic:
The increase in [the code retention rate] initially raises the compensation of code-writing high-tech workers. This draws more high-tech workers into code-writing, thereby raising high-tech worker compensation … Things change over time. As more durable code comes on line, the marginal productivity of code falls, making new code writers increasingly redundant. Eventually the demand for code-writing high-tech workers is limited to those needed to cover the depreciation of legacy code, i.e., to retain, maintain, and update legacy code. The remaining high-tech workers find themselves working in the service sector [and pushing down wages in those occupations]. The upshot is that high-tech workers can end up potentially earning far less than in the [model’s] initial steady state.
As usable code stocks swell, the model indicates, we will at some point pass the cycle’s point of peak code — the moment of maximum demand for new code — and the prospects for employment in programming will begin to decline. Code boom will turn to code bust. (The bust will be even deeper, the economists found, if software is distributed as open source and hence made easier to share.) Even though high-tech workers “start out earning far more than low-tech workers,” they “end up earning far less.”
One thing the economists don’t seem to account for is the automation of programming itself, particularly the use of software to perform many of the tasks necessary to maintain, update, and redeploy legacy code. The automation of coding, which would be encouraged as programmers’ wages increase during the boom period, would likely deepen the bust even further.
Computer models of complex systems are always simplifications, of course, but this study serves to raise important and complicated questions about the long-run demand for programmers. It’s become popular to suggest that all kids should be taught to code as part of their education. That way, the theory goes, they’ll be assured of good jobs in an ever more computerized economy. This study calls into question that hopeful assumption. There can be a glut of coders just as there can be a glut of code.
Image of hackathon: Wikipedia.
From Susan Lerner’s interview with Jonathan Franzen in Booth:
SL: I want to ask you about technology and social media. … I was wondering, given your change of heart about television and its place within our culture, can you comment on this conversion and the possibility that social media might also one day redeem itself?
JF: TV redeemed itself by becoming more like the novel, which is to say: interested in sustained, morally complex narrative that is compelling and enjoyable. How that happens with pictures of you and your friends at T. G. I. Friday’s isn’t clear to me. Twitter isn’t even trying to be a narrative form. Its structure is antithetical to sustained and carefully considered story-telling. How does a structure like that suddenly turn itself into narrative art? You could say, well, Gilligan’s Island wasn’t art, either. But Gilligan’s Island paved the way, by being twenty-two minutes of a narrative, however dumb, to the twenty-two minutes of Nurse Jackie.
SL: You see a trajectory?
JF: Yes, you can see the trajectory there. Which is the same trajectory that the novel itself followed. There was a lot of really bad experimentation in the seventeenth century as we were trying to work out these fundamental problems of “Is this narrative pretending to be true? Is it acknowledging that it’s not true? Are novels only about fantastical things? Where does everyday life fit in?” There were a couple of centuries of sorting that out before the novel really got going in Richardson and Fielding, and then, soon after, culminating in Austen. You can see that maturation in movies as well. You had Birth of a Nation before you had The Rules of the Game. It takes a while for artistic media to mature—I take that point—but I don’t know anyone who thinks that social media is an artistic medium. It’s more like another phone, home movies, email, whatever. It’s like a better version of the way people socially interacted in the past, a more technologically advanced version. But if you use your Facebook page to publish chapters of a novel, what you get is a novel, not Facebook. It’s a struggle to imagine what value is added by the technology itself.
SL: I think there’s an argument that can be made about new technology providing different forms and twists on established ideas, so people can examine—
JF: I’m just looking at the phenomenology of this technology in everyday life.
SL: Pictures of desserts.
JF: Yeah, pictures of desserts and the fact that you can’t sit still for five minutes without sending and receiving texts. I mean, it does not look like any form of engagement with art that I recognize from any field. It looks like a distraction and an addiction and a tool. A useful tool. I’m not a technophobe. I’m on the internet all day, every day, except when I’m actually trying to write, and even then I’m on a computer and using, often, material that I’ve taken from the internet. It’s not that I have technophobia. It’s the notion that somehow this is a transformative, liberating thing that I take issue with, when it seems to me more like a perfection of the free market’s infiltration of every aspect of a human being’s waking life.
It’s interesting — this is an aside — how deeply Gilligan’s Island managed to engrave itself into the cultural worldview of a certain generation of Americans. Despite its surface dumbness, the show, I would suggest, carries a mythical weight, what with the totemic quality of the characters — scientist, celebrity, tycoon, seafarer, etc. — and the Promethean nature of the plot.
O, unscepter’d isle, demi-paradise, demi-hell!