Promoting human error


From a report on a prototype of a self-driving tractor-trailer developed by Daimler as part of its Mercedes-Benz Future Truck 2025 project:

For Daimler, the truck driver of the future looks something like this: He is seated in the cab of a semi, eyes on a tablet and hands resting in his lap …

The Daimler truck retains a steering wheel as a safety measure. This allows a driver to intervene for critical maneuvers …

The experience of guiding a self-driving truck is far less stressful than the vigilance required from a human to respond to traffic conditions. This means that drivers could have enough free time to speak with their families or employers, take care of paperwork or make travel plans …

“It’s strange at first,” said Hans Luft, who sat in the truck’s cab during the demonstration on Thursday. He waved his hands to show observers that he did not need them on the wheel, tapping at his tablet while taking advantage of the 45-degree swivel of his driver’s seat. “But you quickly learn to trust it and then it’s great.”

So you create an automated system that actively undermines the vigilance and situational awareness of the operator while at the same time relying on the operator to take control of the system for “critical maneuvers” in emergencies. This is a textbook case of automation design that borders on the criminally insane. And when an accident occurs — as it will — the crash will be blamed not on “stupid design” but on “human error.”

Image: Randy von Liski.

Leave a Comment

Filed under The Glass Cage

The soma cloud


“The computer could program the media to determine the given messages a people should hear in terms of their overall needs, creating a total media experience absorbed and patterned by all the senses. … By such orchestrated interplay of all media, whole cultures could now be programmed in order to improve and stabilize their emotional climate.” —Marshall McLuhan, 1969

“The experiment manipulated the extent to which people (N = 689,003) were exposed to emotional expressions in their News Feed. This tested whether exposure to emotions led people to change their own posting behaviors, in particular whether exposure to emotional content led people to post content that was consistent with the exposure — thereby testing whether exposure to verbal affective expressions leads to similar verbal expressions, a form of emotional contagion.” —Kramer et al., 2014

“I’m excited to announce that we’ve agreed to acquire Oculus VR, the leader in virtual reality technology. … This is really a new communication platform. By feeling truly present, you can share unbounded spaces and experiences with the people in your life. Imagine sharing not just moments with your friends online, but entire experiences and adventures.” —Mark Zuckerberg, 2014

The strategy behind the Oculus acquisition has become much clearer to me over the last week. Haters gonna hate, worrywarts gonna worry, but I for one am looking forward to Facebook’s Oculus Rift experiments . Once the company is able to manipulate “entire experiences and adventures,” rather than just bits and pieces of text, the realtime engineering of a more harmonious and stabilized emotional climate may well become possible. I predict that the next great opportunity in wearables lies in finger-mountables — in particular, the Oculus Networked Mood Ring. We’ll all wear them, as essential Rift peripherals, and they’ll all change color simultaneously, depending on the setting that Zuck dials into the Facebook Soma Cloud.

I know, I know: this is all just blue-sky dreaming for now. But as the poet said, in dreams begin realities.

At least I think that’s what he said.

This post is an installment in Rough Type’s ongoing series “The Realtime Chronicles,” which began here. A full listing of posts can be found here.

Image: detail of cover of paperback edition of Brave New World.


Filed under Realtime

I feel measurably less emotional now


Sheryl Sandberg, Facebook’s COO, responds to the uproar about the company’s clandestine psychological experiment on its members:

“This was part of ongoing research companies do to test different products, and that was what it was; it was poorly communicated. And for that communication we apologize. We never meant to upset you.”

So an experiment designed to explore how the delivery of information can be programmed to manipulate people’s emotional states was just part of routine product-development testing? No worries. I apologize for getting upset.

Image: Detail of Andre Brouillet’s “Une Leçon Clinique à la Salpêtrière

1 Comment

Filed under Uncategorized

An android dreams of automation


Google’s Android guru, Sundar Pichai, provides a peek into the company’s conception of our automated future:

“Today, computing mainly automates things for you, but when we connect all these things, you can truly start assisting people in a more meaningful way,” Mr. Pichai said. He suggested a way for Android on people’s smartphones to interact with Android in their cars. “If I go and pick up my kids, it would be good for my car to be aware that my kids have entered the car and change the music to something that’s appropriate for them,” Mr. Pichai said.

What’s illuminating is not the triviality of Pichai’s scenario — that billions of dollars might be invested in developing a system that senses when your kids get in your car and then seamlessly cues up “Baby Beluga” — but what the urge to automate small, human interactions reveals about Pichai and his colleagues. With this offhand example, Pichai gives voice to Silicon Valley’s reigning assumption, which can be boiled down to this: Anything that can be automated should be automated. If it’s possible to program a computer to do something a person can do, then the computer should do it. That way, the person will be “freed up” to do something “more valuable.” Completely absent from this view is any sense of what it actually means to be a human being. Pichai doesn’t seem able to comprehend that the essence, and the joy, of parenting may actually lie in all the small, trivial gestures that parents make on behalf of or in concert with their kids — like picking out a song to play in the car. Intimacy is redefined as inefficiency.

I guess it’s no surprise that what Pichai expresses is a robot’s view of technology in general and automation in particular — mindless, witless, joyless; obsessed with productivity, oblivious to life’s everyday textures and pleasures. But it is telling. What should be automated is not what can be automated but what should be automated.

Image: “Communicating with the Beluga” by Bob.


Filed under The Glass Cage

The quarter-of-a-second rule


Mother Jones excerpts my brief essay on the malleability of our sense of time, “The Patience Deficit,” from the anthology What Should We Be Worried About? Here’s the essay’s first paragraph:

I’m concerned about time — the way we’re warping it and it’s warping us. Human beings, like other animals, seem to have remarkably accurate internal clocks. Take away our wristwatches and our cell phones and we can still make pretty good estimates about time intervals. But that faculty can also be easily distorted. Our perception of time is subjective; it changes with our circumstances and our experiences. When things are happening quickly all around us, delays that would otherwise seem brief begin to seem interminable. Seconds stretch out. Minutes go on forever. “Our sense of time,” observed William James in his 1890 masterwork The Principles of Psychology, “seems subject to the law of contrast.”

Read on.

Image: detail of “Forest (4)” by Gerhard Richter.

Comments Off

Filed under Uncategorized

Bringing economics into the world


Throwing his considerable weight behind the post-autistic economics movement, Robert Skidelsky offers a calm but blistering critique of the “mainstream economics” curriculum that has come to dominate university teaching. Arguing that mainstream economics, with its pseudo-scientific mathematical models, is at heart an “ideology of the free market” that can circumscribe thinking and excuse failed policies, Skidelsky argues that the context of economics teaching needs to be broadened to include history, philosophy, politics, and psychology — to reflect the true economic lives of people.

It has become an article of faith that any move toward a more open or “pluralist” approach to economics portends regression to “pre-scientific” modes of thought, just as the results of the European Parliament election threaten to revive a more primitive mode of politics. Yet institutions and ideologies cannot survive by mere incantation or reminders of past horrors. They have to address and account for the contemporary world of lived experience. For now, the best that curriculum reform can do is to remind students that economics is not a science like physics, and that it has a much richer history than is to be found in the standard textbooks.

I suspect that Skidelsky’s piece will provoke a productive debate. Brad Delong has already responded:

We have no business offering a narrow economics B.A. at all. At the undergraduate social-science level, the right way of organizing a major curriculum is to offer some flavor of history and moral philosophy: enough history that students are not ignorant, enough sociology and anthropology that students are not morons, and enough politics and philosophy that students are not fools. (And, I would say, a double dose of economics to ensure that majors understand what is key about our civilization and do not get the incidence of everything wrong.)

And here (pdf download) is the report of the Post-Crash Economics Society that spurred Skidelsky’s comments.

Via The Browser. Image by Penguincakes.


Filed under Uncategorized

Technology as love

For a few years now, I’ve used summertime laziness as an excuse to recycle some of this blog’s old posts. The following post was originally published, under the ponderous headline “God, Kevin Kelly and the Myth of Choices,” in July of 2011. The influence of tools on human possibility is a central theme of The Glass Cage, so it was interesting for me to reread this post in the wake of writing the book. If I were to rewrite the post now, I would shift the focus away from technological progress as a force in itself and place a much greater emphasis on how the design of particular tools determines whether they open or foreclose opportunities and choices for their users.

I suspect it’s accurate to say that Kevin Kelly’s deep Christian faith makes him something of an outlier among the Bay Area tech set. It also adds some interesting layers and twists to his often brilliant thinking about technology, requiring him to wrestle with ambiguities and tensions that most in his cohort are blind to. In a new interview with Christianity Today, Kelly explains the essence of what the magazine refers to as his “geek theology”:

We are here to surprise God. God could make everything, but instead he says, “I bestow upon you the gift of free will so that you can participate in making this world. I could make everything, but I am going to give you some spark of my genius. Surprise me with something truly good and beautiful.” So we invent things, and God says, “Oh my gosh, that was so cool! I could have thought of that, but they thought of that instead.”

I confess I have a little trouble imagining God saying something like “Oh my gosh, that was so cool!” It makes me think that Kelly’s God must look like Jeff Spicoli:


But beyond the curious lingo, Kelly’s attempt to square Christianity with the materialist thrust of technological progress is compelling – and moving. If you’re going to have a geek theology, it seems wise to begin with a sense of the divinity of the act of making. In creating technology, then, we are elaborating, extending creation itself – carrying on God’s work, in Kelly’s view. Kelly goes on to offer what he terms “a technological metaphor for Jesus,” which stems from his experience watching computer game-makers create immersive virtual worlds and then enter the worlds they’ve created:

I had this vision of the unbounded God binding himself to his creation. When we make these virtual worlds in the future — worlds whose virtual beings will have autonomy to commit evil, murder, hurt, and destroy options — it’s not unthinkable that the game creator would go in to try to fix the world from the inside. That’s the story of Jesus’ redemption to me. We have an unbounded God who enters this world in the same way that you would go into virtual reality and bind yourself to a limited being and try to redeem the actions of the other beings since they are your creations … For some technological people, that makes [my] faith a little more understandable.

Kelly’s personal relationship to technology is complex. He may be a technophile in the abstract – a geek in the religious sense – but in his own life he takes a wary, skeptical view of new gadgets and other tools, resisting rather than giving in to their enchantments in order to protect his own integrity. Inspired by the example of the Amish, he is a technological minimalist: “I seek to find those technologies that assist me in my mission to express love and reflect God in the world, and then disregard the rest.” One senses here that Kelly is most interested in technological progress as a source of metaphor, a means of probing the mystery of existence. The interest is, oddly enough, a fundamentally literary one.

The danger with metaphor is that, like technology, it can be awfully seductive; it can skew one’s view of reality. In the interview, as in his recent, sweeping book,What Technology Wants, Kelly argues that technological progress is a force for good in the world, a force of “love,” because it serves to expand the choices available to human beings, to give people more “opportunities to express their unique set of God-given gifts.” Kelly therefore believes, despite his wariness about the effects of technology on his own life, that he has a moral duty to promote rapid technological innovation. If technology is love, then, by definition, the more of it, the better:

I want to increase all the things that help people discover and use their talents. Can you imagine a world where Mozart did not have access to a piano? I want to promote the invention of things that have not been invented yet, with a sense of urgency, because there are young people born today who are waiting upon us to invent their aids. There are Mozarts of this generation whose genius will be hidden until we invent their equivalent of a piano — maybe a holodeck or something. Just as you and I have benefited from the people who invented the alphabet, books, printing, and the Internet, we are obligated to materialize as many inventions as possible, to hurry, so that every person born and to-be-born will have a great chance of discovering and sharing their godly gifts.

There is a profound flaw in this view of progress. While I think that Kelly could make a strong case that technological progress increases the number of choices available to people in general, he goes beyond that to suggest that the process is continuously additive. Progress gives and never takes away. Each new technology means more choices for people. But that’s not true. When it comes to choices, progress both gives and takes away. It closes some possibilities even as it opens others. You can’t assume that, for any given child, technological advance will increase the likelihood that she will fulfill her natural potential – or, in Kelly’s words, discover and share her unique godly gifts. It may well reduce that likelihood.

The fallacy in Kelly’s thinking becomes quickly apparent if you look closely at his Mozart example (which he also uses in his book). The fact that Mozart was born after the invention of the piano and that the piano was essential to Mozart’s ability to fulfill his potential is evidence, according to Kelly’s logic, of the beneficence of progress. But while it’s true that if Mozart had been born 300 years earlier, the less advanced state of technological progress may have prevented him from fulfilling his potential, it’s equally true that if he had been born 300 years later, the more advanced state of technological progress would have equally prevented him from achieving his potential. It’s absurd to believe that if Mozart were living today, he would create the great works he created in the eighteenth century – the symphonies, the operas, the concertos. Technological progress, among other forces, has transformed the world, and turned it into a world that is less suited to an artist of Mozart’s talents.

Genius emerges at the intersection of unique individual human potential and unique temporal circumstances. As circumstances change, some people’s ability to fulfill their potential will increase, but other people’s will decrease. Progress does not simply expand options. It changes options, and along the way options are lost as well as gained. Homer lived in a world that we would call technologically primitive, yet he created immortal epic poems. If Homer were born today, he would not be able to compose those poems in his head. That possibility has been foreclosed by progress. For all we know, if Homer (or Mozart) were born today, he would end up being an advertising copywriter, and perhaps not even a very good one.

Look at any baby born today, and try to say whether that child would have a greater possibility of fulfilling its human potential if during its lifetime (a) technological progress reversed, (b) technological progress stalled, (c) technological progress advanced slowly, or (d) technological progress accelerated quickly. You can’t. Because it’s unknowable.

The best you can argue, therefore, is that technological progress will, on balance, have a tendency to open more choices for more people. But that’s not a moral argument about the benefits of progress; it’s a practical argument, an argument based on calculations of utility. If, at the individual level, new technology may actual prevent people from discovering and sharing their “godly gifts,” then technology is not itself godly. Why would God thwart His own purposes? Technological progress is not a force of cosmic goodness, and it is surely not a force of cosmic love. It’s an entirely earthly force, as suspect as the flawed humans whose purposes it suits. Kelly’s belief that we are morally obligated “to materialize as many inventions as possible” and “to hurry” in doing so is not only based on a misperception; it’s foolhardy and dangerous.

Image: Still from the movie “Fast Times at Ridgemont High.”


Filed under Uncategorized