All the world’s a screen


Do you wear Google Glass, or does Google Glass wear you?

That question came to the fore on October 15, when the U.S. government granted Google a broad patent for “hand gestures to signify what is important.” Now, don’t panic. You’re not going to be required to ask Google’s permission before pointing or clapping or high-fiving or fist-pumping. The gestures governed by the patent are limited to those “used to provide user input to a wearable computing device.” They are, in particular, hand motions that the company envisions will help people use Glass and other head-mounted computers.

One of the challenges presented by devices like Glass is the lack of flexible input devices. Desktops have keyboards and mice. Laptops have touchpads. Smartphones, tablets, and other touchscreen devices have the user’s fingers. How do you send instructions to a computer that takes the form of a pair of glasses? How do you operate its apps? You can move your head around — Glass has a motion sensor — but that accomplishes only so much. There aren’t all that many ways you can waggle your noggin, and none of them are particularly precise. But Glass does have a camera, and the camera can be programmed to recognize particular hand gestures and translate them into instructions for software applications.

To take a particularly literal-minded example, you can frame an object inside a heart formed by your thumbs and fingers in order to register your approval of or fondness for the object. The effect would be the same as clicking a Like button. Google, in its patent filing, provides a couple of illustrations:


You can also “select” some part of the landscape by making a “lasso” gesture with your finger:


You can also do the machine-gun thing with your thumb and index finger, though Google is coy about exactly what that might mean:


The above illustration is described only as “an example user who is depicted as making an example hand gesture, according to an example embodiment.”

Along with excitement and curiosity, the prospect of Glass’s arrival in the mass market has provoked trepidation, stemming mainly from the documentary possibilities of the device’s tiny camera. Some people are nervous about a further loss of privacy and a further expansion of surveillance should the multitudes begin having network-connected cameras strapped to their foreheads. The camera, the patent makes clear, plays a data-input role as well as a documentary one, and the use of jerky hand motions to control Glass and its apps should  be cause for a little added anxiety, if only for the dubious aesthetic merits of having people walking around making weird gestures all the time.

But there’s something deeper going on here. Glass turns the human body into a computer input device more fully and more publicly than anything we’ve seen before — more so than even Microsoft’s Kinect. Kinect focuses a fixed camera on the user, while Glass focuses a mobile camera outward from the user. It’s an important distinction. With Glass, a person’s gaze becomes a computer cursor, with the focus of the gaze also directing the focus of the computer. What that also means is that the person’s surroundings  effectively become a computer display. The world becomes an array of data that can be manipulated by both the cursory gaze of its wearer and the input signals sent by the wearer’s hand gestures. This fulfills the ultimate dream of those who desire, for reasons of ideology or financial gain or both, to make all of existence “machine-readable,” to turn all of reality into a store of digital data. The role of the computer as mediator of experience becomes universal and seamless. Whereas “virtual reality” provided us with a simulation of the real that remained separate from the real, Glass turns the real into a simulation of itself.

“Gradually,” the CUNY media scholar Lev Manovich wrote nearly twenty years ago, “cinema taught us to accept the manipulation of time and space, the arbitrary coding of the visible, the mechanization of vision, and the reduction of reality to a moving image as a given. As a result, today the conceptual shock of the digital revolution is not experienced as a real shock — because we were ready for it for a long time.” We are now, in a similar way, prepared for the further revolution of Glass and its radical transformation of the real. Under the misleading slogan of “reality augmentation,” we are set — should this be the course we choose — to undergo “reality reduction,” as the world becomes a computer display and its sensory richness fades even further away.

Image: still from Contempt by Jean-Luc Godard.

2 thoughts on “All the world’s a screen

  1. yt75

    Always these same kind of titles, time to get a bit through it for you maybe Nick ?
    Words, signs, letters, the forever unwritable common perpetually evolving definitions of them, not a matter of screens.
    And some form of aesthetic need or urge sometimes, as well

Comments are closed.