Remember that Microsoft video on our glassy future? Or that one from Corning? Or that one from Toyota? What they all suggest, and assume, is that our rich natural “interface” with the world will steadily wither away as we become more reliant on software mediation. The infinite possibilities of our sense of touch become reduced to a set of scripted gestures.
Former Apple engineer Bret Victor makes a passionate, and nicely illustrated, case that we need to challenge the reigning visions of future computer interfaces, which he sums up as “Pictures Under Glass”:
Pictures Under Glass sacrifice all the tactile richness of working with our hands, offering instead a hokey visual facade. Is that so bad, to dump the tactile for the visual? Try this: close your eyes and tie your shoelaces. No problem at all, right? Now, how well do you think you could tie your shoes if your arm was asleep? Or even if your fingers were numb? When working with our hands, touch does the driving, and vision helps out from the back seat. Pictures Under Glass is an interaction paradigm of permanent numbness. It’s a Novocaine drip to the wrist. It denies our hands what they do best. And yet, it’s the star player in every Vision Of The Future.
As Anne Mangen has argued, in her work on the tactile aspects of reading and writing, we tend to ignore the importance that our sense of touch plays in our intellectual and emotional lives, probably because we are unconscious of the effects. Unfortunately, that makes it easy for us to sacrifice the richness of our tactile sense when we use, or design, computers. We settle for pictures under glass, for numbness.
I think that the scope of gesture has not been fully explored. We are still largely thinking of a desktop metaphor. What about a card table metaphor? What about a kitchen table metaphor? Or even a three dimensional desk metaphor?
I think any sort of manual task that requires fine motor control of the hands and arms should be considered as a model for abstract computer operations. When you consider how quickly people can learn and excel with the sort of manual dexterity that could be captured in 3D in combination with 2D you can see that a great deal faster and more operations can be controlled. Game developers think this way. Computer UI developers don’t seem to, or at least have no vehicle to show us that they get it.
Nokia has some non-glassy ideas. See this video.
In fairness, any video, especially one for an “arty” audience, is going to be constrained to a presentation that’s easily comprehensible in terms of visual imagery. Hence the focus on simple visual interface. It’s no more than the equivalent of why TV shows and movies often have scenes where billowing smoke is coming out of broken computers (which happens rarely enough in real life that when it does people joke it’s just like TV). In fact, I suspect a real video of people using computers several decades from now might look very dull. This was a great scene (Google/Net information immediacy!)
http://www.inthe80s.com/moviescenes/s.shtml
Star Trek IV: The Voyage Home
* It’s the scene where McCoy and Scotty go to the plastics company to get materials to make a tank for the whales. After offering to make a new material for the company, Scotty tries to use the computer to give him the formula for “Transparent Aluminum”. The scene proceeds as follows:
Scotty: “Computer on! Computer on!”
McCoy: “Mr. Scott.” (hands him the mouse)
Scotty(holding mouse up to his mouth): “Computer on!”
Overall the web has brought back the computer interface to a more “typographic” one, more than “icons and stuff”, and to me what is as much if not more impressive in the videos ‘microsoft one in particular) than the computer interface aspect, is this “fascination for usefulness”, for automation, when I see computers as much if not much more as an interface to knowledge (consulting or editing tools). Usefulness is boring and often consume too much energy.
Don’t think I’m up to speed on this. I read and write. Instead of holding a paper based book in which I write, I switch over to a digital book and highlight and type notes. This is somehow making me a person whose sense of touch isn’t being used in the way it once was. We’re talking about the difference between writing with a pen and paper and typing? Is Bret Victor saying people need to envision futures with computers that make use of touch instead of always imagining us as flicking around on a keypad looking at pictures or video on a screen?
Glad for the Anne M. mention. I’ll check out her work.
“Do you know what I hate about computers? The problem with computers is that there is not enough Africa in them. This is why I can’t use them for very long. Do you know what a nerd is? A nerd is a human being without enough Africa in him or her. I know this sounds sort of inversely racist to say, but I think the African connection is so important. You know why music was the center of our lives for such a long time? Because it was a way of allowing Africa in. In 50 years, it might not be Africa; it might be Brazil. But I want so desperately for that sensibility to flood into these other areas, like computers.
What’s pissing me off is that it uses so little of my body. You’re just sitting there, and it’s quite boring. You’ve got this stupid little mouse that requires one hand, and your eyes. That’s it. What about the rest of you? No African would stand for a computer like that. It’s imprisoning.”
Brian Eno ’95