« Pynchon and the Badass Luddites | Main | Echoes »

TuringCraft

April 10, 2010

Inspired by Weizenbaum's ELIZA, Ben Weber has created a chatbot that converses with other players in online games of StarCraft. The bot's name is EISBot. "Once every thirty seconds," explains Weber, "EISBot randomly selects a message from a pool of 75 messages and sends it to the console." As the following transcript reveals, the bot's randomly chosen messages fit seamlessly into the conversation:

EISBot: What to ally victory?

Player: damn you

Player: im not stupid

EISBot: Wanna join my clan

Player: dunno

Player: im not that great

Player: lol

EISBot: So, this game is pretty much over

Player: yeah

Player: lol

Player: im playing off race

Player: lol

EISBot: Quit wasting time, come kill me

EISBot: What kind of hardware are you using? I’m on a 486 with 16M ram, pure luxory!!!

Player: i dunno

EISBot: PWN PWN PWN!

EISBot: I’m a huge nonyt fanboy

EISBot: Have you played SC 2 yet?

Player: yeah

Player: pretty sick

EISBot: Have you ever played against a C player?

Player: nope

Player: wait

Player: yeah

Player: i lost

Player: badly

Player: haa

EISBot: Did you see the past OSL?

Player: kinda

Note that the bot's one major flaw is that its command of the English language, particularly the use of punctuation marks, is much too sophisticated in comparison with that of the human. The sure way to distinguish the computer's messages from the human's is to recognize that the computer has a rather sentimental attachment to the apostrophe and the comma.

I take this as another indication that I am correct in my suspicion that when computers finally pass the Turing test it won't be because computers have become smarter; it will be because humans have become dumber.

lol

Comments

"...because computers have become smarter; it will be because humans have become dumber."

It's the other way around. Humans can get away writing without punctuation because we are smart. It will be a while before computers can understand text without punctuation. Sort of like how humans can understand warped text in a Captcha, while computers find bad handwriting difficult.

If anything, slang (and text abbreviations are slang, not just laziness) might be the last reveal for determining whether an intelligence is AI or human. (Slang is now often the last reveal for determining nationality or ethnicity among English speakers.)

When bots can understand "pretty sick" then that's when Nick should really start to worry.

Posted by: Kevin Kelly [TypeKey Profile Page] at April 10, 2010 07:22 PM

This story made the rounds last year, passing the reverse Turing test by being picked as real news by a couple of aggregators: http://newstechnica.com/2008/10/13/turing-test-won-with-artificial-stupidity/

Posted by: ragesoss [TypeKey Profile Page] at April 10, 2010 07:37 PM

We are still trapped in the Cartesian belief that ideas can be created and communicated in a "pure" manner without the body.

This test it just tells how poor online communication can be. Trite to say it, but without a real, embodied and energetic presence we miss most of the clues.

Posted by: Ivo Quartiroli [TypeKey Profile Page] at April 11, 2010 12:49 AM

I wouldn't be so optimistic about slang being much help in determining "humanity", Kevin.

It is true that the fact we are smart allows us to extract meaning from an error-ridden transmission (and the typical Web missive tends to be error-ridden to the extreme, if we consider the accepted standards of written communication elsewhere), but this is a double-edged sword.

The problem is that the less actual verifiable information the message contains, the more we are required to supply the author's meaning ourselves. Standards for doing so have evolved, but the consequence is that the author doesn't actually have to make his meaning clear.

In terms of a Turing test, it used to be that we saw the implementation of aforementioned communication standards (grammar and punctuation, for instance) and relation to context as the key problems. These days, the former has completely ceased to be relevant (on the Web anyway) and the latter is diluted by the fact that it's possible to say a lot without actually saying much and still be understood.

Since we've gotten used to the fact that we have to distill the meaning from poorly formulated communications, the construction of a dumb communicator - based on some kind of random or somewhat contextual generation algorithm - is that much more feasible. When faced with a communication that doesn't actually make much sense of itself, we're inclined to bend over backwards to extract meaning from it, since we're used to the fact that many people on the Web actually do communicate like that.

In other words, Nick's assertion may prove true in the sense that a thoroughly stupid program can be created that will generate communications indistinguishable from those sent by a multitude of (supposedly) intelligent human beings. It will pass the Turing test (for a given value of "pass"), but will in no way represent any form of Artificial Intelligence.

Posted by: Krzysztof Wiszniewski [TypeKey Profile Page] at April 11, 2010 12:18 PM

What Krzysztof said.

In addition, re: "It will be a while before computers can understand text without punctuation."

If I enter, say, "im a little teapot" into Google, I receive the same results that I get when I enter "i'm a little teapot." So the Google computers already "understand" text missing standard punctuation as well as they "understand" text containing it. The type of understanding involved (simple symbolic decoding) doesn't, of course, indicate any form of consciousness, much less higher intelligence, but that fact doesn't contradict my hypothesis. In fact, it supports it.

Posted by: Nick Carr [TypeKey Profile Page] at April 11, 2010 04:21 PM

Hey Nick

The Turing Test has been passed! Check it out. Korean Parents mistake computer companion for their own child (who sadly starved to death while they nurtured the virtual one).

http://news.bbc.co.uk/1/hi/world/asia-pacific/8600517.stm

Posted by: mediumkool [TypeKey Profile Page] at April 13, 2010 08:55 AM

Maybe I'm missing something, but there doesn't seem to be a real conversation taking place with the bot.

I don't understand the context of the game, but it appears the bot is just asking questions and making comments but not really engaging in a back and forth dialog which is the hard part.

The bot seems to be stateless.

Can someone point out what I'm missing here?

Posted by: Larry Port [TypeKey Profile Page] at April 14, 2010 11:15 AM

OK Nick, I get it.

The bot isn't doing anything that a bot couldn't have done 20 years ago. It's nowhere near passing the test.

It's just the people are slipping.

Posted by: Larry Port [TypeKey Profile Page] at April 14, 2010 11:51 AM

Right.

The funny thing is that while Weber compares his bot to ELIZA, ELIZA, developed more than 40 years ago, was far more sophisticated. It actually showed a little bit of sense.

Posted by: Nick Carr [TypeKey Profile Page] at April 14, 2010 02:03 PM

This is really sad. StarCraft is a real-time strategy game. Anyone taking the time to construct well-formed sentences is taking time away from game play.

This just shows you are willing to bash anything related to electronic communication without taking the time to investigate for 30 seconds.

Posted by: Jeremy Corbett [TypeKey Profile Page] at April 16, 2010 02:59 PM

Post a comment

Thanks for signing in, . Now you can comment. (sign out)

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)


Remember me?


carrshot5.jpg Subscribe to Rough Type

Now in paperback:
shallowspbk2.jpg Pulitzer Prize Finalist

"Riveting" -San Francisco Chronicle

"Rewarding" -Financial Times

"Revelatory" -Booklist

Order from Amazon

Visit The Shallows site

The Cloud, demystified: bigswitchcover2thumb.jpg "Future Shock for the web-apps era" -Fast Company

"Ominously prescient" -Kirkus Reviews

"Riveting stuff" -New York Post

Order from Amazon

Visit Big Switch site

Greatest hits

The amorality of Web 2.0

Twitter dot dash

The engine of serendipity

The editor and the crowd

Avatars consume as much electricity as Brazilians

The great unread

The love song of J. Alfred Prufrock's avatar

Flight of the wingless coffin fly

Sharecropping the long tail

The social graft

Steve's devices

MySpace's vacancy

The dingo stole my avatar

Excuse me while I blog

Other writing

Is Google Making Us Stupid?

The ignorance of crowds

The recorded life

The end of corporate computing

IT doesn't matter

The parasitic blogger

The sixth force

Hypermediation

More

Nick's first book: Order from Amazon

Visit book site

Rough Type is:

Written and published by
Nicholas Carr

Designed by

JavaScript must be enabled to display this email address.

What?