{"id":2212,"date":"2012-12-01T14:03:08","date_gmt":"2012-12-01T21:03:08","guid":{"rendered":"https:\/\/www.roughtype.com\/?p=2212"},"modified":"2012-12-02T11:57:49","modified_gmt":"2012-12-02T18:57:49","slug":"the-last-invention","status":"publish","type":"post","link":"https:\/\/www.roughtype.com\/?p=2212","title":{"rendered":"The last invention"},"content":{"rendered":"

The human race’s “last invention,” wrote<\/a> the British mathematician and original Singularitarian\u00a0Jack Good in 1964, would be a machine that’s smarter than we are. \u00a0This “ultra-intelligent machine,” by dint of its ability to create even smarter machines, would, “unquestionably,” ignite an “intelligence explosion” that would provide us with innumerable new inventions, improving our lives in unimaginable ways and, indeed, assuring our survival. We’d be able to kick back and enjoy the technological largesse, fat and happy and, one imagines, immortal. We’d never again go bald or forget where we put the car keys.<\/p>\n

That’s assuming, Good threw in, as a quick aside, that “the machine is docile enough to tell us how to keep it under control.” If the machine turned out to be an ornery mofo, then the shit would hit the fan, existentially speaking,\u00a0We’d end up pets, or renewable energy sources. But the dark scenario wasn’t one that Good felt likely. If we could develop an artificial intelligence and set it loose in the world, the future would be bright.<\/p>\n

Nearly fifty years have gone by and, though we’re certainly fat, we’re not particularly happy and we’re not at all immortal. Keys are mislaid. Hair falls out. Graves are dug and filled.<\/p>\n

Worst of all, we’ve lost our optimism about the benevolence of that ultra-intelligent machine that we still like to think we’re going to build. The Singularity is nearer than ever \u2014 40 years out, right? \u2014 and the prospect of its arrival fills us not with joy but with dread. Given our record in such things, it’s hard for us to imagine that the ultra-intelligent machine we design is going to be polite and well-mannered and solicitous \u2014 an ultra-intelligent Mary Poppins. No, it’s going to be an ultra-intelligent Snidely Whiplash.<\/p>\n

So it comes as a relief to hear that Cambridge University is setting up a Centre for the Study of Existential Risk, to be\u00a0helmed<\/a> by the distinguished philosopher Huw Price, the distinguished astrophysicist Martin Reese, and the distinguished programmer Jaan Tallinn, one of the developers of Kazaa and Skype. The CSER<\/a> will be dedicated to examining and ameliorating “extinction-level risks to our species,” particularly those arising from an AI-fueled Singularity.<\/p>\n

In a recent article<\/a>, Price and Tallinn explained why we should be worried about an intelligence explosion:<\/p>\n

We now have machines that have trumped human performance in such domains as chess, trivia games, flying, driving, financial trading, face, speech and handwriting recognition \u2013 the list goes on.\u00a0Along with the continuing progress in hardware, these developments in narrow AI make it harder to defend the view that computers will never reach the level of the human brain. A steeply rising curve and a horizontal line seem destined to intersect! …<\/p>\n

It would be comforting to think that any intelligence that surpassed our own capabilities would be like us, in important respects \u2013 just a lot cleverer. But here, too, the pessimists see bad news: they point out that almost all the things we humans value (love, happiness, even survival) are important to us because we have particular evolutionary history \u2013 a history we share with higher animals, but not with computer programs, such as artificial intelligences. …\u00a0The bad news is that they might simply be indifferent to us \u2013 they might care about us as much as we care about the bugs on the windscreen.<\/p><\/blockquote>\n

At this point in reading the article, my spirits actually began to brighten. An indifferent AI seemed less worrisome than a hostile one. After all, the universe is indifferent to us, and we’re doing okay. But then came the kicker:<\/p>\n

just ask gorillas how it feels to compete for resources with the most intelligent species \u2013 the reason they are going extinct is not (on the whole) because humans are actively hostile towards them, but because we control the environment in ways that are detrimental to their continuing survival.<\/p><\/blockquote>\n

This threw me back into a funk \u2014 and it made me even more eager to see the Centre for the Study of Existential Risk up and running. Until, that is, I began to think a little more about those gorillas. If they haven’t had any luck in influencing a species of superior intelligence with whom they share an evolutionary history (that would be us), isn’t it a little silly to think that we’ll have any chance of influencing an intelligence beyond our own, particularly if that intelligence is indifferent to us and even, from an evolutionary standpoint, alien to us? My mood blackened.<\/p>\n

Writing in the shadow of the bomb,\u00a0Jack Good began his 1964 paper with this sentence: “The survival of man depends on the early construction of an ultra-intelligent machine.” We no longer fear the present as much as Good did, but neither are we able to muster as much confidence in our ability to shape the future to our benefit. As computers have become more common, more familiar, we’ve lost our faith in them. They’ve turned from Existential Hope to Existential Risk. When we imagine our last invention \u2014 the end of human progress \u2014 we sense not our deliverance but our demise. That may actually say more about what’s changed in us than what’s changed about the future.<\/p>\n","protected":false},"excerpt":{"rendered":"

The human race’s “last invention,” wrote the British mathematician and original Singularitarian\u00a0Jack Good in 1964, would be a machine that’s smarter than we are. \u00a0This “ultra-intelligent machine,” by dint of its ability to create even smarter machines, would, “unquestionably,” ignite an “intelligence explosion” that would provide us with innumerable new inventions, improving our lives in […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[1],"tags":[],"class_list":["post-2212","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/posts\/2212","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2212"}],"version-history":[{"count":10,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/posts\/2212\/revisions"}],"predecessor-version":[{"id":2219,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/posts\/2212\/revisions\/2219"}],"wp:attachment":[{"href":"https:\/\/www.roughtype.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2212"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2212"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2212"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}