{"id":7118,"date":"2016-07-03T15:23:01","date_gmt":"2016-07-03T21:23:01","guid":{"rendered":"https:\/\/www.roughtype.com\/?p=7118"},"modified":"2016-08-18T04:01:45","modified_gmt":"2016-08-18T10:01:45","slug":"tesla-and-the-glass-cockpit-problem","status":"publish","type":"post","link":"https:\/\/www.roughtype.com\/?p=7118","title":{"rendered":"Tesla and the glass cockpit problem"},"content":{"rendered":"<p><iframe loading=\"lazy\" src=\"https:\/\/www.youtube.com\/embed\/qQkx-4pFjus?rel=0\" width=\"640\" height=\"360\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><br \/>\nWhen news spread last week\u00a0about the <a href=\"http:\/\/www.wsj.com\/article_email\/teslas-autopilot-flew-under-regulators-radar-1467402007-lMyQjAxMTE2MzA4MTIwMzE5Wj\">fatal crash<\/a> of a computer-driven Tesla, I thought of a conversation I had a couple of years ago with a top computer scientist at Google. We were talking about some recent airliner crashes caused by\u00a0&#8220;automation complacency&#8221; \u2014 the tendency for even very skilled pilots to tune out from their work after turning on autopilot systems \u2014 and the Google scientist noted that the problem of automation complacency is even more acute for drivers than for pilots. If you&#8217;re flying a plane and something unexpected happens, you usually have several seconds or even minutes to respond before the situation becomes dire. If you&#8217;re driving a car, you may have only a second or a fraction of a second to take action before you collide with another car, or a bridge abutment, or a tree. There are far more\u00a0obstacles on\u00a0the ground than in the sky.<\/p>\n<p>With the Tesla accident, the evidence suggests that the crash happened before the driver even realized that he was about to hit a truck. He seemed to be suffering from automation complacency up to the very moment of impact. He trusted the machine, and the machine failed him. Such\u00a0complacency is a <a href=\"http:\/\/www.aerohabitat.eu\/uploads\/media\/Automation_and_Situation_Awareness_-_Endsley.pdf\">well-documented problem<\/a> in human-factors research, and it&#8217;s what led Google to change the course of its self-driving car program a couple of years ago, shifting to a perhaps quixotic goal of total automation without any human involvement. In rushing to give drivers the ability to switch on an &#8220;Autopilot&#8221; mode,\u00a0Tesla ignored or dismissed\u00a0the research, with a predictable result. As computer and car companies push the envelope of automotive automation, driver complacency and skill loss promise to become ever greater challenges \u2014 ones that (as Google appears to have <a href=\"https:\/\/youtu.be\/fwEPnEcVWUg?t=30m36s\">concluded<\/a>) may not be solvable given the fallibility of software, the psychology of human beings, and the realities of driving.*<\/p>\n<p>Following is a\u00a0brief\u00a0excerpt from my book about the human consequences of automation,\u00a0<em><a href=\"http:\/\/www.nicholascarr.com\/?page_id=18\">The Glass Cage<\/a>,<\/em>\u00a0that describes how, as aviation became\u00a0more automated over the years, pilots flying in so-called glass cockpits grew\u00a0more susceptible to automation complacency and &#8220;skill fade&#8221; \u2014 to the point that the FAA is now <a href=\"https:\/\/www.faa.gov\/other_visit\/aviation_industry\/airline_operators\/airline_safety\/safo\/all_safos\/media\/2013\/SAFO13002.pdf\">urging<\/a> pilots to practice manual flying more often.<\/p>\n<blockquote><p>Premature death was a routine occupational hazard for even the most expert pilots during aviation\u2019s early years. Lawrence Sperry died in 1923 when his plane crashed into the English Channel. Wiley Post died in 1935 when his plane went down in Alaska. Antoine de Saint-Exup\u00e9ry died in 1944 when his plane disappeared over the Mediterranean. Air travel\u2019s lethal days are, mercifully, behind us. Flying is safe now, and pretty much everyone involved in the aviation business believes that advances in automation are one of the reasons why. Together with improvements in aircraft design, airline safety routines, crew training, and air traffic control, the mechanization and computerization of flight have contributed to the sharp and steady decline in accidents and deaths over the decades.<\/p>\n<p>But this sunny story carries a dark footnote. The overall decline in the number of plane crashes masks the recent arrival of \u201ca spectacularly new type of accident,\u201d says Raja Parasuraman, a psychology professor at George Mason University and one of the world\u2019s leading authorities on automation. When onboard computer systems fail to work as intended or other unexpected problems arise during a flight, pilots are forced to take manual control of the plane. Thrust abruptly into a dangerous situation, they too often make mistakes. The consequences, as the 2009 Continental Connection and Air France disasters show, can be catastrophic. Over the last thirty years, dozens of psychologists, engineers, and human factors researchers have studied what\u2019s gained and lost when pilots share the work of flying with software. They\u2019ve learned that a heavy reliance on computer automation can erode pilots\u2019 expertise, dull their reflexes, and diminish their attentiveness, leading to what Jan Noyes, a human-factors expert at Britain\u2019s University of Bristol, calls \u201ca deskilling of the crew.\u201d<\/p>\n<p>Concerns about the unintended side effects of flight automation aren\u2019t new. They date back at least to the early days of glass cockpits and fly-by-wire controls. A 1989 report from NASA\u2019s Ames Research Center noted that as computers had begun to multiply on airplanes during the preceding decade, industry and governmental researchers \u201cdeveloped a growing discomfort that the cockpit may be becoming too automated, and that the steady replacement of human functioning by devices could be a mixed blessing.\u201d Despite a general enthusiasm for computerized flight, many in the airline industry worried that \u201cpilots were becoming over-dependent on automation, that manual flying skills may be deteriorating, and that situational awareness might be suffering.\u201d<\/p>\n<p>Studies conducted since then have linked many accidents and near misses to breakdowns of automated systems or to automation complacency or other \u201cautomation-induced errors\u201d on the part of flight crews. In 2010, the FAA released preliminary results of a major study of airline flights over the preceding ten years which showed that pilot errors had been involved in nearly two-thirds of all crashes. The research further indicated, according to FAA scientist Kathy Abbott, that automation has made such errors more likely. Pilots can be distracted by their interactions with onboard computers, Abbott said, and they can \u201cabdicate too much responsibility to the automated systems.\u201d An extensive 2013 government report on cockpit automation, compiled by an expert panel and drawing on the same FAA data, implicated automation-related problems, such as a complacency-induced loss of\u00a0situational awareness and weakened hand-flying skills, in more than half of recent accidents.<\/p>\n<p>The anecdotal evidence collected through accident reports and surveys gained empirical backing from a rigorous study conducted by Matthew Ebbatson, a young human-factors researcher at Cranfield University, a top U.K. engineering school. Frustrated by the lack of hard, objective data on what he termed \u201cthe loss of manual flying skills in pilots of highly automated airliners,\u201d Ebbatson set out to fill the gap. He recruited sixty-six veteran pilots from a British airline and had each of them get into a flight simulator and perform a challenging maneuver\u2014bringing a Boeing 737 with a blown engine in for a landing during bad weather. The simulator disabled the plane\u2019s automated systems, forcing the pilot to fly by hand. Some of the pilots did exceptionally well in the test, Ebbatson reported, but many performed poorly, barely exceeding \u201cthe limits of acceptability.\u201d<\/p>\n<p>Ebbatson then compared detailed measures of each pilot\u2019s performance in the simulator\u2014the pressure exerted on the yoke, the stability of airspeed, the degree of variation in course\u2014with the pilot\u2019s historical flight record. He found a direct correlation between a pilot\u2019s aptitude at the controls and the amount of time that pilot had spent flying without the aid of automation. The correlation was particularly strong with the amount of manual flying done during the preceding two months. The analysis indicated that \u201cmanual flying skills decay quite rapidly towards the fringes of \u2018tolerable\u2019 performance without relatively frequent practice.\u201d Particularly \u201cvulnerable to decay,\u201d Ebbatson noted, was a pilot\u2019s ability to maintain \u201cairspeed control\u201d\u2014a skill crucial to recognizing, avoiding, and recovering from stalls and other dangerous situations.<\/p>\n<p>It\u2019s no mystery why automation degrades pilot performance. Like many challenging jobs, flying a plane involves a combination of psychomotor skills and cognitive skills\u2014thoughtful action and active thinking. A pilot needs to manipulate tools and instruments with precision while swiftly and accurately making calculations, forecasts, and assessments in his head. And while he goes through these intricate mental and physical maneuvers, he needs to remain vigilant, alert to what\u2019s going on around him and able to distinguish important signals from unimportant ones. He can\u2019t allow himself either to lose focus or to fall victim to tunnel vision. Mastery of such a multifaceted set of skills comes only with rigorous practice. A beginning pilot tends to be clumsy at the controls, pushing and pulling the yoke with more force than necessary. He often has to pause to remember what he should do next, to walk himself methodically through the steps of a process. He has trouble shifting seamlessly between manual and cognitive tasks. When a stressful situation arises, he can easily become overwhelmed or distracted and end up overlooking a critical change in circumstances.<\/p>\n<p>In time, after much rehearsal, the novice gains confidence. He becomes less halting in his work and more precise in his actions. There\u2019s little wasted effort. As his experience continues to deepen, his brain develops so-called mental models\u2014dedicated assemblies of neurons\u2014that allow him to recognize patterns in his surroundings. The models enable him to interpret and react to stimuli intuitively, without getting bogged down in conscious analysis. Eventually, thought and action become seamless. Flying becomes second nature. Years before researchers began to plumb the workings of pilots\u2019 brains, Wiley Post described the experience of expert flight in plain, precise terms. He flew, he said in 1935, \u201cwithout mental effort, letting my actions be wholly controlled by my subconscious mind.\u201d He wasn\u2019t born with that ability. He developed it through hard work.<\/p>\n<p>When computers enter the picture, the nature and the rigor of the work change, as does the learning the work engenders. As software assumes moment-by-moment control of the craft, the pilot is relieved of much manual labor. This reallocation of responsibility can provide an important benefit. It can reduce the pilot\u2019s workload and allow him to concentrate on the cognitive aspects of flight. But there\u2019s a cost. Psychomotor skills get rusty, which can hamper the pilot on those rare but critical occasions when he\u2019s required to take back the controls. There\u2019s growing evidence that recent expansions in the scope of automation also put cognitive skills at risk. When more advanced computers begin to take over planning and analysis functions, such as setting and adjusting a flight plan, the pilot becomes less engaged not only physically but also mentally. Because the precision and speed of pattern recognition appear to depend on regular practice, the pilot\u2019s mind may become less agile in interpreting and reacting to fast-changing situations. He may suffer what Ebbatson calls \u201cskill fade\u201d in his mental as well as his motor abilities.<\/p>\n<p>Pilots are not blind to automation\u2019s toll. They\u2019ve always been wary about ceding responsibility to machinery. Airmen in World War I, justifiably proud of their skill in maneuvering their planes during dogfights, wanted nothing to do with the newfangled\u00a0Sperry autopilots. In 1959, the original Mercury astronauts rebelled against NASA\u2019s plan to remove manual flight controls from spacecraft. But aviators\u2019 concerns are more acute now. Even as they praise the enormous gains in flight technology, and acknowledge the safety and efficiency benefits, they worry about the erosion of their talents. As part of his research, Ebbatson surveyed commercial pilots, asking them whether \u201cthey felt their manual flying ability had been influenced by the experience of operating a highly automated aircraft.\u201d More than three-fourths reported that \u201ctheir skills had deteriorated\u201d; just a few felt their skills had improved. A 2012 pilot survey conducted by the European Aviation Safety Agency found similarly widespread concerns, with 95 percent of pilots saying that automation tended to erode \u201cbasic manual and cognitive flying skills.\u201d<\/p>\n<p>Rory Kay, a long-time United Airlines captain who until recently served as the top safety official with the Air Line Pilots Association, fears the aviation industry is suffering from \u201cautomation addiction.\u201d In a 2011 interview with the Associated Press, he put the problem in stark terms: \u201cWe\u2019re forgetting how to fly.\u201d<\/p><\/blockquote>\n<p>What the aviation industry has discovered is that there&#8217;s a tradeoff between computer automation and human skill and attentiveness. Getting the balance right is exceedingly\u00a0tricky. Just because some degree of automation is good, that doesn&#8217;t mean that more automation is necessarily better. We seem fated to learn this hard lesson once again with the even trickier process of automotive automation.<\/p>\n<p>*UPDATE (7\/7): The Times <a href=\"http:\/\/www.nytimes.com\/2016\/07\/08\/automobiles\/wheels\/makers-of-self-driving-cars-ask-what-to-do-with-human-nature.html\">reports<\/a>: &#8220;Experiments conducted last year by\u00a0Virginia Tech\u00a0researchers and supported by the national safety administration found that it took drivers of [self-driving]\u00a0cars an average of 17 seconds to respond to takeover requests. In that period, a vehicle going 65 m.p.h. would have traveled 1,621 feet \u2014 more than five football fields.&#8221;<\/p>\n<p><a href=\"#_ednref1\" name=\"_edn1\"><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>When news spread last week\u00a0about the fatal crash of a computer-driven Tesla, I thought of a conversation I had a couple of years ago with a top computer scientist at Google. We were talking about some recent airliner crashes caused by\u00a0&#8220;automation complacency&#8221; \u2014 the tendency for even very skilled pilots to tune out from their [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[1],"tags":[],"class_list":["post-7118","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"jetpack_publicize_connections":[],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/posts\/7118","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=7118"}],"version-history":[{"count":10,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/posts\/7118\/revisions"}],"predecessor-version":[{"id":7204,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/posts\/7118\/revisions\/7204"}],"wp:attachment":[{"href":"https:\/\/www.roughtype.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=7118"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=7118"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=7118"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}