{"id":8622,"date":"2019-03-14T09:09:10","date_gmt":"2019-03-14T13:09:10","guid":{"rendered":"https:\/\/www.roughtype.com\/?p=8622"},"modified":"2019-03-15T06:38:57","modified_gmt":"2019-03-15T10:38:57","slug":"on-autopilot-the-dangers-of-overautomation","status":"publish","type":"post","link":"https:\/\/www.roughtype.com\/?p=8622","title":{"rendered":"On autopilot: the dangers of overautomation"},"content":{"rendered":"\n

The grounding of Boeing’s popular new 737 Max 8 planes, after two recent crashes, has placed a <\/em>new focus<\/em><\/a> on flight automation. Here’s an excerpt from my 2014 book on automation and its human consequences, <\/em>The Glass Cage<\/em><\/a>, that seems relevant to the discussion.<\/em><\/p>\n\n\n\n

The lives of aviation’s pioneers were exciting but short. Lawrence Sperry died in 1923 when his plane crashed into the English Channel. Wiley Post died in 1935 when his plane went down in Alaska. Antoine de Saint-Exup\u00e9ry died in 1944 when his plane disappeared over the Mediterranean. Premature death was a routine occupational hazard for pilots during aviation\u2019s early years; romance and adventure carried a high price. Passengers died with alarming frequency, too. As the airline industry took shape in the 1920s, the publisher of a U.S. aviation journal implored the government to improve flight safety, noting that \u201ca great many fatal accidents are daily occurring to people carried in airplanes by inexperienced pilots.\u201d<\/p>\n\n\n\n

Air travel\u2019s lethal days are, mercifully, behind us. Flying is safe now, and pretty much everyone involved in the aviation business believes that advances in automation are one of the reasons why. Together with improvements in aircraft design, airline safety routines, crew training, and air traffic control, the mechanization and computerization of flight have contributed to the sharp and steady decline in accidents and deaths over the decades. In the United States and other Western countries, fatal airliner crashes have become exceedingly rare. Of the more than seven billion people who boarded U.S. flights in the ten years from 2002 through 2011, only 153 ended up dying in a wreck, a rate of two deaths for every million passengers. In the ten years from 1962 through 1971, by contrast, 1.3 billion people took flights, and 1,696 of them died, for a rate of 133 deaths per million.<\/p>\n\n\n\n

But this sunny story carries a dark footnote. The overall decline in plane crashes masks the recent arrival of  \u201ca spectacularly new type of accident,\u201d says Raja Parasuraman, a psychology professor at George Mason University and one of the world\u2019s leading authorities on automation. When onboard computer systems fail to work as intended or other unexpected problems arise during a flight, pilots are forced to take manual control of the plane. Thrust abruptly into what has become a rare role, they too often make mistakes. The consequences, as the Continental Connection and Air France disasters of 2009 show, can be catastrophic. Over the last 30 years, scores of psychologists, engineers, and other ergonomics, or \u201chuman factors,\u201d researchers have studied what\u2019s gained and lost when pilots share the work of flying with software. What they\u2019ve learned is that a heavy reliance on computer automation can erode pilots\u2019 expertise, dull their reflexes, and diminish their attentiveness, leading to what Jan Noyes, a human factors expert at Britain\u2019s University of Bristol, calls \u201ca deskilling of the crew.” <\/p>\n\n\n\n

Concerns about the unintended side effects of flight automation aren\u2019t new. They date back at least to the early days of fly-by-wire controls. A 1989 report from NASA\u2019s Ames Research Center noted that, as computers had begun to multiply on airplanes during the preceding decade, industry and governmental researchers \u201cdeveloped a growing discomfort that the cockpit may be becoming too automated, and that the steady replacement of human functioning by devices could be a mixed blessing.\u201d Despite a general enthusiasm for computerized flight, many in the airline industry worried that \u201cpilots were becoming over-dependent on automation, that manual flying skills may be deteriorating, and that situational awareness might be suffering.\u201d<\/p>\n\n\n\n

Many studies since then have linked particular accidents or near misses to breakdowns of automated systems or to \u201cautomation-induced errors\u201d on the part of flight crews. In 2010, the Federal Aviation Administration released some preliminary results of a major study of airline flights over the preceding ten years, which showed that pilot errors had been involved in more than 60 percent of crashes. The research further indicated, according to a report from FAA scientist Kathy Abbott, that automation has made such errors more likely. Pilots can be distracted by their interactions with onboard computers, Abbott said, and they can \u201cabdicate too much responsibility to the automated systems.\u201d<\/p>\n\n\n\n

In the worst cases, automation can place added and unexpected demands on pilots during moments of crisis\u2014when, for instance, the technology fails. The pilots may have to interpret computerized alarms, input data, and scan information displays even as they\u2019re struggling to take manual control of the plane and orient themselves to their circumstances. The tasks and attendant distractions increase the odds that the aviators will make mistakes. Researchers refer to this as the \u201cautomation paradox.\u201d As Mark Scerbo, a psychologist and human-factors expert at Virginia\u2019s Old Dominion University, has explained, \u201cThe irony behind automation arises from a growing body of research demonstrating that automated systems often increase<\/em> workload and create unsafe<\/em> working conditions.\u201d <\/p>\n\n\n\n

The anecdotal and theoretical evidence collected through accident reports, surveys, and studies received empirical backing from a rigorous experiment conducted by Matthew Ebbatson, a young human factors researcher at the University of Cranfield, a top U.K. engineering school. Frustrated by the lack of hard, objective data on what he termed \u201cthe loss of manual flying skills in pilots of highly automated airliners,\u201d Ebbatson set out to fill the gap. He recruited 66 veteran pilots from a British airline and had each of them get into a flight simulator and perform a challenging maneuver\u2014bringing a Boeing 737 with a blown engine in for a landing in bad weather. The simulator disabled the plane\u2019s automated systems, forcing the pilots to fly by hand. Some of the pilots did exceptionally well in the test, Ebbatson reported, but many of them performed poorly, barely exceeding \u201cthe limits of acceptability.\u201d <\/p>\n\n\n\n

Ebbatson then compared detailed measures of each pilot\u2019s performance in the simulator\u2014the pressure they exerted on the yoke, the stability of their airspeed, the degree of variation in their course\u2014with their historical flight records. He found a direct correlation between a pilot\u2019s aptitude at the controls and the amount of time the pilot had spent flying by hand, without the aid of automation. The correlation was particularly strong with the amount of manual flying done during the preceding two months. The analysis indicated that \u201cmanual flying skills decay quite rapidly towards the fringes of \u2018tolerable\u2019 performance without relatively frequent practice.\u201d Particularly \u201cvulnerable to decay,\u201d Ebbatson noted, was a pilot\u2019s ability to maintain \u201cairspeed control\u201d\u2014a skill that\u2019s crucial to recognizing, avoiding, and recovering from stalls and other dangerous situations.<\/p>\n\n\n\n

It\u2019s no mystery\nwhy automation takes a toll on pilot performance. Like many challenging jobs,\nflying a plane involves a combination of psychomotor skills and cognitive\nskills\u2014thoughtful action and active thinking, in simple terms. A pilot needs to\nmanipulate tools and instruments with precision while swiftly and accurately\nmaking calculations, forecasts, and assessments in his head. And while he goes\nthrough these intricate mental and physical maneuvers, he needs to remain\nvigilant, alert to what\u2019s going on around him and adept at distinguishing\nimportant signals from unimportant ones. He can\u2019t allow himself either to lose\nfocus or to fall victim to tunnel vision. Mastery of such a multifaceted set of\nskills comes only with rigorous practice. A beginning pilot tends to be clumsy\nat the controls, pushing and pulling the yoke with more force than is\nnecessary. He often has to pause to remember what he should do next, to walk\nhimself methodically through the steps of a process. He has trouble shifting\nseamlessly between manual and cognitive tasks. When a stressful situation\narises, he can easily become overwhelmed or distracted and end up overlooking a\ncritical change in his circumstances. <\/p>\n\n\n\n

In time, after much rehearsal, the novice gains confidence. He becomes less halting in his work and much more precise in his actions. There\u2019s little wasted effort. As his experience continues to deepen, his brain develops so-called mental models\u2014dedicated assemblies of neurons\u2014that allow him to recognize patterns in his surroundings. The models enable him to interpret and react to stimuli as if by instinct, without getting bogged down in conscious analysis. Eventually, thought and action become seamless. Flying becomes second nature. Years before researchers began to plumb the workings of pilots\u2019 brains, Wiley Post described the experience of expert flight in plain, precise terms. He flew, he said in 1935, \u201cwithout mental effort, letting my actions be wholly controlled by my subconscious mind.\u201d He wasn\u2019t born with that ability. He developed it through lots of hard work.<\/p>\n\n\n\n

When computers enter the picture, the nature and the rigor of the work changes, as does the learning the work engenders. As software assumes moment-by-moment control of the craft, the pilot is relieved of much manual labor. This reallocation of responsibility can provide an important benefit. It can reduce the pilot\u2019s workload and allow him to concentrate on the cognitive aspects of flight. But there\u2019s a cost. Exercised much less frequently, the psychomotor skills get rusty, which can hamper the pilot on those rare but critical occasions when he\u2019s required to take back the controls. There\u2019s growing evidence that recent expansions in the scope of automation also put cognitive skills at risk. When more advanced computers begin to take over planning and analysis functions, such as setting and adjusting a flight plan, the pilot becomes less engaged not only physically but mentally. Because the precision and speed of pattern recognition appear to depend on regular practice, the pilot\u2019s mind may become less agile in interpreting and reacting to fast-changing situations. He may suffer what Ebbatson calls \u201cskill fade\u201d in his mental as well as his motor abilities. <\/p>\n\n\n\n

Pilots themselves are not blind to automation\u2019s toll. They\u2019ve always been wary about ceding responsibility to machinery. Airmen in World War I, justifiably proud of their skill in maneuvering their planes during dogfights, wanted nothing to do with the fancy Sperry autopilots that had recently been introduced. In 1959, the original Mercury astronauts famously rebelled against NASA\u2019s plan to remove manual flight controls from spacecraft. But aviators\u2019 concerns are more acute now. Even as they praise the enormous gains being made in flight technology, and acknowledge the safety and efficiency benefits, they worry about the erosion of their talents. As part of his research, Ebbatson surveyed commercial pilots, asking them whether \u201cthey felt their manual flying ability had been influenced by the experience of operating a highly automated aircraft.\u201d Fully 77 percent reported that \u201ctheir skills had deteriorated\u201d; just 7 percent felt their skills had improved. <\/p>\n\n\n\n

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nThe\nworries seem particularly pronounced among more experienced pilots, especially\nthose who began their careers before computers became entwined with so many\naspects of aviation. Rory Kay, a long-time United Airlines captain who until\nrecently served as the top safety official with the Air Line Pilots\nAssociation, fears the aviation industry is suffering from \u201cautomation\naddiction.\u201d In a 2011 interview, he put the problem in stark terms: \u201cWe\u2019re\nforgetting how to fly.\u201d\n\n\n\n<\/p>\n","protected":false},"excerpt":{"rendered":"

The grounding of Boeing’s popular new 737 Max 8 planes, after two recent crashes, has placed a new focus on flight automation. Here’s an excerpt from my 2014 book on automation and its human consequences, The Glass Cage, that seems relevant to the discussion. The lives of aviation’s pioneers were exciting but short. Lawrence Sperry […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[1],"tags":[],"class_list":["post-8622","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/posts\/8622","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=8622"}],"version-history":[{"count":9,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/posts\/8622\/revisions"}],"predecessor-version":[{"id":8634,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/posts\/8622\/revisions\/8634"}],"wp:attachment":[{"href":"https:\/\/www.roughtype.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=8622"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=8622"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=8622"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}