{"id":5855,"date":"2015-04-01T11:08:42","date_gmt":"2015-04-01T17:08:42","guid":{"rendered":"https:\/\/www.roughtype.com\/?p=5855"},"modified":"2015-04-01T16:38:14","modified_gmt":"2015-04-01T22:38:14","slug":"the-robot-pharmacist","status":"publish","type":"post","link":"https:\/\/www.roughtype.com\/?p=5855","title":{"rendered":"The robot pharmacist"},"content":{"rendered":"

\"robot<\/a><\/p>\n

If you want to understand the complexities and pitfalls of automating medicine (and professional work in general), please read Bob Wachter’s story, adapted from his new book The Digital Doctor<\/em>, of how Pablo Garcia, a 16-year-old patient at the University of California’s San Francisco Medical Center, came to be given a dose of 38 \u00bd\u00a0antibiotic pills rather than the single pill he should have been given. (Part 1<\/a>, part 2<\/a>, part 3<\/a>; part 4 will appear tomorrow.)\u00a0Pretty much every problem with computer automation that I write about in The Glass Cage<\/em> \u2014 automation complacency, automation bias, alert fatigue, overcomplexity, distraction, miscommunication, workload spikes, etc. \u2014 is on display in\u00a0the chain of events that Wachter, himself a physician, describes.<\/p>\n

It’s a complicated story, with many players and many moving parts, but I’ll just highlight one crucial episode. After the\u00a0erroneous drug order\u00a0enters the hospital’s computerized prescription system, the result of (among other things) a poorly designed software template, the order\u00a0is transmitted to the hospital’s pill-packaging robot. Whereas a pharmacist or a pharmacy technician would almost certainly have noticed that something was amiss with the order, the robot dutifully packages up the\u00a038 \u00bd pills as a single dose without a second’s hesitation:<\/p>\n

The robot, installed in 2010 at a cost of $7 million, is programmed to pull medications off stocked shelves; to insert the pills into shrink-wrapped, bar-coded packages; to bind these packages together with little plastic rings; and then to send them by van to locked cabinets on the patient floors. \u201cIt gives us the first important step in eliminating the potential for human error,\u201d said UCSF Medical Center CEO Mark Laret when the robot was introduced.<\/p>\n

Like most robots, UCSF\u2019s can work around the clock, never needing a break and never succumbing to a distraction.<\/p>\n

In the blink of an eye, the order for Pablo Garcia\u2019s Septra tablets zipped from the hospital\u2019s computer to the robot, which dutifully collected the 38 \u00bd Septra tablets, placed them on a half-dozen rings, and sent them to Pablo\u2019s floor, where they came to rest in a small bin waiting for the nurse to administer them at the appointed time. \u201cIf the order goes to the robot, the techs just sort it by location and put it in a bin, and that\u2019s it,\u201d [hospital pharmacist] Chan told me. \u201cThey eliminated the step of the pharmacist checking on the robot, because the idea is you\u2019re paying so much money because it\u2019s so accurate.\u201d<\/p>\n<\/blockquote>\n

Far from eliminating human error, the replacement of an experienced professional with a robot ensured that a\u00a0major\u00a0error went unnoticed. Indeed, by giving the mistaken dose the imprimatur of a computer, in the form of an official, sealed, bar-coded package, the robot pretty much\u00a0guaranteed that the dispensing nurse, falling victim to automation bias<\/a>, would reject her own doubts and give the child all the pills.<\/p>\n

The problems with handwritten prescriptions \u2014 it’s all too easy to misinterpret doctors’ scribbles, sometimes to fatal effect \u2014 are legendary. But solving that very real problem with layers of computers, software templates, and robots introduces a whole new set of problems, most of which\u00a0are never foreseen by the system’s designers. As is often the case in automating complex processes, the computers and their human partners end up working at cross-purposes, each operating under a different set of assumptions. Wachter explains:<\/p>\n

As Pablo Garcia\u2019s case illustrates, many of the new holes in the Swiss cheese weren\u2019t caused by the computer doing something wrong, per se. They were caused by the complex, and under-appreciated, challenges that can arise when real humans\u200a\u2014\u200abusy, stressed humans with all of our cognitive biases\u200a\u2014\u200acome up against new technologies that alter the work in subtle ways that can create new hazards.<\/p><\/blockquote>\n

The lesson isn’t that computers and robots don’t have an important role to play in medicine. The lesson is\u00a0that automated\u00a0systems are also human systems. They work best when designed with a painstaking attentiveness to the skills and foibles of human beings. When people, particularly skilled, experienced professionals, are pushed to the sidelines, in the blind pursuit of efficiency, bad things happen.<\/p>\n

Pablo Garcia survived the overdose, though not without a struggle.<\/p>\n","protected":false},"excerpt":{"rendered":"

If you want to understand the complexities and pitfalls of automating medicine (and professional work in general), please read Bob Wachter’s story, adapted from his new book The Digital Doctor, of how Pablo Garcia, a 16-year-old patient at the University of California’s San Francisco Medical Center, came to be given a dose of 38 \u00bd\u00a0antibiotic […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[1],"tags":[],"class_list":["post-5855","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/posts\/5855","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=5855"}],"version-history":[{"count":10,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/posts\/5855\/revisions"}],"predecessor-version":[{"id":5873,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/posts\/5855\/revisions\/5873"}],"wp:attachment":[{"href":"https:\/\/www.roughtype.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=5855"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=5855"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=5855"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}