Without technology, we would not have many of the things that make life easier today such as microwaves, computers and mobile phones; nor would we have life-saving advancements in the field of healthcare and medical research. People would still be succumbing to preventable diseases at an alarming rate. In today’s society, technology is ubiquitous. Our children are dubbed “digital natives” because they never experienced a world that didn’t rely on computing power to function.
With this rise in the prevalence of technology, we increasingly implement automation – machines running (hopefully) meticulously written code that take over certain tasks or processes for humans because we are fundamentally unpredictable. This is a great call in many situations. It even saves real lives. Consider the consequence if nurses weren’t alerted at hospitals when their patients were in distress! It’s not a pretty thing to ponder.
When not saving lives, automation is often making a process more efficient. (I’d be remiss, though, if I didn’t point out that employers don’t have to pay a salary or benefits to a machine – yet, but that’s another blog post!) Consider the DevOps movement and the problems it tackles – improving the ability to deliver quickly with a particular focus on the time between code commit to software/services being in the hands of customer. The use of specialized tools like build and continuous integration systems is crucial in this space. It is a key tool that helps us keep pace with changing business and customer needs. These things that are now done by automation often used to be done manually by skilled workers, a.k.a. specialists.
On one hand, I want to celebrate! Removing unnecessary 1 bottlenecks around specialists makes achieving agility much easier. But, technology and automation have become such an invaluable part of how we live and work that we wouldn’t know what to do without it. That’s not just a quick quip, we actually could have no idea what to do when our technology or automated processes fail. Therein lies the automation paradox. We don’t want to live without it, but by using it we begin to rely on it. When we rely on it, we often fail to maintain the skills needed to handle unforseen emergencies when they crop up.
In his book “Messy: The Power of Disorder to Transform Our Lives,” Tim Harford shares the story of Air France Flight 447 2 that fell prey to this paradox on May 31, 2009. The aircraft was an Airbus 330, which has an especially sophisticated automation system called “fly-by-wire.” This system observes the actions of the pilot, determines the pilot’s intent and then carries out the intended action flawlessly. It makes for a smooth flight and keeps pilots from doing unsafe things like climbing too steeply – a common cause of stalls. Harford writes that “there had been no crashes in commercial service in the first fifteen years after it was introduced in 1994.” It is safe to say that this automation is beneficial.
But, on this fateful day, a system component iced over, causing the system to operate in a degraded mode. When operating in this degraded mode, the pilot has to take responsibility for more of the plane’s operation. This responsibility was handed to the pilot just as they began to deal with thunderstorms. There were three pilots on the plane, but the one with the least experienced was at the controls. The other pilot in the cockpit was a manager, meaning he had lots of flight hours but not recent ones. The captain was in the back napping to recover from a late night out. The young pilot began to experience problems and acted on his gut instinct to climb to a higher altitude. Normally the fly-by-wire system would prevent him from climbing too quickly and stalling out, but today it couldn’t and the aircraft stalled. The pilots in the cockpit were not skillful enough discern the problem and manage it in time. The captain, upon returning to the cockpit, eventually figured out the issue and began corrective procedures. But, they were too late. Flight 447 crashed into the Atlantic Ocean at 125 miles an hour killing everyone on board – 228 souls.
Though there are many contributing factors to a horrific incident like this one, many key factors can be traced back to a blind reliance on this automated system.
“One machine can do the work of fifty ordinary men. No machine can do the work of one extraordinary man.”
Elbert Hubbard, author of A Message to Garcia
If the automated systems hadn’t been so effective for so long would the captain have felt comfortable going to work having had only one hour of sleep? Would the company provide the pilots situational training to handle unexpected emergencies? I don’t know. But, what we do know is that the more frequently we rely on something to do a particular thing for us, the fewer opportunities we create to maintain and develop that skill.
The moral of this story isn’t to abandon automation because it might be risky. This isn’t a binary decision. Rather we need to consider how we stay prepared as we continue to use more and more automation. Programs like Netflix’s Chaos Monkey are designed to keep skills sharp despite a growing reliance on tools. Other organizations have drills to simulate similar issues in a more controlled environment. No matter how we do it, it seems clear that understanding the need for obliterating proverbial “black boxes” and anticipating how we’d react if our automation failed is a mission that we should wholeheartedly embrace.
I’d love to hear your thoughts and hear stories about successes and failures related to this topic. Share your stories in the comments below.
- I don’t think all specialists are unnecessary, but we definitely over-rely on them as a society.
- I highly recommend the book “Messy: The Power of Disorder to Transform our Lives” by Time Harford. Copyright 2016. This particular story comes from Chapter 7: Automation
0 Comments
1 Pingback