We proudly present another edition of Monday Musings, the occasional feature in which I wax eloquent about the esoteric aspects of business technology.
It’s classic science fiction: Man creates machine. Machine serves man flawlessly. Man turns over control of everything to machine. Machine succumbs to unanticipated flaw and runs amok with humorous or tragic consequences culminating in a lesson about the arrogance of man.
We still haven’t learned our lesson.
Recently, there wan an amusing incident on Amazon.com, where two competing booksellers drove up the price of a paperback book to over twenty-three million dollars! Both sellers had set up automatic pricing algorithms that based their selling price on the other’s current selling price. The result was a reverse price-war between two autonomous computer systems.
The problem could have easily been avoided with some good, old-fashioned reason, but that’s something computers don’t have. No matter how “smart” a machine, it doesn’t have that all-important fail-safe built into every human mind. Even IBM’s Watson (of Jeopardy fame), the “world’s smartest computer” came out with some laughable remarks because it couldn’t evaluate its answers using the filter of reason.
As businesses increase their dependence on computers, this becomes a problem in the marketplace. Look at the 2010 “Flash Crash” for example, in which the Dow dropped around 900 points in just a few minutes. It was precipitated by a computer algorithm that wasn’t designed to handle extreme cases, but which went on dutifully obeying its programming even in the face of a nonsensical outcome. Fortunately, the market recovered almost as quickly as it dropped, but what if it hadn’t?
This type of error occurs every day in business, as people rely on formulas and algorithms to feed them the right answers without checking to see if those answers make sense. Employees, especially the younger generations, are suppressing their ability to reason, blindly accepting computer-generated results as fact. This is something older (and not so much older) executives may not even realize is happening.
The current generation entering the workforce has grown up trusting technology. When they use a calculator, they wouldn’t even consider having to check by hand to make sure the answer is right. They’ve never seen a calculator fail to give the right answer. Similarly, computers (especially computer games) create a world where decisions are limited to a few pre-determined choices, all of which are “correct” because the computer says so. Many interact with the real world, expecting the same sort of limited parameters.
I’m not saying that all younger employees are the same, or that their older counterparts don’t fall victim to the same mentality (I know I certainly do on occasion!), but there is a definite cultural shift that is a direct result of the computer age. It’s something to be aware of.
Computers are very precise, but they can only produce accurate results within specific ranges of known parameters. When values fall outside those ranges, or when the number or nature of the parameters changes, algorithms will fail, sometimes with dramatic results, but sometimes subtly, which can be even more dangerous.