The Myth and Magic of AI-supported Business Decisions

As is the case in all areas of technology, artificial intelligence (AI) now appears constantly in every aspect of decision-making support: embedded in business intelligence (BI) and analytics tools, in information preparation and contextualisation, and as predictive solutions built on a variety of machine learning techniques. The promises are of easier decision making and more certainty in decision outcomes. These promises are, at least in part, mythical.

We are a species invested in predicting the future—as if our lives depended on it. Successful predictions of lurking lions were indeed a matter of survival. Civilization has made us physically safer, but prediction remains a mainstay of culture, from Tarot readings that promise love to genetic tests that warn of future cancers.

The challenge, said Nobel physicist, Niels Bohr, is that “prediction is very difficult, especially if it’s about the future.” For example, we are predisposed to assuming that the future is a largely linear extrapolation of the most familiar past. This is one—or a combination of several—of the near 200 cognitive biases that allegedly afflict us. That we missed or misinterpreted the beginnings of multiple exponential coronavirus pandemic waves is testament to our human shortcomings in prediction.

Sadly, the most common solution today to such shortcomings is to call for more big data and AI.  This is an example of a systemic problem in modern thinking, in both business and IT, that I previously described: “technochauvinism.” Such exclusive focus on technological solutions often blinds us to the non-technical—and often better or only—solutions possible for everyday problems, and leads us to create new problems with inappropriate technology. Such magical thinking is far too common.

I dare to predict that in the decade ahead, one area of technology that is going to lead us deeper into the quagmire of non-functioning and problem-creating technical solutions is an overly wide adoption of artificial intelligence in predictive analytics and our overconfidence in its power in supporting decision makers.

AI is Inherently Conservative
Science fiction author and journalist, Cory Doctorow, offered a “provocation” on New Year’s day 2020, “Our Neophobic, Conservative AI Overlords Want Everything to Stay the Same.” What he said was not new, although his attribution of intention is worrisome.

He gives a succinct and superb summary of how technochauvinism operates in AI. “Machine learning,” he asserts, “is about finding things that are similar to things the machine learning system can already model.” These models are, of course, built from past data with all its errors, gaps, and biases. The premise that AI makes better (e.g. less biased) predictions than humans is already discounted in many cases. The widespread belief that AI can predict novel aspects of the future is magical thinking. Machine learning is fundamentally conservative, based as it is on correlations in existing data; its predictions are essentially extensions of the past. Despite the fact that the pandemic and its effects have thoroughly broken many long-standing commercial trends and behaviours, we continue to see AI being wildly promoted as a potential saviour.

AI is Potentially Controlling
The step beyond predictive analytics (what will happen?) is prescriptive—make it happen and, in particular, efforts to influence and direct human behaviour to a specific outcome, usually to buy something. Here, we enter some murky worlds where the magical thinking is most definitely dark.

Rosalind Picard’s affective computing has AI detecting and classifying human emotions based on facial expressions and intervening to effect behavioural changes. Alex Pentland’s social physics, envisages a utopia where organisations and society are nudged toward full rationality through manipulating relationships.
And an overarching and worrying direction toward manipulating consumer behaviour is named surveillance capitalism in the eponymous and deeply disturbing book by Shoshana Zuboff. Her reworking of a common saying is profoundly shocking: “Forget the cliché that if it’s free, ‘You are the product.’ You are not the product; you are the abandoned carcass. The “product” derives from the surplus [behavioural data] that is ripped from your life.”

AI Dilemmas
Data warehousing and business intelligence first emerged to support decision makers with historical data. Accused of looking too much in the rear-view mirror, we moved to real-time operational data and more recently to embrace predictive analytics. Prescriptive techniques are now gaining in popularity.

Advances in data availability and artificial intelligence are enabling this journey in many cases to good effect. However, implementers must beware of the myths and magical thinking that AI hype encourages. AI technology is not a silver bullet for every problem. In particular, understanding future human intentions and encouraging actions remains a strictly human gift. The myth that AI can solve all the mysteries of human behaviour is the most dangerous magical thinking in which technochauvinists may indulge.

Barry Devlin gives a presentation ‘Would you let AI do your BI?’ during the Data Warehousing & Business Intelligence Summit on July 1st.
He also gives a half day virtual workshop, ‘How to Revamp your BI and Analytics for AI-based Solutions’ on July 2nd.