Get comfortable with ambiguity, it’s the future of knowledge work

1. On Saturday October 15th 2016, a looming weather system in the Pacific braced the northwest for a major windstorm. The Seattle Times warned that The 'murderous' Columbus Day storm of 1962 was spawned by a similar typhoon. Schools cancelled dances. Stores ran short of batteries, candles and emergency supplies.

2. Come the day of the storm, the media suggested a forecast was now difficult. The system tracked about 60 miles off earlier predictions, with the result that Saturday afternoon was gusty, but with little drama. The next day’s headlines spoke of The storm that wasn’t.

3. Seattle’s much-admired local meteorologist, Cliff Mass, admitted that he and others did not effectively communicate the changing threat … and the uncertainty in the forecasts … my profession needs to work hard to find better approaches for communicating uncertainty in our forecasts. Methods that will be effective in this age of social media.

4. The weather forecast was, in some ways, correct. There was a possibility of a disastrous event. However, the inexactness of the prediction was lost in the echoes of social chatter. Uncertainty always lurked in the forecasts, but often as an afterthought to the talk of more dramatic outcomes, which was all that the public heard.

5. Another example. Nate Silver and his team at FiveThirtyEight led into the night of the 2016 US presidential election with the banner There’s A Wide Range Of Outcomes And Most Of Them Come Up Clinton. We know how that worked out. On morning after, Silver’s team defended their methodology under the rubric Why FiveThirtyEight Gave Trump A Better Chance Than Almost Anyone Else. That better chance? A 28% probability of Trump’s victory.

6. Catching our breath, the difficulty is simple to define. Analysts and forecasters must model and predict the likely outcomes of a scenario in motion, but they also need to communicate those outcomes to a public that is functionally illiterate in the language of probability. According to Berwood Yost of Franklin and Marshall College, publishers of a respected poll, The incentives now favor offering a single number that looks similar to other polls … Certainty is rewarded, it seems.

7. This should not be surprising. Research on our reactions to ambiguity show that we are emotionally and physically uncomfortable with uncertainty.

8. And on top of this discomfort, ambiguity is simply not persuasive. As J.M. Coetzee has put it, If you do not already think in probabilistic terms, predictions emerging out of the probabilistic world seem vacuous. Can one imagine the Sphinx foretelling that Oedipus will probably kill his father and marry his mother? Can one imagine Jesus saying that he will probably come again?

9. Business Intelligence has mostly sought to remove ambiguity and uncertainty, favoring clarity over complexity. The best known example is the attempt to deliver a single version of the truth, which implies, in its very phrasing, the existence of alternatives to be suppressed. Certainty, it seems, is again rewarded.

10. Even where uncertainty is unavoidable, Business Intelligence tools and practitioners prefer the solidity of a fixed value. It’s common to see dashboards which predict budgets and expenses to the very dollar for months ahead conjured up who knows how.

11. Of course, we don’t believe that we will ever behold a Gross Margin in January of exactly $1,501,360, but we lack the tools and even the vocabulary, to bring more realistic, more ambiguous, outcomes into action. At best, everyone involved understands this as a game of self-deception. Nevertheless, behind the comfortable illusion lurks the danger of the fallacy of false precision. We may place undue trust in these forecasts because they look accurate, forgetting our own part in the trick.

12. Today, we are only fooling ourselves. But as machine learning and artificial intelligence are innately probabilistic, our societies, not just our businesses, are rapidly reaching the point where such illiteracy will severely limit our work and success.

13. There are indeed black arts in machine learning, which those of us who work in the field should know. But mostly, we need to join Cliff Mass and his fellow meteorologists, in getting better at communicating uncertainty in our work.

Donald Farmer, Principal of TreeHive Strategy, will be presenting at the Data Warehousing & Business Intelligence Summit, on March 20 and 21 in Utrecht, Netherlands.