SPA2030

A. We’re all fortune-tellers

Everything in speech pathology – from assessing a client’s needs, delivering therapy, deciding to start or close a practice or service, or hiring staff – involves forecasting.

The art and science of forecasting has been a hot topic in Australian speech pathology this year as many of us participate in the Speech Pathology 2030 – making futures happen project.

B. But can we beat a chimpanzee?

There’s a major conceptual problem with long-term forecasts:

• In a 20-year study, Wharton Professor Philip Tetlock famously demonstrated that the average expert is only slightly better at predicting the future than a dart-wielding chimpanzee – especially on complex questions looking more than 5 years into the future.
• Former banker, academic and best-selling author, Nassim Nicholas Taleb, tells us that history is determined (or at least significantly shaped) by dramatic “black swans” – events so far outside our experience, we can’t even imagine them until they happen.

So is a project to predict the future of speech pathology in Australia doomed to fail?

Not necessarily.

C. 11 key principles of “Superforecasting”

Recently, Tetlock and colleagues have demonstrated that some ordinary people consistently beat chance with their predictions by more than 60%. He calls these people “Superforecasters”.

So what can we learn from Superforecasters to improve our predictions about the future of our profession? Here are 11 principles – heavily paraphrased – from my Easter reading of Tetlock’s 2015 book about Superforecasting:

1. Adopt the right mindset

Superforecasters tend to be:

• cautious;
• humble;
• non-deterministic: they reject the idea that events are fated;
• open-minded;
• intelligent with a need for critical thinking;
• reflective;
• numerate;
• pragmatic;
• analytical;
• supportive of diverse views;
• used to working with many grades of “maybe”;
• thoughtful updaters – when facts change, they change their minds; and
• intuitive psychologists who check their own thinking to catch cognitive biases.

Sounds like several speech pathologists I know. Good news!

2. Make predictions SMART

Speech pathologists are trained to set SMART goals for clients: i.e. making our goals specific, measurable, attainable, relevant and time-bound. But, like most people, we’re not so good when it comes to making predictions about uncertain future events.

Learning to forecast requires practice. To improve, we need clear feedback that leaves no ambiguity about whether our predictions are correct.  There’s no point predicting that “technology will probably change speech pathology materially in some areas and will maybe leave others fundamentally unchanged” unless we define exactly what we mean by “probably”, “materially”, “in some areas”, “maybe” and “fundamentally” and specify exactly how and to what degree we think change will affect specific areas and leave others unchanged. Otherwise, it will be impossible for our speech pathology “descendants” to test whether our predictions were correct.

3. Focus on worthwhile questions 

Forecasting experts and critics like Taleb and Tetlock agree that there is no evidence that geopolitical or economic forecasters can predict anything ten years out beyond the excruciatingly obvious (e.g. “Speech pathology will be affected by technology changes” or “There will be conflicts”). The accuracy of expert predictions declines towards chance five years out. This has a few implications:

• If we try to plan for a future beyond 2021, we should plan for surprises. That means we should plan for adaptability and resilience – for example, we shouldn’t put all our eggs in one basket. We should plan for the effects of more stringent and less stringent regulation, for example. We should plan for multiple “bad things” to happen at once. As Eisenhower said “Plans are useless, but planning is indispensable”.
• As Taleb posits, we should, where possible, design systems that are “antifragile” – not only resistant to shocks but strengthened by them. This costs money, however. We have to prioritise.
• We shouldn’t waste time and money on easy “clock-like” questions where basic rules of thumb will get us close to the answer.
• We should also be wary of wasting time on predictions about issues that are so uncertain we’ll never be able to beat the dart-throwing chimp. For example, it’s much easier to forecast our future scope of practice in 2020 than 2050. (Could anyone have predicted accurately in 1980 how the profession in Australia would look today?)
• We should break complex problems into manageable sub-problems. This is part of making SMART predictions.

4. Be foxes, not hedgehogs

2500 years ago, Archilochus, the Greek warrior poet is said to have written that a “fox knows many things, but the hedgehog knows one big thing”. Foxes beat hedgehogs when it comes to forecasting. In fact, hedgehogs tend to do slightly worse than random guessing!

To be foxes, we need more than one idea – we need to seek out information and ideas from many sources. We then need to aggregate and synthesise information using intuition and critical thought.

5. Consider inside and outside views

No profession, era or situation is 100% unique. From an outside perspective, we share of lot of similarities with other professions. For example, when looking at the possible effects of technology or increased privatisation of the profession, we need to ask how often do these sorts of things happen in situations of this sort? We then need to think about how speech pathology is different from other examples and precedents.

6.Don’t over- or under-react to emerging evidence and trends

The best forecasters tend to be incremental updaters, often moving their predictions of the likelihood of an event happening in increments of 5% or less. But good forecasters also know how to make big jumps in estimates when compelling new evidence demands it.

7. Work together, encourage diversity, egalitarianism, and challenge groupthink

Teams of forecasters are consistently more accurate than individuals. But we need to avoid groupthink: we need diversity – a particular challenge in speech pathology – and to challenge each others’ ideas regardless of the status of the participants. We also need to treat others with mutual respect when critiquing ideas (Tetlock calls it “constructive confrontation”). The more diverse the teams, the greater the chance that some members will possess scraps of information or relevant experience that others don’t.

Managing teams well to anticipate future events is notoriously difficult. Famous baseball coach Tommy Lasorda once said: “Managing is like holding a dove in your hand. If you hold it too tightly you kill it, but if you hold it too loosely, you lose it.”

8. No plan survives reality – agree big goals but empower each professional to improvise when necessary to pursue them

For big strategic decisions about our future, we need a “mission command” mentality. Broad strategies can and should be agreed after incorporating input from stakeholders. But frequent and rapid tactical decisions can be shaped only on the spot by people on the ground according to estimates of local conditions.

Decision-making power needs to be pushed down so that those of us at the coal face can respond quickly to change. Leaders should set goals – but not tell front-line clinicians how to achieve them.

In other words, as a profession, we need to decide what we want to accomplish. But – and this is a very big but – we should not tell individual speech pathologists how to achieve those goals. This is consistent with the philosophies of innovative organisations like 3M (the inventor of Post-it-Notes) and Amazon. For example, Amazon’s CEO, Jeff Bezos mandates that all employees should “have backbone; disagree and commit….Leaders are obligated to respectfully challenge ideas when they disagree, even when doing so is uncomfortable and exhausting”.

9. Look for clashing forces at work in each problem

For every good policy argument, there is usually a counter-argument to be considered. The tricky thing is to identify, weigh and synthesise all the arguments in a nuanced way, rather than locking ourselves into a binary argument that one or another view is correct in all cases. Nuance matters in big decisions.

10. Plan but then act!

It is not enough to dither indefinitely, trying to avoid mistakes as the world changes around us. We have to make decisions and take action, even if we don’t have all the information to know how things will turn out.

11. Learn from mistakes and wins, but avoid hindsight bias

We should own our failures and conduct unflinching post-mortems, rather than trying to justify mistakes. We also need to study our successes – were they based on sound analysis or merely flukes?

D. Bottom line

Forecasting is difficult, even in the short-term. It’s impossible to lay down binding rules. Guidelines like those above are the best we can do in an unpredictable world. And, even with guidelines, we must be prepared to break them when the circumstances demand it.

Principal source: Tetlock, P. & Gardner, D. (2015). Superforecasting: The Art and Science of Prediction. Random House Books, London. See also: www.goodjudgment.com

Further reading: Taleb, N.N. (2012). Antifragile: Things that gain from disorder. Random House Books, New York, New York. An excellent, (but exhausting) read!

Image: https://bit.ly/1ROAxHB