Why economic forecasting is a flawed science

There is an inevitable lack of precision in trying to come up with an economic forecast

Economists have a dismal record in forecasting recessions and other major events. Why do they so often get it wrong and how do we know when to believe them?

Economists must look on enviously at weather forecasters.

These days, meteorologists only draw attention to themselves when they get things wrong. For economists, it’s noteworthy when they get a prediction right.

It may sound harsh, but it is an assessment backed up by evidence.

Two years ago, a pair of economists from the International Monetary Fund looked at the ability of official and private forecasters to predict recessions. Of 62 recessions that occurred between 2008 and 2009, none were predicted. Even over a longer period of time, from 2008 to 2012 when 88 recessions occurred, just 11 were anticipated by late the preceding year.

As one of the IMF researchers, Prakash Loungani, put it: “The record of failure to predict recessions is virtually unblemished.”

Economists have dismal forecasting records. Why do they so often get it wrong and how do we know when to believe them?

This is not just an academic exercise. Getting forecasts wrong can have serious consequences. Just ask the Europeans.

A cautionary tale

When the European Central Bank (ECB) decided to hike interest rates twice in 2011, it did so expecting the Eurozone economy would pick up and energy prices would surge, forcing up inflation.

As it turned out, the ECB was wrong on all three counts and by year’s end it had been forced to backtrack on both increases. But by then, the damage had been done. In 2012, Europe plunged into recession – its second in three years, which it’s only gradually emerging from.

However, even though the forecasting record of economists is generally abysmal, demand for their predictions is insatiable.

Reserve Bank of Australia governor Glenn Stevens says people are “irresistibly drawn to those who claim to be able to forecast the future, beat the market and give us the illusion of certainty and control.”

One of those under pressure to feed this illusion is National Australia Bank chief economist Alan Oster, who laments that all too often the limitations of economic forecasts are overlooked in the desire for certainty.

Mostly, they are the product of accumulated knowledge and ideas about how the economy works, combined with regular statistical readings on activity and judgements about the influence of major local and international developments, such as the outbreak of war and changes of government.

It is, as Oster readily admits, an inexact science. “My philosophy is that economics is applied psychology with a bit of statistics around it,” he says.

Let’s get sentimental

According to Oster and Commonwealth Bank chief economist Michael Blythe, one of the most difficult aspects of economic forecasting is accounting for the role of human sentiment.
Shifts in mood can have a big influence on the economy. Confident consumers are more likely to borrow and spend; wary businesses may hold off on hiring and investment.

Therefore, Oster and Blythe both keep a close eye on measures of business and consumer sentiment.

But the very nature of sentiment makes things difficult to predict. Mood can be influenced by a myriad of factors that have only a tangential relationship to developments in the economy, such as the plunge on global stock markets at the start of 2016.

It can also be self-reinforcing. If enough people act like there is a recession, there will be one.

Feel the quality

Forecasters also have to grapple with the fact that what is going on in the economy is imperfectly known and understood.

In Australia, economists rely heavily on data provided by the Australian Bureau of Statistics (ABS) to inform them of what is happening, from employment and output to inflation, investment, construction and spending.

The information is only as good as the methods used to collect it and Oster believes there is too often a lack of awareness of the limitations of ABS data.

For example, the ABS labour force survey, used to provide a monthly guide to employment growth and joblessness, draws on data from 26,000 households, which comprise just 0.32 per cent of the civil population of Australia aged 15 years or older.

This relatively narrow statistical base can lead to some volatile outcomes as the sample changes. In August 2014, the ABS reported the economy added 121,000 jobs in just one month, forcing Australia’s unemployment rate down from 6.4 per cent to 6.1 per cent. This result is ridiculed by Oster, who describes it as “bulls**t”.

“There is no way the economy grows by more than 15,000 to 25,000 jobs a month,” he says.

The known unknowns

To arrive at their forecasts, economists typically feed the latest data into models of the economy that seek to emulate the interaction between dozens or hundreds of variables in the “real” economy.

But Blythe points out the quality of results a model produces are only as good as the ideas and assumptions that underpin it. The best it can do is give the forecaster a base from which to work.

Additionally, other considerations, not just economic, come into play.

Oster attributes the IMF so often failing to predict recessions to the political reality of the environment in which it operates. Member governments do not want the IMF to be forecasting a recession on their watch.

Coping with uncertainty

While economists often make very specific forecasts – such as predicting the unemployment rate will drop from 5.8 per cent to 5.7 per cent in February 2016 or building approvals will jump by 18,000 dwellings in March 2016 – these are really an artifice.

Stevens says there is an inevitable lack of precision in trying to come up with an economic forecast and they are subject to significant margins for error.

For example, experience shows the chances of GDP growth actually being within half a percentage point of a forecast made a year earlier is around one in five. Yet, even with these limitations, economic forecasting is more than just a hit-and-hope exercise.

Stevens says an improved understanding of how economies typically behave, along with knowledge of the long-run forces that drive growth (productivity and population) and a good sense of the big forces at play (such as a terms-of-trade shock or global oversupply of oil) means there is plenty economists can say about the future, although "we have to recognise the limits on our capacity to predict … with any precision."

The way the RBA thinks about forecasts and uses them is instructive. Because the central bank has to anticipate developments in the economy in setting interest rates, the use of economic forecasts is unavoidable. But it is not slavish about them, either.

"We have to form a view about the big forces at work, but also operate with due recognition of the limitations of numerical forecasts," says Stevens. "The extent to which policy should respond to forecasts will always have some element of judgement."

Economic forecasts can be a guide, but they should not be viewed as gospel.

Read next: 6 economists who predicted the global financial crisis – and why we should listen to them

Like what you're reading? Enter your email to receive the INTHEBLACK e-newsletter.
December 2021
December 2021

Read the December 2021 issue of INTHEBLACK magazine.

Each month we select the must-reads from the current issue of INTHEBLACK. Read more now.