Flourishing Forecasters – Who makes the most accurate forecasts?

Jonathan ScottBig Data, Forecast Accuracy, Natural Language Processing


5 min read

There are a lot of housing market forecasts. Finding out who is making accurate forecasts doesn’t have to be hard.


If you want to get ahead as a real estate investor or housing industry professional, it’s important to follow major news sources to track latest developments in the housing sector and general economic climate.  These media often feature opinions and predictions about the future course of events, ranging from broad and vague (e.g. “It’s possible that housing starts will fall in the next year”) to specific and measurable (e.g. “We believe seasonally-adjusted housing starts will total 1.21 million in January”).  It’s difficult for any individual to remember the slew of forecasts that are published every week.  Moreover, without a dedicated system, it’s impossible to recall these forecasts, assess who makes accurate forecasts, and use that information to guide understanding of new predictions one encounters. 

How can you tell the well-reasoned forecasts provided by entities with a strong track record of success from those that are not?

How do you know who to trust? Who has a track recored of accurate forecasts?

The Housing Tides team was struggling with these questions when we decided to do something about it.  Enter the Housing Tides Forecaster Report Card.  Leveraging both our system for collecting housing news and our partnership with IBM Watson, we record housing industry forecasts as they’re made, then circle back and grade them when data allow.  The result is a concise report of forecasters, the type of forecasts they make, the median prediction horizon (in other words, how far out their predictions look in time), and the accuracy of those forecasts.

The most common type of forecast we encounter is a “point” forecast, where the forecaster provides a specific value they expect at a certain time (“We believe seasonally-adjusted housing starts will total 1.21 million in January”).  In the table below we’ve listed all forecasters for whom we have more than five graded point forecasts, ranked by the Mean Absolute Percent Error (MAPE) of those predictions.  A lower MAPE means more accurate forecasts and is thus better when comparing forecasters.

Accurate Forecasts - Overview of forecasters with more than 5 point forecasts in the Forecaster Report Card.

Tom Lawler

Accurate Forecasts - Tom Lawler's Forecast Report Card Grade

Tom Lawler, an independent housing economist often quoted in the Calculated Risk blog, scored highest by this measure with a MAPE of 2.9%.  To be sure, Mr. Lawler has a relatively short median forecast horizon of just four days.  He is usually offering predictions on housing starts just before the data is published.  A handful of his recently graded forecasts are listed below.

Accurate Forecasts - Tom Lawler's Forecasts

Wall Street Journal Survey of Economists

Accurate Forecasts - WSJ's Forecast Report Card Grade

Like Mr. Lawler above, the Wall Street Journal survey of economists usually takes place just before data is released and aims to establish a consensus expectation of home starts, permits or sales just before the actual level is known.  As such, it seems appropriate to compare Mr. Lawler’s 2.9% MAPE to the WSJ survey’s MAPE of 6.0%.

Accurate Forecasts - WSJ's Forecasts

National Association of Realtors

Accurate Forecasts - NAR's Forecast Report Card Grade

The National Association of Realtors is a prominent voice in the housing industry, offering guidance and predictions for housing markets.  The NAR has a longer-term forecast horizon than the first two forecasters we highlighted and therefore we should expect more uncertainty in their forecasts; with this context the NAR has demonstrated a reputable MAPE of 10.6%.

Accurate Forecasts - NAR's Forecasts

Reuters Poll

Accurate Forecasts - Reuters' Forecast Report Card Grade

Like the Wall Street Journal, Reuters also polls economists about their expectations for the housing industry.  However, the Reuters survey tends to ask respondents to make estimates further into the future, which we can see by the median forecast horizon of 397 days.  With a longer typical forecast horizon, it’s reasonable that the Reuters poll would perform worse by the MAPE measure, though at 10.7% the survey has a respectable track record.

Accurate Forecasts - Reuters' Forecasts

Check for a Track Record of Accurate Forecasts with Ease

In summary, there are hundreds of forecasts made about the housing market each month.  It’s challenging to find the time to read and process them all, let alone research other opinions and the forecaster’s track record.  In the midst of information and opinion overload, it can be tempting to just “go with your gut” and trust what is familiar or lines up with what you already believe.  But not all forecasts are created equally and the cost of making investment decisions based upon weak forecasts can be huge.  The next time you are trying to decide if you can trust a prediction that the housing market is going to do X or Y, let Housing Tides provide context on the forecaster’s historical accuracy and perspective.

 

Share this Post

About the Author
mm

Jonathan Scott

Jonathan is the Head Analyst for Housing Tides. A key team member from the beginning, Jonathan has been instrumental in shaping the concept of and developing the methodology for Housing Tides. Jonathan is fascinated by the intersection of human behavior and economic outcomes and hopes that Housing Tides will help increase objective decision making by reducing cognitive bias. Jonathan studied economics at CSU.