2017 Media Sentiment – Mining the U.S. Housing Market with Tech Tools

Jonathan ScottBig Data, Contruction, Forecast Accuracy, Natural Language Processing

Housing Tides 2017 Media Sentiment - Mining the U.S. Housing Market with Tech ToolsArtboard 1 copy 14

4 min read

Each month Housing Tides™ leverages IBM Watson® Natural Language Understanding tools to measure the sentiment of over 500 news articles covering the housing and homebuilding sector from over 90 major news sources to “check the pulse” of the industry.  We believe that by quantifying the positivity or negativity of news coverage, housing professionals can gain an understanding of the consensus view of the state of housing markets, while also better recognizing their own biases and those of the news they consume.  Perhaps most importantly, industry pros can save time while seeking out diverse opinions and perspectives.  It’s a careful consideration of the beliefs that differ from one’s own that gives one the best chance to minimize the overconfidence bias. Notably, overconfidence is the first bias that Nobel Prize-winning psychologist and founder of behavioral economics Daniel Kahneman would eliminate if given a magic wand, which speaks to its particularly damning effects on good decision-making.

A recent article published by the Harvard Business Review argues that self-awareness is an important contributor to making better, more confident decisions and can help executives serve as more effective leaders.  As an exercise in introspection, we’ve decided to perform sentiment analysis on the executive summaries that appear monthly in the Housing Tides Report.  By doing so we hope to improve our own self-awareness by better understanding how the content we produce each month matches – or doesn’t match — the larger conversation about U.S. housing markets.

In the chart below we’ve plotted the sentiment of each article captured by the Housing Tides team in 2017, the 30-day moving average sentiment of those articles, and the sentiment of the twelve executive summaries that we published throughout the year.

Housing Tides_Industry media and executive summary sentiment chart for 2017

At a quick glance, one can see that housing news sentiment represents a wide swath of opinions, from the extremely negative to the very positive.  By looking at the moving average it’s apparent that, while there were some ups and downs throughout the year, news coverage of housing writ large was slightly negative with a modest movement towards the neutral value of zero as the year progressed.  Specifically, these 6,000 articles had an average sentiment of -0.06.  (Contrast this with an average sentiment of -0.34 across all news articles we’ve gathered from 2007.)

The Housing Tides executive summaries as a whole were slightly more negative with an average sentiment of -0.16.  This isn’t entirely surprising, given how often we discussed the negative impacts of tight housing supply and high home prices throughout the year.  The most negative summary appeared in April and the most positive summary was published in the August report.

The histogram below shows the portion of content that falls into different sentiment bins, showing these insights in a different way – we can see that the distribution of housing news sentiment is very similar to a normal distribution centered just below a score of zero, while the Housing Tides executive summaries fell within a more narrow range but were more negative overall.

Housing Tides 2017 Distribution of Sentiment Results

Perhaps the most obvious conclusion to draw is that there is no meaningful divergence between the consensus sentiment appearing in housing news media and the Housing Tides executive summaries, with the executive summary sentiment generally fluctuating in a similar range as the average news sentiment in 2017.   

This endeavor in introspection shows that the executive summary content we produce each month is largely consistent with major media outlets’ portrayal of the industry.  Challenges remain, but we’d argue that U.S. housing markets are generally in good shape with a robust labor market and favorable demographics supporting housing demand as the millennial generation reaches a prime home-buying age in coming years.  Sometimes a little context and self-awareness are all you need in order to move the needle from “overconfidence” to just “confidence.”

Share this Post

About the Author

Jonathan Scott

Jonathan is the Head Analyst for Housing Tides. A key team member from the beginning, Jonathan has been instrumental in shaping the concept of and developing the methodology for Housing Tides. Jonathan is fascinated by the intersection of human behavior and economic outcomes and hopes that Housing Tides will help increase objective decision making by reducing cognitive bias. Jonathan studied economics at CSU.