Machine Learning Algos vs. Wall Street Stock Analysts

Predictions hold a pivotal role within the realm of economics. Consider, for instance, the determination of consumption and savings levels, the formulation of pricing strategies, and the critical decisions surrounding workforce management, such as hiring and layoffs. Each of these economic decisions is shaped significantly by the predictions made by economic players regarding the future.

In the past, people used to rely on forecasts based on human experience and expertise. Today, financial institutions and industry professionals are turning to big data and machine learning algorithms to improve their forecasting accuracy and sophistication.

There has been a significant focus on how machine learning algorithms can improve forecast accuracy compared to human forecasts.

However, do machine learning models' forecast errors exhibit similar biases to those seen in human forecasts?

Behavioral Machine Learning

In my research project titled “Behavioral Machine Learning Computer Predictions of Corporate Earnings also Overreact,” in collaboration with two researchers from the University of Minnesota, we discovered that machine learning models outperformed both linear regressions and human stock analysts.

Just like human forecasts, the predictions generated by the machine learning algorithm also tend to overreact to news, showing excessive optimism during favorable times and pessimism during unfavorable times.

In our empirical analysis, we focused on a particular population model that has proven to be highly suitable for financial data—the gradient boosting regression tree model (GBRT).

Our findings revealed that companies that invest more today (those that receive good news), on average, tend to experience more negative earnings surprises in the future. This phenomenon aligns with what we can describe as “overreacting predictions.”

Consistent Overreaction Prediction

We're also exploring other aspects of the overreaction phenomenon we've identified. We're delving into how the model's configurations influence this overreaction. Understanding how different model settings impact the overreaction will provide valuable insights. Our aim is to pinpoint the specific model features that contribute to this overreaction.

Specifically, we're honing in on the aspect of overfitting regularization within the model. To do this, we're adjusting a key model parameter known as the learning rate, which determines how much the model reduces the impact of errors in each iteration. A smaller learning rate results in less model overfitting. When we apply the GBRT model with reduced overfitting, we observe a decrease in the extent of the overreaction.

Similar Pattern in the Real Business World

We've shown that corporate earnings predictions exhibit an overreaction tendency in a simulated setting. The next and more important question to answer is whether we observe a similar pattern in the real world.

To answer this, we utilize manually collected data on stock analysts' training, distinguishing between those trained in machine learning methods and those with a more traditional background.

Analysts with a machine learning background tend to display less overreaction to news, even after adjusting for the firms they cover.

It's important to emphasize that, although the use of machine learning methods offers certain advantages, it doesn't strictly outperform human analysts.

In an article this month in The Financial Times’ Artificial Intelligence coverage, “Robot Stock Analysts: They’re Just Like Us!,” the columnist posed this question based on our study: Are machines better stock analysts than human beings?

One of my co-authors on our study, Murray Frank, was quoted about our findings: “What was a bit surprising to us is that the nature of the bias of the algo is surprisingly similar to the bias in human beings. The behavioral finance guys say the biases humans demonstrate are about the deep psychology of the human brain. But algos, despite the metaphors we use about them, are not human brains.”

We found that human stock analysts’ predictions incorporate information beyond what’s available in standard databases of macroeconomic variables and firm accounting attributes.

Robot stock analysts still can’t beat humans—yet.