Journal of Stock & Forex Trading
Open Access

Evaluating the Performance of Benzinga-Provided Stock Ratings as Trading Indicators

Ashok Arun^*, Jovan M. Tadic, Robert Assaly and Kalea Gore ^*Correspondence: Ashok Arun, Climate and Ecosystem Sciences Division, Lawrence Berkeley National Laboratory, Cyclotron Rd, Berkeley, CA 94720, United States, Email:

Author info »

Introduction

At the moment, 55% of Americans invest in the stock market in various ways. There are many resources available to these investors to research and evaluate their next investment. Numerous private analytical companies offer paid-for market signals based on technical analysis, fundamental analysis, and/or sentiment.

We seek to review applicable literature and evaluate the efficacy of using analyst recommendations as a trading signal for individual investors. In a recent paper the authors examined the performance of the Motley Fool’s Stock Advisor premium service [1]. It concluded that trading based on the Stock Advisor service outperforms both the general market and several groups of companies used as a benchmark during initial and extended periods of time. Another research study evaluated the performance of Jim Cramer’s, the host of CNBC’s Mad Money, security buy and sell recommendations [2]. The authors concluded that Cramer’s buy recommendations resulted in short term profit but had negative alpha over time. The recommendations by the Wall Street Journal were also found to have a statistically significant effect on stock prices [3]. Focused on whether investors could profit from security analysts’ recommendations, using data from 1985 to 1996 researchers noted that recommended securities earned an 18.8% return compared to a 5.78% average market appreciation for securities which were not recommended [4]. However, other papers show evidence that analysts can also systematically underperform. A paper which evaluated sell-side analysts’ stock recommendations during the years 2000 and 2001, found that favored stocks were outperformed by the least favored stocks [5]. The securities recommended underperformed the market by 7.06%. When looking at data between the years of 1996 and 2003, the paper noted that buy recommendations from independent research firms outperformed those from investment banks [6]. It has also been stated that financial analysts’ downward stock recommendation revisions provide superior information to investors than do upward stock recommendation revisions [7].

Another important aspect to focus on is the bias and influence of the individual analyst and not just the company on which they are reporting. One paper found that analysts can credibly convey unfavorable information but can never credibly convey favorable information [8]. Similarly, another paper, found that analysts greatly prefer to move from a buy recommendation to a hold recommendation rather than immediately moving to a sell recommendation [9]. It is also found that the market typically heeds downgrade decisions made by analysts with a higher percentage of buy ratings.

Many different investment recommendation websites and analysts provide a multitude of different indicators for each recommendation. However, these indicators are often hard to understand, and it is even harder to know which indicators to act upon. Researchers found that a singular change in price target (increasing or decreasing), or a singular change in rating (e.g., neutral → buy), is not strong enough to indicate a buy from an investor [10]. Contradictory signals neutralize each other, and confirmatory signals reinforce each other. The researchers also showed that large decreases in price target can be explained by the stock price increasing. However, large price target increases are not based on the stock price increasing. One of the largest stock market relevant resources is Benzinga. Benzinga.com was founded in 2010, and has around 25 million readers per month, as officially published by the company. It provides stock ratings/ rating revisions, multiple charting tools, earnings information, news stories and other information relevant to the market.

In this paper we will focus on evaluating Benzinga Ratings provided through the Benzinga API. The Benzinga Ratings feature provides research analyst stock ratings from established firms, e.g., Morgan Stanley or Goldman Sachs.

Hypothesis

In this paper we measure the success of Benzinga-provided analyst ratings in conjunction with several take-profit/stop-loss pairs of values.

We hypothesize those analysts provided ratings and price targets may contain useful information about future price movements. This effect is well-known and confirmed by our own monitoring of the stock price movements in the period 2020-2021, following recommendations issued by two Motley Fool’s services, Stock advisor and Rule breakers (Figure 1).

Figure 1: Analyst rating vs. Price movement–The immediate effects of Motley Fool's Stock Advisor and Rule Breakers services issued stock recommendations on stock price movements. The plot shows the stock price in USD around and at the exact time when the recommendations were issued (19:00:00 for ZBRA (Stock advisor) and 17:00:00 for CHGG (Rule breakers), in the Central European time zone, on 11/19/20 and 11/25/20, respectively).

In reality, the impact could be mutual, as renowned analysts could impact the behavior of retail investors, causing swings in the stock price. Vice versa, the future price change either affirms or negates the findings by the analyst, improving or diminishing the analyst’s own prestige and future potential to impact the market. For this analysis, it is not important whether the price changes happen because the analyst caused them, or whether the analyst properly predicted price changes that would have happened anyway. We limit our analysis to the performance of the publicly issued ratings and price target changes.

We define success through several indicators, using both expert understanding and various data driven approaches. We anticipate that appropriate indicators will be able to uncover the meaningful ratings changes, those that, in aggregate, have high success rates and where the value of their predictions are significantly higher than a random walk. This leads to our main goal of identifying specific strategies individuals may want to employ in their trading, as well as which analysts and research houses that they may want to follow closely due to their outperformance.

Methodology

The data for our research is retrieved using the Benzinga API and the Alpaca trading API. From the Benzinga API we used the financial_data library to get Benzinga analyst ratings. We collected the ratings for a year’s time, 01/01/2021-12/31/2021. During this period, Benzinga provided 34,049 Analyst ratings and rating revisions from 169 different research houses covering 4,636 different stocks. The two main components of the ratings we will focus on is the rating label and the associated price target. The rating label is a key word with which analysts will classify their report. For example, a rating could be classified as “Buy” or “Outperform”, meaning the analyst suggests a buy, or a rating could be classified as “Sell” or “Underperform”, meaning the analyst suggests selling the security. The price target indicates the price the analyst believes the security will rise or fall too. We used the Alpaca trading API to get the price of the equities being rated by the Benzinga analysts. Security price data was aligned with the time the rating was published or the time closest to the ratings publishing during open market hours (excluding pre- and postmarket activity). We also retrieved security price low and high data from 3 trading days before the rating was published until 179 days after the rating was published.

Note, the analyst price targets provided by Benzinga are not adjusted for stock splits. However, the analysis has included the appropriate adjustments, checking for equities which had splits in the year 2021 and appropriately adjusts the target price if the split date occurred during the expected trading timeline.

We focused our research on three main indicators using the above data:

(I1) Rating pairs

We define a rating pair as the grouping of the second most recent rating of a security and the most recent rating of on security given by the same analyst. Based on expert understanding and common sense, we postulated a set of 39 ratings pairs which indicate that the security should be bought, and a set of 50 ratings pairs which indicate that the security should be sold.

We postulated that the main ratings that signified a buy were in (Table 1).

Table 1: The main ratings that signified a buy.

The main ratings that signified a sell were postulated to be (Table 2).

Table 2: The main ratings that signified a sell were postulated.

A transition from any type of rating (buy, sell, neutral) to a buy rating would signify a buy rating pair, and a transition to a sell rating would signify a sell rating pair:

Buy rating: Buy ≥ strong buy Sell rating: Neutral ≥ sell

All ratings pairs postulated to signify buy or sell actions are provided in the appendix.

(I2) Market value vs. target value

After determining the buy and sell rating pairs we can check the success of analyst ratings using an enter and exit percentage. The enter percentage is defined as the ratio of the target price of the rating compared to the market value of the security at the time when rating was issued. For example, if the new target price was higher than the market price by a given threshold, such a rating constituted a valid buy signal, and equivalent for sell signals. Once the decision to enter has been made we then track how many days the security should be held using daily low and high prices for all subsequent days from the time when the order was placed. If the High daily stock price exceeded the entry price by a given profit threshold, we closed the simulated trade and recorded the number of days the trade lasted. Similarly, if the trade was losing (based on daily low or high) we closed the trade after the loss exceeded the predefined threshold. In this study, we kept profit and stop loss identical for every trade (symmetric around the entry price). The minimum possible trade duration is 1 day and the maximum possible is 180 days. Meaning a trade entered on a specific day must wait until the following day to exit.

p_c : Price at the time the rating is published

p_ct : Target price of the current rating

p_h : High price of day being evaluated for exit

p_l : Low price of day being evaluated for exit

%_in : Enter Percentage

%_out : Exit Percentage

e_c : Enter Criteria

e_s : Successful Exit

e_u : Unsuccessful Exit

e_i : Inconclusive Exit

If buy rating pair:

e_i:e_s and e_u …………… (1)

If sell rating pair:

e_i:e_s and e_u …………… (2)

(I3) Current target value vs. previous target value

We build from indicators (1) and (2) by adding a third criterion which compares the current target value with the previous target value. If the ratio of current target value by previous target value is greater than 1 we consider this an additional buy indicator, if the ratio is less than 1 we consider this an additional sell indicator. This addition to indicator (2) allows us to better evaluate cases where the previous and current ratings both signify a buy or a sell.

p_pt : Target price of the previous rating

If buy rating pair:

If sell rating pair:

Since both indicators market value vs. target value and current target value vs. previous target value depend on indicator rating pairs, the rating pairs indicator was the most important to identify correctly.

Additionally, we explored a combination of the above indicators, evaluating how one rating could affect another. If the same security had 2 or more ratings in a 3-day period, and both ratings passed the same enter criteria for the specific indicator used, we observed whether the newest rating was more or less likely to be successful (synergy and anti-synergy).

r_n : Newest rating

r₀ : Rating published for the same security within 3 days before

Synergy:

Anti-synergy:

To understand the performance of the indicator strategies against a passive strategy, we included a calculation of return above baseline, with the S and P 500 (SPX) acting as the benchmark. Due to the fact that retail investors very rarely have the ability to short stocks, we have prepared a practical simulation: in the case of a “buy”, the investor purchases the security recommended, and in the case of a sell rating, the investor will buy the S and P 500 (SPX) instead of the security.

Return is calculated as the percentage change in the security

b : Security “buy” rating return-SPX return

s : Percent SPX return-security “sell” rating return

Return above baseline:

Mean ^{([∀b, ∀s])}

Results

In total, 73.5 percent of the total ratings over the 1-year period were classified as either a buy or a sell action by our methodology. This forms Dataset1. We found Dataset1 had a much skewed distribution of buy and sell ratings, with the number of buy ratings exceeding the number of sell ratings by a factor of almost 7. 21,835 ratings pairs were classified as a buy action, while only 3,189 ratings pairs were classified as a sell action.

To begin our analysis, we evaluate the success rate of each of our indicator-strategy pairs across Dataset1. Here success is defined as the rating directive matching the real market outcome. I.e. the value of an asset increased following a rating of buy. I1 performed only slightly better than pure chance with a success rate ranging from 50.31% (Exit: 20) to 52.41% (Exit: 10) (Table 3). I2 performed similarly to I1, with a success rate of 47.72% (In: 20 Exit: 20) to 52.51% (In: 20 Exit: 5) (Table 4). I3, which accounts for changing target prices of ratings, performs best of the indicators with a success rate ranging from 52.55% (In: 15 Exit: 5) to 55.56% (In: 20 Exit: 15) (Table 5 and Figure 2).

Table 3: Indicator 1 performance.

Table 4: Indicator 2 performance.

Table 5: Indicator 3 performance.

Figure 2: Indicator 3 success rates split into buy vs. sell trades–It can be seen that almost all measures have a success rate greater than 50%.

On average, Indicator 3 shows the most consistent performance, with all overall success rates well above 50%. Requiring a threshold of 20 to enter but exiting after the stock has moved 15% has the highest success rate, at 55.56%. Not only that, if you assume a baseline 50/50 success rate in a binomial distribution, all thresholds of I3 are statistically significant at a p-value<0.01.

Given that analyst ratings have been shown to contain signal, we investigate indicator performance for specific research houses to see if there are some firms that investors may want to pay more (or less) attention to. The following analysis focuses on the top 15 firms by ratings volume, irrespective of whether their issued ratings resulted in trades. These 15 firms represent 60% of all ratings issued over the course of 2021 (Table 6).

Table 6: Dataset 2.

For all the top 15 analyst firms, I3 performs the best overall. Success rates range from 50% (HC Wainwright and Co.) to 70.15% (Keybanc) (Table 7). Breaking down performance on buy vs. sell ratings reveals that I3 is the best performer for buy ratings across the top 15 firms (Table 8) but the best performer for sell ratings varies (Table 9). This variation may suggest that different indicators are stronger predictors for some analyst firms but, given the small size of the sell ratings set, we advise caution in attributing undue significance to this result.

Table 7: Top 15 analyst best overall performance.

Table 8: Top 15 analyst best buy performance.

Table 9: Top 15 analyst best sell performance.

Looking at the overall and firm-specific ratings results, I2 has significantly worse performance than I1 and I3. Given this, the remaining analysis will focus on I1 and I3 while exploring other dimensions of the data.

Firstly, we will investigate how individual analysts perform. We have focused only on analysts from these top 15 firms, and those whose ratings yielded at least 20 indicator 3 trades over the course of the year.

We can see that, even at these large and overall successful firms, there is a large dispersion in success (Table 10). This indicates that, while in aggregate analyst’s ratings contain meaningful data that can be constructed into a trading signal, more work needs to be done to understand individual analyst performance.

Table 10: Individual analyst best overall performance for indicators 1 and 3.

We note that there are several paid services that score analyst efficacy and offer alerts when analysts change their ratings. Our results are consistent with the idea that some analysts are better than others, and that these services could provide value to end investors. Future versions of this paper may explore analyst-level indicators or other approaches to understanding which analysts may be worth following (e.g. is performance correlated from one year to the next).

An interesting phenomenon that became apparent during analysis was that of ratings clusters, one analyst will issue a rating and then several others will also issue in quick succession. This phenomenon is more pronounced on buys than sells, but happens on both ratings types.

Studying this further, we wanted to explore whether analyst “consensus” (having a confirmatory rating from a second analyst) conferred any benefits to a trading strategy. We define synergy as when entry criteria are hit but has followed another similar recommendation (within 3 days), and anti-synergy as a trade meeting entry criterion with no similar recommendation in the prior 3 days.

Examining the data, it is clear that investors should pay attention to analysts that are either “first movers” or independent thinkers (those that change ratings without other analysts doing the same), as the anti-synergy model outperforms the synergy model in both I1 and I3 and under all entry and exit criteria (Tables 11 and 12). The data suggests that analysts who change their targets independently are more successful than those who might be changing their recommendations because it’s the consensus thing to do.

Table 11: Indicator 1 synergy models best overall performance.

Table 12: Indicator 3 synergy models best overall performance.

As splitting the overall ratings set into synergy and anti-synergy reduces the opportunity set, we examine the proportion of trades that meet each criterion, taking I3 as an example (Figure 3).

Figure 3: Proportion of all I3 trades that are Anti-synergy–Interestingly, the proportion of “anti-synergy” analyst ratings increases with a threshold. This is consistent with our prior findings that a higher threshold/more extreme rating may confer further information not contained in the rating and indicator itself.

Having found an approach and indicators that show high hit rates, readers may be asking “does this outperform a passive strategy?” Due to the fact that retail investors very rarely have the ability to short stocks, we have prepared a practical simulation: In the case of a “buy”, the investor purchases the security recommended, and in the case of a sell rating, the investor will buy the S and P 500 (SPX) instead of the security. We measure the success rate and the profitability of the strategy by looking at the difference in returns between the purchased security, and the alternative (e.g., “buys” look at Security-SPX, “sells” look at SPX-security).

We find that an anti-synergy strategy implemented on either I1 or I3 outperforms the baseline strategy (Tables 13 and 14).

Table 13: I1 anti-synergy performance above baseline.

Table 14: I3 anti-synergy performance above baseline.

Discussion and Conclusion

Throughout this paper we focused on analyst ratings published on Benzinga.com. We defined several different but interconnected indicators to filter out the best analyst ratings for various trading goals. We determined that many individual analysts and overall research firms with ratings on Benzinga that pass the enter criteria for each indicator have high success rates during the studied period of January 1, 2021, through December 31, 2021. This period saw a significant rise in security prices; as such, many of the ratings were buy recommendations. Indicator 1 focused solely on the rating pairs published by the analyst which classified a rating change as either a “buy” or a “sell”. Indicator 2 built off indicator 1 and incorporated the target price of the security in relation to the market value of the security at the time of the rating being published. Indicator 3 built off indicator 2 and added an additional entry criterion which was the target price of the new rating in relation to the previous rating for the security. When testing the indicators on different threshold levels (the amount the security was expected to increase or decrease), the findings showed that all 3 indicators were successful at a rate higher than 50% most of the time. Additionally, we found that the most reliable indicator was indicator 3 with a statistically significant average success rate of 54.37%. Indicator 3 performed best when the target price of the rating was at least 20% greater than the market value, and the trade executed when the price of the security rose to above 15% of the original market value, with a success rate of 55.56%. Overall, we found that ratings performed better when predicting a larger change in the security, 15% or more. When focusing on the top 15 firms with the highest volume ratings, we see that Keybanc reaches an overall success rate of 70.15% (94 successful ratings out of 134 ratings) with indicator 3 at an enter percentage of 20% and an exit percentage of 20%. When looking at individual analysts from these 15 firms who published enough ratings to execute at least 20 trades, we see that David Long from Raymond James reaches an overall success rate of 88.46% (23 successful ratings out of 26 ratings) with indicator 3 at an enter percentage of 15% and an exit percentage of 20%.

We defined a notion of anti-synergy (ratings which are the first or only rating for a specific security in a 3-day span) and synergy (a rating that follows another rating within a 3-day span). Testing these concepts using both indicator 1 and indicator 3, we find that anti-synergy not only had a higher hit rate than synergy trades, they also outperformed the market. This and the rest of our findings throughout the paper strongly suggests that a retail investor using Benzinga ratings can implement not only a winning strategy, but one that outperforms a passive approach.

Further Study

While this paper focuses on identifying the different indicators success rates for the 2021 year and displaying that a viable general trading strategy can be formed from Benzinga ratings for the overall market, future iterations on this paper can investigate several dimensions currently unexplored. One of the most important aspects of this paper that will need to be tested over time is whether analysts who are successful in one year are typically successful in following. Another area of interest which is yet to be investigated is whether specific types of firms (e.g., dedicated research houses vs. investment banks, or privately held institutions vs. publicly traded ones) have more or less success with their predictions. There are also many ways that the indicators we have found can be finetuned and tested to give better performance rates against more specific entry conditions, rather than being run against the entire market. For example, future indicators could be based on exploration of if analysts are better at predicting specific sectors than others, or if analysts are better at predicting moves of larger vs. smaller companies when segmented by market-cap. Additionally, implementing asymmetric stop-losses vs. take-profit levels could significantly improve the performance of trading indicators.

Acknowledgment

Thank you Benzinga.com for providing the APIs needed to research and develop this paper.

References

1. Filbeck G, Lyon A, Zhao X. Evaluating the performance of the Motley Fools Stock Advisor. Invest Manag Financ Innov. 2017;14: 280-290.

   [Crossref][Google Scholar].

2. Bolster P, Trahan E, Venkateswaran A. How mad is mad money? Jim Cramer as a stock picker and portfolio manager. The Journal of Investing. 2012;21(2): 27-39.

   [Crossref][Google Scholar]

3. Desai H, Liang B, Singh AK. Do all-stars shine? Evaluation of analyst recommendations. Financial Anal J. 2000;56(3): 20-29.

   [Crossref][Google Scholar]

4. Barber B, Lehavy R, McNichols M, Trueman B. Can investors profit from the prophets? Security analyst recommendations and stock returns. J Finance. 2001;56(2): 531-563.

   [Cross Ref][Google Scholar]

5. Barber B, Lehavy R, McNichols M, Trueman B. Reassessing the returns to analysts' stock recommendations. Financial Anal J. 2003;59(2): 88-96.

   [Crossref][Google Scholar]

6. Barber BM, Lehavy R, Trueman B. Comparing the stock recommendation performance of investment banks and independent research firms. J financ econ. 2007;85(2): 490-517.

   [Crossref][Google Scholar]

7. Chang YH, Chan CC. Financial analysts’ stock recommendation revisions and stock price changes. Appl Econ. 2008;18(4): 309-325.

   [Cross Ref][Google Scholar]

8. Morgan J, Stocken PC. An analysis of stock recommendations. RAND J Econ. 2003;1: 183-203.

   [Crossref][Google Scholar]

9. Chan CY, Lo HC, Su YR. Distribution of stock ratings and analyst recommendation revision. N Am J Econ Finance. 2014;28: 273-286.

   [Crossref][Google Scholar]

10. Kanne S, Klobucnik J, Kreutzmann D, Sievers S. To buy or not to buy? The value of contradictory analyst signals. Financial Mark Portf Manag. 2012;26(4): 405-428.

    [Crossref][Google Scholar]