January 26, 2017
Robert A. Gillam, Chief Investment Officer McKinley Capital Management, LLC
John B. Guerard Jr., Director of Quantitative Research McKinley Capital Management, LLC
Rochester H. Cahan, U.S. Portfolio Strategist Empirical Research Partners
Quantitative investing is, as the name suggests, all about numbers. Successful quantitative strategies have always had a voracious appetite for numerical data, happily devouring terabytes of ones and zeros without breaking a sweat. But there is a limit to how much of the world’s information is naturally encoded in numbers. After all, most of human experience is expressed in written and spoken language, not sterile numbers. For a long time this imprecise world of emotions was off-limits to quantitative investors. But now advances in natural language processing and machine learning have given quants the tools they need to extract information from unstructured, text-based data sets.
In this research we study whether the sentiment expressed in company conference calls, like the now-ubiquitous quarterly earnings call, can predict the cross-section of future stock returns. Conference calls are a direct interface between the C-suite of a listed company, its shareholders, and the sell-side analysts covering the company. As such, conference calls present a unique opportunity to put context around the cold, hard numbers presented in a company’s financial statements. Our basic question is whether there is useful information in the language used in conference calls that goes above and beyond what one can glean from the numbers alone.
Listen Up: Extracting Sentiment from Conference Calls
The data set for our study is a large archive of conference call transcripts collected by FactSet, a data vendor. The database includes the transcribed proceedings of over 50,000 conference calls for global companies, beginning at the start of 2010 and finishing at the end of June 2016. Around 85% of the calls are quarterly or semiannual earnings calls. An unique feature of the database is that the transcripts are in XML format, which makes it possible to separate out different parts of the call. For example, it is possible to extract the prepared remarks by the company’s CEO from the analyst Q&A session that usually follows.
Exhibit 1 shows the share of stocks in the McKinley Capital global stock universe that have conference call data over time, as well as the coverage by country. About 60% of companies in the universe have conference call data and, as we would expect, the coverage is skewed toward companies domiciled in English-speaking locales.
Exhibit 1: Conference Call Transcript Coverage of the McKinley Capital Global Stock Universe
The first question is how to quantify the information contained in each call. One plausible approach is to try to measure the sentiment expressed unconsciously by each speaker on the call. To do that we deploy the dictionary developed in Loughran and McDonald . In their research the authors classified words in the English language in those with positive, negative, and neutral connotations. An important feature of their dictionary is that the classifications are based on a financial context, for example the word “restatement” is generally more negative when used in finance-speak than when used in general English. Following their lead, we define the sentiment for a body of text to be the count of positive words less the count of negative words scaled by the total number of words.
Armed with this metric we can compute the aggregate sentiment of all the conference calls over time and also the sentiment of the management discussion and analyst Q&A sections separately, see Exhibit 2.
There are two noteworthy results. First, aggregate conference call sentiment tends to track the MSCI World Index, a promising result because it suggests a link between the tone used by participants on the conference calls and market sentiment. Second, the prepared remarks by company managers are consistently more positive than the analyst Q&A that follows, a result that will not come as a huge surprise to anyone who has listened to the carefully-vetted opening monologues that companies prepare these days.
Exhibit 2: Aggregate Conference Call Sentiment
To test whether conference call sentiment predicts future stock returns we first aggregate sentiment at the company level. To do that for a given company at a given point in time we collect all the conference call transcripts for that firm over the prior 12 months and compute the average sentiment across all the calls for the management discussion sections and analyst Q&A sessions separately. We also compute a difference-in-sentiment factor, which is the sentiment in the analyst Q&A session less that of the management discussion.
Exhibit 3 shows the time-series of the monthly rank information coefficients (IC) over time and also the average over the whole sample. An information coefficient is a simple way to measure the predictive power of a stock-selection factor and is defined as the cross-sectional correlation between stocks ranked on the factor at time t and the ranking of stocks by their subsequent return from time t to t+1. A positive correlation indicates that stocks which scored highly on the factor, in this case stocks with highly positive conference call sentiment, went on to deliver the largest returns in the following month.
Exhibit 3: Information Coefficients for Conference Call Sentiment Factors
Based on the simple IC metric, sentiment derived from the analyst Q&A session of the calls has the strongest predictive power for month-ahead stock returns: the time-series is consistently positive and the average IC over the sample is around +4.5%, a high number by the standards of ICs. The IC for management discussion sentiment is lower but still positive, a finding that is somewhat intuitive given CEOs rarely depart from their attorney-vetted remarks these days. Interestingly the difference in sentiment seems to have little predictive power. Our hypothesis was that a big divergence in sentiment between the company’s management and the analysts covering it might tell us something, but that was not borne our in the data.
Because ICs are a very crude measure of predictive power, we also studied the performance of realistic, optimized portfolios based on the sentiment factors. The mean-variance (MV) portfolio construction can be summarized as:
where μ is the expected return vector, \(C\) is the variance-covariance matrix, w is the portfolios weights, and λ is the risk-return tradeoff parameter. The estimation of \(C\) is usually done by a multifactor model, in which the individual stock return Rj of security j, at time t, dropping the subscript t for time, may be written like this:
The non-factor, or asset-specific return on security j, ej is the residual risk of the security after removing the estimated impacts of the K factors. The term ƒk is the rate of return on factor k. The factor model simplifies the \(C\) as the sum of systematic risk covariance and diagonal specific variances,
The portfolio risk is accordingly decomposed into systematic risk and specific risk
If the investor is more concerned about tracking a particular benchmark, mean-variance optimization (1) can be reformulated as mean-variance tracking error at risk (MVTaR) optimization:
Exhibit 4: Real-World Portfolio Performance for Conference Call Sentiment Factors
where wb is the weight vector of benchmark. One can also add equal active weighting constraints (EAW):
The MVTaR with constraints (6) will be noted as EAWTaR. The total tracking error can be decomposed into systematic tracking error and specific tracking error:
Guerard, Markowitz, and Xu  tested several optimization techniques: (1) total risk minimization with no reference to systematic risk, denoted MVM59, which uses mean-variance analysis with a fixed maximum weight (4%) with the same optimization conditions; (2) mean-variance analysis with tracking error at risk, denoted MVTaR; and (3) an equal active weighting tracking error at risk with a maximum deviation of two percent, denoted EAW2TaR. The EAW2TaR optimization technique weights the systematic risk at three times the importance of specific risk and EAW2TaR where x in constraints equation (7) is set to be 2. MVM59, MVTaR, and EAW2TaR have proved to be effective techniques in real-world portfolio construction and management. We seek to maximize the geometric mean of the portfolios, consistent with Latane  and Markowitz 1. We refer to the creation of portfolios with a multifactor model and the generation of the efficient frontier as a Level II test of portfolio construction.
Exhibit 4 shows the results for the sentiment factors using the EAW2TaR methodology. The risk model deployed was the Axioma Statistical Risk Model, composed of 15 factors from a Principal Components Analysis (PCA) estimation, see Guerard, Markowitz, and Xu  for more details. The portfolios delivered excess returns over the MSCI All Country World Index of +2-3% per annum. The information ratios for the signals both have a value 0.4. Both numbers are economically significant in a real-world active management setting.
We conducted attribution analysis on the optimized portfolios to assess the contributions made by stock-selection and factor exposures to active risk and returns. Exhibit 5 shows the results for the Analyst Q&A factor, based on a mean-variance optimization. The left-hand chart decomposes the active return into the specific portion, due to stock-selection ability, and the portion due to factor and style exposures. The specific return is positive and significant at the 5% level while all other contributions are insignificant. That suggests most of the observed alpha in the signal is being produced by picking the right stocks, and not as the byproduct of embedded factor exposures. The right-hand chart shows the same decomposition for the active risk of the portfolio.
Contains MSG: Sentiment as a Flavor-Enhancer
The evidence so far suggests that conference call sentiment, particularly when derived from the analyst Q&A session, has some predictive power for future stock returns. In this section we study the interaction between conference call sentiment and a proprietary earnings forecasting factor called CTEFROIC that is deployed at McKinley Capital. CTEFROIC combines forecast earnings yield, one-month earnings revisions, breadth, and direction of revisions into a single factor that has delivered strong alpha as a stock-selection signal in live, out-of-sample performance (see Guerard et al.  for more details). Thus a factor that can enhance CTEFROIC has cleared a high hurdle rate.
Exhibit 5: Attribution Analysis for Real-World Portfolios Based on Difference in Sentiment Factor
The left hand side of Exhibit 6 shows there is a low correlation of ICs between CTEFROIC and the two conference call sentiment factors. That suggests it may be possible to glean some diversification benefit by combining signals. Indeed that proved to be the case. The right-hand side of Exhibit 6 shows the time-series of monthly ICs for CTEFROIC as a stand-alone signal as well as 50/50 combinations of CTEFROIC with each sentiment factor. Over time the combination of CTEFROIC and the analyst Q&A sentiment factor is consistently better than CTEFROIC on its own.
Exhibit 6: Analysis of Sentiment Combined with CTEFROIC
|CTEFROIC||Analyst Q&A Sentiment||Management Discussion Sentiment|
|Analyst Q&A Sentiment||6||100%|
|Management Discussion Sentiment||7||83||100%|
Source: Compustat, DataExplorers, Russell, Thomson Reuters, Deutsche Bank.
Source: Compustat, DataExplorers, Russell, Thomson Reuters, Deutsche Bank
The improvement in predictive power over the whole sample is depicted in Exhibit 7 (over). The left-hand chart shows the average IC for CTEFROIC increases by +35-45% when combined with the sentiment signals in a 50/50 blend. Note that in this analysis stocks that do not have a sentiment score (around 40% of the sample, as per Exhibit 1) are omitted. This potentially makes the comparison between CTEFROIC and the sentiment factors unfair, since the performance of the latter is measured over a smaller universe. Therefore we repeated the same exercise in the right-hand chart, except this time we gave any stock with missing sentiment data the median score of the universe. The result is qualitatively the same, but the improvement in average IC does come down to about +25%.
Exhibit 7: Enhancement in CTEFROIC Information Coefficient with Sentiment
The Rise of the Machines: Sentiment and Machine Learning Models
The fact conference call sentiment can enhance CTEFROIC is promising, because that factor has been powerful in live performance. However, an even higher hurdle is to benchmark the sentiment factors against a broad universe of factors known to predict future stock returns. To that end we collected 41 different factors, representing a comprehensive survey of the factors used by practitioners. Included are all the usual suspects like price momentum, earnings revisions, valuation ratios, earnings quality metrics, along with a number of proprietary factors developed at McKinley Capital. The first chart in Exhibit 8 shows the top ten factors over the sample period, when ranked by average IC. It turns out the analyst Q&A sentiment factor finishes third out of the pool of 41 factors.
Exhibit 8: Sentiment Factors Benchmarked Against a Broad Universe of Factors
That is a promising start, but to harvest that alpha we would have had to know ex-ante that analyst Q&A sentiment would end up being a good factor, i.e., back in 2010 we would have had to somehow know that we should bet on that factor instead of the 41 other factors. Thus an even stricter test is whether a factor-selection algorithm would have identified the sentiment factor as useful back at each point in time, knowing only what one knew back then. The right-hand chart in Exhibit 8 uses the AdaBoost machine learning algorithm as a factor-selection tool.
At each point in time the model is trained on the prior 12 months of factor performance. The idea is the machine should “learn” which factors are useful, based only on the information known at the time. If there was enough information back through time to suggest the sentiment factor would be a useful factor going forward, then the machine would select it and use it to improve performance.
The white bars in the chart show that when the AdaBoost model was allowed to draw from a pool of factors that included sentiment, it did indeed use it to further enhance performance. In other words, the analyst Q&A sentiment factor showed enough performance even early in the sample to convince a machine learning algorithm that it should use the factor going forward.
Conclusion: A Promising New Alpha Source
Unstructured, text-based data from conference call transcripts appears to contain useful information for forecasting the future returns of global stocks. Specifically, sentiment extracted from the language used in the analyst Q&A session that usually concludes an earnings call has positive predictive power for month-ahead stock returns. In addition to stand-alone alpha, we found evidence that conference call sentiment can be used to enhance a proprietary earnings forecasting factor. Furthermore the factor’s performance consistency was high enough to allow a machine learning model to identify the factor’s potential ex-ante.
Please refer to the full McKinley Capital Disclosure at the end of this article. The information contained herein does not constitute the provision of investment, legal, or tax advice by any of the authors. Any views expressed reflect the current views of the authors which may change without notice.
Mr. Cahan’s contribution and comments do not necessarily correspond to the opinions of Empirical Research Partners or its affiliates. Opinions expressed may differ from views set out in other documents, including separate research published by Empirical Research Partners. Information is provided for informational purposes only and without any obligation, whether contractual or otherwise. No warranty or representation is made as to the correctness, completeness and accuracy of the information given of the assessments made.
1 Guerard, Markowitz, and Xu  referred to the creation of portfolios with a multifactor model and the generation of the efficient frontier as a Level II test of portfolio construction.
Gillam, R.G., Guerard, J.B., Jr., and R. Cahan, 2015. “News Volume Information: Beyond Earnings Forecasting in a Global Stock Selection Model”, International Journal of Forecasting, Vol. 31, pp. 575-581.
Guerard, J.B., Jr., Gultekin, M., and B.K. Stone, 1997. “The Role of Fundamental Data and Analysts’ Earnings Breadth, Forecasts, and Revisions in the Creation of Efficient Portfolios.” In Research and Finance 15, Edited by A. Chen.
Guerard, J.B., Jr., R.T. Rachev, and B. Shao, 2013. “Efficient Global Portfolios: Big Data and Investment Universes”, IBM Journal of Research and Development 57, No. 5, Paper 11, 1-11.
Guerard, J.B., Jr., H.M. Markowitz, and G. Xu, 2015. “Earnings Forecasting in a Global Stock Selection Model and Efficient Portfolio Construction and Management”, International Journal of Forecasting, Vol. 31, pp. 550-560.
Loughran, T., and B. McDonald, 2011. “When is a Liability not a Liability? Textual Analysis, Dictionaries, and 10-Ks.” Journal of Finance, Vol. 66, pp. 35-65.
McKinley Capital Management, LLC (“McKinley Capital”) is a registered investment adviser under the U.S. Investment Advisers Act of 1940. McKinley Capital is not registered with, approved by, regulated by, or associated with the Financial Conduct Authority (“FCA”), the Prudential Regulation Authority (“PRA”), the Securities & Futures Commission of Hong Kong or the China Securities Regulatory Commission. Additionally, none of the authorities or commissions listed in the previous sentence has commented on the firm, the content of any marketing material or any individual suitability assessments.
This report contains back tested and/or model information; any performance is hypothetical and may not be relied upon for investment purposes. Back tested performance was derived from the retroactive application of a model with the benefit of hindsight and does not represent an actual account. Models may not relate or only partially relate to services currently offered by McKinley Capital and model results may materially differ from the investment results of McKinley Capital’s clients. Returns are absolute, were generated using McKinley Capital’s proprietary growth investment methodology as described in McKinley Capital’s Form ADV Part 2A, are unaudited, and do not replicate actual returns for any client. McKinley Capital’s investment methodology has not materially changed since its inception but it has undergone various enhancements.
No securities mentioned herein may be considered as an offer to purchase or sell a firm product or security. Any comment regarding an individual security is presented at the client’s request, may only be used for client reference, and is not reflective of composite or individual portfolio ownership. McKinley Capital may or may not have held or currently hold a specific security. In addition, any positive comments regarding specific securities may no longer be applicable and should not be relied upon for investment purposes. No security is profitable all of the time and there is always the possibility of selling it at a loss. With any investment, there is the potential for loss. Investments are subject to immediate change without notice. Comments and general market related perspectives are for informational purposes only; were based on data available at the time of writing; are subject to change without notice; and may not be relied upon for individual investing purposes.
Global market investing, including developed, emerging and frontier markets, also carries additional risks and/or costs including but not limited to: political, economic, financial market, currency exchange, liquidity, accounting, and trading capability risks. Derivatives trading and short selling may materially increase investment risk and potential returns. These risks may include, but are not limited to, margin/mark-to-market cash calls, currency exchange, liquidity, unlimited asset exposure, and counter-party risk. Future investments may be made under different economic conditions, in different securities and using different investment strategies. McKinley Capital’s proprietary investment process considers factors such as additional guidelines, restrictions, weightings, allocations, market conditions and other investment characteristics. Thus returns may at times materially differ from the stated benchmark and/or other disciplines and funds provided for comparison.
No fees or expenses of any kind have been deducted. Trading activity, asset allocation, and portfolio decisions are based on the management style that McKinley Capital may have followed had it been actively managing a discretionary account for that period. Returns are calculated using the internal rate of return; do not adjust for external cash flows; do not include brokerage commissions; ignore cash interest during adverse states and when deleveraged, are based on fully discretionary accounts; reflect the reinvestment of dividends and interest; are gross of all investment management and all other costs, expenses and commissions associated with client account trading and custodial services fees; and do not take individual investor tax categories into consideration. Returns do include the reinvestment of hypothetical gains, dividends and other income. The currency used to calculate hypothetical performance is the USD, and no specific benchmark is used unless otherwise noted in the presentation. Individual and actual
returns may vary and additional fees or charges will negatively impact an investor’s absolute returns. Clients should realize that net returns would be lower and must be considered when determining absolute returns. Clients should contact the McKinley Capital institutional marketing manager for additional details on such returns.
Decisions and information provided were based on available research at the time and data contains hypothetical results. Back tested and model results do not represent actual trading and may not reflect the impact material economic and market factors might have had on McKinley Capital’s decision-making if it were managing an actual account. Material economic and market factors may have changed and certain investment restrictions may have affected performance. Back tested and model results are not GIPS compliant. Trading strategies that have been retroactively applied may not have been available during the periods presented.
Charts, graphs and other visual presentations and text information were derived from internal, proprietary, and/or service vendor technology sources and/or may have been extracted from other firm data bases. As a result, the tabulation of certain reports may not precisely match other published data. Data may have originated from various sources including but not limited to Bloomberg, FactSet, MSCI, Axioma, Russell Indices, FTSE and/or other systems and programs. With regards to any material, if any, accredited to FTSE International Limited (“FTSE”) ©FTSE : FTSE™ is a trade mark of London Stock Exchange Plc and The Financial Times Limited and is used by FTSE International Limited under license. All rights in the FTSE Indices vest in FTSE and/or its licensors. Neither FTSE nor its licensors accept any liability for any errors or omissions in the FTSE Indices or underlying data. Please refer to the specific service provider’s web site for complete details on all indices. McKinley Capital makes no representation or endorsement concerning the accuracy or propriety of information received from any other third party. With regards to any materials, if any, accredited to MSCI: Neither MSCI nor any other party involved in or related to compiling, computing or creating the MSCI data makes any express or implied warranties or representations with respect to such data (or the results to be obtained by the use thereof), and all such parties hereby expressly disclaim all warranties of originality, accuracy, completeness, merchantability or fitness for a particular purpose with respect to any of such data. Without limiting any of the foregoing, in no event shall MSCI, any of its affiliates or any third party involved in or related to compiling, computing or creating the data have any liability for any direct, indirect, special, punitive, consequential or any other damages (including lost profits) even if notified of the possibility of such damages. No further distribution or dissemination of the MSCI data is permitted without MSCI’s express written consent.
Fees are billed monthly or quarterly, which produces a compounding effect on the total rate of return net of management fees. As an example, the quarterly effect of investment management fees on the total value of a client’s portfolio assuming (a) $1,000,000 investment, (b) portfolio return of 5% a year, and (c) 1.00% annual investment advisory fee would be $10,038 in the first year, and cumulative effects of $51,210 over five years and $110,503 over ten years. Actual client fees vary. A fee schedule, available upon request, is described in the firm’s Form ADV part 2A. To receive a copy of the firm’s ADV or a description of all McKinley Capital’s composites, please contact McKinley Capital at 3301 C Street, Suite 500, Anchorage AK 99503 or 1.907.563.4488. All information is believed to be correct but accuracy cannot be guaranteed. Clients should rely on their custodial statements for the official investment activity records. Clients should contact their custodian with any questions regarding monthly/quarterly receipt of those statements.