Article

Evaluating the quality of investment products: can expert judgment outsmart the market?

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

Purpose This paper aims to deal with expert judgment and its predictive ability in the context of investment funds. The judgmental ratings awarder with a large set of experts was compared to a sample of the dynamic investment funds operating in Central and Eastern Europe with their objective performance, both past and future, relatively to the time of the forecast. Design/methodology/approach Data on the survey sample enabled the authors to evaluate both ex post judgmental validity, i.e. how the experts reflected the previous performance of funds, and ex ante predictive accuracy, i.e. how well their judgments estimated the future performance of the fund. For this purpose, logistic regression for past values estimations and linear model for future values estimations was used. Findings It was found that the experts (independent academicians, senior bank specialists and senior financial advisors) were only able to successfully reflect past annual returns of a five-year period, failing to reflect costs and annual volatility and, mainly, failing to predict any of the indicators on the same five-year horizon. Practical implications The outcomes of this paper confirm that expert judgment should be used with caution in the context of financial markets and mainly in situations when domain knowledge is applicable. Procedures incorporating judgmental evaluations, such as individual investment advice, should be thoroughly reviewed in terms of client value-added, to eliminate potential anchoring bias. Originality/value This paper sheds new light on the quality and nature of individual judgment produced by financial experts. These are prevalent in many situations influencing clients’ decision-making, be it financial advice or multiple product contests. As such, our findings underline the need of scepticism when these judgments are taken into account.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
This paper describes early work trying to predict stock market indicators such as Dow Jones, NASDAQ and S&P 500 by analyzing Twitter posts. We collected the twitter feeds for six months and got a randomized subsample of about one hundredth of the full volume of all tweets. We measured collective hope and fear on each day and analyzed the correlation between these indices and the stock market indicators. We found that emotional tweet percentage significantly negatively correlated with Dow Jones, NASDAQ and S&P 500, but displayed significant positive correlation to VIX. It therefore seems that just checking on twitter for emotional outbursts of any kind gives a predictor of how the stock market will be doing the next day.
Article
Full-text available
Forecasting at the Stock Keeping Unit (SKU) disaggregate level in order to support operations management has proved a very difficult task. The levels of accuracy achieved have major consequences for companies at all levels in the supply chain; errors at each stage are amplified resulting in poor service and overly high inventory levels. In most companies, the size and complexity of the forecasting task necessitates the use of Forecasting Support Systems (FSS). The present study examines monthly demand data and forecasts for 44 fast moving, A-class, durable SKUs, collected from a major U.K. supplier. The company relies upon a FSS to produce baseline forecasts per SKU for each period. Final forecasts are produced at a later stage through the superimposition of judgments based on marketing intelligence gathered by the company forecasters. The benefits of the intervention are evaluated by comparing the actual sales both to system and final forecasts. The findings support the case that adjustments do improve accuracy, particularly under the condition that the adjustment is conservative, in the right direction, but does not overshoot. The question is how best to meet these conditions.
Article
Full-text available
This paper proposes a simple back testing procedure that is shown to dramatically improve a panel data model's ability to produce out of sample forecasts. Here the procedure is used to forecast mutual fund alphas. Using monthly data with an OLS model it has been difficult to consistently predict which portfolio managers will produce above market returns for their investors. This paper provides empirical evidence that sorting on the estimated alphas populates the top and bottom deciles not with the best and worst funds, but with those having the greatest estimation error. This problem can be attenuated by back testing the statistical model fund by fund. The back test used here requires a statistical model to exhibit some past predictive success for a particular fund before it is allowed to make predictions about that fund in the current period. Another estimation problem concerns the use of a single statistical model for all available mutual funds. Since no one statistical model is likely to fit every fund, the result is a great deal of misspecification error. This paper shows that the combined use of an OLS and Kalman filter model increases the number of funds with predictable out of sample alphas by about 60%. Overall, a strategy that uses very modest ex-ante filters to eliminate funds whose parameters likely derive primarily from estimation error produces an out of sample risk-adjusted return of over 4% per annum. Copyright 2007, Oxford University Press.
Article
Full-text available
An advisor is supposed to recommend a financial product in the best interest of her client. However, the best product for the client may not always be the product yielding the highest commission (paid by product providers) to the advisor. Do advisors nevertheless provide truthful advice? If not, will a voluntary or obligatory payment by a client induce more truthful advice? According to the results, only the voluntary payment reduces the conflict of interest faced by advisors.
Article
The benefit of investment advice depends on the quality of advice and the investor's counterfactual portfolio. We use changes in the Oregon University System Optional Retirement Plan to highlight the impact of plan design on the counterfactual portfolios of advice seekers. When brokers are available and target date funds (TDFs) are not, brokers help participants with high predicted demand for advice bear market risk, but they recommend higher-commission options. When brokers are removed and TDFs are added, new high-predicted-demand participants primarily invest in TDFs, which offer similar market risk but higher Sharpe ratios than the broker-advised portfolios within our sample.
Book
Principles of Forecasting: A Handbook for Researchers and Practitioners summarizes knowledge from experts and from empirical studies. It provides guidelines that can be applied in fields such as economics, sociology, and psychology. It applies to problems such as those in finance (How much is this company worth?), marketing (Will a new product be successful?), personnel (How can we identify the best job candidates?), and production (What level of inventories should be kept?). The book is edited by Professor J. Scott Armstrong of the Wharton School, University of Pennsylvania. Contributions were written by 40 leading experts in forecasting, and the 30 chapters cover all types of forecasting methods. There are judgmental methods such as Delphi, role-playing, and intentions studies. Quantitative methods include econometric methods, expert systems, and extrapolation. Some methods, such as conjoint analysis, analogies, and rule-based forecasting, integrate quantitative and judgmental procedures. In each area, the authors identify what is known in the form of `if-then principles', and they summarize evidence on these principles. The project, developed over a four-year period, represents the first book to summarize all that is known about forecasting and to present it so that it can be used by researchers and practitioners. To ensure that the principles are correct, the authors reviewed one another's papers. In addition, external reviews were provided by more than 120 experts, some of whom reviewed many of the papers. The book includes the first comprehensive forecasting dictionary.
Article
Our objective is to penetrate the “black box” of sell‐side financial analysts by providing new insights into the inputs analysts use and the incentives they face. We survey 365 analysts and conduct 18 follow‐up interviews covering a wide range of topics, including the inputs to analysts’ earnings forecasts and stock recommendations, the value of their industry knowledge, the determinants of their compensation, the career benefits of Institutional Investor All‐Star status, and the factors they consider indicative of high‐quality earnings. One important finding is that private communication with management is a more useful input to analysts’ earnings forecasts and stock recommendations than their own primary research, recent earnings performance, and recent 10‐K and 10‐Q reports. Another notable finding is that issuing earnings forecasts and stock recommendations that are well below the consensus often leads to an increase in analysts’ credibility with their investing clients. We conduct cross‐sectional analyses that highlight the impact of analyst and brokerage characteristics on analysts’ inputs and incentives. Our findings are relevant to investors, managers, analysts, and academic researchers.
Article
We conduct a series of field experiments to evaluate two competing views of the role of finan-cial intermediaries in providing product recommendations to potentially uninformed consumers. The first argues that financial intermediaries may provide valuable product education, helping consumers decide which of many complicated products is right for them. Even if commissions influence intermediary recommendations, consumers are sufficiently sophisticated to discount advice. The second, more sinister, view, argues that intermediaries recommend and sell prod-ucts that maximize the agents well-being, with little regard to the need of the customer. Audit studies in the Indian insurance market find evidence consistent with the second view: agents recommend a product that provides them high commissions, though it is strictly dominated by alternative products. Consumers demonstrating lower levels of sophistication are more likely to be offered the wrong product. Agents also appear to cater to the initial preferences of con-sumers even those initial preferences are for products that are not suitable for the consumer. Finally, we exploit a natural experiment that occurred during our audits to test how disclosure requirements affect product recommendations. We find that requiring disclosure of commission levels makes agents less likely to recommend the product for which disclosure is required.
Article
In this paper, an artificial neural network method is applied to forecast the end-of-year net asset value (NAV) of mutual funds. The back-propagation neural network is identified and explained. Historical economic information is used for the prediction of NAV data. The results of the forecasting are compared to those of traditional econometric techniques (i.e. linear and nonlinear regression analysis), and it is shown that neural networks significantly outperform regression models in situations with limited data availability.
Article
Lawsuits alleging illegal and unethical insurance sales practices have received widespread publicity in recent years. Although many observers have argued that one source of ethical conflicts for insurance agents is the industry's reliance on straight commission compensation, there remains a paucity of empirical data to support the claim. Therefore, we tested whether different forms of compensation influence insurance agent recommendations of products. We obtained survey responses from 336 insurance agents. Respondents were presented with a composite sketch of a hypothetical client. Following this description, they were asked to identify which one of eight specified life insurance products they would recommend to the client. A between-groups design was employed to manipulate differences in compensation associated with the eight products. Results indicated that neither amount of coverage nor type of coverage (term life versus cash value) recommended was associated with compensation. However, an unanticipated finding was that amount of coverage recommended was significantly higher when the insured was male than when the insured was female.
Article
Purpose – The paper aims to empirically examine the forecasting ability of US-based world mutual funds during the 2001-2007 time period. Design/methodology/approach – World mutual funds are treated as portfolios composed of two sets of securities, i.e. domestic and foreign and two methodologies are used to measure forecasting ability: domestic differential exposure and assertion rates. Domestic differential exposure is based on the difference between each fund exposure to the domestic market when it is the outperforming market and the portfolio exposure to the domestic market when the foreign market is outperforming. Similar to the differential exposure, assertion rates measure the ability of fund managers to pick, on a monthly basis, an outperforming market. Findings – Although changing economic conditions in both domestic and foreign markets provided plenty of opportunities to outperform market benchmarks, the results of two empirical tests reveal that fund managers fail to effectively manage their exposure to both markets. Some evidence of good forecasting ability is found when funds are examined on a yearly basis. Originality/value – This study provides the first implementation of both methodologies: domestic differential exposure and assertion rates, to examine global funds.
Article
Experts were used as Delphi panellists and asked to present forecasts on financial market variables in a controlled experiment. We found that the respondents with the least accurate or least conventional views were particularly likely to modify their answers. Most of these modifications were in the right direction but too small, probably because of belief-perseverance bias. This paper also presents two post-survey adjustment methods for Delphi method based forecasts. First, we present a potential method to correct for the belief perseverance bias. The results seem promising. Secondly, we test a conditional forecasting process, which unexpectedly proves unsuccessful.
Article
A comparison of 66,100 consensus estimates of Wall Street analysts with reported earnings for a large sample of NYSE, Amex, and OTC companies demonstrates that their forecasts differ significantly from actual reported earnings. A minority of estimates fall within a range around reported earnings considered acceptable to many professional investors. The error rates are not meaningfully affected by the business cycle or industry groupings. The average error also appears to be increasing over time. These findings question the use of finely calibrated earnings forecasts that are integral to the most common valuation models and indirectly question the valuation methods themselves.
Article
The representation of duckweed multiplying itself in a pond is used as a research paradigm to study underestimation of exponential growth. The advantage of this paradigm is that the growth process is presented in a direct non-numerical way. The results show that the underestimation observed in earlier studies where growth was presented by means of tables or graphs, occurs in the pond-and-duckweed situation as well. By manipulating the way the process is presented it is possible to obtain some insight into the sampling strategies used by the subjects when they subjectively extrapolate the perceived processes. These experiments lead to the conclusion that subjects base their extrapolations on three or four samples only.
Article
Purpose – In the personal selling industry, it is particularly difficult for salespeople to manage a conflict of interest that exists between the company and customers. The purpose of this research is to examine the impact of time based compensation on salespeople's selling decisions when the conflict occurs. Design/methodology/approach – Questionnaire surveys were collected from 361 full‐time life insurance salespeople in Taiwan. Findings – The results indicate that compensation, training, and marketing policy may affect salespeople's selling decisions. Originality/value – Very little research addresses what salespeople would do when a conflict of interest occurs between the company and customers. Also, how time based compensation would affect salespeople's selling decisions in company‐customer conflict is unknown. Furthermore, training and marketing policy may affect salespeople's selling decisions in the conflict. This paper relates to these issues and provides some discussions of them.
Article
This research experimentally examines the favorable/unfavorable outcomes of a firm's financial and nonfinancial performance measures on financial analysts' recommendation to divest or invest in a firm. The participants were financial analysts who made recommendations ranging from “definitely sell” to “hold” to “definitely buy.” The results show that financial and nonfinancial performance measures and their favorableness have an interactive impact on analysts' recommendations. To be precise, the recommendations were very close to the “definitely sell” anchor when the performance was unfavorable, irrespective of whether the measures presented were financial or nonfinancial. Further, favorableness of performance on nonfinancial measures appears to be irrelevant when performance on financial measures is unfavorable. However, when performance on financial measures is favorable, the effect of nonfinancial performance had a differential effect on analysts' recommendations depending on whether these measures indicated favorable or unfavorable performance. Specifically, when nonfinancial performance was unfavorable, the recommendations were closer to “hold” on average, but the recommendations were closer to “definitely buy” on average when nonfinancial performance was favorable. These results are consistent with our expectations. Overall, given that more and more firms are disclosing nonfinancial measures along with the traditional financial measures, and with an increasing number of firms reporting unfavorable financial performance, the results of this research underline the importance of considering both financial and nonfinancial measures and their outcomes—favorable and unfavorable—on analysts' recommendations. Data Availability: Please contact the authors.
Article
A broad-based questionnaire survey, which received a high response from German mutual fund companies, sheds light on the black box of institutional equity investing in a systematic manner. The survey asked for fund managers' basic views and practices and for insights into their company's performance-measuring and compensation incentives. While the results suggest that professional equity investors primarily recognise underlying economic information as a source of superior value, there are also strong indications for destabilising behavioural factors arising from the choice of information sources and investment strategies and styles. Attempts at fundamental arbitrage are likely to be constrained significantly by time horizons and the fear of market movements. Agency problems are shown to have a bearing on equity fund managers' investment behaviour.
Article
Investment advisors have significant influence on private investors' financial decisions but empirical studies of their behavior are rare. Based on a large and unique data set of the detailed portfolio compositions and daily transactions of more than 65,000 customers of a large German direct bank, we first compare advised investors with groups of non-advised investors, including matched peers. We find that the effect of investment advice is probably less than assumed. To underline this finding, we conduct an event study comparing 597 investors before and after the decision to take regular investment advice. We find much higher trading activity after the event due to advisors re-structuring clients' portfolios, less risky and speculative trading and rising diversification.
Article
We test a hypothesis that financial analysts use a simple algorithm of an equal growth rate for expenses as is for sales when they forecast corporate earnings by examining the errors in analysts’ earnings forecasts. If expenses change at a lower rate than sales in absolute terms due to the fixed portion of expenses, then analysts’ forecasted expenses will be higher (lower) than actual when they forecast an increase (decrease) in sales, resulting in lower (higher) forecasted than actual earnings assuming that analysts have perfect sales forecasts. Using 3,220 individual financial analysts’ sales and earnings forecasts during the period of 1996-2005 for which sales forecast errors are close to zero, we find that the errors in analysts’ earnings forecasts are positively related to their expected sales growth rate. This result is consistent with the hypothesis that analysts’ imperfect adjustments of cost behavior result in systematic errors in their earnings forecasts.
Article
This study reports the results of an experiment that examines (1) the effects of forecast horizon on the performance of probability forecasters, and (2) the alleged existence of an inverse expertise effect, i.e., an inverse relationship between expertise and probabilistic forecasting performance. Portfolio managers are used as forecasters with substantive expertise. Performance of this ‘expert’ group is compared to the performance of a ‘semi-expert’ group composed of other banking professionals trained in portfolio management. It is found that while both groups attain their best discrimination performances in the four-week forecast horizon, they show their worst calibration and skill performances in the 12-week forecast horizon. Also, while experts perform better in all performance measures for the one-week horizon, semi-experts achieve better calibration for the four-week horizon. It is concluded that these results may signal the existence of an inverse expertise effect that is contingent on the selected forecast horizon.
Article
The main goal of this paper was to examine the accuracy and confidence of financial forecasts during the 2009/2010 crisis. The study was carried out in February 2009 in Poland. The participants represented two groups: financial analysts and laypeople (people without knowledge or skills in finance). All participants were asked to forecast future stock market performance and foreign exchange rates. Additionally, they marked their confidence on a 100-point scale. The results showed that the forecasts significantly differed from the real values. In forecasting both the stock market and the currency exchange market, the prediction error significantly differed from zero. Even if the participants were optimistic in making the directional stock market forecasts, they were pessimistic when making point index predictions, which suggests a judgmental paradox. The experts were slightly better than the non-experts in predicting the stock market. However, their accuracy was generally not better in the exchange market forecasts. The next step of the analysis focused on the confidence factor. The results of this part of the research showed that the laypeople were less confident than the experts in all the judgments.
Article
Exponential growth in numerical series and graphs is grossly underestimated in an intuitive extrapolation task. Subjects’ extrapolations are well described by a model with two parameters only: one for underestimation of the nonlinear growth, the other for linear compensation. The size of the effect is considerable; it is not unusual that two-thirds of the subjects produce estimates below 10% of the normative value. The effect increases with the exponent of the stimulus series, and with addition of a constant to the stimulus series. Neither special instructions about the nature of exponential growth nor daily experience with growth processes enhanced the extrapolations.
Article
The past 25 years has seen phenomenal growth of interest in judgemental approaches to forecasting and a significant change of attitude on the part of researchers to the role of judgement. While previously judgement was thought to be the enemy of accuracy, today judgement is recognised as an indispensable component of forecasting and much research attention has been directed at understanding and improving its use. Human judgement can be demonstrated to provide a significant benefit to forecasting accuracy but it can also be subject to many biases. Much of the research has been directed at understanding and managing these strengths and weaknesses. An indication of the explosion of research interest in this area can be gauged by the fact that over 200 studies are referenced in this review.
Article
Little attention has been paid to a principal decision context in which analysts' earnings forecasts are prepared, namely, as an input to their recommendations. We use two data sets, Value Line, USA, and Research Evaluation Service, Canada, and examine the importance of analysts' earnings forecasts for their stock price forecasts via three hypotheses: (1) analysts' earnings forecasts are important for their stock price forecasts; (2) analysts' long-term earnings forecasts are more important than their short-term earnings forecasts for their predictions of stock prices over a particular stock price forecast horizon; (3) the importance of analysts' earnings forecasts for their stock price forecasts rises as the joint earnings and stock price forecast horizon increases. We show that: (1) when the earnings forecast horizon is the next fiscal year, forecasted earnings explain only 30% of the variation in forecasted price; (2) the importance of forecasted earnings for forecasted price rises as the earnings forecast horizon increases; (3) in the long run, (i.e. three to five years hence), forecasted earnings explain about 60% of the variation in forecasted price. Decision usefulness is an ex ante concept, but tests regarding the usefulness of earnings for stock price generally have used actual (not expectational) data. Our evidence suggests that earnings expectations are decision useful, where the decision context is sell-side analysts' stock price forecasts. Our results are potentially important to users of sell-side analyst research reports. When a stock recommendation is accompanied only by short-run earnings forecasts, investors need to closely examine estimates of non-earnings variables to assess the quality of stock recommendations. In contrast, when stock recommendations are accompanied by both short-run and long-run earnings forecasts, investors need to examine estimates of non-earnings information variables less closely.
Article
As might be expected, surveys show accuracy is the most important criterion in selecting a forecasting strategy. This may lead to the expectation that computer based software would be used to aid the forecasting effort. However, despite the wide availability of forecast software, management judgement appears to be the preferred method. This leads to the question: does management judgement provide the most accurate forecast estimates? This paper reports a field study of judgemental sales forecasting over thirteen manufacturing organisations to investigate whether these forecasts are accurate, unbiased and efficient, and whether better ex ante forecasts could have been developed using a computer based model. The study shows that the company forecasts were not uniformly more accurate than a simple, un-seasonally adjusted, naı̈ve forecast. Most of the source of error is due to both inefficiency (a serial correlation in the errors) and bias in the forecasts. These two factors seemed to mask any contribution of contextual information to accuracy. The results are also discussed in terms of forecasting objectives the organisations may have other than accuracy.
Article
Research on this project was supported by a grant from the National Science Foundation. I am indebted to Arthur Laffer, Robert Aliber, Ray Ball, Michael Jensen, James Lorie, Merton Miller, Charles Nelson, Richard Roll, William Taylor, and Ross Watts for their helpful comments.
Article
Forecasting the returns from investments in mutual funds is a very difficult problem. This study examines a new forecasting approach and system for the performance of mutual funds in Greece. This is accomplished via an application of a variation of the Theta model on a time series composed of the daily values of mutual funds. The proposed models are simple and implemented into an easy-to-use expert forecasting system.
Article
This study aims to explore the differences in various dimensions of forecasting accuracy that may result from the task format used to elicit the probabilistic forecasts. In particular, we examine the effects of using multiple-interval and dichotomous formats on the performance of portfolio managers' probabilistic forecasts of stock prices. Probabilistic forecasts of these experts are compared with those provided by semi-experts comprised of other banking professionals trained in portfolio management, as well as with forecasts provided by a novice group. The results suggest that the task format used to elicit the probabilistic forecasts has a differential impact on the performance of experts, semi-experts, and novices. The implications of these findings for financial forecasting are discussed and directions for future research are given.
Article
This paper compares the relative predictive ability of several statistical models with analysts' forecasts. It is one of the first attempts to forecast quarterly earnings using an autoregressive conditional heteroskedasticity (ARCH) model. ARCH and autoregressive integrated moving average models are found to be superior statistical forecasting alternatives. The most accurate forecasts overall are provided by analysts. Analysts have both a contemporaneous and timing advantage over statistical models. When the sample is screened on those firms that have the largest structural change in the earnings process, the forecast accuracy of the best statistical models is similar to analysts' predictions. Copyright 1995 by MIT Press.
The influence of financial advisers on return chasing
  • S Li
Costly financial advice: conflicts of interest or misguided beliefs?”, Working Paper
  • J T Linnainmaa
  • B T Melzer
  • A Previtero
  • C Grace
Výsledky soutěže Zlatá koruna
  • Zlatá Koruna