PreprintPDF Available

Measuring Communication Quality of Interest Rate Announcements

Authors:
Preprints and early-stage research may not have been peer reviewed yet.

Abstract and Figures

This paper employs text mining analysis to measure the comprehensibility and information quality of the interest rate announcements published by the Bank of Israel over the past two decades. We examine these texts for ease of comprehension and the sentiment conveyed to the public. The findings reveal that readers require fewer years of education to comprehend the Bank of Israel interest rate announcements than they do to understand the interest rate announcements published by the Fed and the ECB. In addition, we show that the sentiment within these announcements is aligned with economic fluctuations and that there is a direct correlation between the uncertainty the communications reflect and the volatility of the domestic market.
Content may be subject to copyright.
1
Measuring Communication Quality of Interest Rate Announcements
1
Jonathan Benchimol,
2
Itamar Caspi
3
and Sophia Kazinnik
4
December 11, 2020
Abstract
This paper employs text mining analysis to measure the comprehensibility and information
quality of the interest rate announcements published by the Bank of Israel over the past two
decades. We examine these texts for ease of comprehension and the sentiment conveyed to
the public. The findings reveal that readers require fewer years of education to comprehend
the Bank of Israel interest rate announcements than they do to understand the interest rate
announcements published by the Fed and the ECB. In addition, we show that the sentiment
within these announcements is aligned with economic fluctuations and that there is a direct
correlation between the uncertainty the communications reflect and the volatility of the
domestic market.
Keywords: text mining, central bank communication, monetary policy, financial stability.
JEL Classification: A12, E44, E52, E58.
1 Introduction
“The Fed continues to help build resilience in the financial system and will communicate its policy
strategy as clearly and transparently as possible to help align expectations and avoid market
disruptions.
5
Jerome Powell, Chair of the Federal Reserve
Central bank transparency supports financial stability (Powell, 2018), mitigates shocks, and
reduces the risk of a post-announcement market overreaction (Mankiw & Reis, 2018). Central
bank transparency is commonly measured by assessing the volume of communications
1
This paper does not necessarily reflect the views of the Bank of Israel, the Federal Reserve Bank of Richmond
or the Federal Reserve System. We thank participants from the Bank of Israel, Harvard University, and the
University of Houston research seminars for their useful comments.
2
Bank of Israel, Jerusalem, Israel. Corresponding author. Email: jonathan.benchimol@boi.org.il
3
Bank of Israel, Jerusalem, Israel.
4
Federal Reserve Bank of Richmond, Richmond, VA, USA.
5
See Powell (2018).
2
published via the bank’s official website, periodic reports, annual publications, and other
documents intended for the broader public (Dincer & Eichengreen, 2014). Central banks
throughout the world have become more transparent over the past two decades. When the
interest rate is close to the lower bound, communication on the expected future path of this
rate (“forward guidance”) has been proven to be an effective policy tool, with its efficacy
depending, to a great extent, on the clarity of the messages shared (Coenen et al., 2017).
Many of the previous studies that analyzed central bank policy announcements involved the
construction of indices of communication quality against which the effect of communications
on financial variables (Brand et al., 2010), financial stability (Born et al., 2011), and the future
path of interest rates (Bennani et al., 2020) were investigated. Using several of the indices
presented in the existing literature, this paper studies the comprehensibility and information
quality of the interest rate announcements published by the Bank of Israel (BoI). In particular,
it evaluates the extent to which announcements are (1) readable by, and accessible to, the
public; and (2) reflect the economic situation and events.
6
2 Communication Quality
Interest rate decisions are published at predetermined dates alongside explanations on the
reasoning behind the decisions.
7
For our analysis, we reviewed the content of the English-
language interest rate announcements that were published on the BoI’s official website
between 2007 and 2018.
8
2.1 Comprehensibility
Announcement comprehensibility was measured using two indices. The first of these was
the type-token ratio (TTR), which describes the variety of the vocabulary employed and is
calculated as the number of word types in the text compared to the number of times these
words appear (tokens), as follows:


A high TTR value indicates the use of a broad vocabulary within the text that requires more
significant effort and knowledge to understand.
6
Benchimol et al. (2020) review methods for analyzing the text in central bank announcements.
7
Several special situations, such as during the Global Financial Crisis (GFC), involved interest rate decisions
outside the preset dates.
8
Text-mining methods are mostly effective for texts in English. We reasonably assume the nature of the
messages the announcements contain is maintained during translation.
3
The second is an index of text complexity that estimates the number of years of U.S. grade-
school-level education required to understand a text (Kincaid et al., 1975; Flesch, 1979). Since
long words and sentences increase the complexity of a given communication, the index
compares the number of words and sentences with the number of syllables and words,
9
as
follows:
 


The outcomes of our analysis reveal that the vocabulary variability (Fig. 1, left panel) rose
sharply in 2017 after years of relative stability, and even decline. This increase coincided with
a change to the announcement format; i.e., shorter texts were used in combination with the
provision of more detailed figures about the state of the economy.
Figure 1. Indices of Variety in Vocabulary in BoI Interest Rate Announcements
Notes: The lines and green dots represent the trend lines and index values, respectively. The TTR is calculated
over each announcement, while the MATTR is determined over a moving average of 100 words in each
announcement.
9
The ECB recently analyzed the readability of their introductory statements with this index (Praet, 2017).
4
The TTR index may be sensitive to text length and may decline with longer texts.
10
To
examine the robustness of the results, we devised a moving average of TTR values in a
window that included 100 words (MATTR). Using this approach, a clear increase in text
complexity since 2017 could be observed (Fig. 1, right panel).
The Flesch-Kincaid text complexity index (Fig. 2, left panel) exhibits relative stability with
little variance over time. Overall, a U.S. undergraduate-level education (14 years) was
required to understand the BoI announcements that were made between 2007 and 2018.
A more precise examination indicated that the average sentence length declined after 2017
(Fig. 2, middle panel) from about 27 to about 22 words per sentence (although levels
increased at the end of the sample), making the texts less complicated. In contrast, the average
number of syllables per word (Fig. 2, right panel) rose, increasing complexity. In sum, the
two factors offset each other such that the complexity index remained stable.
Figure 2. Indices of Text Complexity in BoI Interest Rate Announcements
Notes: The lines and green dots represent trend lines and index values, respectively.
10
Basically, the more tokens a text contains, the higher the repetition of existing types, particularly punctuation
words such as “the” and “and.” This leads to an artificial decline in the TTR-defined complexity of the text as
sentence length increases (See Eq. 1).
5
This complexity index does not consider the content of the text and any field-specific
knowledge or education that may be required to understand it regardless of education level.
However, it represents an average, and aggregated effects may compensate heterogeneities.
Also, excessively low complexity levels militate against proper delivery of technical and
professional information.
Figure 3. International Comparison of Linguistic Variety and Text Complexity Indices
Notes: Yearly averages based on interest rate announcements.
To draw conclusions based on these indices, we turn our attention to the interest rate
announcements published by other central banks. Fig. 3 compares the complexity and
linguistic variety indices of the BoI’s announcements with the indices of the communications
published by the Fed and ECB. The Fed and ECB’s announcements require an average of
around 17 and 19 years of schooling, respectively, to understand. The BoI announcements
are relatively more comprehensible, requiring an average of about 14 years of schooling to
understand.
Furthermore, the Fed and the ECBs linguistic variety indices are almost three times as high
as that of the BoI on average over the sample.
2.2 Information Quality
In this section, we examine the quality of the information in the interest rate announcements
based on the financial context of the words they contain. Instead of analyzing divergence in
6
the tones (Sadique et al., 2013; Tilleman and Walter, 2019), we build a textual uncertainty
index. In the first stage, using common dictionary-based methods that are designed to
analyze financial texts, we identified words in the text that conveyed uncertainty;
11
for
example, risk,uncertainty,volatility, probability, and variable. In the second stage,
we calculated an index of textual uncertainty, which increased as the rate of words classified
in the category of uncertainty increased. The uncertainty index (Fig. 4) shows the
announcements contained relatively high contextual uncertainty around the time of the GFC.
The contextual uncertainty significantly declined thereafter, particularly after 2016, before
rising in response to the global economic uncertainties that transpired toward the end of the
sample period.
Figure 4. Index of Uncertainty in the BoI Interest Rate Announcements and TA-35 VIX
Source: Bank of Israel and Benchimol et al. (2020).
Dictionary-based classifications are generally executed at the single-word level using an
approach that inaccurately negatively construes a two-term phrase that is formed of words
that independently have negative meanings (e.g., “uncertainty” and “declined”) as being
negative when the combination of the words actually communicates a positive message (e.g.,
“uncertainty declined.” Our approach is in line with the literature since phrase-level
classification significantly reduces the information present in short texts, such as interest rate
announcements, and compensation effects may be at work.
11
The classification adopted is based on the categories proposed by Loughran and McDonald (2011).
7
We noted several instances of correlation between our uncertainty index and real-world
economic events. We examined the correlation between uncertainty indices and risk levels
in domestic markets, as reflected in the TA-25 VIX index.
12
,
13
A positive Pearson correlation
of 0.47 was determined across the sample, while a distance correlation of 0.54 confirmed both
linear and nonlinear linkagescausality direction or the assertion that uncertainty leads to
fluctuations in VIX, or vice versa, could not be inferred from the data available. The
announcements may contain analyses that the public are unaware of, thereby causing market
volatility. In contrast, the text may describe VIX fluctuations that would have occurred in any
case, even without the announcement. Further in-depth research is required to identify
causality direction.
14
3 Conclusion
Sharing transparent communications with the public has become an essential central bank
tool that significantly contributes to financial stability. The findings of this study indicate that
understanding the BoI’s interest rate announcements requires a college-level education,
albeit a significantly lower level of education than that required to understand the Fed and
the ECB announcements. Furthermore, we find that the introduction of a new announcement
format at the beginning of 2017 led to a decline in text comprehensibility. Finally, by applying
the text uncertainty index to the BoI’s interest rate announcements, we find a
contemporaneous correlation between these announcements and economic events, and
preliminary evidence that contextual uncertainty correlates with changes in the TA-25 VIX
index.
References
Benchimol, J., S. Kazinnik, and Y. Saadon (2020). Text Mining Methodologies with R: An Application
to Central Bank Texts, Occasional Paper #2020.01, Bank of Israel.
Bennani, H., N. Fanta, P. Gertler, and R. Horvath (2020). Does central bank communication signal
future monetary policy in a (post)-crisis era? The case of the ECB, Journal of International Money and
Finance, 104(C):102167.
12
In February 2017, this index was changed to the TA-35 index.
13
Calculated on the basis of the implied volatility of options on the index.
14
Our uncertainty index positively correlates with the U.S. VIX index, although it is lower (Pearson: 0.44;
Distance: 0.50). This finding, together with the fact the BoI's interest rate announcements do not influence the
U.S. VIX, supports the hypothesis that volatility in the markets causes an increase in the level of uncertainty in
the BoI interest rate announcements, not the other way around.
8
Born, B., M. Ehrmann, and M. Fratzscher (2011). Communicating about MacroPrudential
Supervision A New Challenge for Central Banks, International Finance, 15(2):179203.
Brand, C., D. Buncic, and J. Turunen (2010). The Impact of ECB Monetary Policy Decisions and
Communication on the Yield Curve, Journal of the European Economic Association, 8(6):12661298.
Coenen, G., M. Ehrmann, G. Gaballo, P. Hoffmann, A. Nakov, S. Nardelli, E. Persson, and G. Strasser
(2017). Communication of Monetary Policy in Unconventional Times, Working Paper Series 2080,
European Central Bank.
Dincer, N.N. and B. Eichengreen (2014). Central Bank Transparency and Independence: Updates and
New Measures, International Journal of Central Banking, 10(1):189259.
Flesch, R.F. (1979). How to Write Plain English: A Book for Lawyers and Consumers, New York, NY:
Harper & Row.
Kincaid, J.P., R.P.J. Fishburne, R.L. Rogers, and B.S. Chissom (1975). Derivation of New Readability
Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy
Enlisted Personnel, Tech. Rep. 56, Institute for Simulation and Training.
Loughran, T. and B. McDonald (2011). When is a Liability Not a Liability? Textual Analysis,
Dictionaries, and 10-Ks”, Journal of Finance, 66(1):3565.
Mankiw, N.G. and R. Reis (2018). Friedmans Presidential Address in the Evolution of
Macroeconomic Thought, Journal of Economic Perspectives, 32(1):8196.
Powell, J.H. (2018). Monetary Policy Influences on Global Financial Conditions and International
Capital Flows, speech delivered at the 2018 Eighth High-Level Conference on the International
Monetary System, May 8.
Praet, P. (2017). Communicating the Complexity of Unconventional Monetary Policy in EMU,
speech delivered at the 2017 ECB Central Bank Communications Conference Communications
challenges for policy effectiveness, accountability and reputation, November 15.
Sadique, S., F. In, M. Veeraraghavan, and P. Wachtel (2013). Soft information and economic activity:
Evidence from the Beige Book, Journal of Macroeconomics, vol. 37(C):81-92.
Tillmann, P. and A. Walter (2019). The effect of diverging communication: The case of the ECB and
the Bundesbank, Economics Letters, 176(C):68-74.
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
We review several existing text analysis methodologies and explain their formal application processes using the open-source software R and relevant packages. Several text mining applications to analyze central bank texts are presented.
Article
Full-text available
We examine the European Central Bank’s ad-hoc communication and explore how it informs future monetary policy decisions. Using the rich dataset of the inter-meeting verbal communication by the members of the European Central Bank’s Governing Council between 2008 and 2016, we construct a measure of communication evaluating its inclination towards easing, tightening or maintaining the monetary policy stance. We find that this measure provides useful additional information about future monetary policy decisions, even when we control for market-based interest rate expectations and lagged decisions. Our results also suggest that, in particular, communication related to conventional measures and/or by the ECB President explain the future ECB rate changes well. All aforementioned results hold also in the environment of the zero lower bound. Overall, these results point to the importance of transparent communication in understanding the future course of monetary policy.
Article
When members of monetary policy committees communicate with the public, the resulting cacophony of voices is often considered a source of confusion. We associate each speech delivered by the presidents of the ECB and the Bundesbank since 2008 with a tone score and construct a measure of diverging tone. Shocks to the tone divergence between the Eurosystem's main protagonists drive volatility, policy uncertainty and risk premia.
Article
Milton Friedman's presidential address, “The Role of Monetary Policy,” which was delivered 50 years ago in December 1967 and published in the March 1968 issue of the American Economic Review, is unusual in the outsized role it has played. What explains the huge influence of this work, merely 17 pages in length? One factor is that Friedman addresses an important topic. Another is that it is written in simple, clear prose, making it an ideal addition to the reading lists of many courses. But what distinguishes Friedman's address is that it invites readers to reorient their thinking in a fundamental way. It was an invitation that, after hearing the arguments, many readers chose to accept. Indeed, it is no exaggeration to view Friedman's 1967 AEA presidential address as marking a turning point in the history of macroeconomic research. Our goal here is to assess this contribution, with the benefit of a half-century of hindsight. We discuss where macroeconomics was before the address, what insights Friedman offered, where researchers and central bankers stand today on these issues, and (most speculatively) where we may be heading in the future.
Article
This paper reports updated measures of transparency and independence for more than 100 central banks. The indices show that there has been steady movement in the direction of greater transparency and independence over time. In addition, we show that outcomes such as the variability of inflation are significantly affected by both central bank transparency and independence. Disentangling the impact of the two dimensions of central bank arrangements remains difficult, however.
Article
In response to the financial crisis of 2007–10, many central banks have been given responsibility for macroprudential supervision. This paper argues that central bank communication will play a central role for that purpose, and makes the point that it should be generally geared towards clarity, transparency and predictability, in order to enhance the effectiveness of macroprudential policies and ensure central bank accountability. Moreover, central banks must manage expectations by clearly communicating what macroprudential policy can achieve, and what its limitations are, to counteract reputational risks. This paper also provides empirical evidence showing that financial markets react significantly and systematically to central bank communication about financial stability issues. However, some forms of communication – such as through speeches – may at times raise volatility and uncertainty, in particular during crises, thus underlining the importance of choosing carefully a communication strategy on macroprudential policy which is suited for a given market environment.
Article
This study employs text-analysis software to analyze the contents of the Federal Reserve Beige Book summary of national economic and business conditions, with a particular focus on the predictive content of the text. The empirical results suggest that the Beige Book’s tone changes in response to upswings in industrial production, unemployment, and overall economic conditions. The results further indicate that increases in negative Beige Book tone and decreases in positive Beige Book tone predict recessions one period in advance of their occurrence. Finally, this study shows that financial markets respond to Beige Book tone.
Article
Previous research uses negative word counts to measure the tone of a text. We show that word lists developed for other disciplines misclassify common words in financial text. In a large sample of 10-Ks during 1994 to 2008, almost three-fourths of the words identified as negative by the widely used Harvard Dictionary are words typically not considered negative in financial contexts. We develop an alternative negative word list, along with five other word lists, that better reflect tone in financial text. We link the word lists to 10-K filing returns, trading volume, return volatility, fraud, material weakness, and unexpected earnings.