Methodological Triangulation at the Bank of England:An Investigation

Abstract

This paper investigates the extent to which triangulation takes place within the Monetary Policy Committee (MPC) process at the Bank of England. Triangulation is at its most basic, the mixing of two or more methods, investigators, theories, methodologies or data in a single investigation. More specifically, we argue for triangulation as a commitment in research design to the mixing of methods in the act of inference. The paper argues that there are many motivations for triangulation as well as types of triangulation. It is argued that there is evidence of extensive triangulation of different types within the MPC process. However, there is very little theoretical triangulation present; raising concerns about pluralism. Also, it is argued that the triangulation which occurs is mainly undertaken for pragmatic reasons and does not reflect other, coherent ontological and epistemological positions.

Full text

Page 1

Methodological Triangulation at the Bank of England: An Investigation

Paul Downward,
Institute of Sport and Leisure Policy,
Loughborough University
E-mail: p.downward@lboro.ac.uk

And

Andrew Mearman*,
School of Economics,
University of the West of England-Bristol,
Coldharbour Lane,
Bristol, BS16 1QY
E-mail: Andrew.Mearman@uwe.ac.uk

Abstract: This paper investigates the extent to which triangulation takes place
within the Monetary Policy Committee (MPC) process at the Bank of England.
Triangulation is at its most basic, the mixing of two or more methods, investigators,
theories, methodologies or data in a single investigation. More specifically, we argue
for triangulation as a commitment in research design to the mixing of methods in
the act of inference. The paper argues that there are many motivations for
triangulation as well as types of triangulation. It is argued that there is evidence of
extensive triangulation of different types within the MPC process. However, there is
very little theoretical triangulation present; raising concerns about pluralism. Also,
it is argued that the triangulation which occurs is mainly undertaken for pragmatic
reasons and does not reflect other, coherent ontological and epistemological
positions.


Draft of July 2005.

Copyright held. Please do not cite or quote from without the authors’ prior permission.

*Corresponding author in this case

Page 2

Methodological Triangulation at the Bank of England: An
Investigation1

1. INTRODUCTION

There has been a growing concern that one single method is inadequate for investigating
complex social phenomena. Thus, there has been interest in several quarters, including in
government agencies (such as research councils) and academic departments in the
feasibility of mixing methods in a process of triangulation. In a series of papers, we have
investigated the philosophical and methodological rationale for triangulation (Downward
and Mearman, 2002, 2003, 2004, 2005); and we have attempted to show triangulation in
action, particularly in investigating pricing (Downward and Mearman, 2003).
Specifically, we have argued that triangulation offers a potential solution to the impasse
between the mainstream monist formalist ‘deductivist’ approach and the approach of
critics – such as critical realists – who seem to imply rejection of mainstream methods. In
this way, we work in the spirit of pluralism in economics as advocated by, for instance,
Dow (1985 et passim). For us, triangulation operates within what might be called a
Keynesian or ‘Babylonian’ approach (Dow, 1985).

This paper expands upon our earlier work, by investigating motivations for triangulation.
Most significantly, it investigates the always topical and – often controversial – process
of the Monetary Policy Committee (MPC), and more specifically, the support for this
process provided by the Bank of England, behind the setting of interest rates. This task is
warranted because of the frequent claims (see, for example, Dow, 2004; Backhouse,
2005) that the Bank is an example (albeit rare) of economists using triangulation. Such an
investigation is easier now given the greater transparency of the process following the
independence of the Bank from other arms of government in 1997. We argue: 1) that
there are many examples of triangulation within the MPC process; but that 2) most of
these are of a weak form, involving mainly data triangulation, mainly for pragmatic
reasons; and that (3) the Bank lacks a coherent framework for mixing methods to the
extent to which it does. The paper proceeds as follows: first, types of triangulation are
discussed. Second, motives for triangulation, as found in the existing literature, are
explored. Third, the Bank of England’s MPC process of forecasting inflation is evaluated
in terms of types of and motivations for triangulation.

2. TYPES OF AND MOTIVES FOR TRIANGULATION

2.1 Types of triangulation


1 We acknowledge comments received at a staff seminar at the University of the West of England in
November 2004. In particular, we acknowledge detailed and helpful comments received from Peter
Howells. We acknowledge the participants at a session at the conference of the Association for Heterodox
Economics at City University, London in July 2005. In particular we should like to thank Victoria Chick
for her discussion of the paper. Thanks also to Geoffrey Church. We also acknowledge the comments of
Kevin Butler of the Bank of England. All views expressed here are the authors’ and do not reflect the view
of the Bank. All remaining errors are our own.

Page 3

Triangulation as an applied concept derives from navigation and surveying, whereupon
taking measurements from two separate locations one can derive, or predict, a third
measurement or location2. In social research, in its broadest sense triangulation implies
combining together multiple insights in an investigation3. However, this simple definition
of triangulation masks a range of its meanings and uses. This section presents a non-
exhaustive classification of types of triangulation.

The simplest form of triangulation is also the least extensive, and indeed may not at first
appear to be triangulation. It is the employment of judgement by the economist about
their model, tool, theory or data. For example, an economist might use an econometric
model (which often they have created) to produce an estimation. They could in principle,
passively receive the information from the model, simply report this result and stop their
investigation. However, they can be more active, and apply their judgement to the result,
perhaps to interpret it in a specific way. This employment of judgement can be
interpreted as the interaction of economist with model, and thus is a form of triangulation.
Dow (2004) notes that the use of judgement becomes crucial when faced with
‘Keynesian’ unquantifiable uncertainty.

Denzin (1970) offers four main types of triangulation. Data triangulation is when
different sets (and often types) of data are combined. Different types of data might be
used; for example, survey data might be used alongside time series data. Additionally, the
data could be differentiated spatially or temporally. Thus, the insights of a person at
different times could be triangulated to make an inference about the whole time period.
Clearly, also, different people could be asked once, but at different times. An example
would be the combination of survey and interview data.

Investigator triangulation is the combination of insights from multiple investigators on
the same subject. Imagine three people in the dark examining an unknown object. If the
men perceive that they feel a tail, a thick upright rough surface, and a ivory object, they
might infer that they have, respectively, a donkey, a tree (or umbrella stand) or a horn to
be played. However, the combination of the insights leads the three to conclude that the
mystery object is an elephant. Examples of this type of triangulation abound: for
example, the replication or repeated trials of a study, or the seeking of a second opinion,
are common techniques in medical research. The recent advances in cell biology,
particularly in the discovery of how cells move proteins through a specific pathway in
order to be able to secrete them from the cell (see Pelham, 2001; Del Rio et al, 2004), is
an example of triangulation in ‘natural science’.

2 Blaikie (1991) has some reservations about the appropriateness of this nautical or surveying analogy to
social science. One related criticism of the term is that triangulation implies a precision which is unjustified
in social research. An alternative terminology is of “mixed methods” (Downward and Mearman, 2005);
pluralist (Dow, 2004); or Babylonian (Dow, 1985). However, the term triangulation is well established
now, justifying its use.
3 It is clear that many of the most prominent economists, particularly Smith and Marshall, have broadly
engaged in triangulation, as they drew upon different evidential bases and arguments. Moreover, it can be
argued that, as evidenced by Laidler (1993), a process of triangulation – in this case, the combination of
methods and data types – led directly to the conclusion that the demand for money function is unstable (see
140).

Page 4

In the case of multiple investigators, each may have their own prior theory, or theoretical
paradigms, whose insights are then combined (in various different ways) to reach a
collective conclusion. In the literature this is known as theoretical triangulation (Denzin,
1970). Such theoretical triangulation can also take place within an individual, who might
analyse a finding from multiple perspectives.

Denzin’s (1970) fourth type of triangulation is methodological triangulation, which
involves the combination of different methods. A weak form of methodological
triangulation can be what Denzin calls within-method triangulation, in which different
varieties of the same method are combined. An example of this might the triangulation of
VARs with different specifications or different lag lengths. More adventurous is between-
method triangulation, which involves the use of different methods in combination.
Between-method triangulation is challenging because it often involves the combination of
different underlying methodologies: for example, the combination of an econometric
study with a discourse analysis combines methods based on opposed philosophical
bases4. Our preferred definition adopts the most adventurous position. For us,
triangulation is: the prior commitment in research design to investigation and inference
via the mixing of methods. That, then, is the implicit standard by which apparent
triangulation is assessed. However, in discussing the Bank of England’s practices, all of
the pre-existing definitions will be used.

It is our evaluation that, in the economic mainstream, generally it is the case that
triangulation, beyond the interaction of modeller and model, is limited. The relative lack
of triangulation might reflect positivist philosophical underpinnings (see Frankfort-
Nachmias and Nachmias, 1996). More likely, the lack of triangulation results from the
widely held belief that certain types of method necessarily have higher statistical power;
and that wherever possible such methods should be used. Sophisticated developments of
regression analysis are the best example, perhaps because of their claimed analogy to
controlled experiments.

2.2 Motives for triangulation

Types of triangulation are just one part of the story. In order to evaluate the Bank’s
practices, we also need to ascertain their motives for their actions. A range of motives for
triangulation can be found in the literature. One of the most common is the rationale that
often data are incomplete or inadequate, and that it is necessary to use different data types

4 Typically, it is viewed as controversial to combine different methodologies, because this involves
ontological and epistemological clashes. For example, quantitative data is based on an empiricist ontology,
whereas qualitative data is derived from the presupposition that reality is exhausted by meanings. For the
proponent of quantitative data, qualitative data are not subject to measurement and thus are useless;
whereas for the qualitative researcher, it is impossible to measure meanings, and thus quantitative analysis
is useless. It is beyond the scope of this paper to resolve this clash. However, elsewhere (Downward and
Mearman, 2005) have argued that the ontological clash can be removed if an ontology of complex objects
residing in deep structures is adopted. Specifically, they (see also Downward and Mearman, 2002) argue
that the Critical-Realist logic of retroduction inherently involves triangulation, because various empirical
methods can illuminate both what Critical Realists refer to as the empirical and real domains of reality.

Page 5

to fill the gaps in the original data set. For example, gaps in time series are often filled
with estimates. Additionally, errors in data night be corrected or taken into account by
using other data.

The tactic of econometricians of re-estimating equations under different specifications is
an appeal to the above argument; however, estimating multiple equations can also be
interpreted as an appeal to the law of large numbers. There is an implicit claim made that
increased numbers of confirming estimations increases the ‘validity’ of the estimation
and/or underlying theory (Campbell and Fiske, 1959).

Triangulation can also occur for political reasons. If an investigation is being carried out
amongst a group, as is often the case, in order to convince each participant that their input
is valued, it may be that some part of each party’s view is incorporated into the group
action. This satisfies all stakeholders involved; and it can unwittingly admit theoretical or
even methodological triangulation. Another political factor is that the research group may
have a clear (explicit or implicit) view that pluralism ought to be practised.

Support for pluralism could be based on a prior conviction; but equally it could be based
on epistemological grounds: for example, if the research group adopts a fallibilist
position. Fallibilism is the view that all theories, views, models, etc. are inherently
fallible and that no grounds exist for judging any view completely correct. Often, it is
recognised that no single theory, or more often, one single investigator, has the
computational capacity to deal with the myriad facts in a complex environment. Equally,
such fallibilism may be simply the product of a pragmatic evaluation of experience: if
one’s models – whatever their formulation – have tended in the past always to be
incorrect, the judgement is made that they are likely to be flawed in the future.

Finally, there might be ontological grounds for triangulation, some of which are
suggested above. If objects are complex, it is unlikely that any single datum or observer
can describe them adequately. Therefore, there needs to be several observers/observations
in different locations and/or times, so that a more complete picture can be constructed.
Elsewhere (Downward and Mearman, 2005) we argue that in order to grasp the complex
empirical reality and the (also complex) deeper structures of reality, different empirical
methods are necessary. Also, I argue (2004) that in so-called open systems of reality, in
which the degrees of closure of reality and the methods chosen to examine them are
unlikely to match, triangulation can attempt to compensate for this mis-match by
combining insights from various perspectives.

3. TRIANGULATION AT THE BANK OF ENGLAND

3.1 The Monetary Policy Committee process

The processes under investigation are those surrounding the meetings of the Monetary
Policy Committee (MPC). The Bank, aiming for transparency, has published an extensive
set of papers laying out the process of the MPC’s decision regarding interest rates (see,
for example, Whitley, 1997; Britton, Fisher and Whitley, 1998; Budd, 1998; Bean, 1998;

Page 6

Britton, Cutler and Wardlow, 1999; King, 1999; Kohn, 2000; Bank of England, 1999,
2000, 2003; Bean and Jenkinson, 2001; Pagan, 2003). The Bank’s publications show a
complex, iterative process involving many different models, methods, data types and
people, both MPC members and Bank staff. This paper is based on the interpretation of
the Bank’s publications.

The well-publicised and much-anticipated MPC monthly meetings are the end result of a
month-long (and longer for the production of the Inflation Report) process of data
collection, analysis, presentation and interpretation. The process involves Bank staff and
its Agents collecting and manipulating data to be presented to the MPC, which then
considers the information and makes its decision. The principal tool for decision-making
is the projection of inflation. Every quarter this projection generates the lengthy official
Inflation Report, but in other months, a projection is still required.

The processes by which the Report and the monthly projections are arrived at are rather
similar and their differences will not be considered here (cf. Britton, Fisher and Whitley,
1998; Bean and Jenkinson, 2001). They both are iterative processes, involving a series of
meetings, both with and apart from the MPC members, reflection on past projections,
reconsideration of the projection model, and an amendment of the models after
deliberation on relevant events (or data) from the relevant period. The process culminates
in the production of a numerical projection of inflation.

The final tool for use by the MPC is the “fan chart”. The fan chart is a probability
distribution of projections (Britton et al, 1998; Bank of England, 2000). For each estimate
of inflation (or GDP) which is produced, a probability weight is added, according to the
MPC’s assessment of it (Budd, 1998). The whole fan chart therefore plots the range of
outcomes considered possible by the MPC, together with their subjective assessment of
the likelihood of those outcomes.

3.2 Evidence of triangulation in the process

On a cursory examination, there appears to be considerable evidence of triangulation.
Often, the triangulation appears to be extensive; but on further analysis, it tends to be
fairly superficial, driven by pragmatic concerns. In contrast, for us, ‘deeper’ forms of
triangulation are those which triangulate between methodologies and which do so for
ontological reasons. The grid below summarises our findings. In each case, the motive for
triangulation is found in the rows; the type of triangulation is in the columns. Each box is
either empty, indicating no basis for a judgement; or it contains ‘yes’ or ‘no’. If there is
strong evidence for or against a type of triangulation undertaken for a specific motive, the
yes or no becomes capitalised5. No direct evidence at all was found relating to validity or
the ‘law of large numbers’ as a motive, so they have been deleted. However, in the light
of our other findings, it seems likely that both are relevant in informing the Bank’s use of
triangulation.

5 Clearly, the rigour of such assessments can be questioned. More formal methods, such as content analysis,
are available for assessing quantitatively textual data; however, any content analysis carried out here is
somewhat informal and a matter of judgement.

Page 7

Table 1: Summary of findings – motives for and types of triangulation at the Bank


Data Judgement InvestigatorTheoretical Methodological

Summary

Data
inadequacy

‘Political’
factors

Pragmatism

YES

yes





yes

YES





YES



no

yes

YES

yes

YES

yes

yes

YES

Pluralism





no

no

NO

NO

Epistemology



YES

yes

no

yes

YES

Ontology

yes

yes

no

no

no

no

Summary

YES

YES

YES

no

yes - within
no - between




3.2.1 Triangulation via judgement

We support Dow’s (2004) and Cobham’s (2003) opinion that judgement is an important
part of the Bank of England process. Indeed, the Bank’s own literature portrays the
generation of forecasts as a process of judgement working in tandem with formal
modelling. Whitley (1997: 165) cites approvingly Higgins’ comments on Bryant et al
(1988), that “a formal and quantified framework is an irreplaceable adjunct to the process
of policy thought”; i.e., thought has primacy. Moreover, the projections made by the
Bank’s models are subject to interrogation and interpretation by both Bank staff and the
MPC. Indeed, as the Bank (2004a: 188) notes, the inflation projections are always
ultimately the product of the MPC, not the models. Such an emphasis on judgement
inevitably harks back to the ‘wise men [sic]’ of the pre-independence period (1993-
1997).

The role of judgement introduces a distinctly human element to the setting of interest
rates. Budd (1998) argues that a decision-maker’s mood will always affect the
interpretation of a model. Thus, a MPC member’s decision about the interest rate will be

Page 8

complex. It will be informed by their interpretation of the models (and other data)
presented to them, and by other data they have seen, or by their own modelling done prior
to the meeting. Their decision will also be affected by their pre-suppositions about the
economy and about what their role is in it. Cobham (2003) shows that different
preferences among participants in meetings will affect the final decision taken. However,
also, their decision will be affected by their own feeling or intuition about the current
economic situation. For example, Cobham shows that some members of the MPC have
been instinctively more active than others. Some may have a preference for lower
inflation than others (although Cobham: 481 discounts the importance of possible
different preferences for stability). Some might be more concerned about doing what the
markets expect (see Bell-Kelton, 2005, for a lengthy discussion of this). Finally, some
might be feeling optimistic about the economy, while others will be pessimistic: Bell-
Kelton shows how within the US Federal Reserve’s Open Market (operations)
Committee (FOMC), two camps emerged, one optimistic ‘elves’, the other pessimistic
‘wolves’. All of this suggests a process in which psychology is significant.

3.2.2 Investigator triangulation

The effect of the MPC members using their judgement is multiplied by their interaction.
The MPC listens to the evidence presented to them; each member presents their
assessment of the evidence and their subsequent recommendations; and eventually they
vote. MPC meetings “explore all possible views on the cause and significance of recent
economic developments” (Budd: 1789). The final projection arrived at is therefore the
product of discussion and negotiation amongst the committee. The committee might not
reach a unanimous decision – very often it does not – but its decision can be said to be
“collegiate” (Whitley, 1997: 170); a “collective examination of forces shaping the
outlook to come to a conclusion that belongs to most of the [MPC]” (Kohn, 2000: 24-25).

Although the MPC’s decision is collective, clearly the internal dynamics of the
committee will affect that final decision. More research is required on that, and little is
revealed in the meetings’ minutes; but Bell-Kelton (2005) demonstrates very clearly that
strong personalities or perceived hierarchies of authority in a monetary policy committee
can be significant. The ultimate authority of Chairman Greenspan seems to prevail over
the FOMC. Similarly, in the FOMC, there seems to be some tension between ‘bankers’
and ‘economists’.

The composition of the MPC is similarly varied. the committee as being made up of
academic economists, professional bankers, and professional economists (all with some
degree of academic background). It is possible that such different backgrounds produce
different types of judgements. Cobham (2003: 485) provides some examples of this, for
instance in (Governor) Eddie George’s apparent aversion to interest rate reversals; Buiter
and Vickers, both academic economists, seemed more prone to make interest rate
changes than did ‘career’ bankers. However, Cobham denies any systematic tendencies
of types of MPC members.

Page 9

As Dunne (1991) and Smith (1994) note, economic modelling has traditionally been done
in teams. This tradition continues at the Bank. Bank staff are arranged into departments
(such as Monetary Analysis) responsible for specific areas of study and for the production
of specific data. One of the functions of these teams is to evaluate their models and their
performance in providing accurate projections. In consultation with the MPC, the staff
amend their models ad hoc, according to the models’ past performance (Budd, 1998;
Pagan, 2003). Thus, the interaction within and between teams is a form of triangulation,
and a possible source of tension in the process. Furthermore, the Bank’s agents present
reports from around the country, for the consideration of the Bank staff and MPC
members. Finally, the teams and the MPC discuss all of the data together: there is an
interaction between the groups (Bank, 2004a: 188). In short, there seems to be
considerable investigator triangulation at the Bank.

3.2.3 Theoretical triangulation

There are two main potential sources of theoretical triangulation. One is the fact that
every member of the Bank staff and the MPC will have (possibly subconscious)
theoretical presuppositions which they impose on the evidence they interpret. Second, the
Bank’s suite of models may display theoretical diversity.

There are several ways in which the first could occur. It is clearly possible that different
members of the MPC may have different theoretical backgrounds which they bring into
the process. As discussed above, Cobham (2003) shows that debates between economists
and bankers bring in different perspectives on the economy and the role of the Bank6.
However, in general, it is our view that although there have been small differences
between the perspectives of members, in the wider spectrum of perspectives available,
these differences are small. Second, there might be differences in outlook between
different Bank staff; however, we have found little evidence of this, which in any case
would be difficult to assess given the mainly technical nature of the research of Bank
staff.

The second source of theoretical triangulation would be through the use of multiple
models with different theoretical bases, or a theoretically pluralistic model. Smith (1994)
notes that there was a macro-modelling industry in the 1980s. Effectively, a competition
took place between different macro-models, usually based in different universities with
different traditions in economics. For example, Cambridge models were Keynesian,
while the Liverpool, London Business School and City University Business School
models were strictly monetarist or New Classical (Dunne, 1991; Wallis et al, 1986). The
Bank recognises that the theoretically distinct models of the 1970s and 1980s performed
poorly and that theoretical coherence (as found in the 1980s model) can conflict with

6 On the other hand, perceived differences in theoretical perspective tend to be accentuated more by
academic economists than others. This could work in two ways: one, academic economists could stress
differences, making them appear larger than they are, creating the impression of greater diversity than
actually exists; two, non-academic economists might downplay paradigm differences and present actually
quite different views as being quite similar, creating more apparent consensus than is actually present.
What actually happens is an empirical question.

Page 10

empirical coherence (the ability to provide good forecasts). Thus, a balance must be
struck and a slightly broader model was required (Britton et al, 1998; Pagan, 2003; Bank
of England, 2003). This is an enforced pluralism. Thus, main Bank model is more
pragmatically constructed and less easy to neatly categorise into, a narrow theoretical
camp than previously. The same observation can be made about some of the auxiliary
models. Nevertheless, it is apparent that in practice, there is in evidence little theoretical
diversity. Within the modelling process, there is scope for alternative assumptions to be
made, but the alterations tend to be of a fairly minor nature.

Arestis and Sawyer (2002) analysed the theoretical structure of the Medium-Term
Macroeconomic Model (MTMM), which was until recently the Bank’s main
macroeconometric model. According to Arestis and Sawyer, the MTMM had a number of
key features: long run equilibrium, with short run dynamics captured by ECMs (see also
Pagan, 2003); Cobb-Douglas production functions; vertical Phillips curve at the NAIRU;
sluggish adjustment of nominal and real variables; and significantly, money supply
endogeneity. The model is emblematic of the “new consensus” on money and
macroeconomics. Moreover, Sawyer (private) suggests that there have been shifts in
more subtle ways, for example in the movement from investment functions which were
more Kaleckian (emphasising profits as well as capacity utilisation) to neo-classical
(where investment depends on the price of capital as well as capacity utilisation)
formulations.

In response to Pagan’s (2003) criticisms, the Bank has replaced the MTMM with a new
Bank of England Quarterly Model (BEQM; see Bank, 2004a). As yet, no systematic
study to mirror Arestis and Sawyer’s (2002) has been carried out. However, a cursory
analysis of the model suggests that it is highly similar to the MTMM in its theoretical
orientation. It maintains a ‘new consensus’ approach, i.e., it is a New Keynesian-
orthodox hybrid emphasising optimising behaviour, steady state long run outcomes, a
vertical long-run Phillips curve, structural determinants of industrial competition, open
economy, balanced budgets, a simple monetary policy rule geared to an inflation target,
and wage and price inertia. Indeed, the Bank (2004a: 189) states explicitly that the new
model does not reflect or represent a desire by the Bank to change its view of the
structure of the economy. Thus, Arestis and Sawyer’s comments about the MTMM apply
almost without exception to the BEQM.

The only substantial difference between the BEQM and the MTMM is on expectations.
Expectations played a minimal role in the MTMM, although that might be for practical
reasons of data unavailability. Expectations were considered, for instance, in the
transmission mechanism from interest rates (Bank of England, 1999); however, they
mainly play a role in creating inertia in nominal and real variables (Arestis and Sawyer,
2002: 532) or (implicitly) as bringing about equilibrium. This deployment of expectations
had a very classical flavour to it. In contrast, for Keynesians, confidence plays a crucial
role, for example as a determinant of investment. The BEQM incorporates short term
expectations of demand to affect investment Bank of England (2004a: 189); however, the
main determinants of investment remain the cost of capital and expected return. There is
a greater and more sophisticated role for expectations in the BEQM than in the MTMM.

Page 11

However, the theory of expectations within that model is somewhat unclear. It is
acknowledged that agents have neither perfect foresight nor full information (2004a:191);
but the model falls short of rejecting rational expectations.

Overall, therefore, there is little evidence of theoretical triangulation. It seems that there
is a broad consensus among those involved behind this model as the main tool for
policymaking. Budd (1998) claims that alternative assumptions, when used, are deployed
to explore why forecasts have been inaccurate. Ideally, it seems, a single effective
paradigm, on which everyone agrees, would simplify the process of projection
considerably (Whitley, 1997; Pagan, 2003: 16). Whilst alternative paradigms might sneak
in – for instance via the forecasts of outside economists (Bean and Jenkinson, 2001),
which are used as a comparison for the Bank’s forecast – there is no commitment to
theoretical pluralism or to theoretical triangulation7.

3.2.4 Data triangulation

The Bank’s use of different data types is perhaps its clearest example of triangulation.
The main source of data for the MPC is that produced by the suite of models, principally,
a macro-model. That main model initially tries to create a current picture of the economy,
based on National Accounts data. However, this data is somewhat outdated, capturing
past not current conditions, given the lag in the collection and collation of the raw data
(Britton, et al, 1999; Bean and Jenkinson, 2001; Pagan, 2003). Thus, for the most current
information on existing conditions and trends, other data are required. Furthermore, the
Bank is concerned about measurement error and the consequent revision of incorrect past
data (see Harrison, Kapetanios and Yates, 2004; and Kapetanios and Yates, 2004, for
discussions of how to improve forecasts by taking into account measurement error in past
data). These concerns about data open the door to triangulation, albeit merely based on
the grounds of data deficiency.

At the series of meetings preceding the main meeting, the MPC undertakes a complete
reassessment of all the relevant evidence, and peruses data on, for example, labour
markets, monetary conditions, demand, output, prices, and financial markets. Much of it
is basically descriptive. Some of it might be termed “historical” (Bank, 2004b), whilst
other data is much more recent (Budd, 1998). The MPC has the opportunity to analyse
sectoral, regional and international data which the Bank deems relevant (Kohn, 2000;
Bank, 2004b). Much of this data is on emerging trends. This is data not covered in the
National Accounts. In particular, the Bank utilises independent non-governmental survey
data (see Britton, et al, 1999; Budd, 1998; Whitley, 1997). Such data might have been
collected at different times and places from the official data. Typically, the Bank uses
surveys on business (state of trade surveys: Britton, et al, 1999) and consumer confidence
and sentiment (Bean and Jenkinson, 2001). For example, the Bank employs the CBI
Industrial Trends survey, which is used to ascertain position of the economy in its cycle
(Britton, et al, 1999). The Michigan Consumer Sentiment survey can capture some of the
trends in consumer spending (Bank, 2004b).

7 The suite of models includes small ‘analytical’ models. These models are most commonly based on
optimising assumptions: more evidence of limited use of competing perspectives.

Page 12

Reports by the Bank’s staff utilise a wide range of sources, including press news reports,
which focus on current specific significant events. A recent past example is the
demutualization of the building societies. These events can affect the degree of
uncertainty of the Bank’s forecasts and can bias the fan chart (Budd, 1998; Britton, et al,
1999). These current events also assist the staff in choosing which data should be
presented to the MPC and thereby which issues should be discussed (Budd, 1998). These
presentations are supplemented by data collected from other organisations, such as
building societies, the Royal Institute of Chartered Surveyors (RICS), research institutes,
trades unions and economists from academic and commercial organisations (Bean and
Jenkinson, 2001). These data sources show other examples of triangulation of opinions
and people.

Furthermore, the data from different organisations are often of different types: the RICS
data tends to be on recent house price data, and is often based on recent surveys by RICS
members; building societies draw on recent mortgage completions. On the other hand,
data from other economists is of a more conventional type, often being competing
forecasts with which the Bank’s forecasts are compared. Particular attention is paid to
forecasts and other data from other central banks (Bean and Jenkinson, 2001). Also, the
data might have been collected in different ways from the official data.

A similar role in the decision-making process is played by reports from the Bank’s
Agents around the UK. The Agents’ principle task (in this context) is to visit UK firms
(they make 7000 such visits each year: King, 1999: 10) to gather information. The Bank
values the information they collect in the same way as survey data: it is timely and fills
gaps which would otherwise exist (King, 1999). Indeed, because it tends to be more
anecdotal (Budd, 1998; Bank, 2004b), it is the most recent data at the Bank’s disposal on
current economic conditions. Firms can report to agents their stock levels, recent changes
in demand, their expectations of inflation and above all, their confidence about the
economy and their subsequent intentions for investment. Typically, in each MPC meeting
round, data from 150-200 agents’ reports are presented for consideration (King, 1999;
Bean and Jenkinson, 2001).

Obviously, the Bank does use several types of data from a number of different sources. It
does practice data triangulation. Clearly the main motives for doing so are pragmatic:
official data is often incomplete and/or inaccurate, and suffers from being always
backward-looking and lagged. Thus other data is required to correct for measurement
error and to create a more complete picture. How that data is treated is crucial for
assessing the extent of methodological triangulation.

3.2.5 Methodological triangulation

To reiterate, Denzin (1970) distinguishes between within-method and between-method
triangulation. Within-method triangulation is based on the premise that two trials of the
same test are better than one: it is an inductive exercise. Between-method triangulation
involves the mixing of different methods and is therefore, methodologically at least, more

Page 13

significant. However, between-method triangulation can also be sub-divided: one can
observe the mixing of methods which come from the same methodology (within-
methodology) or from different methodological foundations (between-methodologies).
That raises questions about how the methods are to be combined, and whether one type of
method has primacy. These questions are crucial in understanding the process of the
MPC.

The uncertainty over model specification, and over the reliability of one single model –
all pragmatic concerns about past poor performance, but also hints at fallibilism – have
led to an “eclectic” approach (Whitley, 1997), in which models are combined. The Bank
does not rely on one model: rather, it has a ‘suite of models’ approach. This is common
practice in central banks (Kohn, 2000; Pagan, 2003). The suite includes a main model,
such as the BEQM, which provides the ‘big picture’. The main model provides the initial
average projection of inflation, based on the average response of the model to average
shocks (Whitley, 1997). In addition, a set of as many as thirty-two ‘auxiliary’ (Whitley,
also uses the term ‘analytical’) models, such as a Real Business Cycle model, a labour
market model and others to model specific sectors, providing more specific sectoral or
regional information, add detail which allows the projections of the model to be adjusted.

For example, a small, five-equation macroeconomic forecasting model is employed
(Whitley, 1997), in order to supplement the main model. Other simple, stylised
macroeconomic models are used, to provide an overall picture of the economy. Simple
two equation output gap models are also used (Whitley, 1997). Moreover, a range of
model types is used, including time-varying component models, structural VARs,
Bayesian VARs and factor models (Whitley; Budd, 1998; Pagan, 2003). Pagan claims
that different models are used for different purposes. One such example is the use of
VARs for assigning the probabilities of shocks (Whitley). The smaller models are used to
fill the vacuum – partly in confidence – left by the large-scale models of the 1980s
(Whitley). Final estimates and forecasts would seem, therefore, to result from a
combination of inferences from these other models.

Overall, methodological triangulation seems considerable. A selection of models is used
from within the range of formal models (within-method triangulation). Within each type
a range of modelling techniques is utilised (within-method). A large number of auxiliary
models add considerable specific detail to the information set available to the MPC. From
above, the range of data types used indicates between-method triangulation. The surveys
used tend to be informed by conventional positivist principles; but the anecdotal evidence
presented by Agents is informed by interpretivist philosophy, suggesting between-
methodology triangulation. However, this interpretation must be tempered considerably,
because of the way the models are employed.

A crucial question regarding triangulation is how the triangulated data (or models, etc.)
are to be combined. There is no easy formula for this. In (between-)methodological
triangulation, no assumption is made of the inherent superiority of any methods. As
Downward and Mearman (2004, 2005) argue, the specifics of the question being asked
will determine the method chosen. The Bank has no stated formula for combining data

Page 14

types, so one must be inferred. The treatment of the models used by the Bank suggests
that a fairly clear hierarchy of models exists and that the projection process is geared
around those models at the top of the hierarchy. Specifically, the main macroeconomic
model is the driver of the process. This position is very clear in the Bank (2004a) in
which the BEQM is described as the “main tool in the suite of models” (188; emphasis
added) the Bank uses and is the “primary organisational framework” (188; emphasis
added) for assisting the judgements of the MPC. Other models are largely adjuncts to the
main model (Whitley, 1997).

As a corollary, the data type preferred by the Bank is a time series of official quantitative
data, collected in the usual way. Other data types are adjuncts to the preferred data. These
supplementary data would not, ideally, be used, but the lags and gaps in official data
necessitate a search for other, less reliable data. Overall, while the Bank would prefer to
use only regularly quantified official data, they are forced to take into account other data
types, partly because of data inadequacy and also for pragmatic reasons of poor past
performance. A few examples illustrate the point. The quarterly forecast is “explicitly
quantitative” (Bean and Jenkinson, 2001: 438) as is the fan chart. Admittedly, the initial
modal forecast from the model can be adjusted in the light of other information.
However, only information which will have a “quantitatively significant” effect on the
forecast is considered by the MPC (Bean and Jenkinson, 2001: 439). According to
Whitley (1997), the analytical models provide qualitative data for input into the other
(macro) models; this data will be transformed into a quantitative form or used as proxies
for unavailable data (Britton, et al, 1999). As Whitley notes, only quantifiable shocks can
be included in models. Indeed, this is considered necessary, for survey data to be put to
its “best use” (Britton et al: 179). Survey data is kept in a time series and compared with
other time series data (Britton et al). Quantification occurs via correlation and regression
with other quantitative data (Britton et al). Similarly, the CBI Business Optimism
Balance, a measure of business confidence, is regressed against lags of itself and other
variables (e.g. GDP). Thus, while the merits of surveys per se are acknowledged by the
Bank, in the end they are subsumed under the main, quantitative model. The survey data
have to be subordinated to the quantitative methods, which are apparently superior and
more powerful.

These statements suggest a clear hierarchy of data and models, with the quantitative
macro models at the top. In this light, Higgins’ comment on Bryant et al, quoted above,
that quantitative and formal analyses are an “irreplaceable adjunct to the process of
policy thought” (Whitley, 1997: 165) looks rather different. Rather, policy thought is
based around quantitative analysis; the thought almost looks like an adjunct to the
quantitative analysis, in spite of the many stresses of the role of judgement in Bank
literature. Such an approach is consistent with the way in which methods are used
throughout economics: certain methods have a higher power and intrinsically more value;
and therefore, studies conducted with those methods consequently also have a higher
value. However, the open-systems arguments underpinning triangulation suggest that this
is not the case: methods only have power if they are appropriate to their object.

3.3 Analysis of motives for triangulation at the Bank

Page 15

Whereas Cobham (2003) and Dow (2004) focus on the motives and underlying
conditions for types of decisions taken by the MPC, this paper has taken a twin approach
to the analysis of the MPC process. A taxonomic account of the types of methods used,
and how they are employed, is combined with an analysis of the motivations behind those
actions. This allows us to argue that although there is considerable evidence of
triangulation, particularly of some forms of it, significant questions can be asked about
why.

As Table 1 shows, our analysis finds clear evidence for certain types of motivations. It is
very clear that a concern about data inadequacy has led the Bank to employ several
different types of data from diverse sources. In addition, to the extent that it engages in
methodological triangulation, much of this occurs (indirectly) through the uses of
different types of data, such as anecdotes presented by Agents. This too is inspired by
concern about the adequacy of published official data. It does not seem driven by concern
about formal modelling, either in terms of its ability to provide forecasts (although this is
relevant later) or in terms of its applicability in open systems contexts.

There is some evidence that ‘political’ factors such as satisfying the stakeholders
involved, is relevant to the extent of investigator triangulation. However, it is less clear
how the internal politics of the MPC affect outcomes or affects the choice of methods
used. The choice of methods seems to be a product of organisational convention, self-
evaluation, observation of other comparable organisations, and independent evaluation
(for example the Kohn, 2000, and Pagan, 2003 reports). Further, there seems to be little
evidence of the overriding desire for pluralism per se. As argued above, little theoretical
diversity is present – indeed, the main macro- model seems to be moving away from
pluralism – and the innovations of different data types and auxiliary models is an ad hoc
response to practical problems.

Indeed, pragmatic responses to epistemological problems seem to be the main drivers of
the current approach. The Bank’s process evolves relative to the quality of past
performance. Different data types are necessary because of the existing current practical
limitations of official data. Judgement is necessary because the models cannot be relied
upon as yet to provide good enough forecasts on their own. The job of creating forecasts
is too large and too complex to rely on individuals. Past large-scale models, which relied
on one very narrow theoretical structure, have been unsuccessful in prediction and policy
application (Whitley, 1997). Smaller models which provide detailed information on
specific sectors augment the admittedly and inevitably limited main model. The Bank is
cognisant that all models are abstractions from the complex reality which cannot possibly
capture all the relevant features of the economy; consequently, they are careful not to rely
too heavily on models (Bean and Jenkinson, 2001; Bank of England, 2003)

Of course, such concerns are not merely practical. They are epistemological positions.
There is some evidence in the Bank literature of an awareness of the fallibility of models
and of theory. Whitley (1997) claims that the Bank is more cautious in its claims partly
because modellers in the 1980s contributed to the mistrust about models by making too