Content uploaded by Wouter Van Dooren
Author content
All content in this area was uploaded by Wouter Van Dooren on Jul 18, 2014
Content may be subject to copyright.
330 Int. J. Public Sector Performance Management, Vol. 1, No. 4, 2010
Copyright © 2010 Inderscience Enterprises Ltd.
The quality of local governance – ranking local
governments in Belgium
Steven Van Roosbroek*
Acaciastraat 43, 2800 Mechelen, Belgium
E-mail: Steven.vanroosbroek@vvsg.be
*Corresponding author
Wouter Van Dooren
Department of Political Science, Universiteit Antwerpen,
Sint Jacobsstraat 2, 2000 Antwerpen, Belgium
E-mail: Wouter.VanDooren@ua.ac.be
Abstract: Over the last years, there have been several research efforts to
measure the quality of governance of nations (e.g., World Bank’s Worldwide
Governance Indicators). These efforts have been criticised by several
academics. Less research has been conducted on indicators of the quality of
local governance. In this article, we investigate whether different indicators of
the quality of local governance in Flanders (Belgium) correlate with each other.
Since they all proclaim to measure the quality of governance, we expect strong
correlations. We identified 12 rankings based on eight data sources. This
resulted in 66 correlations, of which only 17 were significant in the expected
the direction. The vast majority of the correlations were insignificant. This
suggests policy makers and academics need to be very careful when
interpreting rankings of the quality of governance. Therefore, future rankings of
governance should be targeted at the micro level of public service delivery,
instead of the macro level of the public sector in general.
Keywords: quality; local governance; Belgium; Flanders; performance;
governance indicator; good governance; validity; reliability; World Bank;
ranking.
Reference to this paper should be made as follows: Van Roosbroek, S. and
Van Dooren, W. (2010) ‘The quality of local governance – ranking local
governments in Belgium’, Int. J. Public Sector Performance Management,
Vol. 1, No. 4, pp.330–345.
Biographical notes: Steven Van Roosbroek is an Auditor for Local
Governments in Flanders and a Former Researcher at the Public Management
Institute, Katholieke Universiteit Leuven, Belgium
Wouter Van Dooren is an Assistant Professor at the University of Antwerp,
Belgium and Co-Chair of the EGPA study group on public sector performance.
His research interest concerns performance and performance measurement in
government.
The quality of local governance – ranking local governments in Belgium 331
1 Introduction
Measurement of the quality of governance has become increasingly popular. Research
has particularly focussed on the governance quality of nations (Arndt and Oman, 2006).
Local government on the contrary has received considerably less attention in the
international research community (Padovani and Scorsone, 2009). Nonetheless, national
initiatives to measure and compare the quality of local governments do exist and may
even provide useful insights for governance rankings in general. Bouckaert (1993) shows
that measurement systems need to be valid, legitimate and functional. Only if these three
conditions are met, a performance measurement is ‘fit for use’. This article studies the
indicators of the quality of local governance (QoLG) in the case of Belgium. We limit the
scope of our research to the validity of these indicators empirically and discuss the
implications for rankings of national governance quality. We will demonstrate that
indicators that proclaim to investigate the QoLG do not correlate, as one might expect
based on the claims of these studies.
This article consists of five parts. First, we examine the concept of ‘quality of
governance’. Subsequently, we will discuss the methods used to measure the QoLG. In
the next paragraph, we outline the methodology of our empirical study on the validity of
governance indicators. The results are presented and discussed in the fourth paragraph. In
the conclusion we discuss the implications of our findings.
2 Conceptual framework: what is quality of governance?
An important drawback of existing empirical studies on the quality of governance is the
lack of conceptual clarity on what is meant by the ‘quality of governance’. We will first
discuss the concept of ‘governance’, after which we will look into the concept of
‘quality’.
Hirst (2000) identifies five uses of the governance concept. Governance as seen by
development economists is linked to the creation of stable and effective institutions that
are important for private economic action. In the field of international affairs, the concept
refers to issues that cannot be dealt with at the national level. The third use of governance
is part of the new public management movement. Privatisation policy and private sector
management steering strategies resulted in a new model of public services. Fourthly,
governance sometimes refers to “new practices of coordinating activities through
networks, partnerships and deliberative forums”. Lastly, the term governance is also
used in the sense of ‘corporate governance’ in the private sector. In the field of
governance indicators, which is the focus of this article, the term governance is mainly
used in the first, third and fourth category outlined by Hirst (2000). These are the
categories this article will focus upon as well.
The extensive variety of uses in a variety of disciplines obviously leads to differences
in the definition of governance. However, there is a common denominator. The World
Bank papers on governance indicators are probably the most widely cited empirical
studies on governance. Kaufmann et al. (2003) define governance as “the traditions and
institutions by which authority in a country is exercised”. Other contributions roughly
332 S. Van Roosbroek and W. Van Dooren
follow this definition
1
, or differ in minor aspects, by i.e., naming the traditions or
institutions. One example is the definition of Lynn et al. (1999): “the term governance
may be defined as regimes of laws, administrative rules, judicial rulings, and practices
that constrain, prescribe, and enable government activity”. Most definitions of
governance refer to the societal capacity to give direction to society (Van Dooren, 2006).
The capacity to steer society is not the sole responsibility of political actors (Bouckaert et
al., 2002). Other actors (private and non-profit) also contribute. ‘Governance’ therefore is
not the same as ‘government’. Indeed, several authors have discussed the occurrence of
‘governance without government’ (Rosenau and Czempiel, 1992; Peters and Pierre,
1998).
While there seems to be some agreement on the basic tenets of the concept of
governance, more problems arise when the ‘quality’ of governance has to be defined.
Different authors emphasise different aspects of this concept. Rothstein and Teorell
(2008) focus on the importance of impartiality in their definition of ‘quality of
government’. Longo (2008) suggests that impartiality is a necessary, but not a sufficient
condition (see also Wilson, 2008), and proposes to include criteria like effectiveness and
efficiency. La Porta et al. (1999) on the contrary narrow their definition of ‘good
government’ to ‘good for economic development’. Other definitions include a variety of
issues like education, healthcare, a stable currency (Stowe, 1992).
In (public) management literature, the meaning of ‘quality’ has evolved through time.
The work of Löffler (2001) has been a major contribution to this field. She shows how
the management of quality has evolved from ‘quality inspection’ to ‘total quality
management’. The definition of quality moved from ‘conformance to specifications’ to
‘achieving maximum (internal and external) customer satisfaction’. Shifts in customer
expectations will result in a different interpretation of ‘quality’. Therefore, ‘quality’ is a
concept that has different meanings, and can be measured in different ways.
At the socio-political level, disagreement on what constitutes ‘good governance’ has
become part of the contemporary democratic process. Schumpeter (1994) compared the
contemporary doctrine of democracy to the doctrine of the 18th century. While the 18th
century philosophy argued that there is a ‘common good’, that is clear to all rational
people; the contemporary doctrine argues that there is “no such thing as a uniquely
determined common good that all people could agree on. (…) [T]o different individuals
and groups the common good is bound to mean different things”. In line with this
observation, Rothstein and Teorell (2008) argue that “if quality of government becomes a
way for experts to define what are to be understood as ‘sound policies’ (…), there is not
much left for political parties and politicians to decide on the representational side of the
democratic system”.
In conclusion, one might argue that there is no generally accepted definition of the
quality of (local) governance. ‘Quality’ means different things to different people, thus,
in the public sector; this gives rooms to political perspectives.
3 How is it measured?
There are different instruments to measure the QoLG. In this paragraph, we will discuss
the most commonly used methods (hard data, surveys, expert opinions and constructing a
composite indicator based on secondary data). None of the existing measurement systems
The quality of local governance – ranking local governments in Belgium 333
can claim to be the ultimate ‘one best way’ of assessing governance quality. All methods
have drawbacks and blind spots.
A first method of collecting data on the quality of governance is by looking at hard
(or objective) data. Examples of indicators of the quality of governance used by authors
include budgetary volatility, contract intensive money and telephone faults (Knack and
Kugler, 2002; Knack et al., 2003). While some defend the use of ‘objective’ hard data,
others are very critical of it. A lot of work needs to be done in standardising definitions.
International institutions, like the OECD or International Organisation for Standardisation
(ISO) play a pivotal role in this effort. However, even ‘objective’ data may be very
subjective, because they are “based on the assumption that benchmarkers have their
priorities right” (Bannister, 2007).
Sometimes surveys are used as indicators of the quality of governance. Respondents
are ordinary citizens, or a specific group of people (e.g., from the business community).
Respondents are asked to assess the quality of governance. As shown by Bouckaert and
Van de Walle (2003) using surveys as indicators of good governance is far from
self-evident. It is hard to assess what rankings of – i.e., – trust in government actually
mean: do they represent differences in culture, expectations, survey methodology or
‘actual’ performance?
Indicators on the quality of governance could also be collected through expert
assessments. Expert assessments can take on many different forms, such as a standardised
survey of experts, application of the Delphi method or conducting a mystery shopping
study (in which an expert, pretending to be a real customer, evaluates the service quality).
Experts are assumed to be neutral and independent in judging governance quality.
However, not everyone believes in this neutrality. According to Arndt and Oman (2006)
expert are influenced by other experts (‘herd behaviour’), similar third parties and sudden
events.
Existing efforts to measure the quality of governance use a variety of these three
methods. The World Bank’s Worldwide Governance Indicators (WGI) is a well known
example. These studies constitute by far the most widely used and best documented
measures of the quality of governance. More than 200 countries are ranked on six
dimensions
2
: voice and accountability, political stability and absence of violence,
government effectiveness, and regulatory quality, rule of law and control of corruption.
The authors collect secondary data, and combine these data in indicators for the different
criteria. The composite indicators (the six dimensions) use surveys, hard data and expert
assessments. Just like with other methods, several studies have been rather critical of
composite indicators. Small changes in the construction of these indicators can have large
effects on the results (see e.g., Lun et al., 2006).
Despite the lack of a clear conceptual framework and of a consensus about which
methods should be used, there have been several efforts to measure the quality of
governance by constructing a composite indicator of the quality of governance (Afonso et
al., 2005; Lopez-Claros et al., 2006; Kaufmann et al., 2007). These indicators are
constructed based on the assumption that the all indicators validly measure (aspects of) a
dimension of the quality of governance. For instance, the WGI use (amongst others)
expert assessments of the management of public debt, citizen evaluations of the
healthcare system, and hard data on government websites to construct a composite
indicator on government effectiveness. There are differences both in terms of the aspects
of governance that are measured, and the methods that are used.
334 S. Van Roosbroek and W. Van Dooren
We may be able to discover in an inductive way some key variables that tell us
something about quality of governance. Based on correlation analysis, we will investigate
the extent to which different indicators of the QoLG do indeed measure the same thing. If
this is the case, we expect high correlations between the different governance indicators.
Rather than deducing quality of governance from theory and conceptual unity, we look
for an ‘empirical truth’.
4 Method
The goal of this article is to assess the validity of indicators of the QoLG. To do this, we
have collected all rankings of the QoLG in Belgium that have been published over the
last decade. We found 12 rankings, based on eight data sources which allowed us to rank
anywhere between 70 and 308 of the local governments in Flanders (Belgium
3
). Table 1
lists the indicators of the QoLG used in our research. A more extensive description of the
indicators is available in Appendix.
Table 1 Indicators of the QoLG in Flanders
Research Year data
Method of data
collection
Focus
Number of
observations
Itinera Institute 2000–2006 Composite index
based on hard data
and
p
erce
p
tions
Efficiency of local
governments
307
De Borger and
Kerstens (2000)
1985 Composite index
based on hard data
Efficiency of local
governments
308
Geys (2006) 2000 Composite index
based on hard data
Efficiency of local
governments
300
General
socio-economic
survey 2001
2001 Survey Satisfaction with
the presence of
‘administrative
amenities’
308
The
Nieuwsblad’s
municipal
survey
2006 Survey Satisfaction with
local government
and local public
administration
306
Test-Aankoop 2000 and 2006 Mystery shopping Quality of
local public
administration from
the citizen
perspective, and the
quality of local
government
webs
it
es
70
Unizo 2006 Mystery shopping Quality of local
public
administration
from the
perspective of
SMEs
101
Indigov 2005 and 2006 Composite index
based on hard data
Quality of local
government
websites
304 and 308
The quality of local governance – ranking local governments in Belgium 335
Just like the WGI, our indicators measure different aspects of governance (efficiency,
citizen/business satisfaction and e-government), using different methods of data
collection (composite index based on hard data, survey and mystery shopping). Normally,
this need not be a problem, since all indicators attempt to measure (aspects of) the QoLG.
In some cases, indicators attempt to measure the same aspect of the QoLG, using the
same method of data collection. Therefore, we would expect moderate to high
correlations between these indicators of the QoLG. However, the more the indicators are
conceptually and methodologically different (e.g., e-government versus the quality of
public administration from the perspective of SMEs), the lower the correlations might be.
We will test this in the next section.
5 Results
Table 3 presents a correlation matrix of the QoLG indicators
4
. We have indicated the
variables that have the same focus (e.g., efficiency) with a bold frame. The variables that
use the same method are shaded in grey. Some variables were recoded, allowing us to
compare positive and negative correlations more easy. Thus, a positive correlation means
that a higher score on governance indicator A is linked with a higher score on governance
indicator B. Significant correlations are marked in the table. A general overview of the
significant and insignificant correlations is shown in Table 2.
Table 2 Overview of significant and insignificant correlations
Similar methods Dissimilar methods
Significant: 6 Significant: 0 Similar concepts
Not significant: 2 Not significant: 4
Significant: 2 Significant: 15, but 6 in the
opposite direction
Dissimilar concepts
Not significant: 11 Not significant: 26
First, we analyse indicators that have a similar focus and use the same method of data
collection. There are eight different correlations in this category, of which six were
significant (all in the expected direction).
• Not surprisingly, the highest correlations were obtained when comparing between
different editions of the same study. Examples are the high correlations between the
results of the assessment of government websites (Indigov, 2005, 2006) and between
the researches of De Borger and Kerstens (1985, 2000; both authors use similar
data).
• Despite the large time gap between the two moments of data collection, the
correlation between the studies of De Borger and Kerstens (1985 and 2000) (0.471)
is quite high. This is much higher than the correlation between Geys and the research
of the Itinera Institute. The data of these two studies were collected during the same
time frame, and the focus (efficiency of local governments) was the same. The
relevance of this finding should not be underestimated. Two studies (Itinera and
Geys) have attempted to measure the efficiency of local governments, partly using
336 S. Van Roosbroek and W. Van Dooren
the same indicators within the same time frame. Yet, correlation was rather low
(0.318), thus suggesting that some slight changes in constructing a composite
indicator have a large effect on the results.
• All correlations involving the research of Test-Aankoop were very low. It was
striking to see the relatively low correlation between the research of 2000 and 2006.
These two studies attempted to measure the same concept, using the same method,
though with slightly different mystery shopping scenarios. Nonetheless, the
correlation was insignificant. There can be two reasons to explain this. On the one
hand, it might be a consequence of large increases/decreases in the evolution of the
quality of different local public administrations. On the other hand, the low
correlation could be explained by possible weaknesses in Test-Aankoop’s
methodology.
Secondly, we look at indicators that have a dissimilar focus, but use the same method of
data collection. Only two of the 13 correlations in this category were significant. There
were no significant correlations between the efficiency indicators and the indicators of
the quality of the local government’s website. These indicators clearly have a different
focus, but use the same methodology (constructing a composite indicator based on hard
data). This finding suggests that it not just the method of data collection that determines
the results, but rather that there is no ‘quality of governance’ concept underlying local
government efficiency on the one hand, and sound e-government policy on the other.
Nonetheless, the analysis of e-government websites is used in other research on the
quality of governance, such as the World Bank’s WGI (West, 2006; Kaufmann et al.,
2007).
The correlation between the two questions in the survey of the ‘Nieuwsblad’
(2006/local public administration and 2006/local government) comes as no surprise, since
it asked for satisfaction with local administration on the one hand and local government
in general on the other. Research has shown that “a high level of trust in one institution
tends to extend to other institutions” (Christensen and Laegreid, 2005). Respondents who
have a high level of trust in ‘local government’ will be more inclined to trust the
‘municipal administration’ too. The correlation between the general socio-economic
survey and the survey of the ‘Nieuwsblad’ was significant as well, though to a lesser
extent.
Thirdly, there are correlations that have a similar focus, but use a different method of
data collection. One can hardly overestimate the importance of this category. One way to
increase the validity of research in social science is to apply different approaches to
investigate a research question. This allows for the researcher to draw more valid
conclusions. Methodological triangulation is the most common way to do this. If different
methods produce similar results, the conclusions can be considered as more valid. There
are four such correlations in Table 3 (bold frame, no shading). All of them are
insignificant. This suggests that the method which is used to assess the QoLG will have
an influence on the results that are obtained.
Lastly, we analyse the correlations between indicators that, even though they try to
measure aspects of the QoLG, have a dissimilar focus and use the different methods of
data collection. This category is important, because in other research (e.g., the World
The quality of local governance – ranking local governments in Belgium 337
Bank Governance Indicators) the implicit assumption is that the indicators that are used,
measure aspects of a similar phenomenon, being the quality of governance, even when
different methodologies are applied and different operational dimensions are studied. If
this is true, we could expect moderate correlations in this category.
There are 41 correlations in this category, of which only 15 were significant.
Moreover, six of these significant correlations were not in the expected direction. For
instance, six of the nine correlations between composite indicators of efficiency and
citizen survey data are significant, yet, only three in the expected direction. This finding
appears to confirm the weak link between actual performance of local government and
citizens’ perceptions of performance (Van de Walle, 2004; Van Roosbroek and Van de
Walle, 2008).
The highest correlations were obtained by the mystery shopping study on the quality
of public administration from the perspective of small and medium-sized businesses
(Unizo 2007). However, here as well some correlations were significant in the opposite
direction. Because of the large number of insignificant correlations (nearly two third in
this fourth category), it is difficult to sustain that all indicators measure aspects of an
underlying concept, being the quality of governance. It does confirm the
multi-dimensionality of the concept and may even point at the existence of trade-offs in
quality of governance.
We explore this multi-dimensionality further by including the role of the
municipalities’ size in our analysis. Size can be related to quality of governance in two
ways. Several empirical studies suggest that increasing the size of organisations has a
positive influence on efficiency (Van Dooren et al., 2007). Increased size could lead to
economies of scale and efficiency gains. In turn this will result in higher organisational
slack, which facilitates innovation (Damanpour, 1991). Larger municipalities could
therefore be linked to better performance. On the other hand, theoretical and empirical
studies on citizens’ trust present a different picture. Survey data from Denmark, the
Netherlands, Norway and the UK find that trust in local government is higher in small
municipalities (Denters, 2002). A multilevel analysis of trust in local government in 55
US cities concluded that population size is inversely related to trust in local government
(Rahn and Rudolph, 2005). Thus, assessments of the quality of governance by citizens
could come to different conclusions than assessments based on performance data.
Local governments in Flanders are very diverse, ranging from less than 100
inhabitants to more than 460,000. It can be hypothesised that size has an effect on the
(evaluations of the) QoLG. In our research, there were high, positive correlations
between size
5
and the studies of Indigov and Unizo (between 0.453 and 0.548).
Correlations between size and the research of the Nieuwsblad were negative. This seems
to be consistent with international research mentioned earlier.
We calculated partial correlations of all indicators of the QoLG, controlling for
population size. Some of the negative correlations in Table 3 can be explained by taking
population size into account. In Table 3, three of the four correlations between the studies
of the Nieuwsblad and Indigov were not significant. The remaining correlation was
significant in the opposite direction than we expected. After controlling for population
size, three of the four correlations were significant in the positive direction, while the
remaining correlation was insignificant.
338 S. Van Roosbroek and W. Van Dooren
Table 3 Correlation matrix of rankings of the QoLG
The quality of local governance – ranking local governments in Belgium 339
Similarly, there was a negative correlation between Unizo’s mystery shopping study and
citizens’ appreciation of local public administration (the Nieuwsblad). Looking at partial
correlations allows us redefine this relation. After we controlled for population size, this
negative relation disappeared. These examples show that quality of governance is a
multi-dimensional concept, and that it is important to take contextual variables into
account when analysing governance indicators. Different values for contextual variables
like size may result in higher scores in indicator A, but lower scores on indicator B.
We will not examine all correlations and partial correlations, since the conclusion is
quite clear. The strength of the correlations between different measurements of the QoLG
is limited at best. Only if indicators have a similar focus and use the same method of data
collection correlations are moderate. In all other cases, the link between two indicators of
the QoLG is hard to predict, and could even be significant in the opposite direction. It
appears that we are not measuring one underlying concept of the QoLG.
6 Conclusions
Quality of governance is in essence a social and political choice. A universally accepted
definition is therefore unlikely to emerge. This problem has not refrained international
organisations from attempting to measure the quality of governance. These measurement
efforts are mainly aimed at the comparison of nations, and not at other government levels.
Our research attempts to fill this gap by focusing on measurements of the QoLG.
Empirically analysing the QoLG could also tell us something about the validity of
comparative governance indicators at the national level. For instance, it is hardly
surprising that the World Bank studies conclude what we already know, i.e., that
corruption in Turkey is higher than in Norway, but lower compared to the Democratic
Republic of Congo. The real challenge lies in being able to distinguish between countries
that are more comparable, like Germany and the UK, or the Scandinavian countries. The
failures and the successes at the extremes are generally self-evident (Besançon, 2003). In
this sense our findings among local governments in Flanders (within a similar cultural
and institutional context) are also relevant for indicators at the (inter)national level.
Several researchers have attempted to measure the QoLG in Flanders (Belgium).
Academics used the indicators to investigate the reasons for high or low QoLG. On the
practitioners’ side, the indicators were discussed in the media, and used as a basis for
management reforms in local governments. If these indicators are both reliable and valid,
we could expect relatively high correlations. However, our analysis has shown that only
17 of the 66 correlations (26%) between the different indicators are significant in the
expected direction. The strength of these correlations was moderate at best. Six
correlations were significant, but in the opposite direction. The vast majority of the
correlations were insignificant. For studies that claim to measure (aspects of) the same
concept, these results are disappointing.
Two explanations are plausible. First, the method could determine the result. In this
case, the low correlations are a ‘measurement problem’. Secondly, the low correlations
could be the result of a conceptual problem, i.e., the lack of conceptual clarity (Van de
Walle, 2008). In this case, there is evidence for both explanations. The fact that the two
mystery shopping studies of Test-Aankoop (2000 and 2006) did not correlate, indicates
that there certainly is room for improving existing measurement systems. In other cases
as well, slight differences in methodology resulted in rather low correlations. These
340 S. Van Roosbroek and W. Van Dooren
findings show, in line with other research, that results of local governance indicators are
highly dependent upon the techniques that are used (Worthington, 2000; Worthington and
Dollery, 2002).
However, we also need to look at the concept of the QoLG. Indicators using the same
method, but with a different focus, did not correlate to the extent one could expect. Since
the method alone does not explain the low number of significant correlations, we have to
look at the underlying concepts of the measurement efforts. Under the banner of QoLG,
different dimensions are measured, ranging from e-government to efficiency of the
municipality. Even indicators that attempt to measure the QoLG as a whole, in fact only
measure a limited aspect of the broad notion of quality. This leads to validity problems. It
appears to be infeasible to measure every aspect of local governance. The concept of the
quality of governance appears to be too broad for operational research.
What needs to be done in future research efforts? First, researchers need to be careful
when constructing composite indicators. One way to ‘solve’ the low correlations between
different governance indicators has been to construct a composite indicator. This may
prove to be ineffective, since -as our study has shown- there is no clear underlying
concept.
Secondly, both policy makers and academics need to be careful in using the rankings
of the quality of governance (Van Dooren and Van de Walle, 2008). These rankings often
receive a lot of attention in the media, and are used in political discourse. If the results are
dependent upon the indicator’s focus and the method that is used, they should be
interpreted with great care. For academics, our findings also have a profound impact for
analyses of the determinants and consequences of high or low quality of governance, or
for the analysis of public management reforms. If we had valid and reliable indicators, we
would be able to give clear answers to questions like ‘what actions should governments
take to increase the quality of governance?’, ‘which reforms should be implemented?’
and ‘how will increased quality of governance influence citizens’ welfare?’. However,
this is difficult because of the role of trade offs between ‘structural variables’ (variables
that cannot be influenced by government like size, as we have shown above), and the
multi-dimensional nature of the concept of the quality of governance.
Finally, this research provides empirical evidence for a growing consensus among
public administration scholars and practitioners on the issue of measuring the quality of
governance. Van de Walle (2008) argues against measuring the quality of governance in
general. Rather, indicators should focus on parts of government that are less broad and
vague than the ‘public sector’. These indicators would also be more useful from a policy
perspective. He argues that “measurement can (…) only work within a well-defined time
frame and in a context where policy learning has created a working agreement on
government objectives”. It is unrealistic to expect such an agreement on government
objectives.
Unlike the studies of the World Economic Forum, the European Central Bank and the
World Bank, the OECD has decided not to follow the old logic of measuring the
performance of the entire public sector by providing a ranking on the basis of ‘overall
government performance’. Their project, ‘Government at a glance’, attempts to go
beyond broad, existing measures that have little relevance for public management
(OECD, 2006). The OECD will “provide a set of key indicators of good government and
efficient public services to help member countries to better assess, plan and measure
their public sector reform agenda”
6
. Measurement systems should not only be reliable
and valid (which was the focus of this article), but also legitimate and functional
The quality of local governance – ranking local governments in Belgium 341
(Bouckaert, 1993). The approach followed by the OECD could result in indicators that
are not only more valid and reliable, but also more useful for the organisations that are
assessed. This may lead to an increased acceptation of measurement systems
(legitimacy). Our contribution has provided more evidence that investigating specific
parts of the public sector could provide more useful information than the construction of
one composite indicator for the quality of (local) governance.
References
Abdellatif, A.M. (2003) ‘Good governance and its relationship to democracy and economic
development’, Global Forum III on Fighting Corruption and Safeguarding Integrity, 20–31
May, Seoul.
Afonso, A., Schuknecht, L. and Tanzi, V. (2005) ‘Public sector efficiency: an international
comparison’, Public Choice, Vol. 123, Nos. 3–4, pp.321–347.
Arndt, C. and Oman, C. (2006) Uses and Abuses of Governance Indicators, Organisation for
Economic Co-operation and Development.
Bannister, F. (2007) ‘The curse of the benchmark: an assessment of the validity and value of
e-government comparisons’, International Review of Administrative Sciences, Vol. 73, No. 2,
pp.171–188.
Besançon, M. (2003) Good Governance Rankings: The Art of Measurement, World Peace
Foundation, Cambridge, Massachusetts.
Bouckaert, G. (1993) ‘Measurement and meaningful management’, Public Productivity and
Management Review, Vol. 17, No. 1, pp.31–43.
Bouckaert, G. and Van de Walle, S. (2003) ‘Comparing measures of citizen trust and user
satisfaction as indicators of ‘good governance’: difficulties in linking trust and satisfaction
indicators’, International Review of Administrative Sciences, Vol. 69, No. 3, pp.329–344.
Bouckaert, G., Van Dooren, W., Verschuere, B., Voets, J. and Wayenberg, E. (2002) ‘Trajectories
for modernizing local governance: revisiting the Flanders case’, Public Management Review,
Vol. 4, No. 3, pp.309–342.
Christensen, T. and Laegreid, P. (2005) ‘Trust in government: the relative importance of service
satisfaction, political factors, and demographics’, Public Performance and Management
Review, Vol. 28, No. 4, pp.487–511.
Damanpour, F. (1991) ‘Organizational innovation: a meta-analysis of effects of determinants and
moderators’, The Academy of Management Journal, Vol. 34, No. 3, pp.555–590.
De Borger, B. and Kerstens, K. (2000) ‘What is known about municipal efficiency? The Belgian
case and beyond’, in Blank, J.L.T. (Ed.): Public Provision and Performance, pp.299–330,
Elsevier Science, Amsterdam.
Denters, B. (2002) ‘Size and political trust: evidence from Denmark, the Netherlands, Norway, and
the United Kingdom’, Environment and Planning C: Government and Policy, Vol. 20, No. 6,
pp.793–812.
Geys, B. (2006) ‘Looking across borders: a test of spatial policy interdependence using local
government efficiency ratings’, Journal of Urban Economics, Vol. 60, No. 3, pp.443–462.
Hirst, P. (2000) ‘Democracy and governance’, in Pierre, J. (Ed.): Debating Governance, pp.13–35,
Oxford University Press, Oxford.
Indigov (2005) 10 Jaar Later: Lokaal eGovernment nog Steeds Onderbenut: Resultaten van een
Doorlichting van de Stedelijke en Gemeentelijke Websites in Vlaanderen en Brussel, Indigov,
Leuven.
Indigov (2006) Indigov Monitor Lokale Besturen 2006. Doorlichting van de Stedelijke en
Gemeentelijke Websites in Vlaanderen in Brussel, Indigov, Leuven.
342 S. Van Roosbroek and W. Van Dooren
Itinera Institute (2006) ‘Het palmares van het lokale beleid in Vlaanderen en Wallonië’, Itinera
Memo.
Kaufmann, D., Kraay, A. and Mastruzzi, M. (2003) ‘Governance matters III: governance Indicators
for 1996–2002’, World Bank.
Kaufmann, D., Kraay, A. and Mastruzzi, M. (2007) ‘Governance matters VI: aggregate and
individual governance indicators 1996–2006’, World Bank Policy Research Working Paper
No. 4280.
Knack, S. and Kugler, M. (2002) Constructing an Index of Objective Indicators of Good
Governance, The World Bank, Washington, DC.
Knack, S., Kugler, M. and Manning, N. (2003) ‘Second-generation governance indicators’,
International Review of Administrative Sciences, Vol. 69, No. 3, pp.345–364.
La Porta, R., Lopez-de-Silanes, F., Shleifer, A. and Vishny, R. (1999) ‘The quality of government’,
Journal of Law, Economics, and Organization, Vol. 15, No. 1, pp.222–279.
Löffler, E. (2001) ‘Defining and measuring quality in public administration’, Teaching and
Research Review, pp.1–21.
Longo, F. (2008) ‘Quality of governance: impartiality is not enough’, Governance, Vol. 21, No. 2,
pp.191–196.
Lopez-Claros, A., Porter, M.E., Sali-i-Martin, X. and Schwab, K. (2006) The Global
Competitiveness Report 2006–2007, Palgrave Macmillan, Basingstoke.
Lun, G., Holzer, D., Tappeiner, G. and Tappeiner, U. (2006) ‘The stability of rankings derived
from composite indicators: analysis of the ‘IL sole 24 ore’ quality of life report’, Social
Indicators Research, Vol. 77, No. 2, pp.307–331.
Lynn, L.E., Heinrich, C.J. and Hill, C.J. (1999) Studying Governance and Public Management:
Challenges and Prospects, Irving B, Harris Graduate School of Public Policy Studies.
OECD (2006) ‘OECD project on management in government: progress report’, Paper presented at
the 34th Session of the Public Governance Committee, 30–31 October, Château de la Muette.
Padovani, E. and Scorsone, E. (2009). ‘Comparing local governments’ performance internationally:
a mission impossible?’, International Review of Administrative Sciences, Vol. 75, No. 2.
Peters, B.G. and Pierre, J. (1998) ‘Governance without government? Rethinking public
administration’, Journal of Public Administration Research and Theory, Vol. 8, No. 2,
pp.223–243.
Rahn, W.M. and Rudolph, T.J. (2005) ‘A tale of political trust in American cities’, Public Opinion
Quarterly, Vol. 69, No. 4, pp.530–560.
Rosenau, J.N. and Czempiel, E.O. (1992) Governance Without Government: Order and Change in
World Politics, Cambridge University Press.
Rothstein, B. and Teorell, J. (2008) ‘What is quality of government? A theory of impartial
government institutions’, Governance, Vol. 21, No. 2, pp.165–190.
Schumpeter, J.A. (1994) Capitalism, Socialism and Democracy, Routledge New York.
Stowe, K. (1992) ‘Good piano won’t play bad music: administrative reform and good governance’,
Public Administration, Vol. 70, No. 3, pp.387–394.
Unizo (2007) De Gemeentelijke Dienstverlening Voor Ondernemers Doorgelicht in 101 Vlaamse
Gemeenten: Algemeen Resultatenrapport, Unizo, Brussel.
Van de Walle, S. (2004) Perceptions of Administrative Performance: The Key to Trust in
Government?, Instituut voor de Overheid, Leuven.
Van de Walle, S. (2008) ‘International comparisons of public sector performance: how to move
ahead?’, Public Management Review, Vol. 10, No. 6, pp.39–56.
Van Dooren, W. (2006) Performance Measurement in the Flemish Public Sector: A Supply and
Demand Approach, Faculty of social sciences, Leuven.
Van Dooren, W. and Van de Walle, S. (2008) Performance Information in the Public Sector: How
it is Used, Basingstoke, Palgrave.
The quality of local governance – ranking local governments in Belgium 343
Van Dooren, W., Lonti, Z., Sterck, M. and Bouckaert, G. (2007) The institutional drivers of
efficiency – OECD technical paper, OECD, Paris.
Van Roosbroek, S. and Van de Walle, S. (2008) ‘The relationship between ombudsman,
government and citizens: a survey analysis’, Negotiation Journal, Vol. 24, No. 3, pp.287–302.
West, D.M. (2006) Global E-Government 2006, Center for Public Policy, Providence.
Wilson, G. (2008) ‘The quality of government’, Governance, Vol. 21, No. 2, pp.197–200.
Worthington, A.C. (2000) ‘Cost efficiency in Australian local government: a comparative analysis
of mathematical programming and econometrical approaches’, Financial Accountability and
Management, Vol. 16, No. 3, pp.201–223.
Worthington, A.C. and Dollery, B.E. (2002) ‘Incorporating contextual information in public sector
efficiency analyses: a comparative study of NSW local government’, Applied Economics,
Vol. 34, No. 4, pp.453–464.
Notes
1 For an overview, see Abdellatif (2003).
2 www.govindicators.org.
3 Because of differences in cultural and institutional context, and for pragmatic reasons (several
indicators are available only for Flemish local governments) we will only investigate
indicators on local governments in the Flemish region. Flanders is the Dutch speaking
northern part of Belgium, with over 6 million of Belgium’s 10.5 million inhabitants.
4 We have also investigated the data using factor analysis, but this did not yield new insights.
5 To reduce the effect of a limited number of large cities we used the natural logarithm of the
number of inhabitants.
6 http://www.oecd.org/document/12/0,3343,fr_2649_33735_37688524_1_1_1_1,00.html.
7 Since this is an ordinal variable, we have used Spearman correlations.
8 http://www.statbel.fgov.be/CENSUS/home_en.asp.
Appendix
Itinera Institute
The Itinera Institute (2006) is an independent and non-partisan Belgian think-tank. It has
created an index of local government efficiency. This index is constructed using five
groups of indicators: financial policy, mobility, education, social and medical services,
and the environment. Each group of indicators uses both hard data as well as survey data.
De Borger and Kerstens (2000) and Geys (2006)
The study of De Borger and Kerstens (2000) is the oldest index in our research. They
analyse five indicators based on hard data: the number of subsistence grants beneficiaries,
the number of students in local primary schools, the surface of public recreational
facilities (in hectare), the total population and the population above 65. The index of
Geys (2006) is related to the index of De Borger and Kerstens (2000). The first three
variables are similar. The fourth variable used by Geys is the total length of municipal
roads (in km). Both studies relate the indicators to the expenditures of local governments,
thus constructing a measure of efficiency. The main difference between both studies is
344 S. Van Roosbroek and W. Van Dooren
the time of data collection: 1985 for the study of De Borger and Kerstens (2000), 2,000
for the Geys study.
General socio-economic survey (GES) 2001
8
The general socio-economic survey (GES) 2001 is the successor of the Belgian
population census. The classic censuses in Belgium have shifted to a general
socio-economic survey. A wide range of questions are included in the survey, such as
citizens’ appreciation of the local environment and ‘administrative amenities’. We use
this last item in our comparative study. There are a number of reasons why this question
is not a perfect indicator. First of all, it is unclear what is meant by ‘administrative
amenities’. Citizens might think of the municipal administration, but also about other
levels of public administration. Secondly, the item measures the provision of
administrative amenities in the neighbourhood, while our research is at the municipal
level.
The Nieuwsblad’s municipal survey
The ‘Nieuwsblad’ is the second most popular newspaper in Flanders. A few months
before the local elections of October 2006 it conducted an internet survey with more than
100,000 respondents. Because of the size of the sample, it was possible to analyse results
at the municipal level. We used two questions for our research: citizen satisfaction with
government, and satisfaction with municipal administration. An important drawback of
this study is its unrepresentative nature. It is likely that readers of the newspaper and
internet users are overrepresented. One might however argue that the skewed sample will
be similar in all municipalities, and that therefore, comparing between municipalities is
possible.
Test-Aankoop
Test-Aankoop is an organisation protecting consumer rights. Based on mystery shopping,
they evaluate the quality of the municipal administration. Their researchers confront the
administration with a range of problems, and assess how well it performs in answering
the mystery shopper’s requests. The quality of local administration was measured like
this in 2000 and 2006. In this last research, the quality of the local government’s website
was ranked as well. This will also be included in our analysis.
Unizo
Unizo is the ‘Organisation for the Self-Employed and SMEs’. Just like Test-Aankoop this
organisation uses mystery shopping research to assess the municipal administration’s
quality. The main difference is its focus, which lies not on citizens, but on small and
medium-sized businesses (SME). Four things were assessed: the municipal government’s
website, its reaction on the mystery shopper’s e-mail, a personal visit and two contacts by
phone. Just like in the case of Test-Aankoop, this research was performed using
standardised scenarios. This time however, the scenarios were representative for
problems of SMEs (Unizo, 2007).
The quality of local governance – ranking local governments in Belgium 345
Indigov
The last ranking we analysed, concerns the quality of e-governance. Indigov is a
consulting agency that annually analyses the quality of local government websites
(Indigov, 2005, 2006). Researchers assess the use of things like e-democracy, electronic
services and the occurrence of information on these websites. Different indicators are
combined in one composite index.
