ArticlePDF Available

Measuring potential sub-unit efficiency to counter the aggregation bias in benchmarking

Authors:

Abstract and Figures

The paper deals with benchmarking cases where highly aggregated decision making units are part of the data set. It is shown that these units – consisting of sub-units which are not further known by the evaluator – are likely to receive an unjustifiable harsh evaluation, here referred to as aggregation bias. To counter this bias, we present an approach which allows to calculate the potential sub-unit efficiency of a decision making unit by taking into account the possible impact of its sub-units' aggregation without having disaggregated sub-unit data. Based on data envelopment analysis, the approach is operationalized in several ways. Finally, we apply our method to the benchmarking model actually used by the Brazilian Electricity Regulator to measure the cost efficiency of the Brazilian distribution system operators. For this case, our results reveal that the potential effect of the aggregation bias on the operators' efficiency scores is enormous.
Content may be subject to copyright.
1
Measuring potential sub-unit efficiency to counter the
aggregation bias in benchmarking
Heinz Ahna, Peter Bogetoftb and Ana Lopesc
a Corresponding Author. Technische Universität Braunschweig, Institute of Management Control and Business Ac-
counting, Fallersleber-Tor-Wall 23, D-83100 Braunschweig, Germany. Phone: +49 531 391-3610. Email:
hw.ahn@tu-bs.de.
b Copenhagen Business School, Department of Economics, Porcelaenshaven 16 A, DK-2000 Frederiksberg,
Denmark. Phone: +45 23326495. Email: pb.eco@cbs.dk.
c Universidade Federal de Minas Gerais, Av. Antonio Carlos, 6627, 31270-901, Pampulha Belo Horizonte, Minas
Gerais, Brazil. Phone: +55 31 99212-7972. Email: analopes.ufmg@gmail.com.
Abstract: The paper deals with benchmarking cases where highly aggregated decision making units
are part of the data set. It is shown that these units – consisting of sub-units which are not further
known by the evaluator – are likely to receive an unjustifiable harsh evaluation, here referred to as
aggregation bias. To counter this bias, we present an approach which allows to calculate the poten-
tial sub-unit efficiency of a decision making unit by taking into account the possible impact of its
sub-units’ aggregation without having disaggregated sub-unit data. Based on data envelopment
analysis, the approach is operationalized in several ways. Finally, we apply our method to the
benchmarking model actually used by the Brazilian Electricity Regulator to measure the cost effi-
ciency of the Brazilian distribution system operators. For this case, our results reveal that the po-
tential effect of the aggregation bias on the operators’ efficiency scores is enormous.
Keywords: Benchmarking; Data envelopment analysis; DEA; Aggregation bias; Potential sub-unit
efficiency; Regulation
2
1. Introduction
In efficiency analysis, it is customary to work with aggregated production data. Not only non-
homogenous inputs and outputs are aggregated into a few input and output categories, but also
spatially distributed production entities as well as temporally consecutive production processes are
aggregated, often into only one overall, consolidated unit to be evaluated. In the terminology of
data envelopment analysis (DEA) which is addressed in the paper, this means that rows (variables)
as well as columns (individual production entities/processes) are aggregated before the actual eval-
uation of the consolidated unit. While the latter is usually referred to as decision making unit
(DMU), its individual production entities and production processes, respectively, are hereinafter
called sub-units.
The above-mentioned aggregations affect the evaluation. We can usually identify more ineffi-
ciency the more we aggregate. However, especially the aspect of sub-unit aggregations has not
received much attention in the efficiency analysis literature. Little is known about the magnitude
of the aggregation impact, and little is known about how this impact can be measured. The present
paper wants to shed some light on this issue, stressing particularly the possible bias resulting from
the evaluation of consolidated DMUs.
This bias occurs under the usual DEA assumption of convexity of the production possibility set.
Here, a DMU can improve its overall profit by adjusting production to variations in prices over
space and time, while its technical efficiency based on physical production data will deteriorate
because a convex technology favors producing the average output using the average input. In other
words, if a consolidated DMU operates several sub-units that serve different areas with different
needs and characteristics, the DMU may easily appear inefficient even though it is in fact operating
3
optimally in the different areas. Such a shortcoming, subsequently referred to as aggregation bias,
analogously applies for different time spans of a period which are to be served differently by a
DMU.
A series of plausible examples can be found which are prone to the aggregation bias, ranging
from special area-related cases like the one of German savings banks ,whose local branches
have to cover an assigned region, no matter how diverse the areas of this region may be (Ahn
and Le, 2015),
to the general case of companies which are present on diverse markets, e.g. worldwide with
corresponding international production sites, as well as
from special time-related cases like the one of Taiwanese design consultancies, for which it has
been shown that it is advantageous to match their the time-based business strategy with the
actual conditions of the market environment (Sung et al., 2010),
to the general case of companies which have to adjust their production to seasonally fluctuations
in demand, e.g. in the travel industry.
As a further example where the aggregation bias matters, we will emphasize the regulatory context
in the paper. On one site, firms are regulated because they benefit from a regional monopoly. On
the other site, these firms have to deal with the specifics of their particular market region. Such a
region may be characterized by a significant heterogeneity of its individual service areas, e.g., due
to variations of the production environment and customer structure. Consequently, a firm affected
by this scenario (as DMU) will develop different strategies for its different service areas (as sub-
units) to maximize its overall profit. Due to the described aggregation bias, however, such a DMU
is likely to receive an unjustifiable harsh evaluation by the regulatory authority which does not take
into account the sub-unit structure.
4
In addition to the negative impact of the aggregation bias on mere efficiency evaluation, at least
two further issues arise. First, aggregated evaluations are difficult to connect with more detailed
evaluations of other evaluators of the same production system: economists using aggregate data
and engineers using detailed data main obtain conflicting results; marketing analysts can be im-
pressed by a firm’s adaptability to changing market conditions, and yet, an aggregate evaluation
may show deficient allocative efficiency because of the underlying convex model. This impedes
the communication and trust between the respective stakeholders.
Second, the aggregation bias induces adverse incentive effects. A contract with a DMU designed
to improve technical and allocative efficiency (cf., e.g., Bogetoft and Otto, 2011; Bogetoft, 2012)
may diminish responsiveness to variations in the market condition because adjustments to inputs
and outputs will manifest as increased technical inefficiency in the aggregated evaluation. Hence,
contracting on technical efficiency may lead to lower profitability and, more generally, poorer goal
fulfillment. It is important that incentive schemes take this problem into account. For example,
leading regulators of electrical networks have made corresponding amendments to their revenue
cap regulation (cf., e.g., the merger approach used by the Norwegian regulator as discussed in
Bogetoft, 2012).
Against this background, the paper suggests an approach to counter the aggregation bias by evalu-
ating the potential impact of sub-unit aggregations on a DMU’s DEA efficiency without actually
having disaggregated sub-unit data. The approach involves hypothetical disaggregations to inves-
tigate whether the conclusions derived from the aggregate information could be significantly al-
tered by more detailed information.
5
The impact our approach can have on efficiency analysis is verified on the basis of data from the
Brazilian Electricity Regulator used in 2015 to measure the cost efficiency of the Brazilian distri-
bution system operators (DSOs). These DSOs are confronted with a – in particular geographically
– heterogeneous business environment that is susceptible to the aggregation bias. Accordingly, our
findings impressively indicate the relevance the aggregation bias may have. In comparison to the
results of the Brazilian DSO model, the number of DSOs identified as efficient clearly raises when
using our approach. Furthermore, the extent by which the particular efficiency scores increase is
noteworthy.
This paper is organized as follows. After an overview of the related literature in Section 2, Section
3 provides a conceptual introduction to the idea of distinguishing between disaggregated sub-unit
efficiency and aggregated, organizational efficiency. We provide simple examples of the aggrega-
tion bias, explain how this bias arises, and outline the measure of potential sub-unit efficiency (PSE)
which serves to correct a biased evaluation of organizational efficiency. As starting point to for-
malize the PSE concept, Section 4 discusses the condition under which the aggregation bias does
not appear. On this basis, Section 5 focuses on the opposite case. After proposing a general ap-
proach to measure PSE referring to Farrell efficiency, we elaborate an operational PSE measure
for the case of missing sub-unit information and also simplify the resulting mixed-integer program.
In Section 6, our model is applied to the case of the Brazilian DSO regulation to investigate to what
extent these DSOs may be affected by the aggregation bias. Final conclusions are provided in Sec-
tion 7.
6
2. Literature review
We are not aware of papers that are directly related to the present study. There are, however, at
least four lines of research with links to the issues we address.
One line of studies relates to sensitivity analysis. To determine a simple but sufficiently detailed
model, one may examine how sensitive the results (efficiency scores etc.) are to aggregations over
space and time, respectively. This issue is of course only one of several that can be addressed by
sensitivity analysis. For other applications in DEA models, see for example Charnes and Neralic
(1990), Ray and Sahu (1992) as well as Charnes et al. (1996). However, our approach does not
calculate the consequences of given aggregations, but we seek for the disaggregation that has the
maximal effect on the results. We construct worst-case scenarios to put the evaluated DMUs in the
best possible light.
Another line of research involves investigating under which conditions it is possible to make exact
aggregations. Many studies address the relationship between the aggregation of variables and the
separability of the DMU into several sub-units. Although aggregating sub-units and variables is
not the same, the former can be considered an instance of the latter if we interpret inputs and outputs
from different sub-units as different inputs and outputs. Then the question is when these can be
aggregated over space and time, respectively, i.e., when it is safe to aggregate the variables de-
scribing the same type of input or output in the different sub-units. In this context, there are a few
studies explicitly linking aggregation and separability issues to the efficiency measurement prob-
lem (cf. Färe and Lovell, 1988). We deviate here by examining specific production entities instead
of general structures. Furthermore, we are mainly concerned with the impact of inexact aggrega-
tions rather than with the circumstances under which exact aggregations are possible. However, in
7
Section 4, we discuss a fundamental condition, sub-unit alignment, that allows for exact aggrega-
tions with respect to efficiency evaluations.
A third line of research is to combine the methods from efficiency analysis, including the estima-
tion of production models using observed productions, with those reported in the production plan-
ning literature, including the use of linear programming to model networks of production processes.
Based on the seminal paper of Färe and Grosskopf (2000a) about network DEA, a series of con-
ceptual variations (cf., e.g., Lewis and Sexton, 2004) as well as application-oriented contributions
(cf., e.g., Mirhedayatian et al., 2014) were provided. We deviate here from this literature by two
aspects. First, we do not address the sequential interrelation of production processes of a DMU.
Instead, we address sub-units of a DMU which are assumed as independent production instances.
Second, we assume that no detailed information about these sub-units are available and investigate
how this limitation could affect the DEA efficiency scores.
A final line of inquiry is that of DEA-based merger analysis. In a series of contributions, bench-
marking models were used to make ex ante predictions of the likely gains from mergers, cf., e.g.,
Bogetoft et al., 2003 as well as Bogetoft and Wang, 2005; the approach suggested in the latter
paper has meanwhile been adopted by regulators in the Netherlands and Norway to guide decision
making and incentive regulation in the context of mergers in the hospital and the energy sector (cf.
Bogetoft, 2012). Furthermore, Andersen and Bogetoft (2007) as well as Bogetoft et al. (2007) ex-
amined the effect of allowing more general reallocations of some of the resources and services
within a large number of market participants. While those studies progressed from less aggregated
to more aggregated production units, the present paper goes in the opposite direction, from aggre-
gated to disaggregated production units. We ask what bias may occur when DMUs are evaluated
as if there are synergies, but when such synergies in fact do not exist because of geographic or
8
temporal barriers. Hence, whereas the merger literature studies hypothetical mergers, the present
paper studies hypothetical disaggregations of consolidated DMUs.
To sum up, we seem to add a new aspect to the vast DEA literature. Yet, from a broader perspective,
our paper can be assigned to the research stream of DEA under the condition of a so-called cen-
tralized management (cf., e.g., Lozano and Villa, 2004, and most recently, Afsharian et al., 2017).
This notion covers contributions in which a central evaluator not merely puts every DMU in its
best possible light, but incorporates management control mechanisms into the efficiency measure-
ment. These mechanisms incentivize the DMUs (as agents) to make decisions in accordance with
the goals of the regulator (as principal). In line with this, our approach applies such a mechanism
which counters the aggregation bias, preventing large companies to reorganize into smaller units
only because this leads to higher efficiency scores. The underlying idea of distinguishing between
disaggregated sub-unit efficiency and aggregated organizational efficiency is illustrated in the fol-
lowing, using two motivational examples.
9
3. Exemplary description of the aggregation bias and its quantification
Most companies comprise individual sub-units operated in parallel as well as over time. When
these sub-units are described in aggregate terms, the adaptation to variations in supply and demand
over space and time will be suppressed. As already outlined, actually favorable sub-unit adaptions
may lead to unfavorable aggregate evaluations of the organization as a whole. In such a setting, we
can distinguish between two efficiency notions:
Sub-unit efficiency: Individual sub-units cannot be improved.
Organizational efficiency: The aggregate of the sub-units cannot be improved.
The issue we investigate is that an organization may have fully efficient sub-units but still appears
inefficient on an aggregate level – we call this the aggregation bias.
A simple example is the so-called Fox paradox (cf. Fox, 1999, 2004). One version of the phenom-
enon is illustrated in Table 1, which describes the case of two companies, e.g. electrical network
firms, both serving rural and urban costumers. DMU A serves 2 rural and 4 urban costumers and
spends 1 on the rural and 1 on the urban costumers. The unit costs (UC) of the two types of con-
sumers are therefore 1/2 and 1/4, respectively. In total, DMU A has spent 2 to serve 6 costumers,
and the aggregate UC are 2/6. The other numbers can be interpreted analogously.
DMU
Rural UC
Urban UC
Aggregate UC
A
1/2
1/4
2/6 = 0.33
B
1,5/2
21/80
22,5/82 = 0.27
Table 1 Fox paradox as example for the aggregation problem
The interesting observation is now that DMU A has
10
lower rural unit costs (cost rural / rural costumer) and
lower urban unit costs (cost urban / urban costumers) but
higher overall unit costs (cost / costumers).
That is, sub-unit by sub-unit, DMU A is more efficient than DMU B, and yet, at the aggregate
level, DMU B is the more efficient unit. The explanation is simple: The relatively more efficient
activity of serving urban costumers plays a larger part in DMU B than in DMU A.
In some cases, we can consider this situation a resolvable allocation problem: DMU A allocates an
excessively large share of its consumers to the least efficient sub-unit. If costumers are freely trans-
ferable, DMU A is indeed responsible for this misallocation, and the aggregate appraisal might be
fair. The important point, however, is that in cases such as the electrical network industry (and
many others, as outlined in Section 1), companies cannot be held responsible for all allocation
problems. For example, they cannot freely reallocate the costumers between rural and urban areas.
In such a situation, the aggregate evaluation becomes biased. DMU A is blamed for performance
aspects it cannot control, one of the most obvious mistakes in adequate performance evaluations.
Indeed, DMU A should appear efficient because for no common composition of costumers can
DMU B outperform DMU A.
Färe and Grosskopf (2000b) describe how to conceptually avoid the Fox paradox by applying
solely additive efficiency measures. However, this way to define efficiency is not compatible with
the classical DEA approach addressed in this paper which applies a ratio efficiency measure.
Hence, the aggregation bias remains a possible pitfall of DEA, as is discussed in the following
example.
Imagine that there are 32 DMUs, each of which has used two inputs to produce the same amount
11
of one output. Fig. 1 illustrates these DMUs as numbered, filled points, and Table 2 lists the re-
spective data set. We observe that all DMUs except for DMU 32 are fully efficient.
Now, assume that we also consider all consolidated versions of the efficient DMUs, i.e., all sums
of two efficient DMUs including the sum of one DMU with itself. When we also allow for
downscaling with 1/2 to scale down the inputs of the consolidated units to give an output of 1, we
obtain the other, unfilled points in Fig. 1.
Fig. 1. Simple DEA example to describe the aggregation problem
Input 1
Input 2
12
!
Input 1
Input 2
Output
PSE* score
DMU
Input 1
Input 2
Output
PSE* score
2.00
0.00
1.00
1.00
17
0.47
0.60
1.00
1.75
1.90
0.03
1.00
1.00
18
0.43
0.70
1.00
1.57
1.80
0.07
1.00
1.00
19
0.40
0.80
1.00
1.42
1.70
0.10
1.00
1.00
20
0.37
0.90
1.00
1.30
1.60
0.13
1.00
1.00
21
0.33
1.00
1.00
1.20
1.50
0.17
1.00
1.00
22
0.30
1.10
1.00
1.11
1.40
0.20
1.00
1.00
23
0.27
1.20
1.00
1.03
1.30
0.23
1.00
1.00
24
0.23
1.30
1.00
1.00
1.20
0.27
1.00
1.03
25
0.20
1.40
1.00
1.00
1.10
0.30
1.00
1.11
26
0.17
1.50
1.00
1.00
1.00
0.33
1.00
1.20
27
0.13
1.60
1.00
1.00
0.90
0.37
1.00
1.30
28
0.10
1.70
1.00
1.00
0.80
0.40
1.00
1.42
29
0.07
1.80
1.00
1.00
0.70
0.43
1.00
1.57
30
0.03
1.90
1.00
1.00
0.60
0.47
1.00
1.75
31
0.00
2.00
1.00
1.00
0.50
0.50
1.00
2.00
32
1.00
1.00
1.00
1.00
* PSE: potential sub-unit efficiency
Table 2 Data set of the simple DEA example and PSE scores
Let us look at one example of these consolidated units: If we add 0.5*DMU 1 and 0.5*DMU 31,
we obtain the input combination (1,1), producing an output of 1. These are precisely the same
inputs and outputs that characterize DMU 32. If, therefore, DMU 32 is really a consolidated DMU,
it might actually be sub-unit efficient even though it is inefficient at the aggregated level. In fact,
if we generalize this example, we can conclude that all unfilled points are potentially sub-unit ef-
ficient. Therewith, our concept of potential sub-unit efficiency (PSE) is basically outlined.1
This concept also takes super-efficiency into account. Concerning the numbered DMUs, e.g., the
ones from 9 through 23 may represent super-efficient performance. Table 1 shows their respective
PSE scores, which are above 1. Thus, these DMUs could have increased their inputs and still could
have been the result of running two efficient sub-units. A PSE score of 1.20, for example, means
1 The model to calculate the PSE scores listed in Table 1 and referred to in the next paragraph will be presented in
Section 5.2.
13
that the corresponding DMU could have expanded the use of both inputs by 20 % and still would
have been potentially sub-unit efficient, i.e., it still could be considered an aggregation of efficient
sub-units.
As an interesting observation, it can be derived that centrally located DMUs have larger aggrega-
tion corrections. The reason is that they are more likely to be the results of a consolidation of sub-
units with rather different input-output profiles. The more extreme DMUs have a smaller correc-
tion, i.e., when evaluated as consolidated units, their aggregation bias is smaller, because they are
less likely to be the result of aggregating sub-units with very different input-output profiles.
In the following, we step by step formalize our PSE concept. As starting point, the next section
formulates and discusses the condition of sub-unit alignment under which the aggregation bias does
not occur.
14
4. Efficiency under the condition of sub-unit alignment
Let us consider a DMU that uses m inputs ! " #$
%&'()*to produce n outputs + " #$
,*&'(). Let T
be the production possibility set, with - " #$
%$, . We assume that T is closed, convex and freely
disposable. (x,y) "*T is weakly efficient (or strictly non-dominated) if
! . /0 + 1 2 3 -*456*788* /0 2 " #$
%$, *0 /0 2 9 ( (1)
where (a,b) >> 0 means that all coordinates of (a,b) are strictly positive. Let the set of weakly
efficient productions in T be denoted W (T). We note that weak efficiency is weaker than the clas-
sical economic notion of efficiency which characterizes a production as efficient when we cannot
increase any output (decrease any input) without decreasing (increasing) another output (input). A
production that can be improved in some but not all dimensions is weakly efficient.
We shall say also that (x,y) "*T is allocatively efficient with respect to a price vector :;0 <= "
*#$
%$, &'() if and only if
!0 + " 76> ?7@
:AB0CB="D <+B . ;!B. (2)
In our theoretical analysis, it is advantageous to use the weak efficiency notion. First, the notion
simplifies several of the results presented below for which we would have to assume unique solu-
tions or solutions with strictly positive prices if we would work with ordinary efficiency to avoid
picking up points on horizontal and vertical segments of the production possibility frontier (cf.,
e.g., Bogetoft and Pruzan, 1991, Appendix A). Second, with convex sets, the weakly efficient pro-
ductions are all those that result from optimal economic behavior (profit maximization) for some
non-negative and non-zero price vector. Finally, in the efficiency analysis literature, it is common
to work with notions of efficiency that are weaker than the classical economic one.
15
To take up again our assumptions, by disposability, W (T) constitutes the ”North-East” boundary
points of T. Moreover, since T is closed and convex, it is the intersection of all halfspaces {(,y´ ) |
vy ux r} containing T (cf. Rockafellar, 1970, Theorem 11.5). Now, note that:
Lemma 1. Any halfspace containing T has a normal (u,v), where (u,v) is non-negative.
Proof. Assume that v has a negative component vi. Let ei = (0,...,0,1,0,...,0), and denote the i’th unit
vector in output space. Thus, for any (x,y) " T and any r, there exists a positive scalar E such that
(y Eei)v xu*F r, violating the free disposability of T. If instead u has a negative component, we
can increase vy ux by increasing the corresponding input coordinate, which again would violate
free disposability.
This finding implies that all supporting hyperplanes {(,y´) | vy´ ux´ = r} through a boundary
point have (u,v) 0. In efficiency terms, this expression may alternatively be written as follows:
Lemma 2. A production (x,y) " T is weakly technically efficient if and only if it is allocatively
efficient with respect to some price vector (u,v)*" #$
%$, &'().
Hence, the weakly technically efficient productions are allocatively efficient under some (non-
negative and non-zero) price vector and vice versa.
Now, to investigate the aggregation problem, let us assume that the production of a DMU can be
split into |H| sub-units, h*"*H = {1,…,|H|}. For any h, let (xh,yh) " #$
%$, be the production vector,
let T h be the set of possible productions for h, and when applicable, let (uh,vh) " #$
%$, \{0} be the
non-negative prices of the inputs and outputs of h. For simplicity, we assume that inputs and out-
puts are freely disposable, i.e., for any T h we assume that !B0 +B " -G0 !BB H !B0 +BB I +B*
* !BB0 +BB " -G. Furthermore, we assume that T h, h*"*H, are closed and convex.
16
We can now provide a necessary and sufficient condition for aggregations to create no biases in
the evaluations: This necessary and sufficient condition for efficiency at the aggregate level is al-
locative efficiency at the sub-unit level with regard to the same set of prices we call this the
alignment condition.
Proposition 1. If T h is convex for all h " H, the aggregate production :!G0 +G=
G"J is weakly
technical efficient at the aggregate level
:!G
G"J 0 +G= " K: -G=
G"J (3)
if and only if the sub-unit productions (xh,yh), h "*H, are allocatively efficient at the sub-unit level
with the same rate of substitutions in all sub-units:
L ;0 < " #$
%$, & ( M !G0 +G* " *76>* *?7@
A0C "DN*<+ .;!**ON " PQ (4)
Proof: When -G is convex for all h " H, so is -G
G"J . Therefore, when the aggregate output is
efficient, :!G
G"J 0 +G= " K: -G
G"J ), we know from Lemma 2 that it is allocativly efficient with
respect to some price vector
L ;0 < " #$
%$, & ( M !G0 +G
G"J * " *76>* *?7@
A0C " DR
R"S
*<+ .;! (5)
In turn, this relationship implies
L ;0 < " #$
%$, & ( M !G0 +G" *76>* *?7@
A0C "DN*<+ .;!**ON " P (6)
because if we use the same price vector as in (5) and :!G0 +G= does not solve ?7@ A0C "DR<+ .;!
for some h, we could have found a better solution in (5) by substituting the solution into
17
?7@ A0C "DR<+ .;! for the old :!G0 +G= in (5). This finding shows that allocative efficiency at the
sub-unit level with the same rate of substitution in all sub-units is a necessary (only if) condition
for aggregate efficiency.
The sufficiency also follows from a contradiction. Let :!G0 +G= * " * 76> *T/! A0C "DR<+ .;!0 N "
P be a solution to (6). Now, if !G0 +G
G"J does not solve (5), there exists an alternative solution
!G0 +G
G"J such that < +G. ; !GU < +G. ; !G
G"JG"JG"JG"J . Thus, for at least one
h, we have <VG. ;!GU<+G.;!G, which contradicts the fact that we had a solution (6) to begin
with. Hence, a solution to (6) yields a solution to (5), and by Lemma 2, the result is weak technical
efficiency. This proves the sufficiency (if) part of the proposition.
Proposition 1 shows that to attain technical efficiency at the aggregate level, it is not enough to be
technical and allocatively efficient at the sub-unit level. Optimal profit-maximizing behavior does
not ensure aggregate technical efficiency. We also need for the sub-units to be aligned by a com-
mon price vector in the sense that the rates of substitution are the same in all sub-units. In more
organizational terms, we can conclude that it is not sufficient to have efficient sub-units; we need
goal concordance among them as well.
Note that goal concordance in this case is considered with respect to a common weighting of the
inputs and outputs, which it is not necessarily advantageous. If prices differ, concordance regarding
the same weights is sub-optimal. In such a case, the alignment condition in Proposition 1, which
implies that scale and scope effects do not come into play, causes a discrepancy between sub-unit
performance and aggregated, organizational performance. In other words: in addition to optimal
behavior in the different sub-units, we need price proportionality in the sub-units to aggregate in-
formation without obscuring the evaluations.
18
One instance of Proposition 1 is particularly clear. If all sub-units are the same, T 1 = T 2 =…= T |H|, a
necessary condition for technical efficiency at the aggregate level is that they all operate if not at
the same point, then at least at the same facet. This follows immediately from the alignment con-
dition in Proposition 1. Furthermore, if the technologies are strictly convex – at least in the relevant
(efficient) part – we conclude that the aggregate production is efficient if and only if all sub-units
are using exactly the same (efficient) production.
This situation illustrates how desirable adjustments to spatial/temporal variations in prices at the
sub-unit level will tend to appear as technical inefficiencies at the aggregate level. Optimal eco-
nomic behavior in the sub-units leads to aggregate inefficiency if prices over space/time are not
accidentally proportional. It is fair to say, therefore, that aggregate efficiency is quite unlikely and
certainly not always desirable; such efficiency may come at the cost of inadequate adaptions to
local variations in prices or, more generally, to variations in demand and supply.
We note that these statements regard aggregate efficiency in an absolute sense, i.e., when compared
with a theoretical production model. In practice, a DMU’s efficiency is usually measured relative
to other DMUs, and because they may be ”handicapped” by variations in the prices on the dis-
aggregated level as well, a DMU may be more likely to be (relatively) as efficient as an aggregate
unit.
In the next section, we propose an approach to make more fair evaluations when DMUs cannot
freely allocate production between different sub-units, i.e., when the condition of sub-unit align-
ment is not valid.
19
5. Potential sub-unit efficiency
5.1. A PSE approach based on Farrell efficiency
In practice, it is uncertain if inefficiency at the aggregate level is the result of slack in the sub-unit
productions or the result of an aggregation of non-aligned but efficient sub-units. We may, how-
ever, ask if a DMU’s observed inefficiency is sufficiently small to be explained by unobserved but
potentially desirable adjustments of its sub-unit productions to local or temporal variations in
prices. Taking this perspective a bit further, we may ask how much of aggregate inefficiency we
can explain by the aggregation of sub-units alone.
If the observed aggregate production can be expressed as an aggregation of efficient sub-unit pro-
ductions, we say that the aggregate production is characterized by potential sub-unit efficiency
(PSE). More precisely, we say that (x,y) " T is potentially :-G0 N " P= sub-unit efficient (PS effi-
cient) if and only if
L !G0 +G"WXX -G0 N " PM ! I !G0 + H +G
G"JG"J (7)
where WXX -G is the efficient production in T h for some notion of efficiency. Hence, a PS efficient
aggregate production of a DMU is one that results from aggregating efficient – but not necessarily
aligned – sub-unit productions. To test such an aggregate production, we let the sub-units be hy-
pothetical and construct sub-unit productions that put a DMU in the best possible light. On this
basis, we can evaluate whether a production (x,y) is possibly sub-unit efficient, and also measure
the corresponding PSE score.
As always, efficiency can be operationalized in different ways. We here refer to Farrell’s idea of
20
determining input efficiency (E) and output efficiency (F) by means of proportional input and out-
put adjustments, respectively. Based on this well-known concept, we can measure the PSE of (x,y)
on the input side, PSEI (x,y), as
YZW[!0 + \ ?7@'W]W! I !G0 + H +G0 :!G0 +G= " WXX:-G
G"J =0 N " P)
G"J Q (8)
The interpretation of this program is straightforward: We seek the largest possible expansion of the
inputs used in (x,y) such that the resulting production still uses no more inputs to produce at least
the same output as some combination of efficient sub-units. A score of PSEI (x,y) = 1 denotes the
case that (x,y) is PS efficient. If PSEI (x,y) exceeds 1, (x,y) is in fact super-PS efficient. If PSEI (x,y)
is less than 1, even when we take limitations of an alignment between the sub-units into account,
there are some savings to be made in the production (x,y).
Likewise, we can measure PSE on the output side by
YZW^!0 + \ ?_`'a]! I !G0a+ H +G0 :! G0 +G= " WXX:-G
G"J =0 N " P)Q
G"J (9)
The interpretation of this program is again simple: We seek the largest possible contraction of the
outputs produced in (x,y) such that the resulting production still uses no more inputs to produce at
least the same output as some combination of efficient sub-units. A score of PSEO (x,y) = 1 denotes
the case that (x,y) is PS efficient. If PSEO (x,y) is less than 1, (x,y) is super-PS efficient because
there exists a combination of efficient sub-units that produce less than y using at least the same
inputs x. If PSEO (x,y) is greater than 1, even when we allow for limitations of an alignment between
the sub-units, there is some potential to expand the outputs in (x,y).
5.2. Operational PSE measures in case of missing sub-unit information
21
In many evaluation settings, detailed information about the sub-units is missing. We do not know
the set of sub-units H or the production possibilities in these, Th. We shall now discuss how to
measure PSE when such sub-unit information is missing.
Taking a usual benchmarking study as starting point, we assume that each of K DMUs,
DMU1,...,DMUK have used inputs !b" #$
% to produce outputs +b" #$
,. Let T be the underlying
production possibility set, and let T* be an estimate of T based on ':!b0 +b=]c " d). If, for exam-
ple, we assume convexity, free disposability and weakly increasing (non-decreasing) returns to
scale, the minimal extrapolation estimate of T using DEA would be
-e\ ':!0 += " #$
%$, ]! H Eb!b0 + I Eb+b0 EbH f)
b"gb"gb"g (10)
To cope with missing information about the sub-unit production possibilities T h, we propose using
the basic idea of non-parametric benchmarking, namely to rely on observed best practices. One
simple approach is to project the original observations on the efficient frontier of T* and to only
use combinations of these observations in the evaluation of PSE. To this end, let the Farrell input
efficiency of DMUk be
Ek = min{E | (Ex,y) " T h} (11)
and consider
Eff (T h) = {(Ekxk,yk), k " K} ON " P. (12)
That is, we let the efficient outcomes of sub-unit h be the efficient versions of the original DMU
observations. Thereby, an observation (xk,yk) is PS efficient if it is the sum of efficient versions of
the actual observations. This direct aggregation approach leads to a PSE measure which is based
22
on a minimum of speculations regarding what is feasible and which is easy to interpret. The greater
Eff (T k), the more PS efficient the DMUs will appear, i.e., we will obtain larger PSEI and smaller
PSEO scores. We can therefore say that the approach of relying solely on the efficient versions of
the original observation is cautious in the sense that it does not lead to excessively good PSE eval-
uations.
Using the described approximation of the efficient sub-unit outcomes, the calculation of PSEI can
be outlined in four steps:
Step 1: Compute the Farrell input efficiencies Ek of each of the original observations (xk,yk), k*"*K.
Step 2: Determine the efficient sub-unit outcomes as
Eff (T h) = {(Ekxk,yk), k " K} ON " P. (13)
Step 3: Determine the PSE reference technology as all possible additions of efficient observations
- \ ': !G
G"J 0 +G
G"J =]:!G0 +G= " WXX -b*ON " P). (14)
Step 4: Determine the PS input efficiency of the original observations as
YZW[!0 + \ ?7@'W]W! I !0 + H +0 :!0 += " -). (15)
This formulation of the direct aggregation approach can be simplified. The last three steeps essen-
tially correspond to the solution of the mixed-integer program
YZW[!0 + \ ?7@
i0jk0l0jmW
nQ oQ****W! I EbWb!p
b0*****q \ f0 l 0 T
g
brs
23
+ H Eb+t
b0*************u \ f0 l 0 v
g
brs
Eb" (0f0w0x0 l 0*********c0 l 0 d
Eb
g
brs I PQ********* (16)
In this program, we look for a combination of efficient productions (Ekxk,yk), k = 1,..., K such that
the combination uses more inputs to produce less outputs than (Ex,y).
Of course, the procedure outlined above can also be executed using projections in the output direc-
tion in Steps 2 and 4:
Step 2*: Determine the efficient sub-unit outcomes as
Eff (T h) = {(xk, Fkyk), k " K} ON " P. (17)
Step 4*: Determine the PS output efficiency of the original observations as
YZWy:!0 += \ ?_`*'a]! I !0 a+ H +0 :!0 += " -). (18)
The mixed-integer program solving PSEO becomes
24
YZWy!0 + \ ?_`
z0jk0l0jma
nQ oQ****! I Ebab!p
b0*********q \ f0 l 0 T*
g
brs
a+ H Eb+t
b0**********u \ f0 l 0 v
g
brs
Eb" (0f0w0x0 l 0*********c0 l 0 d
Eb
g
brs I PQ********* (19)
In applications, the number of underlying sub-units H to consider may be uncertain. Then, we have
different options. One possibility is to choose the most restrictive option and only allow DMUs to
be hypothetically decomposed into two sub-units, i.e., H = 2. The option leads to cautious results
in the sense that it yields the smallest set of PS efficient outcomes and therefore the smallest scores
of the PSEI measure and the largest scores of the PSEO measure. The DEA example in Section 3
was explained using this version of the direct aggregation approach.
Another possibility is to choose the most flexible option and say that we allow DMUs to be hypo-
thetically decomposed into any number of sub-units, i.e., H = any natural number. This option
yields the largest set of PS efficient outcomes and therefore the largest scores of the PSEI measure
and the smallest scores of the PSEO measure.
5.3. A simplified (relaxed) approach
The aggregation approach is conceptually simple because it directly constructs possible PS effi-
cient outcomes by adding together efficient versions of actual outcomes. Still, computationally, the
approach may seem complicated because it involves mixed-integer programming. It may therefore
be interesting to study the following simplified (relaxed) version of the mixed-integer program:
25
YZW[!0 + \ ?7@
i0jk0l0jmW
nQ oQ****W! I EbWb!p
b0*****q \ f0 l 0 T
g
brs
+ H Eb+t
b0*************u \ f0 l 0 v
g
brs
Eb" #{0*************************c0 l 0 d
Eb
g
brs H fQ********** (20)
This problem is relaxed by allowing the E values to be real numbers as opposed to integers, and by
removing the upper constraint on the sum of these values.2
The program is again straightforward to interpret. It involves performing an output-oriented DEA
efficiency analysis of (x,y) in which we assume convexity, increasing (non-decreasing) returns to
scale (IRS), and free disposability. Furthermore the inputs and outputs are reversed, i.e., the inputs
x are treated as outputs and outputs y are treated as inputs.
Because the relaxation leads to slightly larger scores of PSEI, the efficiency measures calculated in
this manner may exceed the scores calculated using the direct aggregation approach. Hence, the
DMUs will tend to appear more efficient. As we will observe, however, the relaxation may be
modest. In fact, in the example presented in Section 6, the relaxed problem leads to the same scores
yielded by the direct aggregation approach with H = 2.
2 Forcing the sum of E values to be at least one is also effectively a relaxation because we could have introduced such
a constraint in the mixed-integer formulations above without affecting the scores of PSE(x,y).
26
Instead of considering the relaxed formulation an approximation, it is possible to motivate the for-
mulation in its own right. One approach could be to assume that the underlying technology is an
IRS technology. The weighted sum of efficient sub-units can now be rewritten as follows:
: EbWb!b0 Eb+b= \ : |b}bWb!b0 |b}b+b
g
brs =
g
brs
g
brs
g
brs (21)
Where }b\ ceiling Eb" f0w0x0 l and 0 αk = Eb~}b 1 for all k=1,…,K. Here, the ceiling
function ceiling(z) is the smallest integer not less than z. Hence, the reference unit used to evaluate
the PSE can be interpreted as the result of two operations:
Downscaling: The efficient versions of the original observations can be downscaled, making
them possibly super-efficient by the increasing return to scale assumption.3
Aggregation: The reference unit can be any direct aggregation of a finite number of efficient
and possibly super-efficient sub-units.
Hence, if we accept the IRS assumption (like in the following example), the simplified approach
is conceptually easy to promote in its own right.
3 Note that if (x,y) is feasible in an IRS DEA technology, any downscaled version k(x,y) for k < 1 is either
efficient or super-efficient.
27
6. Application to the Brazilian DSO model
6.1. Motivation
The Brazilian distribution system operators (DSOs) are regulated on the basis of a DEA model
with weight restrictions to determine efficient cost levels (cf. ANEEL, 2015). This example does
not only serve to merely illustrate our approach based on real-world data, but also sheds some light
on actual issues of benchmarking in the Brazilian energy distribution sector.
First, the size of the Brazilian DSOs entails a quite heterogeneous business environment for deliv-
ering their services. Facing in particular considerably different geographical conditions, it is likely
that many of the DSOs should in fact be regarded as consolidations of diverse sub-DSOs that have
limited possibilities to create synergies. If this is the case, the evaluations based on the Brazilian
DSO model may be affected by the aggregation bias.
Second, the fact that weight restrictions are used in the Brazilian DSO model may mitigate the
heterogeneity problem because the resulting isoquants attain a lower curvature. Therefore, although
the aggregation bias is a reasonable presumption, its importance can only be evaluated by a numeric
analysis based on our new approach.
6.2. The Brazilian DSO model
The Brazilian DSO regulation is in many ways in line with the international literature on regulatory
benchmarking. Corresponding models typically use a series of indicators of the capacity provided,
the transport work undertaken and the customer services delivered as cost drivers (cf. Bogetoft,
2012). The respective input and outputs used in the Brazilian DSO model are shown in Table 3,
which also indicates the tasks covered by the different cost drivers.
28
Model variables
Task
INPUT
x_OPEX_adjusted
OUTPUT
y_Underground_all_tension_levels
Physical assets
y_Air_distribution_network
Physical assets
y_High_network
Physical assets
y_Averaged_market
Transport service
y_Consumers_number
Customer service
z_Neg_non_technical_losses_adjusted
Quality
z_Neg_interruption_adjusted
Quality
Table 3 Brazilian DSO model variables
We note that the use of physical assets to capture capacity provision is quite common in regulatory
practice. It is also noteworthy that the model does not contain direct information about the charac-
teristics of service areas, such as precipitation and vegetation, even though these conditions vary
considerably from DSO to DSO as well as across the areas serviced by the individual DSOs. An-
other remarkable feature of the model is the direct inclusion of quality indicators as negative out-
puts. Quality is usually considered a property of the basic services and is typically handled by
second-stage corrections or add-on regulatory instruments (cf. Bogetoft, 2012).
The Brazilian DSO model also di44ers from common regulatory benchmarking models via the use
of restrictions on the dual weights. In total, seven such restrictions are used, as shown in Table 4.
It can be observed that two of the restrictions limit the possible rate of substitution between outputs,
whereas the remaining five restrict the output costs for individual outputs, compared to the input
OPEX (operational expenditure). The first two constraints are so-called Type I assurance regions,
whereas the latter five are Type II assurance regions.
29
Constraint
Lower limit
Ratio
Upper limit
A
1
<
y_Underground_all_tension_levels/y_Air_distribution_network
<
2
B
0.58
<
y_Air_distribution_network/x_OPEX_adjusted
<
2.2
C
0.4
<
y_High_network/y_Air_distribution_network
<
1
D
0.001
<
y_Averaged_market/x_OPEX_adjusted
<
0.06
E
0.03
<
y_Consumers_number/x_OPEX_adjusted
<
0.145
F
0.01
<
z_Neg_Non_technical_losses_adjusted/x_OPEX_adjusted
<
0.15
G
0
<
z_Neg_interruption_adjusted/x_OPEX_adjusted
<
0.002
Table 4 Weight restrictions used in the Brazilian DSO model
Weight restrictions can be considered either an expression of preferences or an expression of partial
information about rates of substitutions. For example, the last restriction listed in Table 4 can be
an expression that the value of avoiding the loss of an hour cannot exceed 0.002 kBRL, i.e., that
the value of an hour of lost electricity cannot exceed 2 BRL. Alternatively, the restrictions can be
an expression that the actual cost of cutting down on the hours of interruption is never higher than
2 BRL per hour. It is not known whether the restrictions are actually expressions of regulatory
preferences or of specific knowledge of cost effects (cf. Bogetoft and Lopes, 2015).
6.3. Findings
The use of weight restrictions is interesting regarding the aggregation bias because they lead to
more linear isoquants, which one would expect to limit the bias. In that respect, it can be determined
that the constraints have a non-trivial impact on the Brazilian DSO model results, i.e. the constraints
actually matter. For the 61 DSOs of our data set, this is illustrated in Fig. 2. Here, the model results
obtained using weight restrictions (the monotonically increasing black points) are compared with
the pure IRS scores obtained without weight restrictions (the upper series of grey points).
30
Fig. 2. Impact of weight restrictions in the Brazilian DSO model
Next, we have calculated the PSEI scores of the Brazilian DSOs using our simplified (relaxed)
approach. The results are shown in Fig. 3. Here, the DSOs are sorted from the smallest PSEI score
to the largest one. As explained above, this sorting illustrates the expansion of costs that is possible
assuming efficient sub-units. We observe that a large share of the DSOs can in fact be considered
as PS efficient. Only 13 of the 61 DSOs remain inefficient, with a PSEI score less than 1, which
means that 48 DSOs are classified as fully PS efficient (and many of them are super-efficient). By
means of the Brazilian DSO model, only 8 DSOs were classified as efficient. We also observe that
nearly half of the DSOs have PSEI scores greater than 1.5, suggesting that they could in fact have
increased their OPEX by 50% and it would still be possible to consider them as sub-unit efficient.
Brazilian DSOs
Efficiencies without and with weight restrictions
black points:
Brazilian DSO model with weight restrictions
grey points:
model without weight restrictions
31
Fig. 3. PSEI scores
The effects of applying our approach are dramatic. Most DSOs obtain significantly better scores
when we consider them as consolidated units and investigate whether they could in fact be decom-
posed into fully efficient sub-units. This finding is illustrated in Fig. 4, in which we compare the
Brazilian DSO model efficiencies (the monotonic series of black points) with the PSEI efficiencies
derived from the simplified approach (the upper series of non-monotonic grey points).
It is obvious that the derived PS efficiency scores are very lenient on the DSOs. This suggests that
one should consider restricting the number of sub-units H in which the PSE analysis is allowed to
hypothetically disaggregate the DSOs. However, our goal was to demonstrate that the aggregation
of data at the DSO level can have a huge impact on the results, i.e. that the potential aggregation
bias can be enormous.
Brazilian DSOs
PS efficiency of the Brazilian DSOs
32
Fig. 4. Comparison of efficiency scores: the Brazilian DSO model versus the PSE model
Brazilian DSOs
efficiency scores
black points: efficiency scores of Brazilian DSO model
grey points: PSEIscores
33
7. Conclusions
In this paper, we have argued that the occurrence of highly aggregated organizational units in a
benchmarking study may skew the results. Such aggregated DMUs are likely to receive excessively
harsh evaluations. We have illustrated this aggregation bias and reflected upon the condition under
which the bias does not occur, namely the alignment condition. Only with aligned productions of
a DMU’s sub-units, an aggregation of these productions does not affect the efficiency analysis of
the DMU. Basically, price proportionality with respect to the sub-units is needed to allow for an
exact aggregation of their productions without obscuring the evaluation on the DMU level.
As explained in the paper, this condition may be problematic in a many real-world cases, where
DMUs have to manage sub-units with different business environments resulting from, e.g., differ-
ent locations or periods. For such cases, we propose a DEA-based approach for compensating for
the possible aggregation bias by calculating a DMU’s potential sub-unit efficiency PSE. This
concept allows to measuring the extent to which the respective DMU can be viewed as an aggre-
gation of efficient sub-units. A PSE score less than one indicates that – even accounting for given
limitations of an alignment between the sub-units – activities are not performed efficiently. To this
effect, we elaborated how to determine PSE scores under different assumptions.
As example, we applied the PSE concept to the DEA model used by the Brazilian Electricity Reg-
ulator in 2015 to measure the cost efficiency of the Brazilian distribution system operators (DSOs).
Because of the size of these DSOs and the heterogeneity of their service areas, it is highly likely
that many of the DSOs are in fact subject to biased evaluations. Our numerical results showed that
the biases may be considerable. In comparison to the results of the Brazilian DSO model, the num-
ber of DSOs classified as efficient significantly raised, along with a substantial increase of many
34
of the efficiency scores.
The implications of our findings are twofold. From the perspective of a central evaluator, e.g. a
regulator, it is important to be aware of a possible aggregation bias. It seems necessary to investi-
gate whether there are good reasons that the DMUs to be analyzed operate sub-units in different
business environments which require different strategies for performing optimally. In this case,
incorporating the PSE concept into the particular efficiency analysis is a helpful control mechanism
to take the issue into account, inducing fairer and broader accepted evaluations.
Our findings can also be of great relevance from the perspective of particular DMUs under evalu-
ation, since the impact of the aggregation bias on efficiency scores was shown to be potentially
huge. On the one side, it might be in the interest of affected companies to prove that a benchmarking
analysis without considering the bias would be flawed. On the other side, companies may also react
from a strategic point of view, since our findings imply that “playing the regulation” by reorganiz-
ing into smaller sub-units may have a considerable payoff.
As one possibility for further research, our findings could be aligned to bootstrapping in DEA (cf.
the seminal paper of Simar and Wilson, 1999, and for a more recent application, e.g., Gitto and
Mancuso, 2012). We speculate that uncertainty as estimated by bootstrapping is largely inversely
related to the extent of the consolidation bias. While we outlined in Section 3 that the aggregation
bias tends to increase from more to less extreme types of DMUs, the bias correction that can be
derived from a bootstrapping exercise has the opposite tendency. This finding indicates that DMUs
which we are more uncertain about in a typical efficiency analysis are also those that are less likely
to have a large aggregation bias. Vice versa, the firms that we are more certain about in the technical
evaluations will also be those that are more likely to have a large aggregation bias. A thorough
35
investigation of this topic might be fruitful.
36
Acknowledgments
The first author gratefully acknowledges the financial support from the Deutsche Forschungs-ge-
meinschaft (DFG) in the context of the research fund AH 90/5-1.
The third author appreciates the financial support from the Foundation for the Research Support in
Minas Gerais (FAPEMIG) and Companhia Energética de Minas Gerais (CEMIG), Coordination for
the Improvement of Higher Education Personnel (CAPES) and National Council for Scientific and
Technological Development (CNPq) in the context of the research funds APQ-03165-11,
999999.000003/2015-08 and 444375/2015-5, respectively.
37
References
Afsharian, M., Ahn, H., Thanassoulis, E., 2017. A DEA-based incentives system for centrally man-
aged multi-unit organisations. European Journal of Operational Research 259 (2), 587–598.
Ahn, H., Le, M.H., 2015. DEA efficiency of German savings banks: evidence from a goal-oriented
perspective. Journal of Business Economics 85 (5), 953–975.
Andersen, J., Bogetoft, P., 2007. Gains from quota trade: Theoretical models and an application to
the Danish fishery. European Review of Agricultural Economics 34 (1), 105–127.
ANEEL, 2015. Operational costs methodology (translated from Portuguese). Tech. rep., Agncia
Nacional De Energia Elétrica ANEEL, Technical Note 66/2015, Brasilia.
Bogetoft, P., 2012. Performance Benchmarking: Measuring and Managing Performance. Springer,
New York.
Bogetoft, P., Boye, K., Neergaard-Petersen, H., Nielsen, K., 2007. Reallocating sugar beet con-
tracts: can sugar production survive in Denmark? European Review of Agricultural Economics 34
(1), 1–20.
Bogetoft, P., Lopes, A., 2015. Comments on the Brazilian benchmarking model for energy distri-
bution regulation. Fourth cycle of tariff review – technical note 407/2014. Working paper,
nespufmg.com.br/content/upload/p/d3d9446802a44259755d38e6d163e820.pdf (accessed April
2017).
Bogetoft, P., Otto, L., 2011. Benchmarking with DEA, SFA, and R. Springer, New York.
38
Bogetoft, P., Otto, L., 2015. Benchmarking package. Tech. rep., R Project.
Bogetoft, P., Pruzan, P., 1991. Planning with Multiple Criteria. North-Holland, Amsterdam.
Bogetoft, P., Strange, N., Thorsen, B.J., 2003. Efficiency and merger gains in the Danish forestry
extension service. Forest Science 49 (4), 585–595.
Bogetoft, P., Wang, D., 2005. Estimating the potential gains from mergers. Journal of Productivity
Analysis 23 (2), 145–171.
Charnes, A., Rousseau, J., Semple, J., 1996. Sensitivity and stability of efficiency classifications
in data envelopment analysis. Journal of Productivity Analysis 7 (1), 5–18.
Charnes, A., Neralic, L., 1990. Sensitivity analysis of the additive model in data envelopment anal-
ysis. European Journal of Operational Research 48 (3), 332–341.
re, R., Grosskopf, S., 2000a. Network DEA. Socio-Economic Planning Sciences 34 (1), 35–49.
Färe, R., Grosskopf, S., 2000b. Outfoxing a paradox. Economics Letters 69 (2), 159–163.
re, R., Lovell, C.K., 1988. Aggregation and efficiency. In: Eichhorn, W. (Ed), Measurement in
Economics: Theory and Applications of Economic Indices. Physica-Verlag, Heidelberg, pp. 639–
647.
Farrell, M.J., 1957. The measurement of productive efficiency. Journal of the Royal Statistical
Society 120 (3), 253–281.
Fox, K.J., 1999. Efficiency at different levels of aggregation: public vs. private sector firms. Eco-
nomics Letters 65 (2), 173176.
39
Fox, K.J., 2004. Problems with (dis)aggregating productivity, and another productivity paradox.
Working Paper, University of New South Wales.
Gitto, S., Manusco, P., 2012. Bootstrapping the Malmquist indexes for Italian airports. Interna-
tional Journal of Production Economics 135 (1), 403–411.
Lewis, H.F., Sexton, T.R., 2004. Network DEA: efficiency analysis of organizations with complex
internal structure. Computers & Operations Research 31 (9), 1365–1410.
Lozano, S., Villa, G., 2004. Centralized resource allocation using data envelopment analysis. Jour-
nal of Productivity Analysis 22 (1-2), 143–161.
Mirhedayatian, S.M., Azadi, M., Farzipoor Saen, R., 2014. A novel network data envelopment
analysis model for evaluating green supply chain management. International Journal of Production
Economics 147 (2), 544–554.
Ray, P., Sahu, S., 1992. Productivity measurement in multi-product manufacturing firms evalua-
tion and control through sensitivity analysis. International Journal of Production Economics 28 (1),
71–84.
Rockafellar, R.T., 1970. Convex Analysis. Princeton University Press: Princeton, New Jersey.
Simar, L., Wilson, P.W., 1999. Estimating and bootstrapping Malmquist indices. European Journal
of Operational Research 115 (3), 459–471.
Sung, T.-J., Lu, Y.-T., Ho, S.-S., 2010. Time-Based Strategy and Business Performance under En-
vironmental Uncertainty: An Empirical Study of Design Firms in Taiwan. International Journal of
Design 4 (3), 29–42.
40
Appendix
Implementing the abovementioned measures of PSE is very simple. Using the R package ”Bench-
marking” (Bogetoft and Otto, 2015), we can implement the aggregation approach with two sub-
units, H = 2, as follows:
#Initialize
EFF <- Efficiencies of DMUs in T* model, K x 1 vector
mx <- Inputs of original DMUs, K x m matrix
my <- Outputs of original DMUs, K x n matrix
# Calculate the projection on the efficient frontier
mx_eff <- EFF * mx
# Construct all PSE mergers
merge_matrix <- (K(K+1)/2) x K matrix in which each row contains at least
one and at the most two ones and the rest are 0 entries
X_merge <- merge_matrix % * % mx_eff
Y_merge <- merge_matrix % * % my
# Evaluate original DMUs against PSE mergers
PSE_INPUT_BASED_AGGREGATION <-
dea(X=my,Y=mx,XREF=Y_merge,YREF=X_merge,RTS="FDH", ORIENTA-
TION="out")$eff
Furthermore, we can implement the aggregation approach in the general case with arbitrary H by
using this code:
# Initialize
EFF <- Efficiencies of DMUs in T* model, K x 1 vector
mx <- Inputs of original DMUs, K x m matrix
my <- Outputs of original DMUs, K x n matrix
# Calculate the projection on the efficient frontier
41
mx_eff <- EFF * mx
# Evaluate original DMUs against PSE mergers
PSE_INPUT_BASED_AGGREGATION <-
dea(X=my,Y=mx,XREF=my,YREF=mx_eff,RTS="ADD", ORIENTATION="out")$eff
Similarly, we would implement the simplified (relaxed) approach as follows:
# Initialize
EFF <- Efficiencies of DMUs in T* model, K x 1 vector
mx <- Inputs of original DMUs, K x m matrix
my <- Outputs of original DMUs, K x n matrix
# Calculate the projection on the efficient frontier
mx_eff <- EFF * mx
# Calculate the PSE by the aggregation approach
PSE_INPUT_BASED_SIMPLE <-
dea(X=my,Y=mx,XREF=my,YREF=mx_eff,RTS="IRS", ORIENTATION="out")$eff
Hence, it is easy to calculate the PSE of the DMUs and thereby to understand the possible bias
resulting from working with excessively aggregated DMUs.
... The dominance of radial models in the DEA literature seems to have several reasons. For example, Ahn et al. (2018) state: ...
Article
Full-text available
After 40 years of research with thousands of application-oriented scientific papers, empirical evidence that data envelopment analysis (DEA) has really improved the practice of performance measurement and benchmarking in real-life non-production contexts is rare. The main reason for this deficit may be that DEA is founded on the concepts of production theory such as production possibility set or returns to scale. These concepts can hardly be applied to pure multiple-criteria evaluation problems, which are often attempted to be solved using DEA. This paper systematically investigates strengths and weaknesses of DEA in the exemplary case of welfare evaluation using real data on 27 countries of the European Union. We analyze and explain the differences in the results of various frequently used DEA models for two different, but strongly connected sets of welfare indicators, thereby demonstrating the pitfalls, which often arise in the application of DEA, as well as some approaches for avoiding them.
Article
This study focuses on public or private organisations with a large set of operation units. Central managers in these organisations face asymmetric information on the operating costs of sub-units and wish to incentivise them to reveal their minimal costs. In this paper, best practice regulation is implemented to deal with the asymmetric information via a regulation game. In the regulation game, the operating costs of sub-units are divided into several cost levels. To describe the unknown cost levels, a nested frontier-based approach, called context-dependent data envelopment analysis (CD-DEA), is introduced. By using CD-DEA, the efficient feasible incentive contract to the regulation game is proposed, and the condition that guarantees optimality of the proposed contract is also presented. Moreover, the efficient feasible incentive contract shows that the best responses of sub-units in the regulation game are to announce their actual cost levels, which constitute the Nash equilibrium. Several discussions are further provided to ensure the applicability of the proposed approach in realistic environments. Lastly, the proposed approach is illustrated by data from electricity generation and distribution sectors.
Article
Full-text available
In Spain, the average remuneration of large electric distribution companies is much lower than that of smallerones, which leads us to wonder if the smaller ones are managed efficiently. A meta-frontier DEA model is calculated in order to benchmark, in terms of remuneration and quality, different clusters of distributors thatpresent different technologies. The sample is composed of 236 distributors (including the five largest units). Thesavings for the system would be approximately 174 million Euros per year. Moreover, compensation regulation should aim to guide distributors towards their respective intra-cluster frontiers.
Article
Our motivation of this study is to provide a data envelopment analysis (DEA) approach for evaluating and decomposing the operation efficiency, moreover, computing the coordination efficiency of systems with complex internal structure. We consider a two-stage system composed of three sub-systems, where the first stage is comprised of two independent sub-systems in parallel and then linked to the second stage or the third sub-system in series. Additive and multiplicative DEA efficiency measures are proposed to be jointly applied to illustrate the efficiency formation mechanism of the parallel-series system. The definition of coordination efficiency is inspired by the “gap” between accomplished optimal efficiency and idealized maximal efficiency of relevant systems. This study contributes in creating novel means to consider the different structural characteristics in the efficiency assessment of complex network systems, and to measure the externalities in terms of efficiency within their interior. The proposed models are demonstrated by revisiting the case of Taiwanese non-life insurance companies studied by Kao and Hwang. Corresponding implications of the empirical application are also discussed.
Article
In multi-unit organisations such as a bank and its branches or a national body delivering publicly funded health or education services through local operating units, the need arises to incentivize the units to operate efficiently. In such instances, it is generally accepted that units found to be inefficient can be encouraged to make efficiency savings. However, units which are found to be efficient need to be incentivized in a different manner. It has been suggested that efficient units could be incentivized by some reward compatible with the level to which their attainment exceeds that of the best of the rest, normally referred to as “super-efficiency”. A recent approach to this issue (Varmaz et. al. 2013) has used Data Envelopment Analysis (DEA) models to measure the super-efficiency of the whole system of operating units with and without the involvement of each unit in turn in order to provide incentives. We identify shortcomings in this approach and use it as a starting point to develop a new DEA-based system for incentivizing operating units to operate efficiently for the benefit of the aggregate system of units. Data from a small German retail bank is used to illustrate our method.
Article
Now that design is acknowledged as an important consideration for businesses and is recognized as having the potential for driving competitiveness, the only sure thing in adopting a business strategy is that there will always be unknown variables in uncertain environments. Prior findings on the relationship between environmental uncertainty, time-based strategy, and business performance have been inconsistent. Meanwhile, the 'strategic planning' dimension of design consultancy firms has not attracted a great deal of attention. Thus, in this paper, we explore the moderating effects of environmental uncertainty on the relationship between time-based strategy and business performance in design firms. Based on data collected from 80 design firms in Taiwan, the findings first reveal that these firms can adopt different time-based strategies to pursue different performance outcomes. Second, the results show that three of four types of environmental uncertainty have direct effects on measures of business performance such as adaptability and innovation. Third, the findings suggest that appropriate matches between environmental uncertainty and time-based strategy can result in superior business performance. Specifically, the results indicate that the design-differentiation preemptor (DDP) strategy is beneficial for a design firm when it comes to achieving superior adaptability in an environment of high-perceived technological evolutions (TE) or competitor movements (CM) uncertainty. In addition, a design firm can adopt the DDP strategy in order to obtain superior levels of innovation in an environment of low-perceived customer demands (CD) uncertainty. However, the findings reveal that there are no moderating effects of environmental uncertainty found with the design-cost follower (DCF) strategy. Last of all, the limitations of this study and future research possibilities are also discussed.
Book
A production economist focuses on assessment, and will use an aggregate description of technology to answer such questions as: How does the firm compare to its competitors? Has the firm improved its production capabilities? A production engineer focuses on optimizing resources, and will use a detailed description of technology to answer a completely different set of questions: Which operations or plants should produce which products at what time? Should resource capacity be expanded and, if so, which resources should be acquired? Each group could benefit from the other group's perspective. This book offers a unified, integrated point of view that bridges the gap between these two historically distinct perspectives.
Article
We provide one of very few Data envelopment analysis efficiency studies of the German savings banks, thereby contributing evidence on the credit of their business model. This model distinguishes itself by the ultimate purposes to ensure public access to financial services and to support regional economies. To capture the respective goal set of the German savings banks, we propose a framework incorporating rationality concepts of decision making to derive appropriate performance criteria. On this basis, the 2006-2011 analysis reveals the active role of the savings banks in stabilizing the German economy during the financial crisis 2008-2009. The results also suggest that the banks are more efficient in fulfilling their public mandate than in generating profit. Furthermore, a stable scale efficiency pattern is observed, particularly showing that larger banks are experiencing notable decreasing returns to scale.
Article
Green supply chain management (GSCM) has become a method to improve environmental performance. Under stakeholder pressures, forces and regulations, companies need to improve the GSCM practice, which are effected by practices such as green purchasing, green design, product recovery, and collaboration with patrons and suppliers. As companies promote the GSCM, their economic performance and environmental performance will be enhanced. Hence, GSCM evaluation is very important for any company. One of the techniques that can be used for evaluating GSCM is data envelopment analysis (DEA). Traditional models of data envelopment analysis (DEA) are based upon thinking about production as a “black box”. One of the drawbacks of these models is to omit linking activities. The objective of this paper is to propose a novel network DEA model for evaluating the GSCM in the presence of dual-role factors, undesirable outputs, and fuzzy data. A case study demonstrates the application of the proposed model. A case study demonstrates the applicability of the proposed model.