Content uploaded by Jan Rosenow
Author content
All content in this area was uploaded by Jan Rosenow on Jul 06, 2016
Content may be subject to copyright.
An ex-ante evaluation of the EU Energy Efficiency Directive -
Article 7
Published in Economics of Energy & Environmental Policy
Please cite as: Rosenow, J., Leguijt, C., Pató, Z., Eyre, N., Fawcett, T. (2016): An ex-ante
evaluation of the EU Energy Efficiency Directive - Article 7. Economics of Energy &
Environmental Policy 5(2)
Jan Rosenow, Centre on Innovation and Energy Demand, SPRU, University of Sussex,
Ricardo Energy & Environment
Cor Leguijt, CE Delft
Zsuzsanna Pató, REKK
Nick Eyre, Environmental Change Institute, University of Oxford
Tina Fawcett, Environmental Change Institute, University of Oxford
Abstract
The European Union’s Energy Efficiency Directive calls for EU Member States to put in place ambitious energy
efficiency policies and requires them to establish energy saving targets. One of the most important Articles of the
Directive is Article 7, which required Member States to implement Energy Efficiency Obligations and/or
alternative policy instruments in order to reach a reduction in final energy use of 1.5% per year. This paper assesses
how Article 7 has been applied by Member States and what the implications are. Analysing the plans of all 28
Member States we evaluate how Article 7 is implemented across the EU. This includes an analysis of the types of
policies used, the distribution of the anticipated savings across the different policy instruments, and whether or
not the way Article 7 is applied in reality meets the requirements set by the Directive. Our analysis shows that
Member States take very different approaches with some using up to 112 policy measures and others just one. We
also identify areas of concern particularly related to the delivery of the energy savings with respect to the Article
7 requirements, the calculation methods, and the monitoring and verification regimes adopted by Member States.
We model to what extent the projected savings are likely to materialise and whether or not they will be sufficient
to meet the target put forward by Article 7. In our paper we also make suggestions for modifying the Energy
Efficiency Directive in order to address some of the problems we encountered.
Key words
energy efficiency, ex-ante evaluation; energy efficiency directive
Acknowledgements
Parts of this paper are based on two studies evaluating the implementation of Article 7 of the Energy
Efficiency Directive (Rosenow et al. 2015; Forster et al. 2016) funded by the European Commission
(contract numbers ENER.C3.dir(2014)3156530 and ENER/A2/360/2010) and a research paper drafted
for the European Parliament on Article 7 of the Energy Efficiency Directive
(EPRS/IMPT/SER/15/229N). Nick Eyre and Tina Fawcett’s time was funded via the ENSPOL project
(Contract N°: IEE/13/824/SI2.675067).
1 Introduction
Energy efficiency is one of the three key pillars identified in the EU 20-20-20 Strategy - a 20%
reduction of projected primary energy consumption by 2020 (EC 2015a). The Energy Efficiency
Directive (EED) establishes a framework of measures to ensure the achievement of this target (EP
2012). Previous EU policies seek either to set common frameworks for energy efficiency policy in
Member States, e.g. the Energy Performance of Buildings Directive (EPBD) and the Energy Services
Directive (ESD), or to use EU competencies in trade policy to establish common labels and standards,
e.g. through the Ecodesign Directive. Together these have increasingly influenced national energy
efficiency policies of EU Member States. However, the EED intervenes to a much larger extent in
national governance of energy efficiency by a) setting firm energy savings targets and b) suggesting
more exactly the types of policy instruments to be used.
Despite the rising influence of European legislation on national energy efficiency policy, the literature
evaluating energy efficiency policy at the EU level is rather scarce. A recent systematic review of
peer-reviewed energy efficiency programme ex-post evaluations (Wade and Eyre 2015) identified
only four studies analysing the effectiveness of EU energy efficiency policies (Bertoldi et al. 2001;
Saussay et al. 2012; Schiellerup 2001; SRC 2001). All of the other papers found by Wade and Eyre
deal with the evaluation of national energy efficiency policies. There are some studies that undertake
pan-European analyses. For example, Filippini et al. (2014) carried out an econometric analysis of the
level of energy efficiency across EU Member States and the impact of energy efficiency policies.
However, they did not explicitly evaluate the impact of specific EU policies but instead focus on the
role of national policies adopted, some of which are driven by EU initiatives. With regard to ex-ante
evaluations, we are not aware of any peer-reviewed papers carrying out ex-ante evaluations of EU
energy efficiency policies – the available ex-ante evaluations of EU energy efficiency policy are all
located in the grey literature.
In order to address this gap, in this paper we provide an ex-ante evaluation of the EED based on an
extensive review of Member States’ plans for the implementation of the EED. The Directive puts in
place a number of important provisions to be implemented by Member States including the
requirement to establish binding national energy efficiency targets (Article 3), national building
energy efficiency strategies (Article 4), a requirement to renovate 3% of public sector buildings each
year (Articles 5 and 6), the need to establish energy efficiency obligation schemes (Article 7), and
provisions for auditing and metering (Articles 8-12). Instead of evaluating the impact of the whole
Directive (which would be a herculean task), we focus on probably the most important Article of the
Directive (Article 7), which requires Member States to implement Energy Efficiency Obligations
and/or alternative policy instruments in order to reach a reduction in final energy use of 1.5% per year
(EP 2012). Article 7 is expected to deliver more than half of the required energy savings of the 20%
reduction target and is therefore the most important component of the EED in terms of its contribution
(EC 2011a).
We evaluate to what extent Article 7 is likely to deliver its aims and how it is implemented by
Member States. The ex-ante evaluation is based on a substantial amount of information provided by
the 28 EU Member States to the European Commission comprising 7,653 pages of material.
The paper is structured as follows: First, we summarise the provisions made in Article 7 and what it
requires of Member States. Second, we analyse the types of policies implemented and planned by
Member States including a distribution of the anticipated savings across the different policy
instruments, and whether or not they will be sufficient to meet the target put forward by Article 7. Our
analysis shows that Member States take very different approaches with some using up to 112 policy
measures and others just one. Finally, we identify areas of concern particularly related to the
additionality of the energy savings, the calculation methods, and the monitoring and verification
regimes adopted by Member States. We make a number of suggestions for modifying the EED in
order to address some of the problems we encountered.
2 Background
It is has been known for many years that there is significantly less investment in energy efficiency at
the point of end-use than would be required to deliver energy services in the most economically
efficient manner (Blumstein et al. 1980). This phenomenon was named the ‘energy efficiency gap’ in
(Hirst and Brown 1990). Subsequent analysis confirmed that this gap implies the existence of one or
more market failures (Jaffe and Stavins 1994) and that these are to be expected in real energy markets
(Sanstad and Howarth 1994). Subsequent work identified underlying causes from the perspectives of
social psychology (Stern 2000), sociology (Shove 1998) and political economy (Eyre 1997). The
evidence base for the gap has been strengthened by more empirical work summarised in both the
Global Energy Assessment (Urge-Vorsatz et al. 2012) and the IPCC (Lucon et al. 2014). As
opportunities for cost effective energy efficiency investment are taken, innovation generates new
opportunities so that the potential has remained significant over decades (NAS 2010). There is,
however, also emerging evidence that energy efficiency programmes do not always deliver the
anticipated energy savings and fail to meet expectations (Davis et al. 2014; Fowlie et al. 2015).
The policy implications of the energy efficiency gap depend on the nature of the market failure.
Where energy use is subsidised (including by under-pricing of externalities) market based instruments
are the obvious policy response. But there are other important market failures related to trust in supply
chains, information deficits and consumer bounded rationality, see e.g. (Gillingham et al. 2009,
Sorrell et al. 2004) where other policy responses, such as incentives, consumer engagement and/or
product standards are more logical responses. The variety of market failures and types of energy
efficiency investment means that a diverse range of policy instruments is likely to be required.
However, energy efficiency is also important in public policy for reasons distinct from improving
economic efficiency. And it is largely for these reasons that energy efficiency has formed a part of
energy policy at both EU and Member State levels since the 1970s. Whilst the strength of
commitment and justification has varied between Member States and over time, two underlying
concerns separate for economic efficiency arguments stand out as the key drivers.
The first is energy security. The EU is a major energy importer, relying on non-EU sources for more
than half of its primary energy in 2013 (Eurostat 2015). There are specific concerns where there is
reliance on regions viwed as geopolitically problematic. These focussed initially on oil in the wake of
the crises in the 1970s (Hedenus et al. 2010), but are now extended to gas, particularly since the
transit disputes of the last decade (Stern 2006, Yafimava 2011).
The rise in concern about climate change provides another driver for action on energy efficiency,
especially in the content of the global leadership role to which the EU has aspired since the
negotiation of the Kyoto Protocol in 1997. Energy efficiency was been the only significant driver of
greenhouse gas emissions reductions in the first decade of this century (Edenhofer et al. 2014) and
plays a key role in EU climate policy (Delbeke and Vis 2015). These multiple objectives of energy
efficiency justify its separate role in the EU climate policy. There is also increasing interest in a whole
suite of possible benefits from energy efficiency, from macro-economic effects, air quality and health
improvements to delivering jobs (IEA 2014).
The EED (2012/27/EU) was designed to bring the European Union back on track to achieve the 20%
energy consumption reduction target and is one of key steps identified by the Communication on the
Energy Efficiency Plan 2011 and the Roadmap to 2025. Previous analysis by the European
Commission has shown that existing energy efficiency policy measures would not deliver the 20%
target by 2020 and leave a significant gap of more than half of the required reduction (EC 2011b).
Article 7 of the EED requires Member States to establish either energy efficiency obligations (EEOs)
(also known as ‘White Certificates’ and ‘Energy Efficiency Portfolio Standards’) or alternative policy
measures, to achieve new energy savings each year, over the 2014-2020 period, amounting to 1.5% of
the baseline annual energy sales to final customers. In reality the average energy savings are closer to
0.75% because Article 7 allows Member States a) to exclude a range of energy end uses when
calculating their targets (transport, energy for own use etc.) and b) a number of exemptions up to a
maximum of a 25% reduction of the energy savings target. Most Member States made use of both
options.
The European Commission expected initially that Article 7 will deliver an impact of around 10.5% by
2020 (EC 2011a). This figure equals more than half of the 20% target set by the EED. Therefore, it is
the most important Article of the Directive in terms of its estimated impact.
The Member States had to notify to the Commission by 5 December 2013 their detailed plans to reach
the energy savings target under Article 7. These plans included, inter alia, the policy measures that
Member States plan to adopt and their implementation methodology. Further information on Member
States’ plans was provided in their National Energy Efficiency Action Plans (NEEAPs), which had to
be provided by 30 April 2014. In case Member States do not comply with the Directive the European
Commission can refer Member States to the European Court of Justice which can impose penalties.
3 Methodology
Sources used for this evaluation include a formal notification of Member States’ detailed plans to
reach the energy savings target under Article 7 which had to be provided by 5 December 2013, the
relevant additional information on Article 7 provided in the NEEAPs, information and data on
progress provided in the Annual Reports that were due by 30 April 2015, and replies by Member
States to EU pilots requesting additional information on the implementation of Article 7.
In order to assess the plans of the 28 Member States the study team developed a data capture template,
which was used to systematically analyse the documents submitted by Member States. The template
included sections on:
the baseline used to calculate the target and any exclusions made;
the exemptions applied;
the energy savings target;
the list of policy measures used and the projected energy savings of each policy measure; and
each individual policy measure covering the policy type, the calculation methods applied, the way
additionality is addressed; the eligible measures supported by the policy instrument; the lifetimes
used, and the monitoring and verification regime adopted.
For each Member State the template was populated with data, peer-reviewed by another member of
the study team, and updated several times to reflect the most up-to-date information. The information
presented in this paper is based on documents available up until 5th of October 2015.
Based on the 28 data templates we developed a database that forms the basis of the analysis presented
in this paper. For each of the individual policies and measures, we extracted information on the main
characteristics of the measure, and other information relevant to the calculation of the energy savings
of the measure. It is important to note that the information within the database is entirely based upon
the information included within information provided by Member States to the European
Commission; it has not been possible to validate or cross check this information against other sources
given the amount of material reviewed.
For each of the individual policy measures, the following information was captured:
policy instrument type;
expected cumulative energy savings in the period 2014-2020;
target sectors (inferred by expert judgement); and
lifetimes of the energy savings (inferred by expert judgement).
In addition, the templates were used to capture information relating to certain specific requirements
within Article 7 and Annex V, which concern the calculation of the energy savings towards the target
for Article 7. This is important because the savings are calculated bottom-up rather than top-down.
Based on the evidence gathered analysis was carried out on the following aspects:
eligibility;
additionality;
risk of non-delivery; and
risk of double counting.
The data was then used to carry out an aggregate analysis of the total savings that Member States
anticipate. Using qualitative data captured in the 28 data templates on the reliability of the savings
from individual policy measures we calculated the proportion of savings that were ineligible due to
not reflecting end-use energy savings, at risk of not being delivered and at risk of not being additional.
Furthermore, the database allowed us to assess which types of policy measures and sectors are
expected to deliver the energy savings.
Note that energy savings estimates provided by MS in their NEEAPs are highly uncertain for a
number of reasons. One of them is that it is often unclear on which basis the expected savings have
been calculated and only in some cases have Member States used ex-post evaluations of existing
policies to inform estimates of the likely energy savings from future policies. For this reason the
quantitative data on the expected energy savings presented in this paper should be treated with some
caution.
The uncertainty and reliability of policy impact estimates appears to be a general issue in European
energy and climate policy - less than 10% of the entries in the 2011 reporting cycle of the Monitoring
Mechanism on emissions reductions in Member States included quantitative data based on ex post
evaluations (Hilden et al. 2014). This finding is consistent with the analysis by Stern and Vantzis
(2014) who argue that most evaluations carried out in EU Member States rely on ex-ante estimates
whereas the in the US the use of ex-post evaluations is more common. There are also significant
differences with regard to the professional evaluation capabilities in the Member States (Huitema et
al. 2011), which partly explains the inconsistencies in Member States’ approaches.
4 Policy measures
In this section we provide an overview of the types of policy measures implemented across all 28
Member States. In total, Member States implemented or plan to implement 479 policy measures.
Some countries notified very few policy instruments (e.g. Italy) whereas others such as Germany or
Slovakia adopted 112 and 66 policy instruments respectively. Five Member States have notified a
single policy measure for the implementation of Article 7: Denmark, Poland and Bulgaria, and
Luxembourg notified only EEOs whereas Sweden exclusively uses energy/CO2 tax. This shows that
there are significant differences in how Member States comply with Article 7.
There have been attempts to develop criteria for selecting optimal policy measures for compliance
with the Energy Efficiency Directive (Mikucioniene et al. 2014) but in reality Member States do not
use a consistent approach when deciding on which policy measures to implement. In many cases
existing policies determine the selection of policy measures for compliance with Article 7 (75% of all
policy measures (Rosenow et al. 2015)), although some Member States have decided to follow the
implicit recommendation of Article 7 to adopt EEOs as the analysis below illustrates.
4.1 Categorisation
The Directive allows for the use of any policy measures (as alternative measures) that results in end-
use savings equivalent to the target defined by Article 7. It provides a typology of policy measures
that can be considered for implementation, which has also been used in this paper:
EEOs: EEOs oblige energy suppliers and/or distributors to deliver a specified amount of end-use
energy savings within a defined period of time.
Energy efficiency national fund: even though many MSs operate a national fund for financing
energy efficiency measure, in this context it means a fund where obligated parties can make an
annual financial contribution to fulfil their obligation under Article 7 as defined in Article 20(6).
Energy or CO2 taxes: a levy on the energy and/or carbon content of fuels above minimum EU-
requirements that - by increasing the price of the fuels- incentivises fuel saving. Financial stimuli
to energy efficiency investments through the taxation system (e.g. tax rebates for building
renovation) are included in the financing and fiscal incentive policy group.
Financing scheme or fiscal incentive: such schemes provide monetary support from public
sources that are allocated either on the basis of application (e.g. applying for a grant under a
renovation support scheme) or induce energy saving actions automatically (e.g. automatic
eligibility to tax concession when purchasing an electric vehicle).
Regulation or voluntary agreements: voluntary agreements are typically agreements by a sector -
or group of similar actors- with public authorities in which they commit to a) reduce end-use
energy consumption over time, b) design and implement an energy efficiency plan, or c) apply
specific energy efficient technologies. Regulations – in this context - are obligatory and legally
binding measures that do not belong in any of the other categories.
Standards and norms: these administrative measures aim at setting minimum energy efficiency
requirement of products and services in addition to mandatory EU requirements.
Energy labelling schemes: energy labels provide easy-to-understand energy use information of
products that facilitate energy-conscious consumer choices.
Training and education: educational actions that results in the use of efficient technologies or
behavioural changes reducing end use consumption.
Other policy measures: this category comprises any other policy measures that do not fit with the
main categories of policy instruments.
4.2 Share of different policy measures
Following the methodology set out in Section 3, we a) counted the number of policy measures by type
and b) aggregated the notified energy savings by policy instrument type. Note that these data are
purely based on what Member States expect and need to be treated with some caution.
The largest share of the overall savings is expected to be generated by EEOs (33%), financing
schemes or grants (19%), and from taxes (15%). Hence more than half of the savings are expected to
be delivered by policy instruments that provide a direct financial incentive to the target group(s) in
order to persuade the beneficiaries to invest in energy efficiency improvements. EEOs typically
involve a financial contribution from the obligated parties to the overall investment cost of energy
efficiency technologies/improvements. The remainder is paid by the beneficiary. Whilst there are
exceptions to this, for example if EEOs target low-income customers (Rosenow et al. 2013), the
majority of measures delivered by EEOs is only part-funded by the obligated parties (Rohde et al.
2014). From the perspective of the beneficiary EEOs provide them with an economic incentive to
install energy efficiency measures. Taxation measures provide an indirect financial incentive to invest
in energy efficiency as they increase the cost for using energy and reduce the payback periods of
energy efficiency improvements. Together, the instruments changing the cost profile of energy
efficiency investments are expected to generate about 2/3 of the overall savings.
Figure 1 provides an overview of both the number of the different policy measures by policy
instrument category. Figure 2 the share of the overall savings by policy instrument type.
Figure 1: The number of notified policy measures by policy measure type
Source: Forster et al. 2016
Figure 2: The expected energy savings [ktoe] by policy measure type
Source: Forster et al. 2016
The analysis shows that a small number of measures – essentially those genuinely horizontal in nature
in that they promote different energy efficiency improvements across a range of sectors - deliver a
large share of the total savings. In terms of the number of policy instruments, EEOs comprise just 4%
of all policy measures whereas in terms of expected energy savings their share is 33%. Similarly, the
12 notified energy and CO2 taxes (3% of the total number) are expected to deliver 15% of overall
savings. On the other hand, the financing schemes and fiscal measures policy group is more
fragmented (3% of policy measures deliver about 19% of savings): such support schemes are often
very specific according to the type of support (e.g. grant or loan), the target sector and even subsectors
(e.g. public buildings only).
5 Modelling the expected energy savings
The modelling of the expected savings included two separate steps. First, the notified energy savings
by policy measure were aggregated and segmented by policy measure type (see section 4), target
sector, and whether or not the savings were delivered by policy measures that existed prior to the
EED. Second, we carried out an assessment of the credibility of the notified savings using four
indicators (eligibility, additionality, risk of non-delivery, and double counting). The results of each
step are presented below.
The assessments were carried out at the level of all 479 individual policy measures as notified by the
Member States, but we present only the aggregated results on the EU28 level to keep the analysis
manageable. Detailed information for individual policy measures can be found in Forster et al. (2016).
5.1 Aggregation and segmentation of the energy savings
In the first step of the analysis of the savings to be delivered over time we aggregated the energy
savings by sector and developed projections to 2030 and 2050 so that the long-term impact of Article
7 can be demonstrated. In order to do this we had to make some adjustments to the raw data reported
by Member States.
First, Member States did not have to provide a sectoral split of the expected savings in their
notifications, and therefore in a number of cases the sectors had to be inferred by checking each of the
notified policy measures.
Second, the longevity of the energy efficiency improvements had to be derived for each policy
measure. Article 7 (and Annex V) requires the Member States to notify the lifetimes of the energy
savings action that are targeted by the policy measures. However, since the policy measures target
usually more than one type of energy saving action, we had to make expert judgements of the
distribution of lifetimes of the savings that are associated with each policy measure. We did this by
attributing lifetimes to the improvements targeted by each policy measure using standardised lifetimes
for energy efficiency actions provided by CEN (2007). The CEN lifetimes were chosen since they
provide the best available generally accepted overview of lifetimes of energy efficiency
improvements in Europe and Member States use different lifetimes for similar measures.
The likely savings generated by Article 7 have been estimated in the Impact Assessment (EC 2011a).
The Impact Assessment assumed that, by 2020, annual savings in primary energy of 108-118Mtoe per
year will be delivered by Article 7. This figure was based on the Commission’s proposal and does not
include exemptions and policy overlaps. Subsequent analysis by the European Commission (EC
2015b), based on the final negotiated EED text, provides an estimate for annual savings in 2020 of
84.8Mtoe (primary energy).
Those projections can be compared with the individual energy saving targets and policy savings
calculated by MS, and set out in their notifications. A number of adjustments are necessary to do this:
1) Convert to annual savings: The figures provided by MS are cumulative savings by 2020 and need
to be converted to annual savings. We have assumed linear delivery of savings from 2014 to 2020
(that is, the same additional savings are generated every year).
2) Convert final to primary energy savings. The figures in the Impact Assessment are presented in
primary energy savings. The energy savings provided by MS are calculated in final energy
consumption. We have converted the energy savings from final energy to primary energy. To do
this we have:
calculated the share of electricity of the total final energy consumption, which is 21.8% based
on Eurostat data (the other fuels used are already included as primary energy in the final
energy consumption figures);
assumed that the savings would be proportionate according to the share of fuels of final energy
consumption (there may be a discrepancy as MS do not provide a breakdown of the savings
according to fuel); and
applied a conversion factor of 2.5 to convert electricity to primary energy (this is a factor in
line with Annex IV of the Energy Efficiency Directive.
Figure 3 compares the expected energy savings and targets to the original Impact Assessment and the
subsequent estimate by the European Commission based on the final EED text.
Figure 3: Comparison of proposed energy savings targets and policy measures to the Impact
Assessment and Commission estimate based on final EED text
1
Our analysis of the data from MS notifications shows that the energy savings targets and the savings
from policy measures are 10% and 2% lower respectively than the estimate provided by the
Commission, based on the final EED text. This means that a) Member States plan to over-deliver
against their energy saving targets and b) that the expected energy savings are close to the ambition of
the negotiated EED text.
In the period 2014-2020 most of the savings (44%) come from measures that are cross cutting across
more than one sector (such as taxes and financial incentives applying to multiple sectors). 42% of the
savings is expected by the Member States to come from buildings. This is in line with the large
potential for energy efficiency improvements in buildings (Braungardt 2014). Apart from the category
‘cross cutting’, the direct contribution from industry is much smaller (8%), and transport smaller still
(6%).
5.2 Assessment of the credibility of the notified savings
The energy savings presented above are based on the estimates provided by Member States in their
notifications. However, it is necessary to consider whether these estimates of the energy savings are
realistic and credible in all cases, and can be considered additional to what would have happened in
the absence of the EED. In some cases, for example, Member States may have notified measures that
are not eligible for meeting the Article 7 target. It is therefore necessary to make an adjustment of the
overall savings to better reflect what is really expected to be delivered by Article 7, in terms of
cumulative energy savings.
We used four indicators to assess the credibility of the notified energy savings:
Eligibility: This indicator addresses the purpose of the policy measure, i.e. whether the measure is
primarily targeted at achieving end-use energy savings or whether it mainly focuses on other
objectives e.g. renewable energy deployment. Only policy measures that deliver end-use energy
savings are eligible.
1
Hungary has not provided estimates for policy savings yet.
Additionality: This indicator relates to the additionality of the policy measures to minimum EU
standards and in particular whether or not the requirements of the EPBD have been taken into
account when calculating the energy savings.
Risk of non-delivery: This indicator addresses the risk on non-delivery of the notified amount of
savings. This depends on a wide range of issues such as potential over-estimations of energy
savings due to methodological shortcomings.
Risk of double counting: This indicator encapsulates that potential for overlap between policy
measures targeting similar sectors and, as a result, the risk for double counting of energy savings.
For each of the indicators we analysed the evidence provided by Member States for each policy
measure against a set of evaluation questions for each indicator. The results of the analysis for all
indicators are presented in Table 1. Due to the process of the EU Pilots during 2015, there has been a
significant improvement in the completeness and quality of the notified information.
However, currently only 14% of all energy savings have been rated as fully eligible, fully additional,
at low risk of double counting and at low risk of non-delivery. This means that 86% of all savings are
at least partially at risk of not being realised.
Table 1: Credibility assessment of notified energy savings
Indicator
Result
Eligibility
Fully eligible
Mainly eligible (>50% of savings eligible)
Mainly not eligible (>50% of savings not eligible)
Unclear
68%
26%
5%
1%
Additionality
Fully additional
Mainly additional (>50% of savings additional)
Mainly not additional (>50% of savings not additional)
Unclear
43%
24%
14%
19%
Risk of non-delivery
Low
Medium (>50% of savings likely to be delivered)
High (>50% of savings at risk of not been delivered)
Unclear
57%
13%
6%
24%
Risk of double counting
Low
Medium (>50% of savings not at risk of double counting )
High (>50% of savings at risk of double counting)
Unclear
81%
12%
1%
6%
In addition to the issues listed in the table above there are considerable uncertainties around the
calculation of the energy savings including how the performance gap is taken care of, whether
rebound effects are accounted for, if savings are adjusted for free-ridership, and how heterogeneity of
consumers is addressed. We discuss those issues in more detail in section 6.3.
6 Discussion
Assessing the plans of Member States involves considerable challenges both in terms of the
complexity of the subject matter as well as the quantity of material that needs to be assessed. The
results of this paper are based on a detailed analysis over the course of two years analysing 7,653
pages of material submitted by the Member States to the European Commission excluding any
material referenced in the documents. This is equivalent to 274 pages of material per Member State
and given that some Member States which did not yet have fully developed implementation plans
supplied only a minimal amount of information the volume of material is likely to increase over time.
The analysis above illustrates that there are considerable uncertainties around the reliability of the
energy savings estimates provided by Member States. The issue of eligibility of notified savings (e.g.
those from renewable energy technologies) can be expected to be resolved as this is a simple
compliance question. Double counting does not affect a large part of the notified savings. This means
that additionality and the risk of non-delivery are key concerns. The risk of non-delivery identified
here derives from the lack of a consistent approach to monitoring and verification systems set up by
Member States, and multiple methodological issues often not addressed by Member States when it
comes to calculating energy savings from specific policy measures.
Hence the main areas of concern include:
risk of non-additionality of energy savings; and
weak or even absent monitoring and verification regimes; and
methodological issues related to the calculation of energy savings.
We address each of those areas in turn before we provide a number of suggestions for policy reform.
6.1 Additionality
A significant part of the savings is at risk of not being additional to energy efficiency improvements
that would occur even in absence of the policy measures notified by Member States. Although some
Member States designed robust and comprehensive policy packages, additionality appears to be the
most important concern.
The additionality of energy efficiency programmes has been discussed in the literature for some time
(Vine and Sathaye 2000). Given that additionality is recognised as being an important element of
energy efficiency policy the EED makes important provisions for how additionality should be
ensured. First, any savings notified under Article 7 must be additional to existing EU minimum
requirements. In particular, this includes the Energy Performance of Building Directive (Directive
2002/91/EC, and Directive 2010/31/EU) and the Ecodesign Directive (Directive 2009/125/EC).
Second, when calculating energy savings Member States need to give consideration to the potential
impact of free-riders i.e. beneficiaries of the policies that would have undertaken energy efficiency
improvements even in absence of the policies. The issue of free-ridership has been discussed in the
literature at length (e.g. Saxonis 1991) but in our analysis we found only very few Member States who
appear to have systematically excluded free-rider effects from their estimates. This lack of a
counterfactual appears to be a common problem in European climate policy evaluation (Haug et al.
2010).
One reason for the small number of Member States who addressed additionality comprehensively is
likely to be the scarcity of detailed guidance on how to address additionality issued by the European
Commission and, resulting from this, a lack of understanding by Member States of what is required.
6.2 Monitoring and verification
Whilst the information Member States submitted on their energy targets, the policy measures and the
expected savings is relatively complete there are substantial gaps with regard to monitoring and
verification regimes adopted across the EU. In many cases the monitoring and verification system is
described in the NEEAPs and the Article 7 notifications at a very high level only whereas in other
instances even the most basic information is missing. However, partial or missing information on
monitoring and verification does not necessarily imply that there are no robust monitoring and
verification systems. Still, there is a significant risk that monitoring and verification regimes are weak
and do not ensure that the estimated energy savings will be delivered in reality.
Recent analysis by Schlomann et al. (2015) illustrates that this is largely a result of the lack of binding
rules for monitoring and verification at the EU level that provide sufficient detail and clarity to
Member States. While Annex V of the EED sets out the basic requirements for monitoring and
verification and the guidance note on Article 7 provides further explanations of how the requirements
can be addressed, they do not set out in detail how monitoring and verification need to be addressed.
This lack of clarity provides potential loopholes and does not result in a consistent approach to
monitoring and verification across the EU. Member States adopt different approaches to calculate
their energy savings, and report on their methodologies in different ways. This may be well justified,
since some calculation approaches are better suited to some policies than others. However, as a result
of this flexibility, the energy savings that are notified by Member States, and the information reported
on methodologies, are not fully consistent or comparable at an EU level. This inconsistency presents
uncertainty about whether the EU is on track to deliver its target, and reduces the integrity of the
savings that are claimed at an EU level.
There have been attempts to develop detailed guidance on monitoring and verification elsewhere. For
example, the US Environmental Protection Agency has developed measurement and verification
guidance for demand-side energy efficiency which is available in draft (EPA 2015). A similar
document would be useful for the EU context.
6.3 Calculation of energy savings
Energy savings estimates often do not account for factors that reduce the estimated savings. It has not
been possible to review if and how those factors have been accounted for in Member States’
estimations for all policy measures but initial probing suggests that for a large proportion of cases this
may not be the case.
In principle, energy efficiency improvements can be offset by increased demand for energy services
due to the rebound effect (Greening et al. 2000, Sorrell 2007). There are two components. Direct
rebound is caused by reduced energy costs for the service for which energy efficiency has been
improved. Indirect rebound is due to spending of the financial savings and its spillover effects in the
wider economy. Direct rebound effects tend to be in the range 0-30% for major energy services such
as heating and cooling (Sorrell et al. 2009), but more prominent in lower income groups (Hens et al.
2009). Overall, it is a small, but not negligible, effect in EU countries and is increasingly accounted
for in programme evaluation (Wade and Eyre, 2015). Knowledge about indirect rebound effects is
much weaker and therefore it is generally neglected in programme evaluation. Evidence relies very
largely on economic modelling and is very diverse. Indirect rebound effects may be very large for
industrial technologies experiencing very rapid deployment (Sorrell 2007), but there is no basis for
assuming large effects elsewhere. Declining energy consumption trends in the EU as energy
efficiency has improved indicate very small indirect rebound effects.
Assessments of energy efficiency programmes in buildings need to take account of the energy
performance gap, i.e. the growing body of evidence that energy efficiency projects reduce actual
energy consumption by less than the prediction of simple building physics models (e.g. Davis et al.
2014; Fowlie et al. 2016; Wingfield et al. 2008). The effect is partly due to direct rebound, but also
can be affected by the quality of building projects, (lack of) training of users with regard to their new
technologies / measures, by unrealistic assumptions about energy use in poorly heated buildings
before retrofit (Sunikka-Blank and Galvin 2012), and by ignoring heterogeneity of consumer
responses to energy efficiency improvements (Hausman and Joskow 1982). Techniques are under
development to address the effect, including post-occupancy evaluation, e.g. (Menezes et al. 2012)
and feedback to building occupants. (Gupta and Chandiwala 2010).
Initial probing of Member States’ calculation methods suggests that so far only few countries in the
EU systematically account for the effects discussed above. The use of these factors should be taken
into account in future programme evaluation (where this is not already the case) for the purpose of
reporting on Article 7.
6.4 Suggestions for policy reform
As illustrated above, the key issues that affect the reliability of the expected energy savings include
the potential non-additionality of energy savings, and the lack of robust monitoring and verification
regimes. For each of those issues suggestions for policy reform are presented below. An overarching
suggestion is to revisit the requirements in the Directive related to additionality, policy overlaps and
monitoring and verification with the view of providing more clarity and detail. Alongside this
templates covering all of the requirements in a systematic manner accompanied by clear guidance
would a) enable Member States to understand what exactly is required and how they have to report
compliance and b) help the Commission with ensuring that the EED is implemented as intended.
6.4.1 Ensuring additionality
The intention of the EED is to deliver additional energy savings to the status quo. Therefore a number
of provisions are made in the Directive to take into account existing EU minimum requirements and
take free-rider effects into account in the calculation of energy savings from policy measures. In order
to achieve this Member States need to estimate the savings from a policy instrument and subtract the
portion of savings from the policy instrument that would be delivered by existing EU minimum
requirements as well as the estimated free-rider effects. Only some Member States currently
demonstrate they have a comprehensive methodology in place.
One reason for the inconsistent approach to additionality is that the requirements in the Directive are
not always clear. For example, Annex V lists some existing EU minimum requirements explicitly but
not others which has led to confusion and loopholes. For example, the Commission expects Member
States to take into account the cost-optimal path for energy efficiency set by the EPBD when using
building regulations. However, the EPBD is not mentioned in Article 7 and Annex V which is why
some countries argued that there is no legal obligation to include the cost-optimal path of the EPBD in
their calculations.
As a way forward, Annex V should state comprehensively which EU minimum requirements need to
be considered. In addition, clear guidance on how to factor in EU minimum requirements in energy
savings calculations with some worked examples would enable Member States to follow this
approach more consistently. Finally, the EED should require Member States to report to the
Commission in detail how they have ensured that savings from existing EU minimum requirements
are not included in their estimates.
6.4.2 Strengthening the monitoring and verification regime
The inconsistent approach to measuring energy savings and monitoring and verification leads to
considerable uncertainties as to whether the anticipated energy savings will be delivered. Following
the implementation process of the Energy Services Directive in 2006 similar issues were discussed in
the literature (Boonekamp 2006; Thomas et al. 2012). This literature can form the basis of a clear and
consistent approach to monitoring and verification of energy savings across the EU. The Commission
should establish more detailed guidance and clarify the requirements in Article 7 and Annex V to
address the currently incomplete understanding amongst Member States.
6.4.3 Ensuring a more consistent calculation approach
Annex V of the Directive sets out the ‘common methods and principles’ to be used in measurement of
savings. Subject to the issues addressed above, the principles, such as additionality and transparency,
are adequate. However, the methods are less satisfactory. Of the four allowed ‘methods, two are
‘scaled savings’ and ‘surveyed savings’. These are not well-defined and it is not clear why they are
required in addition to the two well-established evaluation approaches of ‘deemed savings’ and
‘metered savings’, for which there is good practice relying on agreed monitoring and verification
protocols that use statistically valid data from previous and current installations respectively. Well-
established national obligation schemes (in Europe and elsewhere) have found it necessary to
developed very detailed rules. It would not be sensible for such set of rules to be fixed in a Directive,
but some common basis is required if the savings rules are to be transparent across Member States. It
would be appropriate to rely on the established EU procedure of ‘comitology’ under which experts
from Member States could agree such rules. These could incorporate guidance, templates and
examples, as well being open to amendment as schemes develop.
7 Conclusions
Given that the Energy Efficiency Directive and particularly Article 7 will be the primary delivery
mechanism at EU level to encourage energy savings, this paper assessed to what extent Article 7 is
likely to fulfil these expectations.
Based on a vast amount of information provided by Member States to the European Commission
(7,653 pages of material from NEEAPs, Article 7 notifications, EU Pilots and additional documents)
we systematically analysed which types of policy measures Member States implemented or plan to
implement in order to comply with Article 7. We also carried out a quantitative analysis of the
notified energy savings by instrument type and sector. The paper illustrated that there are considerable
uncertainties around the reliability of the expected energy savings resulting from the inclusion of non-
energy efficiency measures, the potential non-additionality of savings, double counting, the risk of
non-delivery, and the implications of weak monitoring and verification systems. For each of those
issues we provided an indication of the share of the energy savings that could be affected. Our
analysis illustrates that a significant share of the expected savings is at risk of not being delivered in
practice, although it is impossible to calculate the effect at this stage. This puts into question whether
the EED will achieve its aims.
A number of suggestions for policy reform were developed that would strengthen the Directive and
increase the reliability of the anticipated energy savings. Overall, the lack of clarity of the
requirements with regards to what is required and how it needs to be reported can be addressed by
more detailed provisions, extensive guidance, and reporting templates that ensure Member States
follow a more consistent approach in calculating the savings and reporting them as well as outlining
their monitoring and verification regimes.
In addition, Member States have a responsibility for refining their plans to address the issues
discussed above – they need to respond to the spirit as well as the letter of the legislation. This
includes a more systematic development of evaluation capabilities to reflect the ambitious
requirements in the Energy Efficiency Directive.
8 Bibliography
Bertoldi, P., Waide, P. & Lebot, B. (2001): Assessing the market transformation for domestic
appliances resulting from European Union policies. ECEEE Summer Study
Blumstein, C., Krieg, B., Schipper, L., York, C. (1980): Overcoming social and institutional barriers
to energy conservation. Energy, 5, 355-371.
Boonekamp, P. (2006): Evaluation of methods used to determine realized energy saving. Energy
Policy 34, pp. 3977-3992.
S. Braungardt, R. Elsland, T. Fleiter, M. Klobasa, B. Pfluger, M. Reuter, et al. (2014): Study
evaluating the current energy efficiency policy framework in the EU and providing orientation on
policy options for realising the cost-effective energy efficiency / saving potential until 2020 and
beyond. Fraunhofer ISI, TU Vienna, PWC on behalf of DG ENER, Karlsruhe, Vienna, Rome
CEN (2007): Saving lifetimes of energy efficiency improvement measures in bottom-up calculations,
CWA 15693.
Davis, L.W., Fuchs, A., Gertler, P. (2014): Cash for Coolers: Evaluating a Large-Scale. Appliance
Replacement Program in Mexico, American Economic Journal: Policy 6(4), pp. 207–238
Delbeke, J., Vis, P. (2015): EU Climate Policy explained. Routledge.
EC (2015a): 2020 climate & energy package. Online:
http://ec.europa.eu/clima/policies/strategies/2020/index_en.htm [accessed 15/10/2015]
EC (2015b): Unpublished estimate of expected energy savings based on negotiated text of the Energy
Efficiency Directive
EC (2011a): Impact Assessment accompanying the document Directive of the European Parliament
and of the Council on energy efficiency and amending and subsequently repealing Directives
2004/8/EC and 2006/32/EC {COM(2011) 370 final} {SEC(2011) 780 final}
EC (2011b). Communication from the Commission to the European Parliament, the Council, the
European Economic and Social Committee and the Committee of the Regions. Energy Efficiency
Plan {COM (2011) 109 final}
Edenhofer, O., Pichs-Madruga, R., Sokona et al. (2014): Mitigation of Climate Change. Working
Group III Contribution to the IPCC Fifth Assessment Roport, Climate Change 2014. Summary for
Policymakers, IPCC.
EP (2012): Directive 2012/27/EU of The European Parliament and of The Council on energy
efficiency. L 315/1. OJEU.
EPA (2015): Evaluation Measurement and Verification (EM&V) Guidance for Demand-Side Energy
Efficiency (EE). Draft for public input [Online]. Available:
http://www.epa.gov/sites/production/files/2015-08/documents/cpp_emv_guidance_for_demand-
side_ee_-_080315.pdf [Accessed 7th February 2016]
Eyre, N. (1997): Barriers to energy efficiency more than just market failure. Energy and Environment,
8, 25-43.
Eurostat (2015): Energy production and imports [Online]. Available:
http://ec.europa.eu/eurostat/statistics-explained/index.php/Energy_production_and_imports [Accessed
1st December 2015.
Filippini M., Hunt L., Zoric J. (2014): Impact of energy policy instruments on the estimated level of
underlying energy efficiency in the EU residential sector. Energy Policy 69, pp. 73-81.
Forster, D., Kaar, A.-L., Rosenow, J., Leguijt, C., Pato, Z. (2016): Study evaluating progress in the
implementation of Article 7of the Energy Efficiency Directive. Study for the European Commission.
Fowlie, M, Greenstone, M., Wolfram, C. (2015) `Do energy efficiency investments deliver? Evidence
from the weatherization assistance program. NBER Working Paper No. 21331. [Online]
http://faculty.haas.berkeley.edu/wolfram/Papers/paper_draft_06_15_clean.pdf [Accessed 7thFebruary
2016]
Gillingham, K., Newell, R.G., Palmer, K. (2009): Energy Efficiency Economics and Policy. National
Bureau of Economic Research Working Paper Series, No. 15031.
Greening, L. A., Greene, D.L., Difiglio, C. (2000): Energy efficiency and consumption -- the rebound
effect - a survey. Energy Policy, 28, 389-401.
Gupta, R. & Chandiwala, S. (2010): Understanding occupants: feedback techniques for large-scale
low-carbon domestic refurbishments. Building Research & Information, 38, 530-548.
Haug, C., Rayner, T., Jordan, A., Hildingsson, R., Stripple, J., Monni, S., Huitema, D., Massey, E.,
van Asselt, H. and Berkhout, F. (2010): Navigating the dilemmas of climate policy in Europe:
evidence from policy evaluation studies. Climatic Change 101(3–4), pp. 427–445
Hausman, J., Joskow, P. (1982): Evaluating the costs and benefits of appliance efficiency standards.
American Economic Review 72(2), pp. 220-225
Hedenus, F., Azar, C. & Johansson, D. J. (2010): Energy security policies in EU-25—the expected
cost of oil supply disruptions. Energy Policy, 38, 1241-1250.
Hens, H., Parijs, W. & Deurinck, M. (2009): Energy consumption for heating and rebound effects.
Energy and Buildings, 42, 105-110.
Hildén, M., Jordan, A.J., Rayner, T. (2014): Climate policy innovation: developing an evaluation
perspective. Environmental Politics 23(5), pp. 884–905
Hirst, E. & Brown, M. (1990): Closing the efficiency gap: barriers to the efficient use of energy.
Resources, Conservation and Recycling, 3, 267-281.
Hoos, E. (2012): Energy Efficiency Directive: Making energy efficiency a priority. Unit C3 Energy
Efficiency DG Energy, European Commission 5/10/2012 Stockholm GEODE Conference
Huitema, D., Jordan, A., Massey, E., Rayner,T., van Asselt, H., Haug, C., Hildingsson, R., Monni, S.,
and Stripple, J. (2011) The evaluation of climate policy: theory and emerging practice in Europe.
Policy Sciences 44(2), pp. 179-198
IEA (2014): Capturing the multiple benefits of energy efficiency. Paris: International Energy Agency.
Jaffe, A. B. & Stavins, R. N. (1994): The energy-efficiency gap What does it mean? Energy Policy,
22, 804-810.
Lucon, O., Urge-Vorsatz, D., Ahmed, A. Z., Akbari, H., Bertoldi, P., Cabeza, L. F., Eyre, N., Gadgil,
A., Harvey, L.D.D., Jiang, Y., Liphoto, E., Mirasgedis, S., Murakami, S., Parikh, J., Pyke, C. &
Vilariño, M.V. (2014): Chapter 9: “Buildings”. Intergovernmental Panel on Climate Change, 5th
Assessment Report, Working Group III.: Cambridge University Press.
Menezes, A. C., Cripps, A., Bouchlaghem, D. & Buswell, R. (2012): Predicted vs. actual energy
performance of non-domestic buildings: Using post-occupancy evaluation data to reduce the
performance gap. Applied Energy, 97, 355-364.
Mikucioniene R., Martinaitis V., Keras, E. (2014): Evaluation of energy efficiency measures
sustainability by decision tree method, Energy and Buildings 76, pp. 64-71.
NAS (2010): National Academy of Sciences. Real prospects for energy efficiency in the United
States.
Rohde, C., Rosenow, J., Eyre, N. (2014): Energy Saving Obligations. Cutting the Goardian Knot of
leverage? Energy Efficiency 8(1), pp. 129-140
Rosenow, J., Forster, D., Kampman, B., Leguijt, C., Pato, Z., Kaar, A.-L., Eyre, N. (2015): Study
evaluating the national policy measures and methodologies to implement Article 7 of the Energy
Efficiency Directive. Study for the European Commission.
Rosenow, J., Fawcett, T., Eyre, N., Oikonomou, V. (forthcoming): Energy efficiency and the policy
mix. Building Research & Information.
Rosenow, J., Galvin, R. (2013): Evaluating the Evaluations: evidence from energy efficiency
programmes in Germany and the UK. Energy & Buildings 62, pp. 450-485
Rosenow, J., Platt, R., Flanagan, B. (2013): Fuel poverty and energy efficiency obligations. The case
of the Supplier Obligation in the UK. Energy Policy 62, pp. 1194–1203
Sanstad, A. H. & Howarth, R. B. (1994): Normal Markets, Market Imperfections and Energy
Efficiency. Energy Policy, 22, 811-818.
Thomas, S., Boonekamp, P., Vreuls, H., Broc, J.-S., Bosseboeuf, D., Lapillonne, B., Labance, N.
(2012): How to measure the overall energy savings linked to policies and energy services at the
national level? Energy Efficiency. 5, pp. 19-35.
Saxonis, W. (1991): Free riders and other factors that affect net program impacts. In: Hirst, E., Reed,
J. (eds.), Handbook of Evaluation of Utility DSM Programs. Oak Ridge, TN. Oak Ridge National
Laboratory.
Saussay, A., Saheb, Y. & Quirion, P. (2012): The Impact of Building Energy Codes on the Energy
Efficiency of Residential Space Heating in European Countries – A Stochastic Frontier Approach.
International Energy Program Evaluation Conference. Rome.
Schiellerup, P. (2001): An examination of the effectiveness of the EU minimum standard on cold
appliances: the British Case. ECEEE summer study.
Schlomann, B. Rohde, C., Plötz, P. (2015): Dimensions of Energy Efficiency in a Political Context.
Energy Efficiency 8 (1), pp. 97–115.
Shove, E. (1998): Gaps, barriers and conceptual chasms: theories of technology transfer and energy in
buildings. Energy Policy, 26, 1105-1112.
Sorrell, S. (2007): The Rebound Effect: an assessment of the evidence for economy-wide energy
savings from improved energy efficiency, UK Energy Research Centre.
Sorrell, S., Dimitropoulos, J., Sommerville, M. (2009): Empirical estimates of the direct rebound
effect: A review. Energy Policy, 37, 1356-1371.
Sorrell, S., O'Malley, E., Schleich, J. & Scott, S. (2004): The Economics of Energy Efficiency:
Barriers to Cost Effective Investment, Cheltenham, Edward Elgar.
SRC (2001): A European Ex-post evaluation guidebook for DSM and EE service programmes. report
to the European Commission SAVE programme.
Stern, F., Vantzis, D. (2014): Protocols for Evaluating Energy Efficiency – Both Sides of the Atlantic.
International Energy Policy and Programme Evaluation Conference, 9-11th September 2014, Berlin,
Germany.
Stern, J. (2006): Natural gas security problems in Europe: the Russian–Ukrainian crisis of 2006. Asia-
Pacific Review, 13, 32-59.
Stern, P. C. (2000): New Environmental Theories: Toward a Coherent Theory of Environmentally
Significant Behavior. Journal of Social Issues, 56, 407-424.
Sunikka-Blank, M., Galvin, R. (2012): Introducing the prebound effect: relationships between energy
performance ratings and actual heating energy consumption in German dwellings, and their policy
implications. Building Research and Information 40 (3), pp. 260–273
Urge-Vorsatz, D., Eyre, N., Graham, P., Harvey, D., E., H., Jiang, Y., Kornevall, C., Majumdar, M.,
Mcmahon, J., Mirasgedis, S., Murakami, S. & Novikova, A. (2012): Energy End-Use: Buildings. In:
Johansson, T., Nakicenovic, N., Patwardhan, A. & Gomez-Echeverri, L. (eds.) Global Energy
Assessment. Cambridge: Cambridge University Press.
Vine, E.L., Sathaye, J.A. (2000): The monitoring, evaluation, reporting, verification, and certification
of energy-efficiency projects. Mitigation Adaptation Strategies Global Change 5, pp. 189–216.
Wade, J., Eyre, N. (2015): Energy Efficiency Evaluation: The evidence for real energy savings from
energy efficiency programmes in the household sector. A report by the UKERC Technology & Policy
Assessment Function.
Wingfield, J., Bell, M., Miles-Shenton, D., South, T. & Lowe, R. (2008): Lessons from Stamford
Brook: understanding the gap between designed and real performance.
Yafimava, K. (2011): The transit dimension of EU energy security: Russian gas transit across
Ukraine, Belarus, and Moldova. OUP Catalogue.