Are EMS environmentally effective? The link between environmental management systems and environmental performance in European companies

Abstract

Based on the analysis of a large dataset on the environmental performance of European companies in five industrial sectors, this paper examines the question of whether the presence of an environmental management system (EMS) has a positive impact on the eco-efficiency of companies. It begins with a review of evidence about the link between EMS and environmental performance in business organisations, finding that, despite much research, there is still little quantitative evidence. The second part of the paper uses three independent statistical methods (simple correlations, Jaggi-Freedman indices and a 'trend differences' approach) to assess whether companies and production sites with EMS perform better than those without and whether performance improves after an EMS has been introduced. The paper shows that there is currently no evidence that EMS have a consistent and significant positive impact on environmental performance. Policy action based on the simple assumption that companies with an EMS perform better than those without therefore seems inappropriate.

Full text

Page 1

1

Are EMS environmentally effective?
The link between environmental management systems and environmental
performance in European companies


Julia Hertin1, Frans Berkhout1, 2, Marcus Wagner3, Daniel Tyteca4


1 SPRU – Science and Technology Policy Research
University of Sussex
United Kingdom

2 Institute for Environmental Studies
Vrije Universiteit Amsterdam
The Netherlands*

3 TUM Business School, Technische Universität München and
Centre for Sustainability Management, University of Lüneburg
Germany

4 Université catholique de Louvain
Institut d'Administration et de Gestion
Belgium

* Corresponding author


Published as: Hertin, J., Berkhout, F., Wagner, M., Tyteca, D., Are EMS environmentally
effective? The link between environmental management systems and the environmental
performance of companies, Journal of Environmental Policy & Management, vol 51, no 2,
2008:259-283.

Page 2

2
Abstract
Based on the analysis of a large dataset on the environmental performance of European
companies in five industrial sectors, the paper examines the question of whether the
presence of an environmental management system (EMS) has a positive impact on the eco-
efficiency of companies. It begins with a review of evidence about the link between EMS
and environmental performance in business organisations, finding that, despite much
research, there is still little quantitative evidence. The second part of the paper uses three
independent statistical methods (simple correlations, Jaggi-Freedman indices and a ‘trend
differences’ approach) to assess whether companies and production sites with EMS
perform better than those without and whether performance improves after an EMS has
been introduced. The paper shows that there is currently no evidence that EMS have a
consistent and significant positive impact on environmental performance. Policy action
based on the simple assumption that companies with an EMS perform better than those
without therefore seems inappropriate.

Page 3

3
1. Introduction
Policy instruments that rely on voluntarism, learning processes and procedural change,
rather than direct regulatory control have in recent years come to play a more prominent
role in the environmental policy mix of many industrialised countries. They have been
promoted by those who maintain that traditional hierarchical regulation does not provide
dynamic incentives for environmental improvement, and that voluntary approaches
increase the cost-effectiveness of environmental protection by giving firms flexibility in
making their own choices, reduce the information and administrative costs of regulation
and are a way of distributing the social control of business. Critics, on the other hand, are
sceptical that these ‘soft’ instruments can deliver real environmental improvements.

This paper contributes to the debate on the impact of new environmental policy instruments
by presenting evidence on the effectiveness of one of the most prominent of these
instruments: environmental management systems (EMS). It draws on the Measuring
Environmental Performance of Industry (MEPI) project, which collected and analysed
environmental performance data for 274 companies and 400 production sites (operated by
those firms) in six manufacturing sectors in six EU countries.

The first section discusses the rationale behind the increasing adoption of voluntary and
procedurally-based instruments in general, and EMS in particular. We then review existing
empirical evidence regarding the link between EMS and environmental performance, based
on research carried out in Germany, Austria, Switzerland, the UK and Finland. The main
part of the paper presents the authors' own analysis. It is based on data collected during the
MEPI research project and uses three different methods: 1) Statistical analysis on the firm-
level (multiple regressions); 2) Statistical analysis on the production site-level (simple
regressions); and 3) Longitudinal analysis on the production site-level. The final part draws
conclusions about the potentials and limits of environmental management systems, and
briefly explores the wider implications of the analysis for the role of soft policy instruments
in the environmental policy mix.

Page 4

4
2. New policy instruments and the link between environmental management and
performance
Much has been written about a shift from traditional hierarchical regulation towards a
different set of instruments in environmental policy-making (Andrews, 1998; Gunningham
and Grabosky 1998; Coglianese and Nash, 2001; Khanna, 2001). The widely used term
'new policy instruments' includes a range of different coordinating and steering
mechanisms including economic, procedural, information-based, self-regulatory, co-
regulatory and voluntary instruments. They have in common that they aim to achieve their
objectives by means other than the hierarchical prescription of legally-binding rules and
standards which can be enforced by public authorities.

Although it can be argued whether new policies complement or replace the 'command and
control' approach, it is now widely accepted that the use of alternative instruments is indeed
increasing in many countries (Jordan et al 2003). Looking empirically at the factors that
have led to the adoption of new environmental policy instruments in eight industrialised
countries, Jordan and colleagues (2003a: 202-205) identified a range of different
contributory factors, some of which relate to changing ideas and beliefs, while others stem
from organisational, political and economic factors. One of the key drivers is seen to be the
assumption that new instruments are a more effective way of achieving environmental
improvements.

The effectiveness of these new policy instruments has been widely discussed, especially
with regard to economic instruments (Tietenberg 1991; Newell et al. 1999). Here, we focus
on a different range of policies that have been called 'soft instruments'. This term describes
instruments that aim to achieve environmental aims without employing direct coercion
through law, or induce change by altering relative prices. They include voluntary,
procedural and information-based policies. Although the term ‘soft’ appears vague, it
accurately describes the main characteristics of these policies: to attain environmental
policy objectives without introducing legal or economic (i.e. ‘hard’) constraints. Prominent
examples of soft policy instruments are environmental management systems,
environmental product labelling, public disclosure requirements, best practice
dissemination, industry codes of practice, and voluntary agreements.

Page 5

5
Can soft instruments be more effective than conventional regulation? A variety of opinions
exist. On the one hand, it has been shown that voluntary programmes that place the least
demands on firms have attracted the highest participation rates (Davies and Mazurek,
1996). There is also evidence that even among firms that do participate in voluntary
programmes, there remain a disproportionate number of poor performers and that members
do not improve faster than non-members (King and Lenox, 2000). On the other hand, there
are also those who find that voluntary initiatives have a positive impact on firm
environmental performance. For instance, econometric analysis has shown that voluntary
programmes such as the US EPA’s 33/50 Program, which aimed to reduce releases and
transfers of a list of toxic chemicals, did indeed induce statistically-significant declines in
the releases of these chemicals (Arora and Cason, 1995; Khanna and Damon, 1999).

The intellectual basis of soft instruments is provided by recent cognitive approaches
applied across the social sciences (for example Schön 1983; Dryzek 1987; Fischer 1995;
Weick 1996). Cognitive approaches argue that the behaviour of actors is to a large degree
determined by their subjective interpretation of reality, rather than being the outcome of
'objective' and rationally-determined interests. It follows that any attempt to change
behaviour needs to be based on an understanding of the frames of interpretation, discourses
and knowledge sets which influence how these actors make sense of their world and action
within it, and how they respond to changes in interpretive frames, discourses and so on.
More specifically, soft instruments are based on the assumption that polluting behaviour is
(at least in part) the result of institutionally-situated perceptions of reality (or ignorance
about the state of things). Interpretive frames that stand in the way of environmentally
beneficial decisions could be, for example, the assumption that reducing environmental
damage is always associated with costs, that companies do not have any environmental
responsibilities beyond legal compliance, or that environmental resources are free goods.

Closely related to the first, a second argument is that by changing the sense-making of
individuals and organisations, it is possible to change the attitudes and behaviours of those
individuals and organisations – which, in turn, will ultimately have an impact on the
environmental impacts of behaviours. This could be achieved by providing information
(for example about environmental costs or best practice), through more subtle and long-
term processes of learning and capacity-building, or through processes of awareness-raising
about the liabilities and responsibilities of the polluter.

Page 6

6

By encouraging organisational change, EMS are thought to have a direct impact on
environmental performance. For instance, the preamble to the EMAS (Eco-Management
and Audit Scheme) regulation of the European Union states:

“The objective of EMAS shall be to promote continual improvements in the
environmental performance of organisations” (EMAS regulation, Art 1.2)

These improvements in performance are to be achieved through the imposition of
management controls. However, this link between management and performance cannot be
taken for granted. Research has documented that improving environmental performance is
not usually the principal motive in a company's decision to adopt an EMS. A business
survey carried out amongst Swiss firms identified 14 reasons for implementing an EMS
which were considered to be ‘very important’ or ‘quite important’ by at least half of the
158 respondents (Hamschmidt 2000). The benefits included in this list ranged from
‘strengthening innovation’ and ‘customer loyalty’ to ‘prevention of new environmental
legislation’, with 'enhancement of corporate public image' ranking highest. Only three of
the 14 had a direct relationship with performance ('risk minimisation', 'certainty of legal
compliance' and 'support of ecological transformation of the line of business'), and they
were ranked at positions 4, 9 and 12 (Hamschmidt 2000: 4).

3. EMS and environmental performance: Evidence from other studies
3.1 Studying the effectiveness of EMS
Since the European and international environmental management standards were
introduced in the mid 1990s, it is estimated that approximately 63,500 companies and
production sites have adopted a certified or registered EMS1 worldwide, and many more
systems not audited by third parties exist. The fact that there is substantial experience with
environmental management in companies has triggered a large number of research projects,
evaluations, dissertations and doctoral theses into the effects of EMS. It is surprising that
despite the recent growth of this literature (for recent reviews see for example Dyllick and

1 There are around 3,500 EMAS registrations in the EU and more than 60,000 ISO 14001 certifications
world wide. Data source: EMAS website (http://europa.eu.int/comm/environment/emas) [page accessed 20
June 2004] and ISO World / Umweltbundesamt (http://www.ecology.or.jp/isoworld/english/analy14k.htm)
[page accessed 20 June 2004].

Page 7

7
Hamschmidt 1999; Steger 2000; Ammenberg 2001; Coglianese and Nash, 2001; Ankele et
al 2002; Schaltegger and Synnestvedt, 2002; Thornton et al, 2003; Andrews, 2003; Anton
et al. 2004), empirical evidence about the environmental effectiveness of EMS is still
relatively sparse. One of the reasons is that many studies have focused on the direct
economic costs and benefits associated with EMS. Economic benefits, are not, however, a
reliable indicator of environmental effectiveness because savings can be made without
reducing pollution. For example, a company can save costs by organising environmental
responsibilities better or by identifying cheaper methods of waste disposal. Conversely, the
adoption of an EMS may lead to unanticipated and costly pollution abating measures. For
example if the use of an EMS revealed that the firm was in breach of regulation,
investment in abatement technology might be obligatory.

Even those researchers that have attempted to assess quantitatively the link between EMS
and environmental performance have rarely been able to make valid statements about the
overall environmental effectiveness of EMS. Several reasons may be given for this
apparent anomaly. Of greatest importance, many studies suffer from a shortage of
environmental performance data. In most countries, environmental reporting is not
mandatory and most companies prefer not to publish quantitative performance data. ISO
14001 does not require disclosure of environmental information. Even where data on
emissions, materials use or non-compliance incidents are provided in environmental
reports or EMAS site statements, it is rarely presented in a comparable format. Despite the
activity of organisations such as the Global Reporting Initiative, there is no standard
approach to environmental reporting and measurement. Public emissions registers exist
only in a few countries such as Britain, the United States and the Netherlands, and the
quality and usability of the data in these registers varies. Only a few research projects have
had the capacity to carry out the costly and time-consuming data work necessary to conduct
comprehensive studies of the environmental performance consequences of EMS. Most
studies only look at a small number of companies (Thornton et al, 2003), or rely on data
generated through companies’ self-assessment (Dyllick and Hamschmidt, 1999, Berry and
Rondinelli, 2000; Mohammed, 2000; Florida and Davidson, 2001).

Quantitative analysis of environmental performance of companies poses a series of
conceptual and methodological challenges. First, environmental performance is a complex
and multi-dimensional issue. There is no universally accepted approach to the inherently

Page 8

8
subjective task of weighing different environmental impacts against each other. For
instance, any overall assessment or ranking based on a judgement of how greenhouse gas
emissions compare to chemical spills or special waste will produce highly contested
results. It is also debatable whether the fact that companies operate in different natural
environments should be taken into account when considering pollution that has local
impacts. Second, companies carry out distinct business activities under different economic,
technological and regulatory conditions. Some businesses will always find it more difficult
to improve their environmental performance than others, even if they operate in the same
sector. For example, it may be that the specific demands placed on a company by its
customers prevent the adoption of a cleaner technology. Third, it is difficult to decide
where the system boundaries should be set with regard to environmental performance. Are
companies responsible only for damage caused by production operations, or should issues
such as the supply of raw materials and components, transportation to and from the
company, product use and disposal be included in the assessment of environmental
performance?

Given these difficulties in establishing a robust framework for performance evaluation,
most studies have used self-reported proxies that can be measured through postal or
telephone surveys, for example satisfaction with the EMS, perceived environmental
benefits, or types of measures put in place. Although this is a justifiable response to the
challenges outlined above, the reliance on ‘effort indicators’ and self-assessment limits the
validity of findings. It is important to recognise that conclusions are often based on the
(empirically-informed) judgement of researchers and their interviewees, rather than on
quantitative evidence. In the remainder of this section, we summarise the results of some of
the larger and more performance-oriented studies (see Table 1).
3.2 Results from key studies
There is as yet no consensus on the question of EMS’ impact on environmental
performance. Many researchers reported a moderate improvement of environmental
effectiveness stemming from EMS adoption (e.g. Hamschmidt 2000), even though some
studies (such as UNI/ASU 1997; Kuisma et al 2001, Andrews, 2003; Anton et al, 2004)
adopt a generally more optimistic tone than others (for example FEU 1998; Jäger et al
1998; Steger 2000; Wagner 2002). A considerable variability between companies was also
observed (UNI/ASU 1997; Steinle and Baumast 1997; Kuisma et al 2001; Andrews, 2003).

Page 9

9
Some research has found little evidence that EMS have driven environmental improvement
(Jäger et al 1998; Steger 2000). Matthews (2001) noted ‘…little difference in the toxic
emissions of US automobile assembly facilities with ISO14001 certification and those
without…, in many cases, firms with certified EMS fared worse.’ (p 1927). Wagner (2002)
concludes for a sample of 306 German manufacturing firms that there were no significant
differences in 2001 profitability levels and the ratios of energy consumption to sales or to
employee between firms with and without EMS. Also, no positive trend was found for
either profitability or energy efficiency during the period 1991 to 2001. Against this,
Andrews (2003) established a consistent relationship between EMS adoption and
environmental improvement. Potoski and Prakash also find some evidence that
ISO140001-cerified industrial facilities in the US reduced toxic emissions faster than non-
certified facilities (Potoski and Prakash, 2005), while Anton et al (2004) found a link
between the comprehensiveness of environmental management systems in firms and lower
toxic emissions per unit output. This confirms the suggestion that what counts is the
quality of an EMS (Coglianese and Nash, 2001) and the environmental management style
(Thornton, Kagan and Gunningham, 2003), rather than the presence of such a system.
Matthews et al (2004) make a similar argument and suggest a framework for synthesising
different elements of EMS and so improving performance.

TABLE 1 ABOUT HERE

There is also mixed evidence regarding the effect of EMS on legal compliance with
environmental regulation. Steger (2000), Jäger et al (1998) and Andrews (2003) conclude
that EMS do support compliance. Steger points out, however, that it is difficult to
determine the actual environmental effects of better compliance because non-compliance is
often concerned with formal infringements rather than material breaches. In contrast,
Dahlström et al’s (2003) study was unable to confirm this link. The study - which is one of
the few analyses that draw on a comprehensive set of independent performance
assessments - analyses almost 800 production sites across England and Wales using
databases of operator performance as assessed by Environment Agency enforcement
officers.2 It concluded that having an EMS improves certain procedural aspects of

2 The study linked the Environment Agency’s Operator Performance and Risk Appraisal database and its
enforcement database with the results of an EMS survey of 843 production sites. It used simple observational
techniques as well as ANOVA tests to assess whether an externally validated EMS is associated with higher

Page 10

10
environmental management such as recording and use of information, knowledge and
implementation of authorization requirements, plant maintenance, management and
training, and process operation. Crucially however, they did not find a link between the
presence of an EMS and actual performance measured as complaints, non-compliance
events and the likelihood (as assessed by enforcement officers) of suffering from incidents.

There are also doubts about whether EMS represent an autonomous driver of performance
improvements. In Steger's study (2000), most respondents held the view that the
environmental objectives of the company could also have been attained without an EMS.
Hamschmidt (2000) reports that while most would agree that an EMS had some influence
on environmental performance, only few saw it as a key factor. EMS do not appear to lead
to fundamentally different environmental objectives and strategies, but promote
streamlining of existing environmental responsibilities. Interestingly, external stakeholders
tended to have a more positive view of the costs and benefits than companies themselves
(Steger 2000).

EMS appear to be related to improvements in traditional areas of environmental
management. Empirical studies of EMS in operation show that most companies focus on
on-site production efficiency. The most significant improvements appear to have been
made in the areas of waste management, energy use and water consumption (UNI/ASU
1997; Kuisma 2001; Steinle and Baumast 1997; Dyllick and Hamschmidt 1999; Andrews,
2003). All of these are areas in which direct cost savings can be made because the
environmental goods involved have to be purchased.

There is widespread agreement that EMS have largely failed to broaden the scope of
corporate environmental management because they do not systematically address
environmental concerns outside the factory gate, for example transport and logistics,
sourcing of raw materials and other inputs, product design and end-of-life considerations
(cf Steger 2000; Hamschmidt 2000; Kuisma et al 2001; Jäger et al 1998; Ankele et al
2002).


levels of operator performance (and different aspects of performance), with faster rates of improvement and
with the likelihood of enforcement action.

Page 11

11
4. Analysing the link between EMS and environmental performance
4.1 The MEPI approach
The following analysis reports research carried out in the context of the Measuring
Environmental Performance of Industry (MEPI) study (cf Berkhout et al 2001; Tyteca et al
2001) in which the authors were investigators. The MEPI project operationalised
performance as the environmental efficiency of the production process: the level of input of
energy and materials and the level of output of waste and pollution per unit of product
output. In the fertiliser and printing sectors, where there was insufficient data on production
output, environmental indicators were normalised on turnover and number of employees
respectively. All inputs and outputs were measured in physical terms such as weight or
volume. The project covered six industrial sectors (electricity generation, pulp and paper,
fertilisers, textile finishing, book and magazine printing, and computer manufacture) and
six European countries (Austria, Germany, Italy, Belgium, the Netherlands and the UK).
Within this limited scope, the project team aimed to collect data on environmental
performance for as many companies and production sites as possible. Data were collected
from three types of sources:
• 70 to 80% of the data stems from corporate environmental and financial reports as
well as EMAS statements
• around 10 % of the data were taken from national pollution inventories (UK:
fertilizer and paper, Netherlands: electricity)
• between 10 and 20% of the data was collected through specially-designed
questionnaires in sectors where little public data was available (Italy: all sectors,
UK: printing and textiles).
We estimate that for the more concentrated sectors (paper, electricity and fertilisers) the
MEPI data set covers between 50 and 80 % of production in the six countries. In the
sectors dominated by smaller companies (printing and textile finishing), the data covers
less than 20% of production (see table 2). The computer manufacture sector was not
included in the analysis because very little data was found. The data set covers the years
1985, 1990, and 1994 to 1998.

TABLE 2 ABOUT HERE

Page 12

12
While the database with more than 15,000 performance data points for 274 firms and
around 400 production sites provides a valuable research resource, it also has a number of
significant limitations:

- The data set is incomplete, with many missing values. On average, only 28% of the
performance indicators for which we collected data were available for a given firm or
site in a given year. Principal component analysis (PCA) was carried out to establish
that environmental performance could adequately be reflected by a subset of all
indicators. For example, CO2 emissions were found to be indicative of all air emissions
in the electricity sector (cf Berkhout et al 2001). This enabled us to restrict the analysis
to a smaller number of indicators for which data were more complete and to reduce the
need to aggregate indicators.
- A number of sectors consist of a heterogeneous set of firms, which have a structurally
different environmental profile because they produce different products and/or use
different technologies. Some - but not all - of these differences have been captured
through the analysis of sub-sectors.
- The data set covers a period up to 1998, only few years after the introduction of the
EMAS scheme (1995) and the ISO 14001 standard (1996). The analysis is, therefore,
unable to cover the long-term effects of the adoption of formal EMS. On the other
hand, the use of data from the mid to late 1990s was an opportunity to study a relatively
large number of companies adopting the new EMS standards at that time.
- A significant share of the data has undergone little or no third party validation. Given
that much of the data is disclosed voluntarily, however, we would not expect
companies to consciously falsify performance data.
- In some sectors, the sample of firms is assumed to over-represent large companies and
good performers, since we expect that these would have a higher propensity to publish
data and reports.

Page 13

13
4.2 Analysis
In the remainder of the section, we report three ways in which the link between EMS and
environmental performance was analysed. Throughout the analysis, the 'presence of an
EMS' was operationalised as the presence of a management system that is certified to an
internationally-recognised standard (ISO 14001 or EMAS). A company was counted as
being EMAS certified if all its production sites had adopted this standard. The link
between EMS and environmental performance was investigated through examination of
three hypotheses:
Hypothesis 1: Firms with an EMS have a better environmental performance than those
without an EMS
We aimed to establish whether firms with a certified EMS performed better than those
without. Given the input of management effort in the form of new practices, better
coordination and greater monitoring, we would expect to find that firms with an EMS
would record better performance overall than those without. Andrews (2003) found that
across a panel of firms, performance indicators for which targets and objectives had been
set showed a more consistent pattern of improvement, when compared with all
environmental performance indicators.

Analysing every sector individually, we established significant differences between
individual normalised performance levels achieved by EMS firms and those displayed by
non-EMS firms based on non-parametric analysis. The analysis used those indicators that
were identified as being most suitable by the Principal Component Analysis (PCA).3 Non-
parametric tests, which do not assume a normal distribution, were chosen since some
indicators had skewed distributions. Missing values were treated on a case-by-case basis
(pair-wise exclusion): where data for a specific indicator was missing, this firm was
excluded in the testing for only this variable. Due to the large number of missing variables,
the analysis did not control for any firm characteristics other than industrial sector and firm
size (by means of normalisation, as described in section 4.1).


3 Six performance indicators were identified in the PCA for the fertilisers sector (NOx, VOC, hazardous and
municipal waste, water use and energy use), 9 indicators for the pulp and paper sector (water use, energy use,
solid waste, hazardous waste, CO2, COD, SO2, nitrogen and phosphorus), 9 indicators for the printing sector
(hazardous and total waste, CO2 and SO2, ink use, isopropyl alcohol use, water use, fuel use and energy use),
11 indicators for textile finishing (total and recycled waste, CO2, NOx, VOC, COD, copper, chromium,

Page 14

14
TABLE 3 ABOUT HERE

Overall, three conclusions can be drawn from the results (see Appendix and Table 3). First,
the tests show that, in the large majority of cases, companies with an EMS did not perform
significantly better than those without. In particular, significant differences could be
identified for the environmental performance variables analysed in only three of the five
sectors analysed and for only a minority of indicators. Significant differences (up to the
0.10 level) were found for individual normalised performance levels in the fertiliser sector
(50% of indicators), printing (33%), and pulp and paper (22%), but not in the textiles and
electricity sectors (see table 3). The complete absence of significant differences in two
sectors, together with the fact that only a minority of indicators showed significant
differences in the other three sectors, is notable, and suggests that hypothesis 1 is rejected
on this evidence.

Second, in those sectors where significant differences were found, there were fewer
instances in which EMS firms were significantly more eco-efficient as non-EMS firms:
50% of the significant differences were pro EMS firms in the paper sector and 33% in the
fertilizer and printing sectors. In each sector where significant differences existed, results
were found pointing in both directions, with the slight majority of cases recording negative
correlations between a certified EMS and environmental performance. In the fertiliser
sector, for example, NOx emissions per unit of sales were lower for firms with EMS,
whereas hazardous waste and VOC emissions per unit of sales were higher for firms with
EMS. These different directions of an EMS-effect show that even though there are
significant differences between firms with a certified EMS and those without, these do not
necessarily imply that the certified firms perform better. This is further evidence to reject
hypothesis 1.

Overall, the few and to some extent ambiguous differences suggest that EMS do not
correlate strongly with corporate environmental performance. However, alternative
explanations are possible:


phosphorus, and energy use) and 13 indicators for electricity generation (solid waste, municipal waste,
recycled waste, CO2, NOx, SO2, dust, coal use, gas use, oil use, renewable fuel use and total fuel use).

Page 15

15
- The methods adopted to operationalise both the presence of an EMS and
environmental performance may not be sufficiently precise and nuanced to capture an
EMS-effect. Many companies have EMS, but are not certified, for instance.
- An EMS-effect may have been disguised by the stronger influence of other explanatory
variables some of which have been captured in the data (e.g. country, sub-sector,
company size), while others have not (e.g. technologies, market conditions,
management culture).
- Because EMS are a voluntary instrument, the link between EMS and performance may
be influenced by inverse causality (the problem of endogeneity). If it is the case that
poorly performing firms tend to adopt EMS because they feel the need to improve
performance or to signal commitment, the lack of significant differences could be due
to a lower performance baseline, rather than the ineffectiveness of EMS. It seems more
likely, however, that in the majority of cases the act of EMS adoption suggests
stronger environmental commitment (Hamschmidt, 2000). In this case, the absence of
a strong correlation between EMS and performance is all the more surprising. In light
of the current lack of one unified theory of EMS adoption (Russo (2001), for example,
provides arguments for both lines of reasoning), these last two aspects seem to be
essentially empirical questions. The trend analysis testing Hypothesis 3 tries to address
this problem of causality.

Another issue which could influence the validity of results is whether EMS firms have
reported data on a significantly lower number of indicators than non-EMS firms, perhaps
even choosing not to report data in areas of poor performance. If this were the case, then a
test for significant differences would likely underestimate a positive effect of EMS
certification, since the performance levels of uncertified firms would be upward-biased.

Table A.5 reports the results of tests for significant differences in reporting between EMS
and non-EMS firms. Overall, it is found that only in one instance was there a significant
difference in the average number of indicators reported - with ISO-certified firms in the
paper industry having a significantly lower number. This largely rejects the notion that
uncertified firms under-report their performance strategically. In cases where differences
are insignificant, both patterns of reporting are observed, i.e. there are sectors where the
average number of indicators is higher for EMS firms (e.g. fertilizers) but also sectors
where the average number of indicators is higher for non-EMS firms (e.g. electricity and