Rating the Raters: Evaluating how ESG Rating Agencies Integrate Sustainability Principles

Abstract

Environmental, social, and governance (ESG) rating agencies, acting as relevant financial market actors, should take a stand on working towards achieving a more sustainable development. In this context, the objective of this paper is, on the one hand, to understand how criteria used by ESG rating agencies in their assessment processes have evolved over the last ten years and, on the other hand, to analyze whether ESG rating agencies are contributing to fostering sustainable development by the inclusion of sustainability principles into their assessment processes and practices according to the ESG criteria. This research is based on a comparative descriptive analysis of the public information provided by the most representative ESG rating and information provider agencies in the financial market in two periods: 2008 and 2018. The findings show that ESG rating agencies have integrated new criteria into their assessment models to measure corporate performance more accurately and robustly in order to respond to new global challenges. However, a deep analysis of the criteria also shows that ESG rating agencies do not fully integrate sustainability principles into the corporate sustainability assessment process.

Full text

sustainability
Article
Rating the Raters: Evaluating how ESG Rating
Agencies Integrate Sustainability Principles
Elena Escrig-Olmedo * , MaríaÁngeles Fernández-Izquierdo , Idoya Ferrero-Ferrero ,
Juana María Rivera-Lirio and María Jesús Muñoz-Torres
Sustainability of Organizations and CSR Management Research Group-IUDESP, University Jaume I,
Campus del Riu Sec-Avda. Vicent Sos Baynat s/n, Castellón 12071, Spain; afernand@uji.es (M.A.F.-I);
ferrero@uji.es (I.F.-F.); jrivera@uji.es (J.M.R.-L.); munoz@uji.es (M.J.M.-T.)
*Correspondence: eescrig@uji.es; Tel.: +34-964-387-145
Received: 31 December 2018; Accepted: 5 February 2019; Published: 11 February 2019


Abstract:
Environmental, social, and governance (ESG) rating agencies, acting as relevant financial market
actors, should take a stand on working towards achieving a more sustainable development. In this context,
the objective of this paper is, on the one hand, to understand how criteria used by ESG rating agencies in
their assessment processes have evolved over the last ten years and, on the other hand, to analyze whether
ESG rating agencies are contributing to fostering sustainable development by the inclusion of sustainability
principles into their assessment processes and practices according to the ESG criteria. This research is
based on a comparative descriptive analysis of the public information provided by the most representative
ESG rating and information provider agencies in the financial market in two periods: 2008 and 2018.
The findings show that ESG rating agencies have integrated new criteria into their assessment models
to measure corporate performance more accurately and robustly in order to respond to new global
challenges. However, a deep analysis of the criteria also shows that ESG rating agencies do not fully
integrate sustainability principles into the corporate sustainability assessment process.
Keywords:
Environmental, social and governance (ESG) factors; ESG rating agencies; sustainability
principles; sustainable and responsible investment; sustainable development
1. Introduction
There has been a significant development of sustainable and responsible investment (SRI)
in the last ten years. Investors, shareholders, governments and firms have benefitted from this
since they request accurate information not only regarding financial performance but also about
environmental, social and governance (ESG) aspects, which has become part of their competitive
strategy [1]. These factors have given rise to the inevitable appearance of ESG rating agencies.
ESG rating agencies scrutinize businesses and assess corporate sustainability performance
by using their own research methodologies. This expertise has turned ESG rating agencies into
a key reference for companies, financial markets and academia in terms of corporate sustainability
assessments. Consequently, the sustainability rating market has grown considerably in the last
decade—in keeping with their influence—to the point of being studied not only as economic actors
but also as social actors, which have an impact on the behavior of other social actors in society [
2
].
However, if social impact is not internalized and ESG rating agencies act only as economic actors,
the messages launched by rating agencies about what could a sustainable company be or how corporate
sustainability performance could be measured might be misrepresented. This could be overcome if the
expectations held by society and rating agencies about sustainability and sustainable development are
matched. Consequently, providing society with misleading information about corporate sustainability
can affect the social legitimacy and trust of both companies and ESG rating agencies.
Sustainability 2019,11, 915; doi:10.3390/su11030915 www.mdpi.com/journal/sustainability

Sustainability 2019,11, 915 2 of 16
In this context and considering the remarkable changes the rating agency industry has undergone in
the last years, some issues have arisen: What concept of corporate sustainability has consolidated regarding
the ESG assessment criteria in the last decade? Have these assessment criteria used by ESG rating agencies
changed over time considering the processes of mergers and acquisitions or the new challenges in terms of
sustainability? Are these rating agencies contributing to achieving a more sustainable development?
Previous studies have already been conducted with the aim of analyzing the historical evolution, growth
and consolidation of ESG rating agencies and their strategies [
3
,
4
], their assessment frameworks and weighing
systems [
5
–
8
]. Moreover, the question regarding the implications in terms of the definition of corporate
sustainability derived from the criteria adopted by ESG rating agencies has previously been addressed [
9
],
by analyzing their contribution to more sustainable business models. However, to our knowledge, the analysis
of their efforts to achieve adequate sustainable development has not yet been analyzed.
In this paper, the theoretical foundation for the study of the ESG rating agency market is based
on the institutional theory, since this theory emphasizes the relationship between organizations and
how companies conform to norms as a consequence of pressures placed by institutions on the setting
to which they belong [
10
]. Specifically, the focus is over the study of the relationship between ESG
rating agencies and society in general, beyond their business scope, by means of the reflection around
how ESG rating agencies are operationalizing sustainability in their assessment frameworks. In this
paper, we use the terms “sustainable development” and “sustainability” interchangeably even though
the debate regarding possible differences between the terms remains unsolved; sustainability and
sustainable development are commonly used as synonymous [11].
Therefore, the purpose of this paper is twofold: First, to know how the criteria used by ESG
rating agencies in their assessment process have evolved over time and second, to analyze whether
ESG rating agencies are contributing to achieving a more sustainable development by the inclusion of
sustainability principles [
12
] into their assessment processes and practices. As a result, we analyze
whether ESG rating agencies integrate the main sustainability principles into their assessment processes:
(i) the sustainability dimensions (financial economic, environmental and social) and the balance among
them (ii) the intergenerational perspective, (iii) the stakeholder approach and (iv) the life-cycle thinking.
This analysis is based on a comparative study of the most representative ESG rating and
information provider agencies in the financial market. This paper first explores how their assessment
criteria have evolved in the last 10 years, followed by an examination of how ESG rating agencies have
worked towards achieving sustainable development.
The main findings of this paper allow us to understand the integration of sustainability principles
into the financial markets system and to extend the prior insights from Escrig-Olmedo et al. [
7
] that
present an overview on the different assessment criteria adopted by sustainability indices and ESG
rating agencies; Avetisyan and Hockerts [
4
] explain the causes and impact of the ESG rating agency
consolidation process, and Muñoz-Torres et al. [
9
] analyze whether assessment methods employed by
ESG agencies are identifying and/or driving more sustainable business models.
The results of this work are especially relevant to understanding the behavior of these
organizations and their evaluation processes, considering the EC Sustainable Finance Action Plan,
The Sustainable Development Goals (SDGs), Paris 2016 Agreement on Climate Change and the
increasing emphasis on the monitoring of corporate sustainability performance.
The paper is structured as follows. The next section presents the extant literature and includes
a descriptive analysis of the ESG rating agencies industry evolution. After that, the data and methods
are presented and the main findings are remarked. Finally, the implications, contributions and
limitations are discussed in the concluding section of the paper.
2. Theoretical Background
According to Nawaz and Koc [
13
], who adopted the systematic methodology proposed by
Tranfield et al. [14], we developed our literature review following the next steps:

Sustainability 2019,11, 915 3 of 16
Step 1—Identification of keywords: In particular, the search included the following terms: “rating
agencies + corporate social responsibility” or “rating agencies + sustainability” or “rating agencies +
sustainable development” or “rating + development goals”. Step 2—Screening criteria: The search was
limited to papers published in the English language between 2000 and 2018. Step 3—Identification of
search engines: The academic databases used by the literature search were Web of Science and Google
Scholar. Step 4—Execution of search (total number of returned results): The search was carried out in
November 2018 and led to the identification of 78 articles in Web of Science and 478 articles in Google
Scholar. Step 5—Number of articles selected after initial screening: In this step, we reviewed the title
and abstract to demonstrate a clear connection between ESG rating agencies and corporate sustainability.
Finally, 45 articles were selected after this systematic review. Step 6-Number of articles selected after final
screening and research gap detection: After a careful reading of the identified articles, 44 papers were
selected as relevant publications to set the theoretical framework of this paper in order to develop the body
of knowledge concerning the behavior of ESG rating agencies. As was stated in the previous section, most
of the papers are focused on assessment methodologies and their results; however, scarce information is
provided related to how aligned they are with the concept of sustainability.
2.1. ESG Rating Agency Industry
The current context demands companies to contribute to sustainable development [
15
] by means
of corporate strategies that integrate sustainable practices into their activities with the aim of achieving
corporate sustainability. Corporate sustainability, therefore, requires a commitment between the
present environmental, social and economic needs of a firm’s stakeholders and their future needs.
According to Lozano [
16
], corporate sustainability refers to “corporate activities that proactively
seek to contribute to sustainability equilibria, including the economic, environmental and social
dimensions of today, as well as their interrelations within and throughout the time dimension
(i.e., the short-, long- and longer-term), while addressing the company’s systems, i.e., operations
and production, management and strategy, organizational systems, procurement and marketing,
and assessment and communication, as well as with its stakeholders”.
Assessing corporate sustainability is gaining momentum since the financial market is paying
growing attention to this issue [
17
]. However, most existing tools, frameworks and mechanisms to
measure corporate sustainability are not adequate [18].
ESG rating and information provider agencies (the so-called corporate social responsibility (CSR)
ratings, social ratings, sustainability ratings or SRI ratings agencies) have emerged in response to
the demands of socially responsible investors that require social and environmental information of
companies so to invest in more sustainable companies [4].
ESG rating agencies assess the corporate sustainability performance of a large number of
companies. Some ratings are based exclusively on extra-financial information while others combine
financial and extra-financial data to assess long-term value and sustainability [19].
They use a large amount of information obtained from the companies themselves through
questionnaires that involve a laborious process and the analysis of public information (reports, news,
etc.) examined by interdisciplinary work teams in different geographical areas.
In the last decade, the ESG rating industry has grown considerably and has already undergone
a phase of consolidation, not only with merger and acquisition processes among the existing ESG rating
agencies but also through the new entrance of financial rating and information provider agencies.
The beginning of the financial crisis in 2008 brought about a positive shift in capital market perceptions
and attitudes towards corporate sustainability [
20
]. As Figure 1shows, the ESG rating agency market has
gone through a concentration process in the last 10 years (from 2008 to 2018). Financial stakeholders and
investment decision makers tend to be the main incentives to this growth and concentration process [
21
,
22
],
which shows the permanence of the corporate social responsibility movement [19].

Sustainability 2019,11, 915 4 of 16
Sustainability 2018, 10, x FOR PEER REVIEW 4 of 17
Figure 1. Cont.

Sustainability 2019,11, 915 5 of 16
Sustainability 2018, 10, x FOR PEER REVIEW 5 of 17
Figure 1. The environmental, social and governance (ESG) rating agencies market concentration.
Figure 1. The environmental, social and governance (ESG) rating agencies market concentration.

Sustainability 2019,11, 915 6 of 16
ESG rating agencies stop being isolated market actors oriented to a limited financial market niche
to become a promising and dynamic business even for “traditional” rating agencies. This trend could
be illustrated, for example, by the case of Morgan Stanley Capital International (MSCI). MSCI is one of
the current ESG rating agencies, and MSCI analyzes the environmental, social and governance-related
business practices of thousands of companies worldwide.
MSCI is the result of the absorption of several ESG research providers. In 2010, MSCI acquired
RiskMetrics Group, a provider of risk management and governance products and services. RiskMetrics
had previously bought ISS (Institutional Shareholder Services) in 2007, Innovest Strategic Value
Advisors in February 2009, and Kinder Lydenberg Domini (KLD) Research & Analytics in November
2009. The last two are now known as MSCI ESG Research. In addition, in July 2010, MSCI acquired
MeasureRisk, a provider of risk transparency and risk measurement tools for hedge fund investors.
Afterwards, in August 2014, MSCI acquired Governance Holdings Co. (GMI Ratings), a provider
of corporate governance research and ratings, and in January 2013, it bought InvestorForce, a provider
of performance reporting tools to the institutional investment community in the United States.
MSCI is an example of how a big data provider has expanded their scope as a provider of ESG
information for institutional investors, including the world’s most important mutual funds, pension
funds and hedge funds. Furthermore, MSCI ESG Research data and ratings are used in the construction
of the MSCI ESG Indices.
This example shows how a large number of agencies have cropped up, while others have
disappeared—most often taken over by a competitor. According to Avetisyan and Hockerts [
4
],
it is possible to observe two growth strategies: (i) organic growth and partnerships, which is
the establishment of a network of alliances (e.g., RobecoSAM), and (ii) mergers and acquisitions,
that happen when two or more ESG rating agencies merge and join forces (e.g., Vigeo-EIRIS merge)
or when prior financial data providers and assessment managers decide to enter into the ESG rating
industry (e.g., MSCI).
This concentration process has allowed ESG rating agencies to develop wider and integral
assessments of corporate sustainability, considering that sustainability is a multidimensional concept.
As Figure 1shows, current ESG rating agencies have integrated specialized actors in corporate
governance, data management, risk or communication into their systems. In addition, this market
change has led to the emergence of more professional, multidisciplinary and multicultural work teams
and the extension of the geographic and sectorial reach.
In this context, the first research question arises: How have the criteria used by ESG rating agencies
in their corporate sustainability assessment process evolved over time, considering the processes of
mergers and acquisitions?
The analysis of this evolution will evidence whether the concept of corporate sustainability
understood by rating agencies has changed over time and in what sense.
2.2. Have ESG Rating Agencies Been Able to Integrate Sustainability Principles into Corporate
Sustainability Assessment?
The development of the ESG industry is mainly due to the combination of financial factors and
market power [
4
]. On the one hand, the efforts made by large listed companies in terms of sustainability
management and reporting during the last years [
23
,
24
] confirm the existence of an active strategy
for being well-positioned in ESG ratings. As Mackenzie et al. [
25
] highlight, from the institutional
theoretical view, the propensity to invest in sustainability management may be influenced by factors
such as pressure groups, regulation, the organization’s competitive position and the dialogue with
stakeholders; in particular, the authors found that belonging to an ESG stock index encourages
companies to improve their sustainability management since this fact sends positive signals to
shareholders and the rest of the stakeholders. On the other hand, outcomes derived from ESG
rating agencies assessments have been frequently used by academics and the research community
(e.g., Reference [26]).

Sustainability 2019,11, 915 7 of 16
Consequently, ESG rating agencies influence not only the behavior of financial market actors [
27
]
but also the institutionalization of sustainability management in companies and are being considered
as “institutional entrepreneurs” [
2
]. Therefore, the responsibility of ESG rating agencies cross financial
market boundaries since the consequences of dynamic changes in corporate sustainability assessments
by ESG rating agencies go beyond the scope of financial markets, affecting the society as a whole.
Bearing this influence in mind, the analysis of how ESG rating agencies could make a positive
contribution to the sustainable development is a key challenge to overcome. Consequently, a second
research question arises: Are ESG rating agencies contributing to achieving a sustainable development
by the inclusion of sustainability principles into their assessment processes and practices?
Each ESG rating agency uses its own corporate sustainability assessment methodology. The diverse
methodologies seem to be associated to a market-led strategy of differentiation [
22
] and to cultural and
ideological factors [
28
]. However, in the different assessment processes of these ESG rating agencies, three
aspects of measurement are always considered: the high-level categories evaluated (environmental, social
and governance) and the positive criteria included in each category; the controversial activities and practices
evaluated; and the normalization process of their ratings by the industry [6].
The proliferation of these rating agencies and the diversity of their assessment methodologies
have posed several challenges to be met:
(i) Lack of transparency. ESG rating agencies do not offer complete and public information about the
criteria and the assessment process developed by them to evaluate the corporate sustainability
performance. This makes understanding what ESG rating agencies are measuring and making
comparisons between them difficult [7,19,22,29].
(ii)
Commensurability. ESG rating agencies may measure the same concept in different ways.
Therefore, if the assessments of ESG ratings are not consistent, which involves evidence of low
commensurability, the hypothesized benefits of CSR cannot occur [6].
(iii)
Trade-Offs among criteria. ESG ratings methodologies may compensate higher scores in one
domain with very low scores in another domain [17,30,31].
(iv)
Lack of an overall score. Most of the ESG rating agencies provide environmental, social and
governance rates to each domain, but they do not provide an overall score of the corporate
sustainability performance [32].
(v)
Stakeholders’ preferences. ESG rating agencies do not address the different stakeholders’ expectations
in their evaluation processes, which influences their acceptance and usefulness [17,33].
These shortcomings recommend caution in the interpretation of ESG rating agency outcomes.
As Busch et al. [
34
] wonder, “If ESG data do not reliably and validly reflect organizational
reality [ . . . ]
how can sustainable investment practices contribute to sustainable development?” Moreover, taking
into account the abovementioned evolution of the ESG rating industry in the last decades (Figure 1),
together with the increase of power of the financial rating industry, “The desired new norms of the SRI
movement are therefore at risk of being reabsorbed by the traditional collective norms of the financial
rating industry,” as Avetisyan and Hockerts [4] highlight.
Our proposition is that the inclusion of sustainability principles into ESG rating agency assessment
processes and practices is a necessary condition for minimizing this risk: that is also a risk in the
process of the institutionalization of the corporate sustainability concept.
2.3. Sustainability Principles
In the last years, sustainable development has become a strategic objective for business and
governments and since the launch of the UN Sustainable Development Goals (SDGs) in 2015, the global
community has had a new framework to address the most urgent global problems.
Sustainable development is defined in the Anthropocene as “the development that meets the
needs of the present while safeguarding Earth’s life-support system, on which the welfare of current
and future generations depends” [35].

Sustainability 2019,11, 915 8 of 16
This definition describes sustainability within a broad and complex context. The work of
Muñoz-Torres et al. [
12
] presents a debate regarding the main sustainability principles associated to the
concepts of sustainability and sustainable development. The authors, based on seminal references in
the sustainability field [
36
–
39
], propose four basic and common conceptual Principles of Sustainability,
which could be generally accepted across different disciplines [40]. These principles are as follows:
(i)
Sustainability dimensions and the balance among them. Sustainability is a multidimensional
concept [
41
] where three domains can be distinguished: financial economic, environmental and
social. Achieving a balance among the three domains implies not prioritizing or undermining
a dimension over the other ones.
(ii)
Intergenerational perspective. This principle implies identifying, evaluating and managing the
future and the current needs [
39
], considering the long-term effects of today’s decisions and
a balance between both short- and long-term ones.
(iii)
Stakeholder approach. Sustainability involves identifying the current stakeholders’ needs and
expectations and the future generations’ needs.
(iv)
Life-cycle thinking (LCT). Decision-makers should shoulder economic, environmental and social
responsibilities in order to achieve sustainability which cross legal organizational boundaries.
In this regard, sustainability involves managing the impacts of upstream and downstream
activities and, accordingly, adopting an LCT approach.
These principles are consistent with previous works such as Wass et al. [
11
], who highlight the
following sustainability principles: i) the normativity principle: sustainable development is influenced
by our values and reflects the kind of world we want; ii) the equity principle among generations and
species; iii) the integration principle opposed to the concepts such as “balancing” or “trading-off”
sustainability dimensions; and iv) the dynamism principle: sustainable development is a process
submitted to change.
Nevertheless, the present paper follows Muñoz et al.’s [
12
] sustainability principles because they
facilitate the operationalization of the sustainability concept in an organizational and life-cycle thinking
context, and it is consistent with previous proposals.
3. Methodology
In order to answer our research questions, we developed a qualitative analysis in two stages.
First, we selected eight ESG rating and information provider agencies that are the most
representatives of the European and USA SRI market, considering the number of companies analyzed
(between 4,000 and 20,000 companies), their clientele, countries and markets covered (see Table 1).
These rating agencies, despite their merger and acquisition processes, have maintained their assessment
structures or have integrated their expertise in other comprehensive assessment structures (Figure 1).
These allow the comparability of the ESG rating agencies criteria along the time.

Sustainability 2019,11, 915 9 of 16
Table 1. The sample.
ESG Rating Agencies and
Information Provider Agencies 1
(2008) →(2018)
Number of Companies Analyzed
by ESG Rating Agencies Other Important Facts
1. ASSET4 →REFINITIV
2. ECP →ECP
3. EIRIS →FTSE Russell ESG
Ratings 2
4. KLD Research & Analytics
Inc. →MSCI ESG Research
5. Oekom →ISS-oekom
6. SAM →RobecoSAM
7. SIRI Company
→Sustainalytics
8. Vigeo →Vigeo EIRIS
Around 22,000 companies
Around 4000 companies
Around 4100 companies
Around 7000 companies
Around 20,000 companies
Around 4500 companies
Around 9000 companies
Around 4000 companies
Companies across 87 countries
Companies across 47 developed
and emerging markets
It is used by 46 of the top 50 asset
managers and by 1200
investors worldwide.
2000 institutional clients,
115 markets covered.
Companies across 60 countries
Expanded to 11,000 companies in
Q2 2019.
A multicultural team of
145 analysts
Notes:
1
Colum 1 shows the ESG rating agencies analyzed and their evolution over the time after a convergence
process.
2.
In 2008, the FTSE4 Good Index was carried out by EIRIS. The current FTSE Russell uses its own
methodology. Source: Own creation (data December 2018–January 2019).
The analysis was developed in two periods: 2008 and 2018. These two periods are relevant for
the ESG rating agency industry because, after the fall of Lehman Brothers in 2008, which marked the
beginning of the financial crisis and revealed the limitations of traditional measurement models of
corporate performance and risk analysis, the financial market began to consider corporate sustainability
as an important factor [
20
]. For instance, the way KLD Research & Analytics Inc., a leader in ESG
research, was acquired by RiskMetrics (2009) and later by MSCI (2010) showed how the traditional
financial information and risk analysis agencies began to consider ESG information necessary for
giving a better assessment of corporate performance. Moreover, in 2018, the SRI market has grown
exponentially due to stakeholders’ demands of accurate information regarding company performance.
In fact, sustainable investing assets reached $12 trillion in the United States, which represented a 38%
increase in 2016 [42]. In Europe, SRI assets represented around 11 trillion euros [43].
Continuing the process, we examined the relevant environmental, social and corporate governance
criteria used by these ESG rating and information provider agencies to evaluate corporate sustainability
through a content analysis method [
44
]. We developed a thematic content analysis that led to the
identification of common themes in different types of texts. Furthermore, as Beattie et al. [
45
] highlight,
it is a holistic content analysis in which the complete text is analyzed.
Following Bardin [46], the thematic content analysis is divided in three steps:
(i)
Pre-analysis: We established the objectives of the content analysis, and we selected the material
for analysis. Concretely, we analyzed the qualitative and public information provided by the
ESG rating agencies on their corporate websites concerning their corporate sustainability analysis
criteria in 2008 (see Escrig-Olmedo et al. [
7
]) and 2018. We browsed the company website in order
to seek information on the corporate sustainability assessment, sustainability and ESG data, ESG
research and ratings until five clicks.
(ii)
Exploration: We defined the unit of analysis and codes. First, we identified themes for examination
considering current sustainability global goals, and apart from this, following Escrig-Olmedo et al. [
7
]
and Muñoz-Torres et al. [9], we defined certain codes (categories).
(iii)
Treatment and interpretation: We examined the relevance and the presence or absence of the
themes in the content analyzed. We considered that a theme is relevant if it is public on the ESG
rating agency website and, moreover, if it is highlighted as a key criterion in the assessment
process of the ESG rating agency. Finally, we quantified the percentage of ESG rating agencies in
the sample that offered information about specific assessment criteria.

Sustainability 2019,11, 915 10 of 16
Secondly, following Muñoz-Torres et al. [
12
], we examined whether ESG rating agencies integrate
the main sustainability principles into their sustainability assessment frameworks. To that end,
we studied whether the ESG criteria used by ESG rating and information provider agencies are
consistent with these sustainability principles. To this analysis, three researchers acted as primary
coders and after that, two researchers checked the result and resolved inconsistencies [
47
,
48
]. On the
other hand, we examined specific aspects of their methodologies (such as the aggregation process of
scores).
To determine if an ESG rating agency integrates sustainability principles into their assessment
methodologies, the following arguments were taken into account:
(i)
Sustainability dimensions and the balance among them. An ESG rating agency integrates this
principle if in its assessment process and methodologies, environmental, social and governance
criteria are equally important.
(ii)
The intergenerational perspective. An ESG rating agency integrates this principle if the criteria
or assessment methodology aspects such as the future and the current needs or specific risks
considering both the short and long term are explicitly laid down.
(iii)
Stakeholder approach. An ESG rating agency integrates this principle if stakeholders’ needs and
expectations are integrated in its criteria and assessment process.
(iv)
Life-cycle thinking (LCT). An ESG rating agency applies this principle if the way the company
impacts upstream and downstream activities, in accordance with the adoption of a LCT approach,
are examined in its criteria and assessment frameworks.
4. Results
This section first shows the results of the comparative study of the positive assessment criteria
used by the ESG rating and information provider agencies in 2008 and 2018 and then discusses the
main findings about the integration of the sustainability principles into their assessment processes
and practices.
The ESG criteria in the two years analyzed are the same; however, the integration of these criteria
in the assessment frameworks of ESG rating agencies has changed in these two periods.
By performing an analysis of the results by dimensions (environmental, social and governance),
we can highlight that regarding the evolution of environmental criteria (Figure 2), a change of trend
is observed in the screening criteria used by the ESG rating and information provider agencies.
Specifically, in 2008, the most widely used analysis criteria were environmental policy/management
(100%), emissions (62.5%) and climate change (50%) while in 2018, the main criteria analyzed were
environmental policy/management (87.5%), water use and management (87.5%) and protection
of biodiversity (87.5%). In addition, the aspects that have been incorporated into the assessment
process of ESG rating agencies in 2018 were climate change (75%), emissions (75%) and waste
management/reduction (75%). The results show a greater interest in environmental concerns than
10 years ago. As a result, in 2008, the assessment of the corporate environmental performance
focused principally on the analysis of environmental management policies or systems while in 2018,
this assessment framework was complemented by the inclusion of new criteria linked to the efforts
of large companies to reduce emissions, consumption, etc. This greater interest in combating climate
change and in mitigating greenhouse-gas emissions reflects how the agreements reached at the
21st Conference of the Parties (COP21) in Paris (December 2015) are having a strong impact on the
assessment of corporate sustainability performance.

Sustainability 2019,11, 915 11 of 16
Sustainability 2018, 10, x FOR PEER REVIEW 11 of 17
Figure 2. The environmental positive criteria.
Regarding the social pillar criteria (see Figure 3), the aspects that have been mainly considered
in the assessment process of ESG rating agencies analyzed in 2008 were human capital development
and training (100%), human rights (87.5%) and community relations (87.5%). On the contrary, the
aspects incorporated into the assessment process of all ESG rating agencies in 2018 were labor
management, human rights and quality working condition, health and safety. After the definition of
Sustainable Development Goals (SDGs), aspects related to the improvement of health and education,
the reduction of inequalities and the necessity to spur economic growth seem to be crucial to measure
how companies contribute to sustainable development.
However, issues related to business behavior, talent attraction and market ethics seem less
important. It is also interesting to highlight that criteria related to supply-chain management and
security and data protection have arisen in the last ten years, which reflects the new trends in
sustainability assessment that focus on more complex and integrated productive configurations
instead of on companies as isolated structures.
Figure 2. The environmental positive criteria.
Regarding the social pillar criteria (see Figure 3), the aspects that have been mainly considered in the
assessment process of ESG rating agencies analyzed in 2008 were human capital development and training
(100%), human rights (87.5%) and community relations (87.5%). On the contrary, the aspects incorporated
into the assessment process of all ESG rating agencies in 2018 were labor management, human rights and
quality working condition, health and safety. After the definition of Sustainable Development Goals (SDGs),
aspects related to the improvement of health and education, the reduction of inequalities and the necessity to
spur economic growth seem to be crucial to measure how companies contribute to sustainable development.
However, issues related to business behavior, talent attraction and market ethics seem less
important. It is also interesting to highlight that criteria related to supply-chain management and
security and data protection have arisen in the last ten years, which reflects the new trends in
sustainability assessment that focus on more complex and integrated productive configurations
instead of on companies as isolated structures.
Sustainability 2018, 10, x FOR PEER REVIEW 12 of 17
Figure 3. The social positive criteria.
Finally, concerning the corporate governance aspects (Figure 4) considered by most ESG rating
and information provider agencies in the period 2008, we can highlight that corporate governance
functions and committees (100%), board structure (75%) and remuneration/compensation policy
(62.5%) were the most valuable criteria. These criteria, generally included in the corporate governance
codes, were regarded as important in 2018. By contrast, those criteria that involve abstract concepts
with measurement challenges (e.g., brand management or vision and strategy) are not usually used
as a key criterion in the public information of ESG rating agencies. However, the increase in the
prevention of corruption and bribery issues and in the transparency issues stresses a significant
difference between 2008 and 2018, since they are now the second most analyzed aspects in the
assessment process (87.5%). As in the case of environmental concerns, aspects related to the
governance of companies seem to be gaining importance into the assessment frameworks, especially
those related to the prevention of corruption and bribery; international organizations such as OCDE
are defining these concerns through a series of recommendations to successfully attack corruption
both in the public and private sector.
Figure 3. The social positive criteria.

Sustainability 2019,11, 915 12 of 16
Finally, concerning the corporate governance aspects (Figure 4) considered by most ESG rating
and information provider agencies in the period 2008, we can highlight that corporate governance
functions and committees (100%), board structure (75%) and remuneration/compensation policy
(62.5%) were the most valuable criteria. These criteria, generally included in the corporate governance
codes, were regarded as important in 2018. By contrast, those criteria that involve abstract concepts
with measurement challenges (e.g., brand management or vision and strategy) are not usually used as
a key criterion in the public information of ESG rating agencies. However, the increase in the prevention
of corruption and bribery issues and in the transparency issues stresses a significant difference between
2008 and 2018, since they are now the second most analyzed aspects in the assessment process (87.5%).
As in the case of environmental concerns, aspects related to the governance of companies seem to
be gaining importance into the assessment frameworks, especially those related to the prevention of
corruption and bribery; international organizations such as OCDE are defining these concerns through
a series of recommendations to successfully attack corruption both in the public and private sector.
Sustainability 2018, 10, x FOR PEER REVIEW 13 of 17
Figure 4. The corporate governance positive criteria.
However, are these criteria consistent with the sustainability principles? To answer this
question, it is necessary to thoroughly examine each ESG rating agency individually.
By drawing on the concept of how each ESG rating agency integrates sustainability principles
into their assessment processes and practices, we followed the conceptual framework developed by
Muñoz-Torres et al. [9] (see Table 2) who have been able to state that they do not fully integrate them
into their corporate sustainability assessment process.
Although all ESG rating agencies incorporate the sustainability dimensions (or pillars) in their
assessment process, not all of them evaluate sustainability in a balanced way. For example,
RobecoSAM does not consider the three dimensions of sustainability in a balanced way, since each
dimension has a specific weight in the assessment and it is not clear how it has been defined, as public
information about assessment is based on sectoral matters.
Concerning the stakeholder approach principle, all the ESG rating agencies in the sample are
gradually integrating the needs and expectations of a variety of stakeholders, as can be seen from the
analysis of the assessment criteria used by them (Figures 3–5). These criteria assess the degree of
compliance with the expectations and needs of different stakeholders, such as the rights of
indigenous people, local suppliers, customer relationship management and satisfaction, etc.
ESG rating agencies partially integrate risk assessment processes that incorporate the
intergenerational perspective. The public information of Refinitiv, FTSE Russell, ISS-Oekom and
Vigeo-EIRIS do not allow analysts to perceive how they manage risks connected to the future
generation needs or expectations.
Finally, there is no evidence that ESG rating agencies integrate life-cycle thinking as an explicit
principle in their sustainability assessment. As mentioned above, some of these agencies are, to some
extent, beginning to analyze the company management of their supply chains, but this does not mean
applying an explicit life-cycle thinking that requires different and specific assessment methodologies
[12]. Therefore, there is a clear need to advance in the definition of tools and initiatives that allow
practitioners to integrate sustainability principles into the assessment of corporate performances.
Figure 4. The corporate governance positive criteria.
However, are these criteria consistent with the sustainability principles? To answer this question,
it is necessary to thoroughly examine each ESG rating agency individually.
By drawing on the concept of how each ESG rating agency integrates sustainability principles
into their assessment processes and practices, we followed the conceptual framework developed by
Muñoz-Torres et al. [9] (see Table 2) who have been able to state that they do not fully integrate them
into their corporate sustainability assessment process.
Although all ESG rating agencies incorporate the sustainability dimensions (or pillars) in their
assessment process, not all of them evaluate sustainability in a balanced way. For example, RobecoSAM
does not consider the three dimensions of sustainability in a balanced way, since each dimension has
a specific weight in the assessment and it is not clear how it has been defined, as public information
about assessment is based on sectoral matters.
Concerning the stakeholder approach principle, all the ESG rating agencies in the sample are
gradually integrating the needs and expectations of a variety of stakeholders, as can be seen from
the analysis of the assessment criteria used by them (Figures 2–4). These criteria assess the degree of
compliance with the expectations and needs of different stakeholders, such as the rights of indigenous
people, local suppliers, customer relationship management and satisfaction, etc.

Sustainability 2019,11, 915 13 of 16
ESG rating agencies partially integrate risk assessment processes that incorporate the intergenerational
perspective. The public information of Refinitiv, FTSE Russell, ISS-Oekom and Vigeo-EIRIS do not allow
analysts to perceive how they manage risks connected to the future generation needs or expectations.
Finally, there is no evidence that ESG rating agencies integrate life-cycle thinking as an explicit principle
in their sustainability assessment. As mentioned above, some of these agencies are, to some extent,
beginning to analyze the company management of their supply chains, but this does not mean applying
an explicit life-cycle thinking that requires different and specific assessment methodologies [
12
]. Therefore,
there is a clear need to advance in the definition of tools and initiatives that allow practitioners to integrate
sustainability principles into the assessment of corporate performances.
Table 2. The sustainability principles in ESG rating agencies.
ESG Rating
Agencies
(1) Sustainability
Dimensions 1(2) Balance (3) Intergenerational
Perspective
(4) Stakeholder
Approach
(5) Life-Cycle
Thinking
REFINITIV
EC
EN
SO
CG
× × X×
ECP
EN
SO
CG
X X X ×
FTSE Russell
ESG Ratings
EN
SO
CG
X×X×
MSCI ESG
Research
EC
EN
SO
CG
X X X ×
ISS-oekom
EN
SO
CG
X×X×
RobecoSAM
EN
SO
CG
×X X ×
Sustainalytics
EN
SO
CG
×X X ×
Vigeo EIRIS
EN
SO
CG
X×X×
ESG rating
agencies
EC
EN
SO
CG
6=6=X×
X
Yes,
×
No,
6=
Partially and —Not explicit;
1
Financial Economic (EC), Environmental (EN), Social (SO) and
Corporate Governance (CG). Source: Own work.
5. Discussion and Conclusions
This paper explores how the ESG rating agency industry and the criteria they use in the assessment
process have evolved over the last ten years and examines whether ESG rating agencies are now
contributing to a more sustainable development by the inclusion of sustainability principles into their
assessment processes and practices. This analysis has been developed from a comparative study of
the most representative ESG rating and information provider agencies in the financial market in two
periods of time: 2008 and 2018.
The ESG rating agency industry has gone through a lengthy merger and acquisition process, which has
redefined the industry map with bigger, more professionalized and finance industry-connected companies.
Moreover, ESG rating agencies also have a clear commercial character [
17
], since they market diverse

Sustainability 2019,11, 915 14 of 16
products and services (sustainability indices, sector and thematic research reports, benchmarks, etc.).
The result is a clearer business case, and ESG rating agencies’ bargaining powers have grown exponentially.
This could imply a biased concept of sustainability if sustainability principles are not guaranteed in the
assessment business.
ESG rating agencies have integrated new criteria into their assessment models (mainly
environmental and governance criteria) in order to assess corporate performance in a more robust
and accurate way. These results support the findings of Saadaoui and Soobaroyen [
22
] who point out
that ESG rating agencies focus on analyzing environmental, social and governance criteria, whereas
the economic dimension is less studied; in addition, Attig et al. [
49
] have found that each ESG rating
agency regards different individual components of CSR as relevant.
However, we can observe that ESG rating agencies do not fully integrate the sustainability
principles into the corporate sustainability assessment process; therefore, they should include these
principles in their work to contribute to the sustainable development of the companies that they
rate. We encourage ESG rating agencies to improve the measurement of corporate sustainability
performances by integrating sustainability principles into their assessment processes and practices.
For instance, there are assessment methodologies like footprint analysis that allow appraisers to tackle
the life-cycle thinking principle. Using those methodologies will contribute to achieving a more
sustainable development.
This study makes a significant contribution to the extant literature on ESG rating agencies, offering
information to academics about the metrics that they use in their researches. Moreover, from a practical
standpoint, this paper is expected to contribute to the way the financial market faces extra-financial
assessment, providing useful insights to policy-makers and other market actors such as fund managers
or investors who want to build their portfolios and are more committed to social and environmental
aspects. This paper also contributes to promoting an open debate over who rates the raters, adding
sustainability principles as key elements to the analysis, which help to clarify which concept of
corporate sustainability is being institutionalized in the SRI market and beyond.
Finally, a number of limitations need to be considered. The constant changes in the sector (mergers,
acquisitions and disappearance of ESG rating agencies) and the selection of limited cases do not show
all the particularities of ESG rating agencies, although our selection covers the main trend. The current
study was limited by analyst bias in the content analysis developed. We encourage other researchers to
reply and extend the sample of study. Moreover, this study is limited by the lack of public information
made available by ESG agencies about their evaluation criteria.
Our research poses an important question that future studies could address: How are ESG rating
agencies integrating sustainability principles into their assessments of corporate performance along supply
chains? Current supply-chain performance-assessment systems are not suitable tools to integrate new global
challenges. Therefore, further efforts are needed to develop more comprehensive corporate sustainability
assessment frameworks which integrate sustainability principles along the whole supply chain.
Author Contributions:
This article is a joint work of the five authors. E.E.-O., M.Á.F.-I., I.F.-F., J.M.R.-L. and
M.J.M.-T. contributed to the research ideas, literature review and analysis and to writing the paper. All authors
read and approved the final manuscript.
Acknowledgments:
This paper is supported by the European Union’s Horizon 2020 Research and Innovation
Programme under Grant Agreement No. 693642, project SMART (Sustainable Market Actors for Responsible Trade).
Conflicts of Interest: The authors declare no conflict of interest.
References
1. Galbreath, J. ESG in focus: The Australian evidence. J. Bus. Ethics. 2013,118, 529–541. [CrossRef]
2.
Elbasha, T.; Avetisyan, E. A framework to study strategizing activities at the field level: The example of CSR
rating agencies. Eur. Manag. J. 2018,36, 38–46. [CrossRef]
3.
Avetisyan, E.; Ferrary, M. Dynamics of stakeholders’ implications in the institutionalization of CSR Field in
France and in the United States. J. Bus. Ethics. 2013,115, 115–133. [CrossRef]

Sustainability 2019,11, 915 15 of 16
4.
Avetisyan, E.; Hockerts, K. The consolidation of the ESG rating industry as an enactment of institutional
retrogression. Bus. Strateg. Environ. 2017,26, 316–330. [CrossRef]
5.
Chatterji, A.K.; Levine, D.I.; Toffel, M.W. How well do social ratings actually measure corporate social
responsibility? J. Econ. Manage. Strat. 2009,18, 125–169. [CrossRef]
6.
Chatterji, A.K.; Durand, R.; Levine, D.I.; Touboul, S. Do ratings of firms converge? Implications for managers,
investors and strategy researchers. Strateg. Manage. J. 2016,37, 1597–1614. [CrossRef]
7.
Escrig-Olmedo, E.; Muñoz-Torres, M.J.; Fernandez-Izquierdo, M.A. Socially responsible investing:
Sustainability indices, ESG rating and information provider agencies. Int. J. Sustain. Econ.
2010
,2, 442–461.
[CrossRef]
8.
Semenova, N.; Hassel, L.G. On the validity of environmental performance metrics. J. Bus. Ethics.
2014
,132,
1–10. [CrossRef]
9.
Muñoz-Torres, M.J.; Fernández-Izquierdo, M.A.; Rivera-Lirio, J.M.; Escrig-Olmedo, E. Can environmental,
social and governance rating agencies favor Business Models that promote a more Sustainable Development?
Corp. Soc. Resp. Env. Ma 2019. online first.
10.
DiMaggio, P.; Powell, W. The New Institutionalism in Organizational Analysis; University of Chicago Press:
Chicago, IL, USA, 1991.
11.
Waas, T.; Hugé, J.; Verbruggen, A.; Wright, T. Sustainable development: A bird’s eye view. Sustainability
2011,3, 1637–1661. [CrossRef]
12.
Muñoz-Torres, M.J.; Fernández-Izquierdo, M.Á.; Rivera-Lirio, J.M.; Ferrero-Ferrero, I.; Escrig-Olmedo, E.;
Gisbert-Navarro, J.V.; Marullo, M.C. An Assessment Tool to Integrate Sustainability Principles into the
Global Supply Chain. Sustainability 2018,10, 535. [CrossRef]
13.
Nawaz, W.; Koc, M. Development of a systematic framework for sustainability management of organizations.
J. Clean. Prod. 2018,171, 1255–1274. [CrossRef]
14.
Tranfield, D.; Denyer, D.; Smart, P. Towards a methodology for developing evidence-informed management
knowledge by means of systematic review. Brit. J. Manage. 2003,14, 207–222. [CrossRef]
15.
Abdelkafi, N.; Täuscher, K. Business models for sustainability from a system dynamics perspective.
Organ. Environ. 2016,29, 74–96. [CrossRef]
16.
Lozano, R. A holistic perspective on corporate sustainability drivers. Corp. Soc. Resp. Env. Ma.
2015
,22,
32–44. [CrossRef]
17.
Windolph, S.E. Assessing corporate sustainability through ratings: Challenges and their causes.
J.Environ. Sustain. 2011,1, 61–81. [CrossRef]
18.
Ben-Eli, M.U. Sustainability: definition and five core principles, a systems perspective. Sustain. Sci.
2018
,13,
1337–1343. [CrossRef]
19.
Scalet, S.; Kelly, T.F. CSR Rating Agencies: What is Their Global Impact? J. Bus. Ethics.
2010
,94, 69–88.
[CrossRef]
20.
Lopatta, K.; Kaspereit, T. The world capital markets’ perception of sustainability and the impact of the
financial crisis. J. Bus. Ethics. 2014,122, 475–500. [CrossRef]
21.
Scott, M.E.; Cocchi, D.; Campbell-Gemmell, J. Defining a fit for purpose statistically reliable sustainability
indicator. Sustain. Account. Mana. 2014,5, 262–267.
22.
Saadaoui, K.; Soobaroyen, T. An analysis of the methodologies adopted by CSR rating agencies.
Sustain. Account. Mana. 2018,9, 43–62. [CrossRef]
23.
Rivera, J.M.; Muñoz, M.J.; Moneva, J.M. Revisiting the relationship between corporate stakeholder
commitment and social and financial performance. Sustain. Dev. 2017,25, 482–494. [CrossRef]
24.
Taylor, J.; Vithayathil, J.; Yim, D. Are corporate social responsibility (CSR) initiatives such as sustainable
development and environmental policies value enhancing or window dressing? Corp. Soc. Resp. Env. Ma.
2018. [CrossRef]
25.
Mackenzie, C.; Rees, W.; Rodionova, T. Do responsible investment indices improve corporate social
responsibility? FTSE4Good’s impact on environmental management. Corp. Gov.
2013
,21, 495–512. [CrossRef]
26.
Orlitzky, M.; Schmidt, F.L.; Rynes, S.L. Corporate social and financial performance: A meta-analysis.
Organ. Stud. 2003,24, 403–441. [CrossRef]
27.
Slager, R.; Gond, J.-P.; Moon, J. Standardization as institutional work: The regulatory power of a responsible
investment standard. Organ. Stud. 2012,33, 763–790. [CrossRef]

Sustainability 2019,11, 915 16 of 16
28.
Sandberg, J.; Juravle, C.; Hedesstrom, T.M.; Hamilton, I. The heterogeneity of socially responsible investment.
J. Bus. Ethics. 2009,87, 519–533. [CrossRef]
29.
Chatterji, A.; Levine, D. Breaking down the wall of codes: evaluating non-financial performance
measurement. Calif. Manage. Rev. 2006,48, 29–51. [CrossRef]
30.
Delmas, M.; Blass, V.D. Measuring corporate environmental performance: The trade-offs of sustainability
ratings. Bus. Strateg. Environ. 2010,19, 245–260. [CrossRef]
31.
Escrig-Olmedo, E.; Muñoz-Torres, M.J.; Fernández-Izquierdo, M.Á.; Rivera-Lirio, J.M. Lights and shadows
on sustainability rating scoring. Rev. Manag. Sci. 2014,8, 559–574. [CrossRef]
32.
Liern, V.; Pérez-Gladish, B. Ranking corporate sustainability: a flexible multidimensional approach based on
linguistic variables. Int. T. Oper. Res. 2018,25, 1081–1100. [CrossRef]
33.
Escrig-Olmedo, E.; Muñoz-Torres, M.J.; Fernández-Izquierdo, M.Á.; Rivera-Lirio, J.M. Measuring corporate
environmental performance: a methodology for sustainable development. Bus. Strateg. Environ.
2017
,26,
142–162. [CrossRef]
34.
Busch, T.; Bauer, R.; Orlitzky, M. Sustainable development and financial markets: Old paths and new avenues.
Bus. Soc. 2016,55(3), 303–329. [CrossRef]
35.
Griggs, D.; Stafford-Smith, M.; Gaffney, O.; Rockström, J.; Öhman, M.C.; Shyamsundar, P.; Steffen, W.;
Glaser, G.; Kanie, N.; Noble, I. Policy: Sustainable development goals for people and planet. Nature
2013
,
495, 305–307. [CrossRef] [PubMed]
36.
Elkington, J. Cannibals with Forks: The Triple Bottom Line of 21st Century Business; Capstone Publishing Ltd.:
Oxford, UK, 1997.
37. Freeman, E. Strategic Management: A Stakeholder Approach; Pitman: Boston, MA, USA, 1984.
38.
International Organization for Standardization (ISO). International Organization for Standardization, ISO
26000:2010—Guidance on Social Responsibility; ISO: Genève, Switzerland, 2010.
39.
World Commission on Environment and Development (WCED). Our Common Future. 1987. Available
online: www.un-documents.net/our-common-future.pdf (accessed on 11 January 2019).
40.
Lindsey, T.C. Sustainable principles: Common values for achieving sustainability. J. Clean Prod.
2011
,19,
561–565. [CrossRef]
41.
Boggia, A.; Cortina, C. Measuring sustainable development using a multi-criteria model: A case study.
J. Environ. Manage. 2010,91, 2301–2306. [CrossRef]
42.
US SIF. Report on US Sustainable, Responsible and Impact Investing Trend. 2018. Available online:
https://www.ussif.org/files/Trends/Trends%202018%20executive%20summary%20FINAL.pdf (accessed
on 20 January 2019).
43.
Eurosif. European SRI Study. 2018. Available online: http://www.eurosif.org/wp-content/uploads/2018/
11/European-SRI-2018-Study.pdf (accessed on 20 January 2019).
44.
Abbott, W.F.; Monsen, R.J. On the Measurement of Corporate Social Responsibility: Self reported Disclosures
as a Method of Measuring Corporate Social Involvement. Acad. Manage. J. 1979,22, 501–515.
45.
Beattie, V.; McInnes, B.; Fearnley, S. A methodology for analysing and evaluating narratives in annual
reports: a comprehensive descriptive profile and metrics for disclosure quality attributes. Account. Forum
2004,28, 205–236. [CrossRef]
46. Bardin, L. Análise de conteúdo (Edição revista e actualizada); Lisboa: Edições, Portugal, 2009; p. 70.
47.
Moneva, J.M.; Rivera-Lirio, J.M.; Muñoz-Torres, M.J. The corporate stakeholder commitment and social and
financial performance. Ind. Manage. Data Syst. 2007,107, 84–102. [CrossRef]
48.
García-Pérez, I.; Muñoz-Torres, M.J.; Fernández-Izquierdo, M.Á. Microfinance literature: A sustainability
level perspective survey. J. Clean Prod. 2017,142, 3382–3395. [CrossRef]
49.
Attig, N.; El Ghoul, S.; Guedhami, O.; Suh, J. Corporate social responsibility and credit ratings. J. Bus. Ethics.
2013,117, 679–694. [CrossRef]
©
2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).