Maturity Models of Healthcare Information Systems and Technologies: a Literature Review

Abstract

The maturity models are instruments to facilitate organizational management, including the management of its information systems function. These instruments are used also in hospitals. The objective of this article is to identify and compare the maturity models for management of information
systems and technologies (IST) in healthcare. For each maturity model, it is identified the methodology of development and validation, as well as the scope, stages and their characteristics
by dimensions or influence factors. This study resulted in the need to develop a maturity model based on a holistic approach. It will include a comprehensive set of influencing factors to reach all areas and subsystems of health care organizations.

Full text

SYSTEMS-LEVEL QUALITY IMPROVEMENT
Maturity Models of Healthcare Information Systems
and Technologies: a Literature Review
João Vidal Carvalho
1
&Álvaro Rocha
2
&António Abreu
1
Received: 13 February 2016 /Accepted: 31 March 2016
#Springer Science+Business Media New York 2016
Abstract The maturity models are instruments to facilitate
organizational management, including the management of its
information systems function. These instruments are used also
in hospitals. The objective of this article is to identify and
compare the maturity models for management of information
systems and technologies (IST) in healthcare. For each maturity
model, it is identified the methodology of development and
validation, as well as the scope, stages and their characteristics
by dimensions or influence factors. This study resulted in the
need to develop a maturity model based on a holistic approach.
It will include a comprehensive set of influencing factors to
reach all areas and subsystems of health care organizations.
Keywords Stages of growth .Maturity models .Hospital
information systems .Management
Introduction
Health institutions together with government organiza-
tions are realizing that a certain inability to properly
manage the processes of health is directly related to
technological infrastructure limitations and management
inefficiency [1,2]. Hospital Information systems man-
agers usually look at the mistakes made in these orga-
nizations and ask themselves on what they should have
done to prevent them. It appears that these errors are
usually symptoms of natural growth and organizations
maturation. It seems to be the result of the development
of the organization to its current maturity [3,4]. The
changes that an organization experiences, from its be-
ginning to maturity, fit perfectly into the principles of
Stages of Growth theory. Also, they occur in the current
context of healthcare IST.
The maturity models are based on the premise that peo-
ple, organizations, functional areas, processes, etc., evolve
through a process of development and growth towards a
more advanced maturity accomplishing several stages [5].
Mutafelija and Stromberg [6] reports that the concept of
maturity has been applied to more than 150 areas of IST.
Obviously, the maturity models have also been applied in
various fields of IST in the health field.
This article first presents a brief overview of the Maturity
Models in IS area (second section). Then in third section, the
research methodology adopted in literature review is de-
scribed. Fourth section presents the results of the literature
review, that is, the 14 Maturity Models of IST in health care
are described. Finally, in fifth section is presented the summa-
ry and closing remarks.
This article is part of the Topical Collection on Systems-Level Quality
Improvement
*João Vidal Carvalho
cajvidal@iscap.ipp.pt
Álvaro Rocha
amrocha@dei.uc.pt
António Abreu
aabreu@iscap.ipp.pt
1
Instituto Politécnico do Porto, ISCAP, S. Mamede de Infesta,
Matosinhos, Portugal
2
Departamento de Engenharia Informática, Universidade de Coimbra,
Coimbra, Portugal
JMedSyst (2016) 40:131
DOI 10.1007/s10916-016-0486-5

Maturity models in IST management
Richard Nolan is considered the principal architect of the IST
maturity approach. In fact, after studying the use of IS in major
organizations in US, Nolan proposed a four stages maturity
model [7]. Later, he improved the first version adding two new
stages to the initial model [8]. In this second version, the
model states that organizations begin slowly in the Initiation
stage, which is followed by a period of rapid spread of the use
of IT in Contagion stage. Later, the need for Control arises.
Then, it is followed by Integration of different technological
solutions. The Data Administration stimulates development
with no costs increasing associated with IST. Finally, steady
growth produces Maturity.
While this approach to Nolan maturity models has been
recognized as highly innovative, it also has generated discus-
sion and controversy. Some researchers have validated the
model but others have suggested additions. Another re-
searchers have developed new models (e.g. [9–13]).
Among the new models after the Nolan first version model,
the most consensual, comprehensive and understanding is the
Galliers and Sutherland Revised Model [4,5]. This model
provides a better view of how an organization plans, develops,
uses and organizes an IS and provides suggestions for pro-
gression toward higher maturity stages. This method consists
of six stages. It assumes that an organization can stand in
differentstages of maturity at any time and can be conditioned
by different factors of influence. Besides, it has features
matching modern network organizations and provides a data
collection tool to assess the maturity [5]. Recently, after the
Galliers and Sutherland [13] model, other models have been
proposed (e.g. [14–17]), including a new model of Nolan with
nine maturity stages [18] to meet technological developments
of IST and its management. With regard to IS management,
another good example of maturity model is Khandelwal and
Ferguson model [15] which features nine stages of maturity. It
combines the theory of stages with critical success factors.
Nevertheless, the Galliers and Sutherland [13] continues to
be considered the most complete and updated IST manage-
ment model [3,4].
The maturity models are applied in one department alone
or in the whole organization. They fit in various types of
organizations. In fact, there are several examples of maturity
models focused on different areas of the organization and IST:
Damsgaard and Scheepers [19]maturitymodelforIntranets,
Holland and Light [20] maturity model for ERP systems,
CMMI [21] maturity model and ISO / IEC 15504 [22] for
software development processes. There are also maturity
models in the areas of software maintenance [23], business
management [24], project management ([25,26]), project
management for portfolios and programs [27], information
management [28], management of IS/ICT [29], e-business
[12,30–32], e-learning [33], knowledge management [34,
35], business process management (BPM) [36], enterprise ar-
chitecture [37,38], etc.
Methodology adopted for the literature review
Aiming to conduct a comprehensive and wide literature re-
view, it was necessary to define a strategy in order to identify
and analyze systematically the available literature on maturity
models of healthcare IST. An initial review provided criteria to
choose the approach and establish the strategies to be applied
to this project.
The first strategy by Webster and Watson [39]suggestsa
structured approach in three basic steps: to identify the rele-
vant literature in main sources (i.e. Bleading journals^)and
recognized conferences. Then, the authors suggest conducting
a search in the reference section of the studies identified in the
first step in order to identify potential works related; finally, it
is suggested the search via Web of Science of works which
cite the works identified in the previous two steps.
The second strategy, proposed by Tranfield et al. [40], sug-
gests five steps for a systematic review of the literature. The
first stage defines terms, keywords and combinations to be
used as criteria to be applied in the literature review. A second
phase is to identify relevant works that contain the keywords
and terms defined above. Inthe third phase, it is carried out an
assessment of identified papers and made a selection of works
that meet certain criteria of quality. In the fourth phase, it must
be extracted the relevant information from the selected litera-
ture. Finally, in the fifth phase a synthesis of data is done.
The analysis of both strategies described above shown that
the approach of Webster and Watson [39], although simple
and easy to implement, is not completely suited to this work.
The literature on maturity models of healthcare information
systems is limited in major journals and conferences. With
regard to Tranfield et al. [40] approach, it was found that there
is not a clear procedure for the identification of relevant work
in the second phase. On the other hand, when assessing the
quality of studies, the authors state that it is a challenge to
define quality criteria for qualitative work. It caused some
apprehension due to the fact that most of the work in this area
has a qualitative approach. Despite the concerns referred
above, the literature review was carried out based on this ap-
proach with minor modifications and simplifications (Fig. 1).
The second phase of Tranfield et al. [40] was replaced by the
three basic steps described by Webster and Watson [39].
Therefore, the terms and keywords were defined as litera-
ture searching criteria, taking in account that most of the rel-
evant literature on maturity models of health care information
systems is written in English. BMaturity Model^and BStages
of Growth^combined with other terms of this knowledge area
were used for the search iterations (Table 1).
131 Page 2 of 10 J Med Syst (2016) 40:131

The searching criteria were applied to the literature review.
Given that Tranfield et al. [40] did not suggest any procedure
for this stage, it was followed the approach proposed by
Webster and Wats on [39] introducing two changes: in the first
step, the main sources were replaced by major web platforms
of scientific literature; and in the third step of this approach,
Web of Science platform was replaced by the search engines
Google and Google Scholar.
Then, we look for research works across the platforms AIS
Electronic Library, ISI Web of Knowledge, SCOPUS,
Springer, Elsevier/Science Direct and IEEE Computer
Society Digital Library. Afterwards, we proceeded to a data
analysis to identify related references, as suggested by
Webster and Watson [39]. Finally, given that the disclosure
of much of the information on MaturityModels of health care
information systems has been accomplished through technical
reports, research and white papers projects, we move to a more
extended search through the search engine Google Scholar
and Google to ensure identification of other relevant work
for the study. It should be noted that our study found that
research on overall maturity models is in increasing, however,
much of the publishing related to health care are not present in
the IST leading journals.
After identifying a wide range of work in this area, accord-
ing to the approach of Tranfield et al. [40] it was necessary to
define quality criteria for the selection of suitable studies for
this research. However, despite the difficulty in defining qual-
ity criteria for qualitative work, it was found that few models
presented details of their design process and decisions taken in
its development [41]. It was understood that it was convenient
to apply a simple and comprehensive criterion of quality. It
was established to gather all the studies when it was possible
to clearly identify the context (motivation, goal, results, and
benefit) and where maturity models were mentioned directly
or indirectly. The characterization of each model was done
taking in account description, scope, identification of stages
and their characteristics, size, influencing factors, methods
adopted in the development and validation process.
In the end, after processing of all cases, to some extent
conditioned by the perception of researcher on maturity
models in the IST health field, we selected 14 models which
are described below.
Maturity models of IST in health care
The maturity models identified in the literature review as the
most relevants are the Quintegra MaturityModel for electron-
ic Healthcare [2] and the Healthcare IT (HIT) Maturity Model
developed by IDC Health Industry Insights. They present a
wide scope on Hospital IST [42].
The HIMSS Maturity Model for Electronic Medical
Record [43] and the Continuity of Care Maturity Model [44]
are examples of applications for the Electronic Medical
Record (EMR). Both were developed by the same HIMSS.
Besides, there is the maturity model for Electronic Patient
Record (EPR) [45] for systems that manage all patient infor-
mation and Patient Records/Content Management Maturity
Model [46].
Also, there are national health services that have begun to
develop and adopt maturity models for the health area. For
example, the National E-health Transition Authority of
Australia [47] has created the Interoperability Maturity
Model. This model was designed for interoperability associ-
ated with the technical, informational and organizational ca-
pacities of the different players involved in health services.
Another example is the NHS Infrastructure Maturity Model
[48]. It is a maturity assessment model that helps organiza-
tions of the United Kingdom National Health Service to con-
duct self-assessment of their technological infrastructure.
Fig. 1 Methodology adopted for
the systematic literature review
Table 1 Research criteria for the systematic literature review
Research criteria
BMaturity Model^AND BHealth^
BMaturity Model^AND BHealthcare^
BMaturity Model^AND BHospital^
BMaturity Model^AND BeHealth^
BStages of Growth^AND BModel^AND BHealth^
BStages of Growth^AND BModel^AND BHospital^
JMedSyst (2016) 40:131 Page 3 of 10 131

Finally, there are other models for different areas: maturity
model for the PACS area developed by Wetering and
Batenburg [49], the Healthcare Usability Maturity Model
[50] in the area of usability in health systems, Healthcare
Analytics Adoption Model [51] in the area of data analysis,
Hospital Cooperation Maturity Model [41] in the networking
area, Telemedicine Service Maturity Model [52]intelemedi-
cine and IDC’s Mobility Maturity Model for Healthcare [53]
in the area of mobile platforms and devices.
Quintegra maturity model for electronic healthcare
(eHMM)
The Maturity Model for electronic Healthcare is a model that
incorporates all service providers associated with the health
process. It is adaptable to any provider at any level of maturity
[2]. The eHMM Maturity Model provided by Quintegra illus-
trates the transformation of an e-health process from an im-
mature stage to a nationwide stage. According to its authors,
the stages of maturity of this model provide a roadmap for
health organizations to adopt continuous improvement of
healthcare processes.
Based on the study conducted by Quintegra we have iden-
tified several features that illustrate the nature of the progres-
sion of maturity. According to this model, the areas that
showed progression in maturity are: timeliness of process,
data access and accuracy of data, process effort, cost effective-
ness, quality of process results and utility or value to
stakeholders.
IDC healthcare IT (HIT) maturity model
IDC (Health Industry Insights) developed a maturity model
that describes the five developmental stages of hospitals IS.
Each step is supported by the capabilities of the previous
stage. This maturity model, called Healthcare IT (HIT)
Maturity Model, has been used worldwide by IDC to assess
the maturity of the hospitals IS (HIS). Also, it has been used to
compare the average maturity between regions and countries
of different continents [42]. This model has five stages, name-
ly: basic HIS, advanced HIS, clinical HIS, and digital hospital
and virtual hospital.
IDC’s mobility maturity model for healthcare
More recently, IDC Health Insights proposed a maturity mod-
el for health care organizations. It consists of stages, measures,
results and actions to advance along the path of maturity in the
context of mobility toward a mobile culture. This model re-
sulted as consequence of new opportunities associated with
the value of mobility. It is an answer to the need for exploring
alternative technologies, reengineering of business processes,
availability of qualified personnel and development and
implementation management of platforms and mobile appli-
cations [53].
To help healthcare organizations achieving their mobility
strategies, IDC Health Insights has developed a maturity mod-
el consisting of five stages (ad hoc, opportunistic, repeatable,
managed and optimized) and four critical measures (strategic
intent, technology, people, and processes). In addition to the
model, IDC also has featured a guide with actions for
healthcare organizations to move effectively through the
stages of maturity model.
HIMSS electronic medical record maturity model
(EMRAM)
HIMSS Maturity Model for Electronic Medical Record is a
model for the identification of various stages of maturity in the
area of Electronic Medical Record (EMR) of hospitals [43]. In
these times, understanding the performance of EMR in hospi-
tals is a challenge in the health care context [43]. The HIMSS
Analytics (Healthcare Information and Management Systems
Society) developed an adoption model to identify the stages of
maturity of the EMR from the limited ancillary department
systems to paperless EMR environment [54]. The model pro-
posed by HIMSS Analytics is named EMR Adoption Model
(EMRAM) and consists of 8 stages. According to HIMSS
Analytics, the structure of this model ensures that a stage is
reached only when all their applications are operational.
HIMSS continuity of care maturity model (CCMM)
It was created to help the optimization of results in health
systems and patient satisfaction. The HIMSS Continuity of
Care Maturity Model (CCMM) goes beyond Stage 7 of
EMRAM [44]. It consists of 7 stages and it is based on the
EMRAM structure. This global maturity model addresses the
convergence of interoperability, exchange of information, co-
ordinationof care, patient involvement. Its goal is the efficient
management of health for the whole of the population and also
at the individual level [44]. This model also has the ability to
assess the implementation and use of IT by the health service
providers in order to optimize clinical and financial outcomes.
With regard to the benefits of using this model, we can
highlight the guidelines for the design of a solid strategy, at
national and regional levels. Appropriate measures are taken
in a timely manner and include all stakeholders [44]. As an
example of these guidelines, we highlight the standardization
of: IT systems, privacy, patient involvement, etc.
Electronic patient record maturity model (EPRMM)
According to the NHS (United Kingdom National Health
Service), there are six different stages of functionality imple-
mented cumulatively until a complete and exhaustive
131 Page 4 of 10 J Med Syst (2016) 40:131

Electronic Patient Record (EPR) [45] is achieved. The adop-
tion of an ERP system has been seen as a goal of health care
organizations. In fact, it is intended to improve the efficiency
of the organizations in the treatment of patient information,
timely provision and needs at the point of care. As it pro-
gresses, more information will be available in the information
system, whether using traditional computers, mobile phones
or portable devices. The EPR system functions as the main
source of all patient information. It keeps the complete med-
ical record and will be available online at the point of contact
with the patient.
Patient records/content management maturity model
(forrester model)
Forrester Research Inc. has developed a model with three
stages for the area of EMR. This model was developed in
order to help health care providers to assess their systems,
the way they collaborate and interact, the state of the
workflow, and most important, determining the map to get
to the next phase. According to Clair [46], this three stages
model includes four dimensionsor influencing factors: access,
interoperability, content features and planning and strategy. In
addition to the model itself, Forrester Research Inc. has also
developed a manual to drive systems to the next stage. The
three stages of this model are: Paper- or imaged-based patient
records dominate, Access to standalone repositories improves
and Access to the complete digital medical record is role-
based.
NEHTA interoperability maturity model (IMM)
The provision of health care involves many different stake-
holders, including both the technical and organizational infor-
mational area. The ability of these actors to interoperate will
have a strong impact on the delivery of health care safely and
confidence along the stages [47]. The constant evolution of
technology and the changes in clinical practice bring us to
assess the ability to take advantage of these developments.
The National E-health Transition Authority of Australia
(NEHTA) produced an Interoperability Maturity Model
(IMM) which is based on three components: five stages
CMMI (Capability Maturity Model Integration), a set of inter-
operability goals, and an evaluation model focused at the na-
tional level.
The five stages of this model are constrained by organiza-
tional, informational and technical dimensions at local, corpo-
rate and national level. Interoperability targets for reuse, evo-
lution, standards, scope, scalability, configurability and expla-
nation are shared between the three dimensions. The objec-
tives associated with business and governance are set to the
organizational dimension. Informational dimension targets are
classified as: data format and semantics, meta-data, ownership
and rights, common building blocks. Targets associated with
the technical dimension are classified as: interface specifica-
tion, functional decomposition, communication protocol. n-
tier architecture and technical policy separation.
NHS infrastructure maturity model (NIMM™)
The NHS Infrastructure Maturity Model (NIMM) aims to pro-
vide a coherent framework for healthcare organizations. The
organization will be able to measure its own current techno-
logical infrastructure capabilities in specific areas and conse-
quently, to identify and prioritize activities that enhance these
capabilities [48]. Therefore, the NIMM is a model of evalua-
tion of maturity technological infrastructure.
This model adopts the Key Capabilities Self-Assessment
Tool to support IT organizations associated with NHS. It is
used for preparing a self-assessment of technology infrastruc-
ture assessing the maturity of their capabilities. Furthermore, it
helps in the identification of improvement maturity projects.
The NIMM has a holistic approach: it takes in account
technological and IT infrastructure organizational sides. In
fact, it has 72 evaluation capabilities grouped in 13 categories.
The categories are divided into technological aspects and or-
ganizational issues. The technological aspects are: Common
Applications & Services; Operating Systems; Infrastructure
Hardware Platforms; Network Devices & Services; IT
Security & Information Governance; Infrastructure Patterns
& Practices; End User Devices. The organizational issues
are: Infrastructure Governance; Business Alignment;
Procurement; People & Skills; Financial Management;
Principles, Standards, Procedures & Guidelines.
Healthcare analytics adoption model (HAAM)
Health care has moved through three phases of computeriza-
tion and data management, i.e., data collection, data sharing
and more recently data analysis. The data collection phase is
characterized by the implementation of EMRs. It does not
have a significant impact on the quality or the cost of health
care [51]. According to these authors, it will be necessary to
invest in practices associated with data analysis and use of
data warehouses. In this sense, the HAAM model was devel-
oped to accelerate the progress of maturity analytical data in
health care organizations.
Healthcare Analytics Adoption Model (HAAM) is a model
to measure the adoption and use of data warehouses and data
analysis in health care [51]. This model was initially devel-
oped by Sanders in 2012 [55] as result of years of work in this
area. He anticipated foreseeable needs of the healthcare indus-
try. This model is based on EMRAM model [43]. It received
numerous contributions from several healthcare consultants
resulting in an update version in 2013. This model has a sim-
ilar approach as EMRAM to assess the adoption of data
JMedSyst (2016) 40:131 Page 5 of 10 131

analysis in health. It is structured in 8 stages. Each one of them
performs through several capabilities that define the path of
health organizations to data analysis maturity. In addition,
each stage includes a progressive expansion of analytical ca-
pabilities in the following four areas: new data sources, com-
plexity, data literacy and data timeliness.
Hospital cooperation maturity model (HCMM)
This model aims to conceptualize an evolutionary path for
improving cooperation within hospital and between hospitals
[41]. The authors felt the need to develop the model because
of the real and observable changes hospitals are suffering. It
was intended to cope with increased competition and market
dynamics. The model application would force specialization
and cooperation.
The Hospital Cooperation Maturity Model helps hospitals
in the evolution of strategic, organizational and technical ca-
pabilities in a systematic way. The model contributes to struc-
tures and collaborative processes become efficient and effec-
tive. HCMM consults a total of 36 reference points, reflecting
three distinct organizational dimensions relevant to the ability
to cooperate. On the one hand, the model can be used as the
basis for comparative evaluation of the quality of cooperation
between a specific hospital and their business partners; on the
other hand, it may be applied as a common basis for sharing
learning and improvement actions.
As mentioned above, the HCMM is structured in three
layers or dimensions. The first one is a strategic layer set to
measure the ability of a hospital to cooperate with external
partners. The second one is the organizational layer set to
measure the ability to cooperate within the hospital (i.e., be-
tween different departments, divisions, etc.). Finally, the third
layer is an information layer used to measure the technical
capabilities of a hospital to provide the IT infrastructure need-
ed for internal and external cooperation efficiently and
effectively.
PACS maturity model (PMM)
The PACS maturity model (PMM) describes the process ma-
turity of hospitals based on PACS. The analysis is developed
in terms of functionality and integration of the work flow
practice. PMM is a descriptive and normative model. It was
developed as a guide for evaluation and strategic planning
[49]. In this regard, the PMM can be used for strategic plan-
ning. The model incorporates growth paths to reach higher
stages of PACS maturity. However, this model omits a rele-
vant issue. The development used in this maturity model will
be different in different areas of the same organization.
Besides, the maturity maximization cannot be effective or
Bideal^in all circumstances [56].
On the basis of 34 scientific papers literature review on
PACS and subsequent meta-analysis, Wetering and
Batenburg [49] identified three major trends in the evolution
and maturity of PACS: (1) Radiological and hospital-wide
process improvements, (2) Integration optimization and inno-
vation, and (3) Enterprise PACS and EPR. From there, the
authors defined five dimensions (strategy and policy, organi-
zation and processes, monitoring and control; information
technology, and people and culture) and five PACS maturity
stages that hospital can achieve: infrastructure, process, clini-
cal process capability, integrated managed innovation and op-
timized enterprise chain.
Telemedicine service maturity model (TMSMM)
The authors [52] consider that this maturity model can be
implemented to measure and manage the health system capa-
bility to provide clinical health care at a distance. Indeed, this
model can be used to measure, manage and optimize all com-
ponents of a telemedicine system and the health system in
which it is applied. The term Btelemedicine^was first used
in 1970 and refers to the provision health services (medicine)
at a distance (tele).
The TMSMM model is based on three dimensions. The
intersection of each pair forms a matrix, each one with specific
meaning and function. First, five domains are defined to pro-
vide a holistic view of all the factors that impact the imple-
mentation of telemedicine services. Secondly, the telemedi-
cine service dimension is built by five micro-level processes,
a meso-levelprocess and one macro-level process per domain.
The third domain is the maturity scale, which provides assess-
ment standards for maturity measurement. The domain
adopted by this model is the 5 M’s(BMan - Users
Communities^,BMachine - Infrastructures ICT^,BMaterial -
EHR systems,^BMethod - Change Management^and
BMoney - Financial Sustainability^). The maturity scale is
based on the stages indicators of CMM maturity model
(Capability Maturity Model). There are 5 stages. Stage 1: ad
hoc - service is unpredictable, experimental, and poorly con-
trolled; stage 2: managed - the service is characterized by
projects and is manageable; stage 3: standard - the service is
defined as a standard business process; stage 4: quantitatively
managed - the service is quantitatively measured and con-
trolled; stage 5: optimizing - focus on continuous
improvement.
Healthcare usability maturity model (UMM)
The Healthcare Usability Maturity Model helps healthcare
professional to assess the usability stages of IST of organiza-
tions and how they can advance to the next stage [57]. The
authors of this Maturity Model led a Usability Taskforce cre-
ated by HIMSS [50]. Its objective was to develop a new model
131 Page 6 of 10 J Med Syst (2016) 40:131

Tab le 2 Summary and comparison of maturity models for IST healthcare
Designation Health Field Stages Research method Influencing factors / dimensions Assessment tool Reference
model
Author/Year
Quintegra Maturity Model
for electronic Healthcare
(eHMM)
General 7 n/a Entities; Department; Infrastructure n/a n/a Sharma [2]
IDC Healthcare IT (HIT)
Maturity Model
General 5 n/a Types of IS n/a n/a [42]
IDC’s mobility maturity
model for healthcare
mHealth 5 Survey, Case study Intent; Technology; People; Processes IDC’s Mobility Maturity
Model Guidance
CMM [53]
HIMSS Maturity Model for
Electronic Medical
Record (EMRAM)
EMR 8 n/a Types of IS EMR Penetration
Assessment Tool
n/a [54]
HIMSS Continuity of Care
Maturity Model (CCMM)
General 8 n/a Types of IS n/a EMRAM [44]
Patient records/content man-
agement maturity model
(Forrester Model)
EMR 3 Interviews with US healthcare providers Access; Interoperability; Content Features;
Planning & Strategy
Guidance To Get To The
Next Phase
n/a [46]
Maturity Model for
Electronic Patient Record
(EPRMM)
EMR 6 n/a EPR System n/a n/a [45]
NEHTA Interoperability
Maturity Model (IMM)
Interoperability 5 ODP standards (open distributed
processing)
Organisation; Information; Technical Yes IMM / CMMI [47]
NHS Infrastructure Maturity
Model (NIMM)
Infrastructure
IT
5 n/a Process; People & Organisation; Technology;
Security & Information Governance; Strategy
Alignment & Business Value
Key Capabilities Self-
Assessment Tool
CMM [48]
Healthcare Analytics
Adoption Model
(HAAM)
Data
Warehouse
&Analysis
9 Data gathered by observation and
learned in a structured educational
curriculum, experts opinions
New Data Sources; Complexity; Data Literacy;
Data Timeliness
Healthcare Analytics
Adoption Model Self
Inspection Guide
EMRAM [51]
Hospital Cooperation
Maturity Model (HCMM)
Networking/
Cooperation
4 Interviews, Focus Group, prototype Strategic; organizational; Information HCMM Instantiation CMM [41]
PAC S Maturit y Mo de l
(PMM)
PACS 5 Literature review, qualitative meta-
analysis approach
Strategy & Policy; Organization & Processes;
Monitoring & Control; IT; People & Culture
Yes CMMI [ 49]
Telemedicine Service
Maturity Model
(TMSMM)
Telemedicine 5 Literature review, workshop with health
and IT professionals, case study
Man; Machine; Material; Method; Money Yes CMM [52]
Healthcare Usability
Maturity Model (UMM)
Usability 5 Literature review, case study Focus on users; Management; Process &
Infrastructure; Resources; Education
Yes Scha ff er UM ,
Nielsen UM,
Earthy UM
[50]
JMedSyst (2016) 40:131 Page 7 of 10 131

for identifying elements and main steps involved in successful
integration of usability in a healthcare organization.
The development of this model was based on the evalua-
tion of the characteristics of three usability maturity models
[58–60] and how they could be adopted in healthcare. In this
model, each phase enables organizations to identify their cur-
rent stage of usability and also includes guidelines to advance
to the next stage. The five stages are: unrecognized, prelimi-
nary, implemented, integrated and strategic. Within each
stage, these elements are taken in account: focus on users,
management, process and infrastructure, resources and
education.
Summary and closing remarks
The 14 Maturity Models of IST in health care resulting from
this literature review are summarized in Table 2. Besides the
identification of each model and its authors, is presented the
health field (general or specific), number of stages, research
method adopted in its development, influence factors consid-
ered, assessment tool and model that was used as reference for
its development.
As a result of this literature review, it was found that the
maturity models for health care ISTare developed by different
types of entities, including national and international health
care companies, research organizations in ICT as well as aca-
demic experts in this domain.
It was also found that there are two approaches: in one
hand, the highly specialized models that have resulted in a
health subsystem and in the other hand, the more comprehen-
sive models, i.e. models representing the hospital IS as a
whole (e.g. eHMM, IDC HIT, CCMM). Also, it was found
that most of the analyzed maturity models does not disclose
the design process nor the research options for development
and validation [41], thus compromising the researcher work.
It appears that CMM and CMMI his successor, is the ref-
erence model for the design of Maturity Models in the health
sector. This model has served as inspiration for dozens of
maturity models in the various areas of IST [61]. In fact, 6
of 14 identified models base its structure on the CMM model.
Regarding the number of maturity stages, there are models
from 3 stages as the Forrester Model [46] up to 9 stages of
HAAM [51].
It is noted that not all the identified maturity models with
various dimensions or influencing factors have explicitly bro-
ken down the characteristics for each stage of maturity. In fact,
from 11 maturity models with influence factors, only 5 dis-
criminate characteristics for each stage [2,41,46,50,52].
With regard to influence factors, it was detected entries with
the same name in different maturity models and entries with
different names but with the same meaning or interpretation
(result of using different terminology adopted by the authors).
Also, the authors did not apply weights to each of the influenc-
ing factors, that is, in the assessing process of the overall
maturity of health IST, all influencing factors have the same
importance.
In the case of adoption of a tool for assessing the system
maturity, it was found that most of the models, besides focus-
ing on the assessment of the system’s maturity, they pay at-
tention to an improvement path of such maturity. However,
not all have a properly systematized process to move to a
higher maturity level.
Some maturity models are developed by health national
and supranational organizations, mainly corporations, who
are dedicated to technological developments, such as IDC
Health Insights and HIMSS or even by national health orga-
nizations as the NHS or NEHTA. This fact complicates the
process of search and analysis of their respective models,
since access to information is restricted. Consequently, it is
not possible to know the development methodology and val-
idation adopted. Moreover, only a small part of the models
were published in IS Journals ([41,45,49], while the rest are
published mostly in white papers, making it impossible thus
attest to its validity in the context of peer review.
As a result of this study, none of the identified models has a
sufficiently broad scope covering all areas and subsystems of
health care organizations. In this sense, a maturity model with
a holistic approach including a comprehensive set of influenc-
ing factors is missing. In this perspective, a new model to fill
the gap should be designed. This new model, should include
the main influence factors with different weights depending
on their relative importance and its development should be
supported by rigorous scientific methods of conceptualization
and validation.
References
1. Freixo, J., and Rocha, Á., Arquitetura de Informação de Suporte à
Gestão da Qualidade em Unidades Hospitalares. RISTI - Revista
Ibérica de Sistemas e Tecnologias de Informação 14:1–18, 2014.
2. Sharma, B., Electronic Healthcare Maturity Model (eHMM).
Quintegra Solutions Limited, 2008.
3. Rocha, Á., Evolution of information systems and technologies ma-
turity in healthcare. Int. J. Healthc. Inf. Syst. Inform. 6(2):28–36,
2011.
4. Carvalho, J. V., Rocha, A., and Vasconcelos, J. B., Towards an
encompassing Maturity Model for the Management of Hospital
Information Systems. J. Med. Syst. 39(9):1–9, 2015.
5. Rocha, Á., and Vasconcelos, J., Os Modelos de Maturidade na
Gestão de Sistemas de Informação. Revista da Faculdade de
Ciência e Tecnologia da Universidade Fernando Pessoa. 1:93–
107, 2004.
6. Mutafelija, B., and Stromberg, H., Systematic Process Improvement
Using ISO 9001:2000 and CMMI. Artech House, Boston, 2003.
7. Nolan, R., Managing de computer resource: A stage hypotesis.
Commun. ACM 16(7):399–405, 1973.
131 Page 8 of 10 J Med Syst (2016) 40:131

8. Nolan, R., Managing the crisis in data processing. Harv. Bus. Rev.
57(2):115–126, 1979.
9. McKenney, J. L., and McFarlan, F. W., The information archipelago
- Maps and bridges. Harv. Bus. Rev. 60(5):109–119, 1982.
10. King, J., and Kraemer, K., Evolution and organizational informa-
tion systems: An assessment of Nolan’s stage model. Commun.
ACM 27(5):466–475, 1984.
11. Huff, S. L., Munro,M. C., and Martin, B.H., Growth stages ofend-
user computing. Commun. ACM 31(5):542–550, 1998.
12. Earl, M. J., Management Strategies for Information Technologies.
Prentice Hall, Upper Saddle River, 1989.
13. Galliers, R. D., and Sutherland, A. R., Information systems man-
agement and strategy formulation: The ‘stages of growth’model
revised. J. Inf. Syst. 1(2):89–114, 1991.
14. Auer, T., Beyond IS Implemention: A Skill-Based Aproach to IS
Use. 3rd European Conference on Information Systems, Athens,
Greece., 1995.
15. Khandelwal, V., and Ferguson, J., Critical Success Factors (CSFs)
and the Growth of IT in Selected Geographic Regions. Proceedings
of 32nd Hawaii International Conference on Systems Sciences
(HICSS-32), USA., 1999.
16. King, W. R., and Teo, T. S. H., Integration between business plan-
ning and information systems planning: Validating a stage hypoth-
esis. Decis. Sci. 28(2):279–307, 1997.
17. Mutsaers, E., Zee, H., and Giertz, H., The evolutionof information
technology. Inform. Manag. Comput. Secur. 6(3):115–126, 1998.
18. Nolan, R., and Koot, W., Nolan stages theory today: A framework
for senior and IT management to manage information technology.
Holland Manag. Rev. 31:1–24, 1992.
19. Damsgaard, J., and Scheepers, R., Managing the crises in intranet
implementation: A stage model. Inf. Syst. J. 10(2):131–149, 2000.
20. Holland, C., and Light, B., A stage maturity model for enterprise
resource planning systems. Data Base Adv. Inf. Syst. 32(2):34–45,
2001.
21. SEI Software Eng Institute, CMMI® for Development, Version 1.3,
Improving processes for developing better products and services.
Tech. Rep. No. CMU/SEI-2010-TR-033), Carnegie Mellon
University, 2010.
22. SPICE, SPICE Project: Software Process Assessment: Part 2 - A
model for process management, Version 1.0. 1995.
23. April, A., Abran, A., and Dumke, R., Assessment of software main-
tenance capability: A model and its architecture. Proceedings of the
8th European Conference on Software Maintenance and
Reengineering (CSMR2004), Los Alamitos CA: IEEE Computer
Society Press, 2004: p. 243–248.
24. Levin, G., and Nutt, H., Achieving Excellence in Business
Development: The Business Development Capability Maturity
Model. 2005. Available from: http://www.maturityresearch.com/
novosite/biblio/CMM_Achieving%20Excellence%20in%
20Business%20Development.pdf. Accessed Sep 2014.
25. Kerzner, H., Using the Project Management Maturity Model:
Strategic Planning for Project Management, 2nd edition. Wiley,
New York, 2005.
26. Brookes, N., and Clark, R., Using Maturity Models to Improve
Project Management Practice. POMS 20th Annual Conference.
Orlando Florida USA., 2009.
27. Murray, A., Capability Maturity Models - Using P3M3 to Improve
Performance. Outperform: MakingStrategy Reality, 2006. 2.
28. Venkatesh, V., et al., User acceptance of information technology:
Toward a unified view. MIS Q. 27(3):425–478, 2003.
29. Renken, J., Developing an IS/ICT management capability maturity
framework. . Research conference of the South African Institute for
Computer Scientists and Information Technologists (SAICSIT).
Stellenbosch, 2004: p. 53–62.
30. Earl, M. J., Evolving the EBusiness. Bus. Strateg. Rev. 11:33–38,
2000.
31. Gardler, R., and Mehandjiev, N., Supporting Component-Based
Software Evolution.In Aksit, M., Mezini, M. & Unland, R.
(Eds.), Objects, Components, Architectures, Services, and
Applications for a Networked World, 2003, Series: Lecture Notes
in Computer Science, 2591. Springer Verlag., 2003: p. 103–120.
32. Ludescher, G., and Usrey, M., Towards an ECMM (E-Commerce
Maturity Model). Proceedings of the First International Research
Conference on Organizational Excellence in the Third
Millennium. Estes Park: Colorado State University, 2000.
33. Marshall, S., E-Learning maturity model. Process descriptions.
Victoria University of Wellington, New Zealand, 2007.
34. Berztiss, A.T., Capability Maturity for Knowledge Management.
DEXA Workshop, IEEE Computer Society, 2002: p. 162–166.
35. Maybury, M.T., Knowledge Management at the MITRE
Corporation. The MITRE Corporation, 2002. https://www.
researchgate.net/publication/228858770_Knowledge_
Management_at_The_MITRE_Corporation.
36. Rosemann, M., and deBruin, T., Business Process Management
Maturity - A Model for Progression. Proceedings of the 13th
ECIS, Regensburg, 2005.
37. Nascio, NASCIO Enterprise Architecture Maturity Model, Version
1.3. National Associationof State Chief Information Officers, 2003.
38. DOC, IT Architecture Capability Maturity Model. Department of
Commerce, USA Government Introduction, 2003.
39. Webster, J., and Watson, R. T.,Analyzing the past to prepare forthe
future: Writing a literature review. MIS Q. 26(2):13–23, 2002.
40. Tranfield, D., Denyer, D., and Smart, P., Towards a methodology
for developing evidence-informed management knowledge by
means of systematic review. Br. J. Manag. 14:207–222, 2003.
41. Mettler, T., and Blondiau, A., HCMM –A Maturity Model for
Measuring and Assessing the Quality of Cooperation between
and within Hospitals. 25th IEEE International Symposium on
Computer-Based Medical Systems (CBMS), 2012.
42. Holland, M., Dunbrack, L., and Piai, S., Healthcare IT Maturity
Model: Western European Hospitals - The Leading Countries.
European IT Opportunity: Healthcare Healthcare Provider IT
Strategies Health Industry Insights, an IDC Company, 2008.
43. HIMSS, The EMR Adoption Model. HIMSS Analytics: Innovative
Research/Informed Decisions, 2008.
44. Etin, D., Quality of Care with IDC & HIMSS models - Where are
eHealth projects going in EMEA? EMC Spark, 2014. Available:
http://sparkblog.emc.com/2014/05/quality-care-idc-himss-models-
ehealth-projects-going-emea/. Accessed Sep 2015.
45. Priestman, W., ICT Strategy 2007–2011 for The Royal Liverpool
and Broadgreen University Hospitals NHS Trust. Trust Board
Meeting 6th November 2007., 2007. Document Number: V1.4.
46. Clair, C.L., Electronic Medical Records Need More To Support
BMeaningful Use^.Forrester Research Inc, 2010.
47. NEHTA, Interoperability Maturity Model: Version 2.0. National E-
Health Transition Authority Ltd, Sydney, 2007.
48. NHS, National Infrastructure Maturity Model [Online]. Available:
http://systems.hscic.gov.uk/nimm.Accessed 16 Sep 2015. 2011.
49. Wetering, R., and Batenburg, R., A PACS maturity model: A sys-
tematic meta-analytic review on maturation and evolvability of
PACS in the hospital enterprise. Int. J. Med. Inform. 78:127–140,
2009.
50. HIMSS, Promoting Usability in Health Organizations: Initial Steps
and Progress Toward a Healthcare Usability Maturity Model.
Healthcare Information and Management Systems Society, 2011
51. Sanders,D., Burton, D.A., andProtti,D., The Healthcare Analytics
Adoption Model: A Framework and Roadmap (white paper). 2013.
Available from: https://www.healthcatalyst.com/whitepaper/
healthcare-analytics-adoption-model/. Accessed Oct 2015.
52. van Dick, L., and Schutte, C.S.L., The Telemedicine Service
Maturity Model: A Framework for the Measurement and
JMedSyst (2016) 40:131 Page 9 of 10 131

Improvement of Telemedicine Services. INTECH: open science/
open minds, 2013. Chapter 10: 217–238.
53. Dunbrack, L. and L. Hand, A Maturity Model for Mobile in
Healthcare. IDC Health Insights: Business Strategy, 2013. Doc #
HI241777.
54. Garets, D., and Davis, M., Electronic Medical Records vs.
Electronic Health Records: Yes, There Is a Difference. AHIMSS
AnalyticsTM White Paper, 2006.
55. Sanders, D., A Model for Measuring Industry-Wide Adoption and
Capability of Healthcare Analytics and Data Warehousing in the
USA. ElectronicHealthcare 11(2), 2012.
56. Wetering, R., and Batenburg, R., Towards a Theory of PACS
Deployment: An Integrative PACS Maturity Framework. JDigit
Imaging. 27(3):337–350, 2014.
57. Staggers, N. and M. Rodney, Promoting Usability in Organizations
with a New Health Usability Model: Implications for Nursing
Informatics. NI 2012: Proceedings of the 11th International
Congress on Nursing Informatics, 396, 2012.
58. Nielsen, J., Corporate Usability Maturity Stages: 1–4 and 5–8.
2006. Available from: https://www.nngroup.com/articles/usability-
maturity-stages-1-4/. Accessed Nov 2015.
59. Earthy, J., Usability Maturity Model: Human-Centredness Scale.
IE2016 INUSE Deliverable D5.1.4s, 1998.
60. Schaffer, E., Instituionalization of Usability: A Step-By-Step Guide.
Addison-Wesley, Boston, 2004.
61. Becker, J., Niehaves, B. et al., Maturity Models in IS Research. 18th
European Conference on Information Systems, 2010.
131 Page 10 of 10 J Med Syst (2016) 40:131