ArticlePDF AvailableLiterature Review

Abstract and Figures

Objective: The purpose of this scoping review was to locate, examine and describe the literature on indicators used to measure prehospital care quality. Introduction: The performance of ambulance services and quality of prehospital care has traditionally been measured using simple indicators, such as response time intervals, based on low-level evidence. The discipline of paramedicine has evolved significantly over the last few decades. Consequently, the validity of utilizing such measures as holistic prehospital care quality indicators (QIs) has been challenged. There is growing interest in finding new and more significant ways to measure prehospital care quality. Inclusion criteria: This scoping review examined the concepts of prehospital care quality and QIs developed for ambulance services. This review considered primary and secondary research in any paradigm and utilizing any methods, as well as text and opinion research. Methods: Joanna Briggs Institute methodology for conducting scoping reviews was employed. Separate searches were conducted for two review questions; review question 1 addressed the definition of prehospital care quality and review question 2 addressed characteristics of QIs in the context of prehospital care. The following databases were searched: PubMed, CINAHL, Embase, Scopus, Cochrane Library and Web of Science. The searches were limited to publications from January 1, 2000 to the day of the search (April 16, 2017). Non-English articles were excluded. To supplement the above, searches for gray literature were performed, experts in the field of study were consulted and applicable websites were perused. Results: Review question 1: Nine articles were included. These originated mostly from England (n = 3, 33.3%) and the USA (n = 3, 33.3%). Only one study specifically aimed at defining prehospital care quality. Five articles (55.5%) described attributes specific to prehospital care quality and four (44.4%) articles considered generic healthcare quality attributes to be applicable to the prehospital context. A total of 17 attributes were identified. The most common attributes were Clinical effectiveness (n = 17, 100%), Efficiency (n = 7, 77.8%), Equitability (n = 7, 77.8%) and Safety (n = 6, 66.7%). Timeliness and Accessibility were referred to by four and three (44.4% and 33.3%) articles, respectively.Review question 2: Thirty articles were included. The predominant source of articles was research literature (n = 23; 76.7%) originating mostly from the USA (n = 13; 43.3%). The most frequently applied QI development method was a form of consensus process (n = 15; 50%). A total of 526 QIs were identified. Of these, 283 (53.8%) were categorized as Clinical and 243 (46.2%) as System/Organizational QIs. Within these categories respectively, QIs related to Out-of-hospital cardiac arrest (n = 57; 10.8%) and Time intervals (n = 75; 14.3%) contributed the most. The most commonly addressed prehospital care quality attributes were Appropriateness (n = 250, 47.5%), Clinical effectiveness (n = 174, 33.1%) and Accessibility (n = 124, 23.6%). Most QIs were process indicators (n = 386, 73.4%). Conclusion: Whilst there is paucity in research aiming to specifically define prehospital care quality, the attributes of generic healthcare quality definitions appear to be accepted and applicable to the prehospital context. There is growing interest in developing prehospital care QIs. However, there is a need for validation of existing QIs and de novo development addressing broader aspects of prehospital care.
Content may be subject to copyright.
Downloaded from https://journals.lww.com/jbisrir by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AWnYQp/IlQrHD3XI41p+sDLxav672OkrT/877EIC+J5ow99jAiYEWnKbirmBjL+1Tu1A== on 11/18/2018
Downloadedfromhttps://journals.lww.com/jbisrir by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AWnYQp/IlQrHD3XI41p+sDLxav672OkrT/877EIC+J5ow99jAiYEWnKbirmBjL+1Tu1A== on 11/18/2018
Indicators to measure prehospital care quality: a scoping
review
Robin Pap
1,2
Craig Lockwood
1
Matthew Stephenson
1
Paul Simpson
2
1
Joanna Briggs Institute, Faculty of Health and Medical Sciences, University of Adelaide, Adelaide, Australia,
2
School of Science and Health,
Western Sydney University, Sydney, Australia
ABSTRACT
Objective: The purpose of this scoping review was to locate, examine and describe the literature on indicators used
to measure prehospital care quality.
Introduction: The performance of ambulance services and quality of prehospital care has traditionally been
measured using simple indicators, such as response time intervals, based on low-level evidence. The discipline of
paramedicine has evolved significantly over the last few decades. Consequently, the validity of utilizing such
measures as holistic prehospital care quality indicators (QIs) has been challenged. There is growing interest in finding
new and more significant ways to measure prehospital care quality.
Inclusion criteria: This scoping review examined the concepts of prehospital care quality and QIs developed for
ambulance services. This review considered primary and secondary research in any paradigm and utilizing any
methods, as well as text and opinion research.
Methods: Joanna Briggs Institute methodology for conducting scoping reviews was employed. Separate searches
were conducted for two review questions; review question 1 addressed the definition of prehospital care quality and
review question 2 addressed characteristics of QIs in the context of prehospital care. The following databases were
searched: PubMed, CINAHL, Embase, Scopus, Cochrane Library and Web of Science. The searches were limited to
publications from January 1, 2000 to the day of the search (April 16, 2017). Non-English articles were excluded. To
supplement the above, searches for gray literature were performed, experts in the field of study were consulted and
applicable websites were perused.
Results: Review question 1: Nine articles were included. These originated mostly from England (n ¼3, 33.3%) and the
USA (n ¼3, 33.3%). Only one studyspecifically aimed at defining prehospital care quality. Five articles (55.5%) described
attributes specific to prehospital care quality and four (44.4%) articles considered generic healthcare quality attributes
to be applicable to the prehospital context. A total of 17 attributes were identified. The most common attributes were
Clinical effectiveness (n ¼17, 100%), Efficiency (n ¼7, 77.8%), Equitability (n ¼7, 77.8%) and Safety (n¼6, 66.7%).
Timeliness and Accessibility were referred to by four and three (44.4% and 33.3%) articles, respectively.
Review question 2: Thirty articles were included. The predominant source of articles was research literature (n ¼23;
76.7%) originating mostly from the USA (n ¼13; 43.3%). The most frequently applied QI development method was a
form of consensus process (n ¼15; 50%). A total of 526 QIs were identified. Of these, 283 (53.8%) were categorized as
Clinical and 243 (46.2%) as System/Organizational QIs. Within these categories respectively, QIs related to Out-of-
hospital cardiac arrest (n ¼57; 10.8%) and Time intervals (n ¼75; 14.3%) contributed the most. The most commonly
addressed prehospital care quality attributes were Appropriateness (n ¼250, 47.5%), Clinical effectiveness (n ¼174,
33.1%) and Accessibility (n ¼124, 23.6%). Most QIs were process indicators (n ¼386, 73.4%).
Conclusion: Whilst there is paucity in research aiming to specifically define prehospital care quality, the attributes of
generic healthcare quality definitions appear to be accepted and applicable to the prehospital context. There is
growing interest in developing prehospital care QIs. However, there is a need for validation of existing QIs and de
novo development addressing broader aspects of prehospital care.
Keywords Ambulance; emergency medical services; healthcare quality assessment; prehospital care; quality indicators
JBI Database System Rev Implement Rep 2018; 16(11):2192 2223.
Correspondence: Robin Pap, robin.pap@adelaide.edu.au
There is no conflict of interest in this project.
DOI: 10.11124/JBISRIR-2017-003742
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2192
SYSTEMATIC REVIEW
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
Introduction
An all-inclusive definition of prehospital care
includes all care that is provided by any service
to a patient before they reach a hospital. For the
purpose of this scoping review, prehospital care is
the care that ambulance services provide for patients
with urgent or emergency care needs.
1
It starts when
someone calls the ambulance service and ends when
the patient has been conveyed to hospital. In some
cases, all the care a patient needs can be provided
before being conveyed to hospital and there is no
need to transport them. Historically, ambulance
services were established to provide swift transport
of the sick and injured to hospital. Timely and safe
conveyance of patients with urgent and emergency
care needs to an appropriate healthcare facility
remains the central function of modern ambulance
services. However, the scope of prehospital care and
coverage that ambulance services provide has
evolved significantly over the last few decades.
2-6
The primary drivers of these developments have been
the professionalization of the paramedic industry,
improvements in how ambulance services integrate
into the wider healthcare systems and increasing
demand due to a range of factors, including an
aging and growing population and the expanding
burden of chronic disease.
7-11
Despite this growth,
the relatively new formation of the paramedicine
profession and consequent lack of research capacity
coupled with the complexities of conducting
data collection in the prehospital emergency care
setting have led to paucity of discipline-specific,
scientific evidence.
12-18
Consequently, the perfor-
mance and quality of ambulance services has tradi-
tionally been measured using naı
¨ve indicators based
on no or low-level evidence, e.g. response time
intervals.
13,16,19
These simple types of measures have
dominated ambulance services’ performance reports
because they are easily obtained and readily under-
stood by the public and policymakers alike.
13,19-21
Although shorter prehospital time intervals may be
associated with better outcomes in certain, time-
critical patient cohorts,
22,23
the validity of response
time as a holistic prehospital care quality indicator
(QI) has been challenged.
24,25
As a result, there is a
need for and growing interest in finding new and
more significant ways to measure prehospital care
quality.
26-29
A clear definition of quality is crucial for the
development of meaningful QIs. Donabedian
30
argued that quality cannot be assessed until it is
decided how it is to be defined. The concept of
quality is easily understood; however, defining it is
challenging because quality is highly contextual. In
the context of healthcare, the formulation of a defi-
nition has been a perpetual problem among health-
care managers and researchers.
31-36
This has led to
two approaches in defining quality in healthcare
generic and disaggregated definitions.
34
Generic def-
initions are broad and all-encompassing, whereas
disaggregated definitions recognize the multidimen-
sionality of the concept and focus on individual
components.
34
These components or attributes of
quality allow these definitions to be operationalized
in the form of quality frameworks, which are essen-
tial for the development of a balanced suite of
QIs.
37,38
The boundaries of each attribute may vary
depending on how the attribute itself is defined. This
can cause overlap which has led researchers to bun-
dle or aggregate attributes with significant common-
alities into dimensions. Campbell et al.
34
suggested
there are two principle dimensions of quality of
care for individual patients: access and effective-
ness. When discussing healthcare for populations,
additional dimensions are introduced: equity and
efficiency.
34
Quality indicators are measurable aspects
which provide a quantitative basis for clinicians,
organizations and planners aiming to improve the
processes by which patient care is provided and their
outcomes.
38
An ideal QI is characterized by a num-
ber of desirable features. Most importantly, a QI
should be meaningful, evidence-based, interpretable
and generalizable.
39-41
Quality indicators can be
classified in a range of different ways. Donabedian’s
approach of assessing the structures, processes and
outcomes of medical care is widely accepted as the
pre-eminent model for the measurement of quality
in healthcare. Donabedian defined ‘‘structure’’
as the attributes of the setting in which care is
provided (e.g. material resources, human resources
and organizational characteristics), ‘‘process’’ as
the activities that contribute to healthcare carried
out by healthcare practitioners (e.g. diagnosis,
treatment and patient education), and ‘‘outcomes’’
as the effects of healthcare on individuals or
populations.
30
The current scoping review sought to locate,
examine and describe the literature on indicators
used to measure prehospital care quality. Prior to
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2193
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
the development of the protocol,
42
a preliminary
search of the JBI Database of Systematic Reviews
and Implementation Reports and the Cochrane
Database of Systematic Reviews for previous scop-
ing or systematic reviews on the topic was performed
and revealed no results. This scoping review is useful
for healthcare professionals of ambulance services
who are involved in quality improvement programs
and researchers investigating ways of measuring
ambulance service quality and performance. It forms
part of a wider research project, the AuStralian
Prehospital care quality IndicatoR projEct
(ASPIRE), which aims to develop and test prehospi-
tal care QIs for the Australian setting.
Review questions
The purpose of this scoping review was to locate,
examine and describe the literature on indicators
used to measure prehospital care quality. Specifi-
cally, the review sought to answer the following
questions:
1) What attributes have been used to define or
describe the concept of quality in the context
of prehospital care provided by ambulance
services?
2) What processes have been used to develop pre-
hospital care QIs and what are the character-
istics of the QIs and of their underlying
frameworks?
Inclusion criteria
Participants
This scoping review examined definitions and
descriptions of prehospital care quality and QIs or
quality measures developed for ambulance services
providing prehospital care. Both paramedic (Anglo-
Saxon model) and physician systems (Franco-Ger-
man model) were considered. Countries of origin
were limited to those similar to Australia in terms of
economy (high-income based on World Bank open
data
43
) and healthcare system (western).
Concept
The concepts of interest for this scoping review were
‘‘quality’’ and ‘‘quality indicators/measures’’. For
the purpose of this review, indicators that were
intended to measure performance or quality were
considered. Indicators utilized to measure ambu-
lance service activity were excluded.
Context
The context of this scoping review was prehospital
care provided by ambulance services. Indicators
developed for other services providing prehospital
care (e.g. event medical management), other emer-
gency services (e.g. fire and rescue services) or
in-hospital emergency care (e.g. emergency depart-
ments) were excluded.
Types of studies
This review considered primary and secondary
research in any paradigm and utilizing any methods
as well as text and opinion research. The review also
considered documents developed by professional
organizations/accrediting bodies and governments.
Documents written by ambulance services for their
own, service-specific purposes were excluded.
Methods
This review employed Joanna Briggs Institute (JBI)
methodology for conducting scoping reviews.
44
The
objectives, inclusion criteria and methods for this
scoping review were specified in advance and docu-
mented in a protocol.
42
Search strategy
The search strategy aimed to find both published and
unpublished studies. A three-step search strategy
was utilized for each of the two review questions.
For each, an initial limited search of PubMed and
CINAHL was undertaken followed by analysis of
the text words contained in the titles and abstracts,
and of the index terms used to describe the articles. A
second search for each review question using
all identified keywords and index terms was then
undertaken in the following databases: PubMed,
CINAHL, Embase, Scopus, Cochrane Library and
Web of Science. Thirdly, the reference lists of all
selected reports and articles were searched for addi-
tional studies. Only English language papers were
included in this review due to this being the only
language all reviewers understand, as well as time
and budget constraints. The searches were limited
to publications from January 1, 2000 until the day
of the search (April 16, 2017) since more wide-
spread application of quality improvement techni-
ques, throughout all sectors of healthcare, has
occurred in the 21
st
century.
45
To supplement the
above, searches for gray literature on ProQuest
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2194
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
Dissertations and Theses, OpenThesis, Networked
Digital Library of Theses and Dissertations were
performed. The search strategy is detailed in Appen-
dix I. Furthermore, experts in the field of study were
consulted and the following websites of professional
organizations, accrediting bodies and government
agencies were manually searched:
Agency for Healthcare Research and Quality
(AHRQ), National Quality Measures Clearing-
house (NQMC)
46
Association of Ambulance Chief Executives
(AACE)
47
Australian Commission for Safety and Quality in
Health Care (ACSQHC)
48
Australian Government Productivity Commis-
sion
49
Care Quality Commission (CQC)
50
Council of Ambulance Authorities (CAA)
51
International Associating of Fire Fighters
(IAFF)
52
National Highway Traffic Safety Administration
(NHTSA) Office of Emergency Medical Services
(EMS)
53
National Health Service (NHS) England.
54
Study screening and selection
Screening and selection for inclusion was conducted
by two reviewers in accordance with the inclusion/
exclusion criteria above. Due to the large volume of
initial search results, the second reviewer was asked
to screen a random sample (20%) of all titles and
abstracts. Full-text reviews were done for all poten-
tial articles by both reviewers. Disagreements
between the two reviewers were resolved through
discussion and a third reviewer, when required.
Data extraction
Two charting tables were developed as part of the
protocol; one for each of the two review questions.
These were amended during piloting. In the charting
table for review question 1, the attribute categories
were changed to ‘‘explicitly stated’’ and ‘‘extrapo-
lated from generic healthcare definitions’’. In the
charting table for review question 2, a generic QI
framework consisting of Clinical and System/Orga-
nizational categories and relevant sub-categories, as
well as the identified attributes or prehospital care
quality, was compiled. These refinements resulted
from the iterative review and charting process typi-
cally performed in scoping reviews.
55
The refined
charting tables are shown in Appendix II. The tables
were converted to electronic data extraction forms
(Microsoft
1
Excel 2016 for Mac [Redmond, Wash-
ington, USA]). Relevant data were extracted from
the included articles and web-based sources to
address the review questions.
Concept-related data extracted for review ques-
tion 1 were definitions and/or attributes of preho-
spital care quality. Furthermore, the intended EMS
system was recorded, and the attributes were cate-
gorized into those that were explicitly stated in the
articles and those that were extrapolated from
generic definitions of healthcare quality and rea-
soned to be applicable to prehospital care setting
by the articles’ authors. For review question 2,
concept-related data extracted were characteristics
of the QIs. This included the origin, intended EMS
system, method of development and the Donabedian
type. Each indicator was categorized by the scoping
review authors into the QI framework (Clinical or
System/Organizational category and sub-category),
assigned to one or more of the identified prehospital
care quality attributes, and classified according to
Donabedian’s model.
Presentation of results
Search results and article selections were summa-
rized in flowcharts adapted from the Preferred
Reporting Items for Systematic reviews and Meta-
Analyses (PRISMA) flowchart developed by Moher,
et al.
56
Article characteristics, prehospital care qual-
ity attributes and QI characteristics were summa-
rized in tabular form showing counts and
proportions. Bar charts were compiled to illustrate
distribution of prehospital care quality attributes
and framework categories using Microsoft
1
Excel
for Mac 2016 (Redmond, Washington, USA).
Results
Review question 1
Selection
The database searches yielded 1301 citations in total
(Figure 1). An additional 26 records were found
through other sources. After duplicates were
removed, 1185 citations remained. The titles and
abstracts for these were screened, 1170 papers had
irrelevant titles and/or abstracts and hence were
excluded. The remaining 15 citations were consid-
ered for further detailed assessment of the full article
and six were excluded as they did not meeting the
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2195
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
inclusion criteria, i.e. not prehospital care provided
by ambulance services. The search yielded a total of
nine articles for inclusion in the review.
Description of articles
Two (22.2%) articles stemmed from primary
research studies (Table 1). The remaining seven were
secondary research articles (n ¼4; 44.4%) and gov-
ernment/industry reports (n ¼3; 33.3%). Only one
study (Owen
57
) specifically aimed at defining pre-
hospital care quality and its attributes. Haugland
et al.
58
developed QIs for prehospital care and
aligned these to attributes of quality taken from a
generic healthcare quality definition. The two
included literature reviews (O’Meara
19
; El Sayed
59
)
explored performance frameworks for ambulance
services and referred to several framework dimen-
sions or quality attributes, either specific to preho-
spital care or healthcare in general. Spaite et al.
18
proposed a conceptual model for prehospital care
outcomes research. Outcome categories can provide
a useful framework for measurement and hence
attributes of prehospital care quality. Milner
60
provided an opinion piece on improving ways to
evaluate the quality of emergency services, including
ambulance services. Lastly, a national ambulance
services performance report,
61
an ambulance service
quality inspection framework,
62
and a report on the
future of emergency medical services
63
were concep-
tually and contextually appropriate for inclusion.
Proportionally, most articles were published after
2014 (n ¼3; 33.3%) and originated from England
(n ¼3; 33.3%) and the USA (n ¼3; 33.3%). Two
articles originated from Australia (22.2%). The
majority (n ¼8; 88.9%) of articles referred to para-
medic systems. Five articles (55.5%) described
attributes to specifically define prehospital care qual-
ity and four (44.4%) articles considered attributes of
healthcare quality in general as applicable to the
prehospital context.
Records idenfied through
database searching
Addional records idenfied
through other sources
Records aer duplicates removed
(n = 1185)
Records screened
(n = 1185)
Records excluded
(n = 1170)
Full-text studies assessed
for eligibility
(
n = 15
Full-text studies excluded, with
reasons
(n = 6)
not prehospital n = 4:
not ambulance service n = 2:
Studies included
(n = 9)
Screening
Included Eligibility Idenficaon
(n = 1301) (n = 26)
Figure 1: Search results and study selection and inclusion process for review question 1
36
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2196
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
Descriptions of prehospital care quality
attributes
A total of 17 attributes of prehospital care quality
were identified via the article review ranging from
two to nine attributes per article (Table 2). All
articles referred to Clinical effectiveness. Other com-
mon attributes were Efficiency (n ¼7; 77.8%), Equi-
tability (n ¼7; 77.8%) and Safety (n ¼6; 66.7%).
Timeliness and Accessibility were referred to by four
and three (44.4% and 33.3%) articles, respectively.
Table 2 details which attributes were specifically
described as those of prehospital care quality and
which were extrapolated from generic healthcare
quality definitions but considered applicable to the
prehospital context by the articles’ authors. The
generic healthcare quality definitions the articles’
authors referred to originated from the Institute of
Medicine (IOM)
64
and Maxwell.
65
Table 1: Characteristics of articles (review question 1)
Characteristic
No. (%) out of a total of
nine articles
a
Type of research/project:
Primary research 2 (22.2)
Consensus method 1 (11.1)
Mixed qualitative methods 1 (11.1)
Secondary research 4 (44.4)
Literature/systematic review 2 (22.2)
Editorial, focus, perspective, commentary, other 2 (22.2)
Government sources/industry report 3 (3.33)
Year of publication:
20002004 2 (22.2)
20052009 2 (22.2)
20102014 2 (22.2)
20152017 3 (33.3)
Country of origin:
b
England 3 (33.3)
USA 3 (33.3)
Australia 2 (22.2)
Norway 1 (11.1)
International 1 (11.1)
EMS system:
Paramedic 8 (88.9)
Physician 1 (11.1)
Attributes of pre-hospital care quality:
Explicitly stated 5 (55.5)
Extrapolated from generic health care quality definitions 4 (44.4)
a
Percentages may not total 100 due to rounding.
b
Categories are not mutually exclusive.
EMS, emergency medical services.
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2197
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
Review question 2
Selection
The database searches identified a total of 10,359
potential records for review (Figure 2). An additional
six records were found through other sources. After
duplicates were removed, 7594 articles remained.
Following title and abstract screening, 7540 records
did not meet inclusion criteria and were excluded.
The full-text articles of the remaining 54 citations
were read and 24 were excluded due to not contain-
ing any QIs, being set in an irrelevant context or
being specific to an ambulance service. The search
produced 30 articles for inclusion in the review.
Description of articles
All included articles aimed at producing QIs or
quality measures, either exclusively or in part for
ambulance services providing prehospital care.
Where only part of the indicators was intended
for prehospital care, details of only those indicators
were extracted. The predominant source of articles
was the research literature (n ¼23; 76.7% of
included articles) and the most common method
applied to develop QIs was a form of consensus
process (n ¼15; 50%) (Table 3). There was an
increase in publications over time with 20
(66.7%) articles being published since the year
2010. The prevalent country of origin was the
USA (n ¼13; 43.3%). Three articles originated from
Australia (10%). The majority of articles presented
QIs that were developed for paramedic systems
(n ¼25; 83.3%).
Description of quality indicators
A total of 526 QIs were identified in the review
(median per article 12.5; interquartile range 6.3)
ranging from one to 101 QIs per article. The major-
ity (n ¼436; 82.9%) of QIs originated from research
literature identified in the database searches
(Table 4). The remaining 90 (17.1%) were devel-
oped by government agencies (n ¼69; 13.1%) and
professional organizations or accrediting bodies
(n ¼21; 4%). Four hundred and nine QIs (77.8%)
were developed by means of a consensus process.
Literature, scoping, or systematic reviews were used
for the development of 281 QIs (53.4%). Most QIs
Table 2: Attributes of prehospital care quality (review question 1)
Attribute
Owen,
2010
57
O’Meara,
2005
19
Spaite,
et al.,
2001
18
AGPC/
CAA,
2016
61
CQC,
2016
50
IOM,
2006
63
El-Sayed,
2012
59
Milner,
et al.,
2001
60
Haugland,
et al.,
2017
58
No. (%)
out of a
total of
nine
articles
Clinical effectiveness x x x x x xx xx xx xx 9 (100)
Efficiency x x x xx xx xx xx 7 (77.8)
Equitability x x x xx xx xx xx 7 (77.8)
Safety x x x xx xx xx 6 (66.7)
Appropriateness x x x xx 4 (44.4)
Timeliness x xx xx xx 4 (44.4)
Accessibility x x x 3 (33.3)
Patient-centeredness xx xx xx 3 (33.3)
Responsiveness x x xx 3 (33.3)
Acceptability x xx 2 (22.2)
Continuity/Sustainability x x 2 (22.2)
Availability x 1 (11.1)
Capability x 1 (11.1)
Caring x 1 (11.1)
Cost-effectiveness x 1 (11.1)
Interpersonal effectiveness x 1 (11.1)
Well-led x 1 (11.1)
x: specifically described as prehospital care quality attributes.
xx: extrapolated from generic health care quality attributes by the ar ticles’ authors.
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2198
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
Records idenfied through
database searching
Addional records idenfied
through other sources
Records aer duplicates removed
(n = 7594)
Records screened
(n = 7594)
Records excluded
(n =7540)
Full-text studies assessed
for eligibility
Full-text studies excluded,
with reasons
(n = 24)
no quality n=14:
indicators
not prehospital/ n=9:
ambulance
service
service-specific n=1:
Studies included
(n = 30)
Screening
Included Eligibility Idenficaon
(n = 6)(n = 10,359)
(n = 54)
Figure 2: Search results and study selection and inclusion process for review question 2
36
Table 3: Characteristics of articles that inform the development of prehospital care quality indicators
and their underlying frameworks (review question 2)
Characteristic No. (%) out of a total of 30 articles
a
Literature origin:
Research literature 23 (76.7)
Government 5 (16.7)
Professional association/accrediting body 2 (6.7)
Type of research/project:
Consensus method 15 (50)
Systematic/scoping/literature review 5 (16.7)
Observational cohort study 4 (13.3)
Retrospective case series/audit 3 (10)
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2199
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
Table 3. (Continued)
Characteristic No. (%) out of a total of 30 articles
a
Not reported 3 (10)
Year of publication:
20002004 3 (10)
20052009 7 (23.3)
20102014 11 (36.7)
20152017 9 (30)
Country of origin:
USA 13 (43.3)
Canada 4 (13.3)
England 4 (13.3)
Australia 3 (10)
Netherlands 2 (6.7)
Denmark 1 (3.3)
Ireland 1 (3.3)
Israel 1 (3.3)
Norway 1 (3.3)
EMS system:
Paramedic 25 (83.3)
Physician 5 (16.7)
a
Percentages may not total 100 due to rounding.
Table 4: Characteristics of quality indicators
Characteristic No.(%) out of a total of 526 QIs
a
Literature origin:
Research literature 436 (82.9)
Government 69 (13.1)
Professional association/accrediting body 21 (4)
Indicator development method:
b
Consensus process 409 (77.8)
Literature/Scoping/Systematic review 281 (53.4)
Guidelines-based 45 (8.6)
Case audit 20 (3.8)
Unclear/not reported 38 (7.2)
EMS system:
Paramedic 464 (88.2)
Physician 62 (11.8)
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2200
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
Table 4. (Continued)
Characteristic No.(%) out of a total of 526 QIs
a
Framework component:
Clinical QIs: 283 (53.8)
Airway management and oxygenation 27 (5.1)
Asthma 23 (4.4)
Acute coronary syndrome 36 (6.8)
Out-of-hospital cardiac arrest 57 (10.8)
Pain management 17(3.2)
Seizures 11 (2.1)
Stroke 27 (5.1)
Trauma 35 (6.7)
Hypoglycemia 11 (2.1)
General 27 (5.1)
Other disease-specific 12 (2.3)
System/Organizational QIs: 243 (46.2)
Communication/Dispatch 7 (1.3)
Documentation 12 (2.3)
Education 3 (0.6)
Financial 2 (0.4)
Hospital notification 11 (2.1)
Paramedic health and safety 10 (1.9)
Patient safety 14 (2.7
Patient satisfaction 11 (2.1)
Personnel performance evaluation/audit 11 (2.1)
Research 1 (0.2)
Resources/Deployment 66 (12.5)
Time intervals 75 (14.3)
Other 20 (3.8)
Prehospital care quality attributes:
b
Acceptability 11 (2.1)
Accessibility 124 (23.6)
Appropriateness 250 (47.5)
Availability 48 (9.1)
Caring 33 (6.3)
Capability 35 (6.7)
Clinical effectiveness 174 (33.1)
Continuity/Sustainability 15 (2.9)
Cost-effectiveness 12 (2.3)
Efficiency 11 (2.1)
Equitability 36 (6.8)
Interpersonal effectiveness 13 (2.5)
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2201
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
Table 4. (Continued)
Characteristic No.(%) out of a total of 526 QIs
a
Patient-centeredness 34 (6.5)
Responsiveness 32 (6.8)
Safety 36 (6.8)
Timeliness 86 (16.3)
Well-led 24 (4.6)
Reported Donabedian type:
b
Structure 49 (9.3)
Process 268 (51)
Outcome 57 (10.8)
Not reported 154 (29.3)
Assigned Donabedian type:
Structure 63 (12)
Process 386 (73.4)
Outcome 77 (14.6)
a
Percentages may not total 100 due to rounding.
b
Categories are not mutually exclusive.
EMS, emergency Medical Services; QI, quality indicator.
0 102030405060
Seizures
Hypoglycemia
Other disease-specific
Pain management
Asthma
Stroke
General
Airway management and oxygenation
Trauma
Acute coronary syndrome
Out-of-hospital cardiac arrest
Figure 3: Distribution of quality indicators within the Clinical framework component (total quality
indicators n¼526, Clinical quality indicators n¼283)
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2202
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
were developed in countries or for ambulance ser-
vices with paramedic systems (n ¼464; 88.2%).
Amongst the 526 QIs there was an almost even
distribution between Clinical QIs (n ¼283; 53.8%)
and System/Organizational QIs (n ¼243; 46.2%).
Further distribution amongst the Clinical and Sys-
tem/Organizational sub-categories areas is detailed
in Table 4 and illustrated in Figures 3 and 4. The
Clinical conditions for which most QIs were devel-
oped were Out-of-hospital cardiac arrest (n ¼57;
10.8%), Acute coronary syndrome (n ¼36; 6.8%)
and Trauma (n ¼35, 6.7%).
Within the System/Organizational category, the
most frequent sub-categories were Time intervals
(n ¼75; 14.3%), Resources/Deployment (n ¼66;
12.5%) and Other (n ¼20; 3.8%) which comprised
many low-acuity transport and referral aspects.
The most commonly addressed prehospital care
quality attribute was Appropriateness (n ¼250;
47.5%). This was followed by Clinical effectiveness
(n ¼174; 33.1%) and Accessibility (n ¼124; 23.6%).
Figure 5 shows the distribution of prehospital care
quality attributes amongst the QIs. Appendices III-V
respectively show itemization of Clinical sub-
categories, System/Organizational sub-categories,
and prehospital care quality attributes by article.
The Donabedian type was reported for 372 QIs
(71.1%). Two QIs were classified as both Structure
and Process indicators. The remaining 154 QIs
(29.3%) were assigned a Donabedian type by the
scoping review authors. Ultimately, QIs assessing a
Process were the predominant type (n ¼386; 73.4%).
When bundled into an Access dimension (Availabil-
ity, Accessibility and Timeliness) and an Effectiveness
dimension (Appropriateness, Clinical effectiveness,
Interpersonal effectiveness), the number of QIs from
the research literature (n ¼436) which addressed at
least one of the attributes within the Access dimension
was 109 (25%) and the number of QIs which
addressed at least one of the attributes within the
Effectiveness dimension was 260 (59.6%). For QIs
stemming from government agencies (n ¼69), these
numbers were 26 (37.7%) and 41 (59.4%), respec-
tively. For QIs developed by professional organizations
or accrediting bodies (n ¼21)theywerefive(23.8%)
and seven (30%). This is illustrated in Figure 6.
0 1020304050607080
Research
Financial
Education
Communication/Dispatch
Paramedic health and safety
Personnel performance evaluation/Audit
Patient satisfaction
Hospital notification
Documentation
Patient safety
Other
Resources/Deployment
Time intervals
Figure 4: Distribution of quality indicators within the System/Organizational framework component
(total quality indicators n¼526, System/Organizational quality indicators n¼243)
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2203
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
0 50 100 150 200 250 300
Acceptability
Efficiency
Cost-effectiveness
Interpersonal effectiveness
Continuity/Sustainability
Well-led
Responsiveness
Caring
Patient-centeredness
Capability
Equitability
Safety
Availability
Timeliness
Accessibility
Clinical effectiveness
Appropriateness
Figure 5: Distribution of quality indicators amongst prehospital care quality attributes (total quality
indicators n¼526, categories are not mutually exclusive)
QI: quality indicator
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
QIs from professional
organisations/accrediting bodies
QIs from governments
QIs from research literature
Access dimension Effectiveness dimension Other dimensions
Figure 6: Percentage of quality indicators under Access, Effectiveness and Other dimensions
(research literature n¼436, government agencies n¼69 and professional organizations/accrediting
bodies n¼21)
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2204
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
As detailed in the protocol for the review,
42
the
authors had intended to present a table combining
duplicate QIs and showing frequency counts. How-
ever, due to significant heterogeneity amongst the
QIs this synthesis was deemed infeasible.
Discussion
This scoping review identified and examined the
literature on indicators to measure prehospital care
quality. Given that the development of meaningful
QIs requires clear understanding of how quality is
being defined, the first part of the review addressed
attributes of prehospital care quality. It has been
argued that characteristics of prehospital care qual-
ity should be no different to those of healthcare
quality in other parts of the system.
66
When com-
pared to attributes of quality in performance frame-
works of wider healthcare systems internationally,
67
none of the attributes identified in this review which
were specifically described as prehospital care qual-
ity attributes can be considered exclusive to this
context. Thus, it could be said that as a component
of healthcare, prehospital care has common attrib-
utes with generic definitions of healthcare quality.
The prehospital setting, however, is different and
unique in many ways. Ambulance services deal pre-
dominantly with urgent and emergency calls, either
real or perceived, and often are required to provide
coverage for communities spread over large geo-
graphical areas. Prehospital care practitioners fre-
quently work in austere environments and with
relatively limited resources. Besides being responsi-
ble for initial access to the healthcare system, in most
cases ambulance services need to provide transport
and facilitate further access to appropriate health-
care services. Although the search results indicate
significant scarcity of research that defines quality in
this specific context, the findings suggest that timely
access to appropriate, safe and effective care, which
is responsive to patients’ needs and efficient and
equitable to populations are the key quality attrib-
utes in the prehospital context. These key attributes,
or dimensions encompassing them, may be mapped
to a routine prehospital care pathway.
57
Further-
more, they should be addressed in prehospital care
quality indicator frameworks to facilitate holistic
performance measurements and quality improve-
ment. Campbell et al.
34
and Owen
57
developed such
frameworks for general healthcare and prehospital
care, respectively. The frameworks were created by
combining the dimensions of quality (access and
effectiveness) with Donabedian’s structure, process
and outcomes model. Integrating the key attributes
of prehospital care quality identified in this scoping
review to such frameworks may provide useful mod-
els for QI developers and ultimately ambulance
services endeavoring to systematically evaluate the
quality of their care.
The increase in publications on prehospital care
QIs in recent years confirms that, at least in the
research community, there is growing interest in
measurement of quality in this context. Considering
the relative paucity of QIs available from govern-
ments and professional organizations or accrediting
bodies, the evidently increasing capacity to develop
QIs using systematic, evidence-based methods could
be seen as an opportunity for ambulance services or
professional associations to collaborate with aca-
demic institutions.
The majority of QIs identified in this review were
developed in English-speaking countries and for
paramedic systems. However, these findings are
likely to have been influenced by the language
restrictions in the database searches. Paramedic sys-
tems, as opposed to physician systems, are the more
common EMS model found in English-speaking
countries.
68,69
Ideally, the content of a QI should
be based on clinical evidence. However, in health-
care disciplines with a limited clinical evidence base,
such as paramedicine, QIs may need to be developed
using available clinical evidence alongside expert
judgement.
70
It is therefore unsurprising that con-
sensus processes were the most frequent method
being applied in the development of QIs. Whilst
several consensus methods exist, the RAND/UCLA
Appropriateness Method (RAM), developed by the
Research and Development (RAND) Corporation in
collaboration with the University of California, Los
Angeles (UCLA),
71
is the only method combining
available evidence with expert opinion. Originally
designed to investigate expert consensus on the
appropriateness of medical interventions, RAM is
a validated method to develop quality indica-
tors,
70,72,73
including those specific to prehospital
care.
57
There was reasonable balance overall between
QIs categorized as Clinical and those categorized
as System/Organizational. However, within the
Clinical category there was a strong focus on
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2205
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
Out-of-hospital cardiac arrest and within the Sys-
tem/Organizational component the most frequent
sub-category was Time intervals. Although these
QIs address vital aspects of care for small cohorts
of time-critical patients, the results suggest that even
in the new millennium these indicators continue to
dominate what is meant to be holistic and balanced
prehospital care quality measurement.
The attributes of prehospital care quality which
were addressed most frequently by the QIs appeared
to correspond somewhat with the key attributes
identified in the first part of the review. The excep-
tion to this was Efficiency, which was included in
seven (77.8%) articles describing prehospital care
quality but addressed by only 11 (2.1%) of all QIs.
When bundled into Access (Accessibility, Availabil-
ity, Timelines) and Effectiveness (Appropriateness,
Clinical effectiveness, Interpersonal effectiveness)
dimensions, a comparison between the different
QI origins suggests that professional organizations
and accrediting bodies appeared to have relatively
less focus on QIs addressing aspects of Effectiveness
(Figure 6), strengthening the argument for more
collaboration between academic and non-academic
institutions.
Process was the most common Donabedian type
amongst the QIs, both before and after the scoping
review authors assigned a type. Considering the
short patient contact time in prehospital care and
the complexities of relating hospital-based outcome
measures to preceding prehospital care, a prevalence
of process QIs in this context can be expected. For
these to be true QIs though, they need to relate to
improved outcomes. A valid process indicator is one
which previously has been demonstrated to produce
a better outcome.
38
Similar principles apply to struc-
tural indicators for quality assessment in that the
structural component needs to show increased like-
lihood of resulting in a desirable outcome or related
process.
38
An assessment of the underlying evidence
and validation of the QIs was beyond the scope of
this review. Considering the historical perspectives
of quality measurement in prehospital care, there is a
need for research appraising the validity of preho-
spital care QIs.
Scoping reviews are subject to the limitations of
any review. The search may not have been exhaus-
tive due to date range settings and language restric-
tions. This may be especially true for data
originating from physician EMS systems (Franco-
German system) which are more likely to be pub-
lished in languages other than English. Being a
scoping review, no rating of the quality of evidence
was performed.
Conclusion
There is a paucity of research investigating how
prehospital care quality is defined or which generic
attributes of healthcare quality are perceived to be
most important in prehospital care. Literature
reviewed in this study suggests that high-quality
prehospital care involves timely access to appropri-
ate, safe and effective care, which is responsive to
patients’ needs and efficient and equitable to pop-
ulations. There is growing interest in how preho-
spital care quality can be measured. Considering
the limited evidence base of paramedicine, the
prevalence of consensus methods being used in
the development of QIs and the advances of
the profession, there is a need for validation of
existing QIs and scientifically rigorous de novo
QI development.
Acknowledgments
The authors would like to thank Mr. James Pearce
for being the second reviewer in the article selection
process.
Funding
This review, and the larger project it is a part of, is
receiving support through an Australian Govern-
ment Research Training Program Scholarship.
References
1. Sayre MR, White LJ, Brown LH, Armacost M, Dean JM, Frame
SB, et al. The National EMS Research Agenda. Prehospital
Emerg Care 2002;6(sup3):1– 43.
2. O’Meara PF, Tourle V, Stirling C, Walker J, Pedler D. Extend-
ing the paramedic role in rural Australia: A story of flexibility
and innovation. Rural Remote Health 2012;12(2):1978.
3. Cooper SJ, Grant J. New and emerging roles in out of
hospital emergency care: A review of the international
literature. Int Emerg Nurs 2009;17(2):908.
4. O’Meara P, Grbich C. Paramedics in Australia. Frenchs Forest,
NSW: Pearson Education Australia; 2009.
5. Woollard M. The role of the paramedic practitioner in the
UK. J Emerg Prim Heal Care 2006;4(1):19.
6. Trevithick S, Flabouris A, Tall G, Webber CF. International
EMS systems: New South Wales, Australia. Resuscitation
2003;59(2):165–70.
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2206
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
7. Joyce CM, Wainer J, Piterman L, Wyatt A, Archer F. Trends in
the paramedic workforce: A profession in transition. Aust
Heal Rev 2009;33(4):53340.
8. Williams B, Onsman A, Brown T. From stretcher-bearer to
paramedic: the Australian paramedics’ move towards pro-
fessionalisation. J Emerg Prim Heal Care 2009;7(4):12.
9. Stirling CM, O’Meara P, Pedler D, Tourle V, Walker J. Engaging
rural communities in health care through a paramedic
expanded scope of practice. Rural Remote Health 2007;7
(4):839.
10. The Council of Ambulance Authorities. The factors affecting
the supply of health services and medical professionals in
rural areas - submission to the senate standing committee
on community affairs. Melbourne 2011.
11. IBIS World. Ambulance Services in Australia: Market
Research Report (Snapshot) [Internet]. 2017 [cited 2017
Nov 4]. Available from: https://www.ibisworld.com.au.
12. Callaham M. Quantifying the scanty science of prehospital
emergency care. Ann Emerg Med 1997;30(6):78590.
13. Dunford J, Domeier RM, Blackwell T, Mears G, Overton J,
Rivera-Rivera EJ, et al. Performance Measurements in Emer-
gency Medical Services. Prehospital Emerg Care 2002;6(1):
92– 8.
14. Tippett V, Clark M, Woods S, FitzGerald G. Towards a national
research agenda for the ambulance and prehosital sector in
Australia. J Emerg Prim Heal Care 2003;1(1). Tippett.
15. MacFarlane C. The advances and evidence base for preho-
spital care. Emerg Med J 2003;20(2):1145.
16. Myers JB, Slovis CM, Eckstein M, GoodloeJM, Isaacs SM, Loflin
JR, et al. Evidence-based performance measures for emer-
gency medical services systems: A model for expanded EMS
benchmarking. Prehospital Emerg Care 2008;12(2):141– 51.
17. Turner J. Building the evidence base in pre-hospital urgent
and emergency care: a review of research evidence and
priorities for further research Sheffield: University of Shef-
field Medical Care Research Unit; 2010.
18. Spaite D, Maio RF, Garrison H, Desmond J, Gregor M, Stiell I,
et al. Emergency Medical Services Outcomes Project
(EMSOP) II: Developing the Foundation and Conceptual
Models for Out-of-Hospital Outcomes Research. Ann Emerg
Med 2001;37(6):657–63.
19. O’Meara P. A generic performance framework for ambu-
lance services: an Australian health services perspective.
Australas J Paramed 2005;3(3):1– 13.
20. Moore L. Measuring quality and effectiveness of prehospital
ems. Prehospital Emerg Care 1999;3(4):32531.
21. Maio R, Garrison H, Spaite D, Desmond J, Gregor M, Cayten
C, et al. Emergency Medical Services Outcomes Project I
(EMSOP I): Prioritizing Conditions for Outcomes. Ann Emerg
Med 1999;33(4):423–32.
22. Studnek JR, Garvey L, Blackwell T, Vandeventer S, Ward SR.
Association between prehospital time intervals and ST-
elevation myocardial infarction system performance. Circu-
lation 2010;122(15):1464–9.
23. Takahashi M, Kohsaka S, Miyata H, Yoshikawa T, Takagi A,
Harada K, et al. Association between prehospital time
interval and short-term outcome in acute heart failure
patients. J Card Fail 2011;17(9):742– 7.
24. Blanchard IE, Doig CJ, Hagel BE, Anton AR, Zygun DA,
Kortbeek JB, et al. Emergency medical services response
time and mortality in an urban setting. Prehospital Emerg
Care 2012;16(1):142–51.
25. Pons PT, Haukoos JS, Bludworth W, Cribley T, Pons KA,
Markovchick VJ. Paramedic response time: Does it affect
patient survival? Acad Emerg Med 2005;12(7):594 600.
26. Archer F, Woollard M, Snooks H, Evans A, Wells B, Peconi J,
et al. What are the highest priorities for research in pre-
hospital care? Results of a review and Delphi consultation
exercise. Australas J Paramed 2008;6(4):3.
27. Snooks H, Evans A, Wells B, Peconi J, Thomas M, Woollard M,
et al. What are the highest priorities for research in
emergency prehospital care? Emerg Med J 2009;26(8):
549– 50.
28. Pickering A, Mason S, Turner J, Bradley P, Irving S. Emer-
gency services review: A comparative review of interna-
tional ambulance service best practice. London 2009.
29. Coster JE, Irving AD, Turner JK, Phung V-H, Siriwardena AN.
Prioritizing novel and existing ambulance performance
measures through expert and lay consensus: A three-stage
multimethod consensus study. Health Expectations 2017;
21(1):249–60.
30. Donabedian A. The quality of care: How can it be assessed?
J Am Med A 1988;260(12):17438.
31. Deming WE. Out of the Crisis. Cambridge, MA: Massachu-
setts Institute of Technology; 1986.
32. Donabedian A. Evaluating the quality of medical care.
Millbank Meml Fund Q 1966;44(3):166– 206.
33. Blumenthal D. Quality of health care - part 1: Quality of care
- What is it? N Engl J Med 1996;335(12):891 4.
34. Campbell S, Roland M, Buetow S. Defining quality of care.
Soc Sci Med 2000;51(11):161125.
35. Cooperberg MR, Birkmeyer JD, Litwin MS. Defining high
quality health care. Urol Oncol Semin Orig Investig
2009;27(4):411–6.
36. Australian Institute for Health and Welfare. Definitions of
safety and quality of health care [Internet]. 2016 [cited 2016
Feb 23]. Available from: http://www.aihw.gov.au/sqhc-def-
initions/.
37. Rubin HR, Pronovost P, Diette GB. From a process of care to a
measure: The development and testing of a quality indica-
tor. Int J Qual Health Care 2001;13(6):489 96.
38. Mainz J. Defining and classifying clinical indicators for
quality improvement. Int J Qual Heal Care 2003;15(156):
523– 30.
39. Hearnshaw HM, Harker RM, Cheater FM, Baker RH, Grim-
shaw GM. Expert consensus on the desirable characteristics
of review criteria for improvement of health care quality.
Qual Heal Care 2001;10(3):173–8.
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2207
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
40. Mainz J. Developing evidence-based clinical indicators: a
state of the art methods primer. Int J Qual Heal Care
2003;15(Supp 1):i5– 11.
41. Jones P, Shepherd M, Wells S, Le Fevre J, Ameratunga S.
Review article: what makes a good healthcare quality
indicator? A systematic review and validation study. Emerg
Med Australas 2014;26(2):113–24.
42. Pap R, Lockwood C, Stephenson M, Simpson P. Indicators to
measure pre-hospital care quality: a scoping review proto-
col. JBI Database Syst Rev Implement Reports 2017;15(6):
1537–42.
43. The World Bank. The World Bank Open Data [Internet]. 2016
[cited 2016 Nov 23]. Available from: http://www.worldban-
k.org/.
44. Aromataris E, Munn Z, editors. The Joanna Briggs Institute
Reviewer’s Manual [Internet]. The Joanna Briggs Institute;
2017. Available from: https://reviewersmanual.joannabriggs.
org/.
45. Sollecito WA, Johnson JK. McLaughlin and Kaluzny’s Con-
tinuous Quality Improvement in Health Care. 4th ed.
Burlington, MA: Jones & Bartlett Learning; 2013.
46. Agency for Healthcare Research and Quality - National
Quality Measures Clearinghouse. NQMC website [Internet].
[cited 2017 Apr 16]. Available from: https://www.quality-
measures.ahrq.gov/.
47. Association of Ambulance Chief Executives. AACE website
[Internet]. [cited 2017 Apr 16]. Available from: https://
aace.org.uk/.
48. Australian Commission for Safety and Quality in Health
Care. ACSQHC website.
49. Australian Government Productivity Commission. PC web-
site [Internet]. [cited 2017 Apr 17]. Available from: https://
www.pc.gov.au/.
50. Care Quality Commission. Care Quality Commission website
[Internet]. 2016 [cited 2017 Apr 16]. Available from: http://
www.cqc.org.uk/.
51. Council of Ambulance Authorities. CAA website [Internet].
[cited 2017 Apr 16]. Available from: http://www.caa.net.au/.
52. International Association of Fire Fighters. IAFF website
[Internet]. [cited 2017 Apr 16]. Available from: http://client.
prod.iaff.org/.
53. National Highway Traffic Safety Administration’s Office of
Emergency Medical Services. NHTSA’s Office of EMS website
[Internet]. [cited 2017 Apr 16]. Available from: https://www.
ems.gov/.
54. National Health Service England. NHS England website
[Internet]. [cited 2017 Apr 16]. Available from: https://
www.england.nhs.uk/.
55. Peters M, Godfrey C, McInerney P, Baldini Soares C, Khalil H,
Parker D. Chapter 11: Scoping Reviews. In: Aromataris, E.,
Munn, Z., editors. Joanna Briggs Institute Reviewer’s Manual
[Internet]. 2017. Available from: https://reviewersmanual.
joannabriggs.org/.
56. Moher D, Liberati A, Tetzlaff J, Altman DG, Altman D, Antes
G, et al. Preferred reporting items for systematic reviews
and meta-analyses: The PRISMA statement. BMJ [Internet]
2009;6(7):339; b2535. Available from: https://doi.org/
10.1136/bmj.b2535.
57. Owen RC. The development and testing of indicators of
prehospital care quality [dissertation]. Manchester: Univer-
sity of Manchester; 2010; 340 p.
58. Haugland H, Rehn M, Klepstad P, Kru¨ ger A. Developing
quality indicators for physician-staffed emergency medical
services: a consensus process. Scand J Trauma Resusc
Emerg Med 2017;25(1):14.
59. El Sayed MJ. Measuring quality in emergency medical
services: a review of clinical performance indicators. Emerg
Med Int (2012): 2012; 1–7.
60. Milner P, Ffphm M. Evaluating emergency services activity at
the health district level. J R Soc Med FORUM Qual Healthc
2001;94(Suppl 39):31– 7.
61. Steering Committee for the Review of Government
Service Provision. Report on Government Services [Inter-
net]. Vol. D. 2016. Available from: http://www.pc.gov.au/
research/ongoing/report-on-government-services/2016/
emergency-management/rogs-2016-volumed-emergency-
management.pdf.
62. Care Quality, Commission., Inspection framework:, NHS.,
Ambulance Services - Core service: Emergency and Urgent,
Care. Newcastle: Care Quality Commission; 2016; 43.
63. Institute of Medicine. Future of Emergency Care: Emergency
Medical Services at the Crossroads [Internet]. Washington
DC: The National Academies Press; 2006. 285 p. Available
from: http://www.nap.edu/catalog/11629
64. Crossing the Quality Chasm: A New Health System for the
21st Century. 2001; 364(9):0–309. Available from: http://
www.nap.edu/catalog/10027.html.
65. Maxwell RJ. Dimensions of quality revisited: from
thought to action. Qual Saf Heal Care [Internet] 1992;1(3):
171– 7; Available from: http://qualitysafety.bmj.com/
lookup/doi/10.1136/qshc.1.3.171.
66. Pirrallo R, Forster R, Swor R. Reconciling the Persuit of
Excellence: Essentials of a Quality EMS Organisation.
In: Improving Quality in EMS. Dubuque: Kendall/Hunt;
2005.
67. Braithwaite J, Hibbert P, Blakely B, Plumb J, Hannaford N,
Long JC, et al. Health system frameworks and performance
indicators in eight countries: A comparative international
analysis. SAGE Open Med 2017;5:110.
68. Roudsari BS, Nathens AB, Arreola-Risa C, Cameron P, Civil I,
Grigoriou G, et al. Emergency Medical Service (EMS) systems
in developed and developing countries. Injury 2007;38(9):
1001– 13.
69. Dick WF. Anglo-American vs. Franco-German emergency
medical services system. Prehosp Disaster Med 2003;18(1):
29– 35; discussion 35 37.
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2208
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
70. Campbell SM, Braspenning J, Hutchinson A, Marshall M.
Research methods used in developing and applying quality
indicators in primary care. Br Med J 2003;326(7393):816 9.
71. Fitch K, Bernstein SJJ, Aguilar MDD, Burnand B, LaCalle
JRR, Lazaro P, et al. The RAND /UCLA Appropriateness
Method User ’ s Manual [Internet]. Transformation. 2001.
109 p. Available from: http://www.rand.org.
72. Campbell S, Kontopantelis E, Hannon K, Burke M, Barber A,
Lester H. Framework and indicator testing protocol for
developing and piloting quality indicators for the UK quality
and outcomes framework. BMC Fam Pract [Internet] BioMed
Central Ltd 2011;12(1):85; Available from: http://www.bio-
medcentral.com/1471-2296/12/85.
73. Ko
¨tter T, Blozik E, Scherer M. Methods for the guideline-
based development of quality indicators – a systematic
review. Implement Sci 2012;7(1):21.
74. Colwell C, Mehler P, Harper J, Cassell L, Vazquez J, Sabel A.
Measuring quality in the prehospital care of chest pain
patients. Prehospital Emerg Care England 2009;13(2):
237– 40.
75. Daudelin DH, Kulick ER, D’Amore K, Lutz JS, Barrientos MT,
Foell K. The Massachusetts emergency medical service
stroke quality improvement collaborative. Prev Chronic
Dis 2013;10(130126):E161.
76. National Highway Traffic Safety Administration; EMS Com-
pass [Internet]. [cited 2016 Apr 16]. Available from: https://
www.ems.gov/projects/ems-compass.html.
77. Frischknecht Christensen E, Berlac PA, Nielsen H, Christian-
sen CF. The Danish quality database for prehospital emer-
gency medical services. Clin Epidemiol 2016;8:667 71.
78. Grudzen CR, Liddicoat R, Hoffman JR, Koenig W, Lorenz KA,
Asch SM. Developing quality indicators for the appropri-
ateness of resuscitation in prehospital atraumatic cardiac
arrest. Prehosp Emerg Care 2007;11(4):43442.
79. Hoogervorst EM, van Beeck EF, Goslings JC, Bezemer PD,
Bierens JJLM. Developing process guidelines for trauma
care in the Netherlands for severely injured patients: results
from a Delphi study. BMC Health Serv Res 2013;13:79.
80. Maio R. Emergency Medical Services Outcomes Evaluation.
Washington DC; 2003.
81. Murphy A, Wakai A, Walsh C, Cummins F, O’Sullivan R.
Development of key performance indicators for prehospital
emergency care. Emerg Med J 2016;33(4):28692.
82. Norris RM. A new performance indicator for acute myocar-
dial infarction. Heart 2001;85(4):395 401.
83. Oostema JA, Nasiri M, Chassee T, Reeves MJ. The quality of
prehospital ischemic stroke care: Compliance with guide-
lines and impact on in-hospital stroke response. J Stroke
Cerebrovasc Dis 2014;23(10):2773–9.
84. Patterson PD, Lave JR, Martin-Gill C, Weaver MD, Wadas RJ,
Arnold RM, et al. Measuring Adverse Events in Helicopter
Emergency Medical Services: Establishing Content Validity.
Prehospital Emerg Care 2014;18(1):35–45.
85. Rosengart MR, Nathens AB, Schiff MA. The identification of
criteria to evaluate prehospital trauma care using the Delphi
technique. J Trauma 2007;62(3):70813.
86. Santana MJ, Stelfox HT. Development and evaluation of
evidence-informed quality indicators for adult injury care.
Ann Surg 2014;259(1):186–92.
87. Siriwardena AN, Shaw D, Donohoe R, Black S, Stephenson J.
Development and pilot of clinical performance indicators
for English ambulance services. Emerg Med J 2010;27(4):
327– 31.
88. Stelfox HT, Bobranska-Artiuch B, Nathens A, Straus SE. A
systematic review of quality indicators for evaluating pedi-
atric trauma care. Crit Care Med 2010;38(4):1187–96.
89. Stelfox HT, Straus SE, Nathens A, Bobranska-Artiuch B.
Evidence for quality indicators to evaluate adult
trauma care: A systematic review. Crit Care Med 2011;
39(4):846–59.
90. Gitelman V, Auerbach K, Doveh E. Development of road
safety performance indicators for trauma management in
Europe. Accid Anal Prev 2013;60(Nov 2013):412– 23.
91. Patterson PD, Probst JC, Moore CG. Expected annual emer-
gency miles per ambulance: An indicator for measuring
availability of emergency medical services resources. J Rural
Heal 2006;22(2):102–11.
92. Stelfox HT, Bobranska-Artiuch B, Nathens A, Straus SE.
Quality indicators for evaluating trauma care: A scoping
review. Arch Surg 2010;145(3):28695.
93. Van der Eng D, van Beeck E, Hoogervorst E, Bierens J.
Education, exposure and experience of prehospital teams
as quality indicators in regional trauma systems. Eur J Emerg
Med 2016;23(4):274–8.
94. Willis CD, Gabbe BJ, Cameron PA. Measuring quality in
trauma care. Injury 2007;38(5):527– 37.
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2209
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
Appendix I: Search strategies
Review question 1: Search A; Review question 2: Search B
Database Search terms
PubMed emergency medical services[mh] OR emergency medical technicians[mh] OR emergency treatment[mh] OR
ambulances[mh] OR air ambulances[mh] OR prehospital[tiab] OR pre-hospital[tiab] OR paramedic
[tiab] OR
ambulance
[tiab] OR out-of-hospital[tiab] OR out of hospital[tiab] OR ems[tiab] OR emt[tiab] OR emergency
service
[tiab] OR emergency medical service
[tiab] OR emergency technician
[tiab] OR emergency practitio-
ner
[tiab] OR emergency dispatch[tiab] OR emergency despatch[tiab] OR emergency resus
[tiab] OR at the
scene[tiab] OR emergency care[tiab] OR medical emergency[tiab] AND Health Care Quality, Access and
Evaluation[mh:noexp] OR
Quality of Health Care[mh:noexp] OR quality[tiab] AND
Search A Search B
(definition
[tiab] OR description
[tiab] OR attribu-
te
[tiab] OR factor
[tiab] OR characteristic
[tiab] OR
feature
[tiab] OT formulation
[tiab] OR frame-
work
[tiab] OR outline
[tiab] OR statement
[tiab])
AND quality[tiab]
Benchmarking[mh] OR Quality Indicators, Health
Care[mh] OR Healthcare Quality Indicator[tiab] OR
Health Care Quality Indicator[tiab] OR quality
assessment
[tiab] OR quality indicator
[tiab] OR
quality measure
[tiab] OR performance indica-
tor
[tiab] OR performance measure
[tiab] OR clinical
indicator
[tiab] OR clinical measure
[tiab] OR effec-
tiveness indicator
[tiab] OR effectiveness measur-
e
[tiab] OR structure indicator
[tiab] OR structure
measure
[tiab] OR process indicator
[tiab] OR pro-
cess measure
[tiab] OR outcome
indicator
[tiab] OR
outcome measure
[tiab]
CINAHL MH emergency careþOR MH emergency medical services OR MH emergency medical technicians OR TI
prehospital OR AB prehospital OR TI ‘‘pre-hospital’’ OR AB ‘‘pre-hospital’’ OR TI paramedic
OR Ab
paramedic
OR TI ambulance
OR AB ambulance
OR TI ‘‘out-of-hospital’’OR AB ‘‘out-of-hospital’’ OR TI
‘‘out of hospital’’ OR AB ‘‘out of hospital’’ OR TI ems OR AB ems OR TI emt OR AB emt OR TI
‘‘emergency service
’’ OR AB ‘‘emergency service
’’ OR TI ‘‘emergency medical service
’’ OR AB ‘‘emergency
medical service
’’ OR TI ‘‘emergency technician
’’ OR AB ‘‘emergency technician
’’ OR TI ‘‘emergency
practitioner
’’ OR AB ‘‘emergency practitioner
’’ OR TI ‘‘emergency dispatch’’ OR AB ‘‘emergency dispatch’’
OR TI ‘‘emergency dispatch’’ OR AB ‘‘emergency dispatch’’ OR TI ‘‘emergency resus
’’OR AB ‘‘emergency
resus
’’OR TI ‘‘at the scene’’ OR AB ‘‘at the scene’’ OR TI ‘‘emergency care’’ OR AB ‘‘emergency care’’ OR
TI ‘‘medical emergency’’ OR AB ‘‘medical emergency’’ AND MH Quality of Care Research
OR TI quality OR AB quality AND
Search A Search B
TI ‘definition
N5 quality’ OR AB ‘definition
N5
quality’ OR TI ‘description
N5 quality’ OR AB
‘description
N5 quality’ OR TI ‘attribute
N5 qual-
ity’ OR AB ‘attribute
N5 quality’ OR TI ‘factor
N5
quality’ OR AB ‘factor
N5 quality’ OR TI
‘characteristic
N5 quality’ OR AB ‘characteristic
N5
quality’ OR TI ‘feature
N5 quality’ OR AB ‘feature
N5 quality’ OR TI ‘formulation
N5 quality’ OR AB
‘formulation
N5 quality’ OR TI ‘framework
N5
quality’ OR AB ‘framework
N5 quality’ OR TI
‘outline
N5 quality’ OR AB ‘outline
N5 quality’ OR
TI ‘statement
N5 quality’ OR AB ‘statement
N5
quality’
MH clinical indicatorsþ
OR TI ‘‘quality indicator
’’ OR AB ‘‘quality
indicator
’’ OR TI ‘‘quality measure
’’ OR AB ‘‘qual-
ity measure
’’ OR TI ‘‘clinical indicator
’’ OR AB
‘‘clinical indicator
’’ OR TI ‘‘clinical measure
’’ OR
AB ‘‘clinical measure
’’ OR TI ‘‘performance
indicator
’’ OR AB ‘‘performance indicator
’’ OR TI
‘‘performance measure
’’ AB ‘‘performance measure
’’
OR TI ‘‘effectiveness indicator
’’ OR AB ‘‘effective-
ness indicator
’’ OR TI ‘‘effectiveness measure
’’ OR
AB ‘‘effectiveness measure
’’ OR TI ‘‘structure
indicator
’’ OR AB ‘‘structure indicator
’’ OR TI
‘‘structure measure
’’ OR AB ‘‘structure measure
’’
OR TI ‘‘process indicator
’’ OR AB ‘‘process
indicator
’’ OR TI ‘‘process measure
’’ OR AB ‘‘pro-
cess measure
’’ OR TI ‘‘outcome indicator
’’ OR AB
‘‘outcome indicator
’’ OR TI ‘‘outcome measure
’’
OR AB ‘‘outcome measure
’’
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2210
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
(Continued)
Database Search terms
Embase ‘emergency health service’/de OR ‘emergency medical dispatch’/de OR ‘emergency treatment’/exp OR
ambulance/exp OR ‘air medical transport’/de OR prehospital:ti,ab OR pre-hospital:ti,ab OR paramedic
:ti,ab
ambulance
:ti,ab OR out-of-hospital:ti,ab OR ‘out of hospital’:ti,ab OR ems:ti,ab OR emt:ti,ab OR
‘emergency service
’:ti,ab OR ‘emergency medical service
’:ti,ab OR ‘emergency technician
’:ti,ab OR
‘emergency practitioner
’:ti,ab OR ‘emergency dispatch’:ti,ab OR ‘emergency despatch’:ti,ab OR ‘emergency
resus
’:ti,ab OR ‘at the scene’:ti,ab OR ‘emergency care’:ti,ab OR ‘medical emergency’:ti,ab AND ‘health care
quality’/de OR quality:ti,ab AND
Search A Search B
(definition
near/5 quality):ti,ab OR (description
near/5 quality):ti,ab OR (attribute
near/5 quality):-
ti,ab OR (factor
near/5 quality):ti,ab OR
(characteristic
near/5 quality):ti,ab OR (feature
near/
5 quality):ti,ab OR (formulation
near/5 quality):ti,ab
OR (framework
near/5 quality):ti,ab OR (outline
near/5 quality):ti,ab OR (statement
near/5 quality):-
ti,ab
benchmarking/de OR
‘clinical effectiveness’/de OR
‘clinical indicators’/de OR
‘performance measurement system’/exp OR
‘healthcare quality indicator
’:ti,ab OR ‘health care
quality indicator
’:ti,ab OR
‘quality assessment
’:ti,ab OR
‘quality indicator
’:ti,ab OR
‘quality measure
’:ti,ab OR
‘performance indicator
’:ti,ab OR ‘performance mea-
sure
’:ti,ab OR
‘clinical indicator
’:ti,ab OR
‘clinical measure
’:ti,ab OR
‘effectiveness indicator
’:ti,ab OR ‘effectiveness mea-
sure
’:ti,ab OR
‘structure indicator
’:ti,ab OR
‘structure measure
’:ti,ab OR
‘process indicator
’:ti,ab OR
‘process measure
’:ti,ab OR
‘outcome
indicator
’:ti,ab OR
‘outcome measure
’:ti,ab
Scopus TITLE-ABS (prehospital) OR TITLE-ABS (pre-hospital) OR TITLE-ABS (paramedic) OR TITLE-ABS
(ambulance) OR TITLE-ABS (‘‘out of hospital’’) OR TITLE-ABS (ems) OR TITLE-ABS (emt) OR TITLE-ABS
(‘‘emergency service’’) OR TITLE-ABS (‘‘emergency medical service’’) OR TITLE-ABS (‘‘emergency techni-
cian’’) OR TITLE-ABS (‘‘emergency practitioner’’) OR TITLE-ABS (‘‘emergency dispatch’’) OR TITLE-ABS
(‘‘emergency despatch’’) OR TITLE-ABS (‘‘emergency resuscitation’’) OR TITLE-ABS (‘‘at the scene’’) OR
TITLE-ABS (‘‘emergency care’’) OR TITLE-ABS (‘‘medical emergency’’) AND TITLE-ABS (quality) AND
Search A Search B
TITLE-ABS (definition W/5 quality) OR TITLE-ABS
(description W/5 quality) OR TITLE-ABS (attribute
W/5 quality) OR TITLE-ABS (factor W/5 quality) OR
TITLE-ABS (characteristic W/5 quality) OR TITLE-
ABS (feature W/5 quality) OR TITLE-ABS (formula-
tion W/5 quality) OR TITLE-ABS (framework W/5
quality) OR TITLE-ABS (outline W/5 quality) OR
TITLE-ABS (statement W/5 quality)
TITLE-ABS (‘‘healthcare quality indicator’’) OR
TITLE-ABS (‘‘health care quality indicator’’) OR
TITLE-ABS (‘‘quality assessment’’) OR TITLE-ABS
(‘‘quality indicator’’) OR TITLE-ABS (‘‘quality mea-
sure’’) OR TITLE-ABS (‘‘performance indicator’’) OR
TITLE-ABS (‘‘performance measure’’) OR TITLE-ABS
(‘‘clinical indicator’’) OR TITLE-ABS (‘‘clinical mea-
sure’’) OR TITLE-ABS (‘‘effectiveness indicator’’) OR
TITLE-ABS (‘‘effectiveness measure’’) OR TITLE-ABS
(‘‘structure indicator’’) OR TITLE-ABS (‘‘structure
measure’’) OR TITLE-ABS (‘‘process indicator’’) OR
TITLE-ABS (‘‘process measure’’) OR TITLE-ABS
(‘‘outcome indicator’’) OR TITLE-ABS (‘‘outcome
measure’’)
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2211
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
(Continued)
Database Search terms
Cochrane Library MeSH descriptor: [Emergency Medical Services] this term only OR MeSH descriptor: [Emergency Medical
Service Communication Systems] this term only OR MeSH descriptor: [Emergency Treatment] explode all
trees
OR prehospital OR pre-hospital OR paramedic OR ambulance OR out-of-hospital OR ‘‘out of hospital’’ OR
ems OR emt OR ‘‘emergency service’’ OR ‘‘emergency medical service’’ OR ‘‘emergency technician’’ OR
‘‘emergency practitioner’’ OR ‘‘emergency dispatch’’ OR ‘‘emergency despatch’’ OR ‘‘emergency resuscita-
tion’’ OR ‘‘at the scene’’ OR ‘‘emergency care’’ OR ‘‘medical emergency’’ AND MeSH descriptor: [Quality of
Health Care] explode all trees OR quality AND
Search A Search B
definition near/5 quality OR description near/5 qual-
ity OR attribute near/5 quality OR factor near/5
quality OR characteristic near/5 quality OR feature
near/5 quality OR formulation near/5 quality OR
framework near/5 quality OR outline near/5 quality
OR statement near/5 quality
MeSH descriptor: [Quality Indicators, Health Care]
explode all trees
‘‘healthcare quality indicator’’ OR ‘‘health care qual-
ity indicator’’ OR
‘‘quality assessment’’ OR
‘‘quality indicator’’ OR
‘‘quality measure’’ OR
‘‘performance indicator’’ OR ‘‘performance measure’’
OR
‘‘clinical indicator’’ OR
‘‘clinical measure’’ OR ‘‘effectiveness indicator’’ OR
‘‘effectiveness measure’’ OR ‘‘structure indicator’’ OR
‘‘structure measure’’ OR ‘‘process indicator’’ OR
‘‘process measure’’ OR ‘‘outcome
indicator’’ OR
‘‘outcome measure’’
Web of Science TI ¼prehospital OR TI ¼pre-hospital OR TI ¼paramedic
OR
TI ¼ambulance
OR TI ¼out-of-hospital OR TI ¼’’out of hospital’’ OR TI¼ems OR TI ¼emt OR
TI ¼’’emergency service
’’ OR TI ¼’’emergency medical service
’’ OR TI ¼’’emergency technician
’’OR
TI ¼’’emergency practitioner
’’ OR TI ¼’’emergency dispatch’’ OR TI ¼’’emergency dispatch’’ OR
TI ¼’’emergency resus
’’ OR TI ¼’’at the scene’’ OR TI ¼’’emergency care’’ OR TI ¼’’medical emergency’’
AND TI ¼quality AND
Search A Search B
TI ¼(definition near/5 quality) OR TI ¼(description
near/5 quality) OR TI ¼(attribute near/5 quality) OR
TI ¼(factor near/5 quality) OR TI ¼(characteristic
near/5 quality) OR TI ¼(feature near/5 quality) OR
TI ¼(formulation near/5 quality) OR TI ¼(framework
near/5 quality) OR TI ¼(outline near/5 quality) OR
TI ¼(statement near/5 quality)
TI ¼’’Healthcare Quality Indicator
’’ OR
TI ¼’’Health Care Quality Indicator
’’ OR
TI ¼’’quality assessment
’’OR
TI ¼’’quality indicator
’’ OR
TI ¼’’quality measure
’’ OR
TI ¼’’performance indicator
’’ OR TI ¼’’performance
measure
’’ OR
TI ¼’’clinical indicator
’’ OR
TI ¼’’clinical measure
’’ OR
TI ¼’’effectiveness indicator
’’ OR TI ¼’’effectiveness
measure
’’ OR
TI ¼’’structure indicator
’’ OR
TI ¼’’structure measure
’’ OR
TI ¼’’process indicator
’’ OR
TI ¼’’process measure
’’ OR
TI ¼’’outcome
indicator
’’ OR
TI ¼’’outcome measure
’’
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2212
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
(Continued)
Database Search terms
ProQuest Dissertations
and Theses
SU(‘‘emergency medical services’’) OR SU(‘‘emergency medical technician’’) OR SU(‘‘emergency treatment’’)
OR SU(ambulance) OR SU(‘‘air ambulance’’) OR TI,AB(prehospital) OR TI,AB(pre-hospital) OR
TI,AB(paramedic
) OR TI,AB(ambulance
) OR TI,AB(out-of-hospital)OR TI,AB(‘‘out of hospital’’) OR
TI,AB(ems) OR TI,AB(emt) OR TI,AB(‘‘emergency service
’’) OR TI,AB(‘‘emergency medical service
’’) OR
TI,AB(‘‘emergency technician
’’) OR TI,AB(‘‘emergency practitioner
’’) OR TI,AB(‘‘emergency dispatch’’) OR
TI,AB(‘‘emergency despatch’’) OR TI,AB(‘‘emergency resus
’’) OR TI,AB(‘‘at the scene’’) OR TI,AB(‘‘emer-
gency care’’) OR TI,AB(‘‘medical emergency’’) AND SU(‘‘Health Care Quality, Access and Evaluation’’) OR
SU(‘‘Quality of Health Care’’) OR TI,AB(quality) AND
Search A Search B
TI,AB(definition near/5 quality) OR TI,AB(description
near/5 quality) OR TI,AB (attribute near/5 quality)
OR TI,AB (factor near/5 quality) OR TI,AB (charac-
teristic near/5 quality) OR TI,AB (feature near/5
quality) OR TI,AB (formulation near/5 quality) OR
TI,AB (framework near/5 quality) OR TI,AB (outline
near/5 quality) OR TI,AB (statement near/5 quality)
SU(Benchmarking) OR SU(‘‘Quality Indicators,
Health Care’’) OR TI,AB(‘‘Healthcare Quality
Indicator
’’) OR TI,AB(‘‘Health Care Quality
Indicator
’’) OR
TI,AB(‘‘quality assessment
’’) OR
TI,AB(‘‘quality indicator
’’) OR
TI,AB(quality measure
’’) OR
TI,AB(‘‘performance indicator
’’) OR TI,AB (‘‘perfor-
mance measure
’’) OR
TI,AB(‘‘clinical indicator
’’) OR
TI,AB(‘‘clinical measure
’’) OR
TI,AB(‘‘effectiveness indicator
’’) OR TI,AB (effective-
ness measure
’’) OR
TI,AB (structure indicator
’’) OR
TI,AB (structure measure
’’) OR
TI,AB (process indicator
’’) OR
TI,AB (process measure
)OR
TI,AB (outcome
indicator
)OR
TI,AB (outcome measure
)
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2213
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
(Continued)
Database Search terms
OpenThesis TI(prehospital) OR ABST(prehospital) OR TI(pre-hospital) OR ABST(pre-hospital) OR TI(paramedic) OR
TI(ambulance) OR ABST(ambulance)
TI(out-of-hospital) OR ABST(out-of-hospital) OR TI(‘‘out of hospital’’) OR ABST(‘‘out of hospital’’) OR
TI(ems) OR ABST(ems) OR TI(emt) OR ABST(emt) OR TI(‘‘emergency service’’) OR ABST(‘‘emergency
service’’) OR TI(‘‘emergency medical service’’) OR ABST(‘‘emergency medical service’’) OR TI(‘‘emergency
technician’’) OR ABST(‘‘emergency technician’’) OR TI(‘‘emergency practitioner’’) OR ABST(‘‘emergency
practitioner’’) OR TI(‘‘emergency dispatch’’) OR ABST(‘‘emergency dispatch’’) OR TI(‘‘emergency despatch’’)
OR ABST(‘‘emergency despatch’’) OR TI(‘‘emergency resuscitation’’) OR ABST(‘‘emergency resuscitation’’)
OR TI(‘‘at the scene’’) OR ABST(‘‘at the scene’’) OR TI(‘‘emergency care’’) OR ABST(‘‘emergency care’’) OR
TI(‘‘medical emergency’’) OR ABST(‘‘medical emergency’’) AND TI(‘‘Health Care Quality, Access and
Evaluation’’) OR ABST(‘‘Health Care Quality, Access and Evaluation’’) OR TI(‘‘Quality of Health Care’’) OR
ABST (‘‘Quality of Health Care’’) OR TI(quality) OR ABST(quality) AND (TI(definition
AND quality) OR
ABST(definition
AND quality)) OR (TI(description
AND quality) OR ABST(description
AND quality)) OR
(TI(attribute
AND quality) OR ABST(attribute
AND quality)) OR (TI(factor
AND quality) OR
ABST(factor
AND quality)) OR (TI(characteristic
AND quality) OR ABST(characteristic
AND quality)) OR
(TI(feature
AND quality) OR ABST(feature
AND quality)) OR (TI(formulation
AND
Search A Search B
quality)) OR (TI(feature
AND quality) OR
ABST(feature
AND quality)) OR (TI(formulation
AND quality) OR ABST(formulation
AND quality))
OR (TI(framework
AND quality) OR
ABST(framework
AND quality)) OR (TI(outline
AND quality) OR ABST(outline
AND quality)) OR
(TI(statement
AND quality) OR ABST(statement
AND quality))
TI(‘‘Healthcare Quality Indicator’’) OR
ABST(‘‘Healthcare Quality Indicator’’) OR
TI(‘‘Health Care Quality Indicator’’) OR
ABST(‘‘Health Care Quality Indicator’’) OR
TI(‘‘quality assessment’’) OR ABST(‘‘quality assess-
ment’’) OR
TI(‘‘quality indicator’’) OR ABST(‘‘quality indicator’’)
OR TI(‘‘quality measure’’) OR ABST(‘‘quality mea-
sure’’) OR
TI(‘‘performance indicator’’) OR ABST(‘‘performance
indicator’’) OR TI(‘‘performance measure’’) OR
ABST(‘‘performance measure’’) OR
TI(‘‘clinical indicator’’) OR ABST(‘‘clinical indicator’’)
OR TI(‘‘clinical measure’’) OR ABST(‘‘clinical mea-
sure’’) OR
TI(‘‘effectiveness indicator’’) OR ABST(‘‘effectiveness
indicator’’) OR TI(‘‘effectiveness measure’’) OR
ABST(‘‘effectiveness measure’’) OR
TI(‘‘structure indicator’’) OR ABST(‘‘structure indica-
tor’’) OR TI(‘‘structure measure’’) OR ABST(‘‘struc-
ture measure’’) OR
TI(‘‘process indicator’’) OR ABST(‘‘process indica-
tor’’) OR TI(‘‘process measure’’) OR ABST(‘‘process
measure’’) OR
TI(‘‘outcome indicator’’) OR ABST(‘‘outcome indica-
tor’’) OR TI(‘‘outcome measure’’) OR ABST(‘‘out-
come measure’’)
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2214
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
(Continued)
Database Search terms
Networked Digital
Library of Theses and
Dissertations
title:’’prehospital’’ OR title:’’pre-hospital’’ OR title:’’paramedic’’ OR title:’’ambulance’’ OR title:’’out-of-
hospital’’ OR title:’’out of hospital’’ OR title:’’ems’’ OR title:’’emt’’ OR title:’’emergency service’’ OR
title:’’emergency medical service’’ OR title:’’emergency technician’’ OR title:’’emergency practitioner’’ OR
title:’’emergency dispatch’’ OR title:’’emergency dispatch’’ OR title:’’emergency resus’’ OR title:’’at the scene’’
OR title:’’emergency care’’ OR title:’’medical emergency’’ AND title:’’quality’’ AND
Search A Search B
(title:’’definition’’ AND title:‘‘quality’’) OR (title:’’de-
scription’’ AND title:‘‘quality’’) OR (title:’’attribute’’
AND title:‘‘quality’’) OR (title:’’factor’’ AND title:‘‘-
quality’’) OR (title:’’characteristic’’ AND title:‘‘qual-
ity’’) OR (title:’’feature’’ AND title:‘‘quality’’) OR
(title:’’formulation’’ AND title:‘‘quality’’) OR (title:’’-
framework’’ AND title:‘‘quality’’) OR (title:’’outline’’
AND title:‘‘quality’’) OR (title:’’statement’’ AND
title:‘‘quality’’)
title:’’Healthcare Quality Indicator’’ OR title:’’Health
Care Quality Indicator’’ OR
title:’’quality assessment’’ OR
title:’’quality indicator’’ OR
title:’’quality measure’’ OR
title:’’performance indicator’’ OR title:’’performance
measure’’ OR
title:’’clinical indicator’’ OR
title:’’clinical measure’’ OR
title:’’effectiveness indicator’’ OR title:’’effectiveness
measure’’ OR
title:’’structure indicator’’ OR
title:’’structure measure’’ OR
title:’’process indicator’’ OR
title:’’process measure’’ OR
title:’’outcome
indicator’’ OR
title:’’outcome measure’’
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2215
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
Appendix II: Charting tables
Review question 1
Article/Source characteristics
Author(s)/Organization
Year of publication
Country of origin
Type of research/project
Results extracted from article/source (definition/description of prehospital care quality)
Intended EMS system &Paramedic &Physician
Definition of prehospital care quality (if provided)
Attribute(s) prehospital care quality
Attribute(s) category &Explicitly stated
&Extrapolated from generic healthcare
quality definitions
EMS, Emergency medical services
Review question 2
Article/Source characteristics
Author(s)/Organization
Year of publication
Country of origin
Type of research/project
Results extracted from article/source (quality indicators)
Literature origin &Research literature
&Professional association/accrediting
body
&Government
Indicator development method
Intended EMS system &Paramedic &Physician
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2216
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
Indicator 1 Framework components/matrix:
&Clinical
&Airway management and oxy-
genation
&Asthma
&Acute coronary syndrome (ACS)
&Out-of-hospital cardiac arrest
(OHCA)
&Pain management
&Seizures
&Stroke
&Trauma
&Hypoglycemia
&General
&Other disease-specific
&System and organizational
&Communication/dispatch
&Documentation
&Education
&Financial
&Hospital notification
&Research
&Paramedic health and safety
&Patient safety
&Patient satisfaction
&Personnel performance evalua-
tion/audit
&Time intervals
&Other
Prehospital care quality attributes:
&Acceptability
&Accessibility
&Appropriateness
&Availability
&Caring
&Capability
&Clinical Effectiveness
&Continuity/Sustainability
&Cost-Effectiveness
&Efficiency
&Equitability
&Interpersonal Effectiveness
&Patient-Centeredness
&Responsiveness
&Safety
&Timeliness
&Well-led
Type reported: &Structure &Process
&Outcome
Type assigned: &Structure &Process
&Outcome
Indicator 2, 3, ... (as for indicator 1 above)
EMS, Emergency medical services
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2217
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
Appendix III: Clinical framework components by article
Article
No. of
QIs
Airway
mgmt. &
oxygenation
Acute
coronary
syndrome Asthma
Out-of-
hosp.
cardiac
arrest Hypoglycemia
Pain
management Seizure Stroke Trauma General
Other
disease-
specific
Australian Com-
mission for
Safety and Qual-
ity in Health
Care (2017)
48
6x x
Agency for
Healthcare
Research and
Quality –
National Quality
Measures Clear-
inghouse
(2017)
46
12 x x x x
Colwell et al.
(2009)
74
7x
Australian Gov-
ernment Produc-
tivity Commis-
sion (2015)
61
6xx
Daudelin et al.
(2013)
75
5 x
Emergency Med-
ical Services
Compass
(2016)
76
13
Frischknecht
Christensen et
al. (2016)
77
9x x
Grudzen et al.
(2007)
78
28 x
Haugland et al.
(2017)
58
26 x x
Hoogervorst et
al. (2013)
79
13 xx
International
Association of
Fire Fighters
(2001)
52
15 x
Maio et al.
(2003)
80
18 x x x x x
Murphy et al.
(2016)
81
101 x x x x x x x x x x
Myers et al.
(2008)
16
12 x x x x x
National Health
Service England
(2017)
54
16 x x x
National High-
way Traffic
Safety Adminis-
tration (2009)
53
22 x x x x x x
Norris (2001)
82
2 x
Oostema et al.
(2014)
83
8 x
Owen (2010)
57
72 x x x x x x x x x
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2218
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
(Continued)
Article
No. of
QIs
Airway
mgmt. &
oxygenation
Acute
coronary
syndrome Asthma
Out-of-
hosp.
cardiac
arrest Hypoglycemia
Pain
management Seizure Stroke Trauma General
Other
disease-
specific
Patterson et al.
(2014)
84
13 x x
Rosengart et al.
(2007)
85
25 x x x x
Santana et al.
(2014)
86
8xxx
Siriwardena et
al. (2010)
87
20 x x x x x x x x x
Stelfox et al.
(2010)
88
29 x x x x
Stelfox et al.
(2011)
89
21 x x x x
QI, Quality indicator
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2219
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
Appendix IV: System/Organizational framework components by article
Article
No. of
QIs
Communi-
cation/
Dispatch
Documen-
tation
Educa-
tion
Fina-
ncial
Hosp.
notifi-
cation
Paramedic
health &
safety
Patient
safety
Patient
satis-
faction
Personnel
perf.
eval./
audit Research
Resources/
Deploy-
ment
Time
intervals Other
Australian
Commission for
Safety and
Quality in
Health Care
(2017)
48
6 x
Australian Gov-
ernment Pro-
ductivity Com-
mission
(2015)
61
6xxxx
Daudelin et al.
(2013)
75
5x
Emergency
Medical Ser-
vices Compass
(2016)
76
13 x x
Frischknecht
Christensen et
al. (2016)
77
9x x x
Gitelman et al.
(2013)
90
11 xx
Haugland et al.
(2017)
58
26 x x x x x x x x x
Hoogervorst et
al. (2013)
79
13 xx
International
Association of
Fire Fighters
(2001)
52
15 x x x x x
Murphy et al.
(2016)
81
101 x x x x x x x x x x x
Myers et al.
(2008)
16
12 x
National
Health Service
England
(2017)
54
16 xxx
National High-
way Traffic
Safety Adminis-
tration (2009)
53
22 x x x x x
Oostema et al.
(2014)
83
8x x
Owen (2010)
57
72 x x x x x x x x
Patterson et al.
(2006)
91
1 x
Patterson et al.
(2014)
84
13 x x x x
Rosengart et al.
(2007)
85
25 x x x x
Santana et al.
(2014)
86
8x x
Siriwardena et
al. (2010)
87
20 xx
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2220
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
(Continued)
Article
No. of
QIs
Communi-
cation/
Dispatch
Documen-
tation
Educa-
tion
Fina-
ncial
Hosp.
notifi-
cation
Paramedic
health &
safety
Patient
safety
Patient
satis-
faction
Personnel
perf.
eval./
audit Research
Resources/
Deploy-
ment
Time
intervals Other
Stelfox et al.
(2010)
92
2xx
Stelfox et al.
(2010)
88
29 x x x x x x x x
Stelfox et al.
(2011)
89
21 xxx
Van der Eng et
al. (2016)
93
3 x
Willis et al.
(2007)
94
2x x
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2221
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
Appendix V: Prehospital care quality attributes by article
Article
No. of
QIs
Accep-
tability
Acces-
sibility
Appro-
priateness
Availa-
bility Caring
Capa-
bility
Clinical
effectiv-
eness
Conti-
nuity/
sustain-
ability
Cost-
Effective-
ness
Effi-
ciency
Equita-
bility
Inter-
personal
effective-
ness
Patient-
Centered-
ness
Respon-
siveness Safety
Timeli-
ness Well-led
Australian
Commission for
Safety and
Quality in
Health Care
(2017)
48
6xx x x
Agency for
Healthcare
Research and
Quality –
National Qual-
ity Measures
Clearinghouse
(2017)
46
12 x x x xx
Colwell et al.
(2009)
74
7x x
Australian Gov-
ernment Pro-
ductivity Com-
mission
(2015)
61
6xx x xx x x x x x x x x
Daudelin et al.
(2013)
75
5xx x
Emergency
Medical Ser-
vices Compass
(2016)
76
13 x x x x x x x x
Frischknecht
Christensen et
al. (2016)
77
9xxx x xx xx
Gitelman et al.
(2013)
90
11 x x x x x
Grudzen et al.
(2007)
78
28 x
Haugland et al.
(2017)
58
26 x x x x x x x x x x x x x x x
Hoogervorst et
al. (2013)
79
13 x x x x x x
International
Association of
Fire Fighters
(2001)
52
15 x x x x x x x x x x x x x x x
Maio et al.
(2003)
80
18 x x x x x
Murphy et al.
(2016)
81
101 x x x x x x x x x x x x x x x x x
Myers et al.
(2008)
16
12 x x x x
National Health
Service England
(2017)
54
16 x x x x x x x x x
National High-
way Traffic
Safety Adminis-
tration (2009)
53
22 x x x x x x x x x x x x x x x
Norris (2001)
82
2x
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2222
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
(Continued)
Article
No. of
QIs
Accep-
tability
Acces-
sibility
Appro-
priateness
Availa-
bility Caring
Capa-
bility
Clinical
effectiv-
eness
Conti-
nuity/
sustain-
ability
Cost-
Effective-
ness
Effi-
ciency
Equita-
bility
Inter-
personal
effective-
ness
Patient-
Centered-
ness
Respon-
siveness Safety
Timeli-
ness Well-led
Oostema et al.
(2014)
83
8xx x
Owen (2010)
57
72 x x x x x x x x x x x x x x
Patterson et al.
(2006)
91
1 x
Patterson et al.
(2014)
84
13 x x x xx
Rosengart et al.
(2007)
85
25 x x x x x x x x x
Santana et al.
(2014)
86
8xxxxx xxx
Siriwardena et
al. (2010)
87
20 x x x x x x x x
Stelfox et al.
(2010)
92
2x x x
Stelfox et al.
(2010)
88
29 x x x xx x
Stelfox et al.
(2011)
89
21 x x x x x x xx
Van der Eng et
al. (2016)
93
3x
Willis et al.
(2007)
94
2x xx
SYSTEMATIC REVIEW R. Pap et al.
JBI Database of Systematic Reviews and Implementation Reports ß2018 THE JOANNA BRIGGS INSTITUTE 2223
©2018 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
... Correspondingly, ambulance service performance measurement has historically focused on operational aspects and time intervals such as response time. 1 Modern prehospital care that ambulance services provide often involves complex out-of-hospital and mobile healthcare to patients across the lifespan presenting with injury or illness across a spectrum of acuity. For the purpose of this project, the context of prehospital care is limited to that of healthcare provided by ambulance services. ...
... Since meaningful performance measurement not only produces data to ensure the maintenance of quality but also provides information on whether or not change is effective in achieving improvement, there has been growing interest in more sophisticated clinical and non-clinical QIs to parallel ambulance services developments and the progressive prehospital care they provide. 1 As such, the requirement for valid quality measurement to gauge improvements and monitor performance is echoed in the Australian prehospital care setting. ...
... A scoping review was conducted to map the attributes of 'quality' in the context of prehospital care and to establish a list of internationally existing prehospital care QIs. 1 The review employed the JBI methodology for conducting scoping reviews. 14 The review's systematic search confirmed paucity in literature that defines prehospital care quality or that examines what dimensions of generic healthcare quality definitions are important in prehospital care. ...
Article
Background Globally, the measurement of quality is an important process that supports the provision of high-quality and safe healthcare services. The requirement for valid quality measurement to gauge improvements and monitor performance is echoed in the Australian prehospital care setting. The aim of this study was to use an evidence-informed expert consensus process to identify valid quality indicators (QIs) for Australian prehospital care provided by ambulance services. Methods A modified RAND/UCLA appropriateness method was conducted with a panel of Australian prehospital care experts from February to May 2019. The proposed QIs stemmed from a scoping review and were systematically prepared within a clinical and non-clinical classification system, and a structure/process/outcome and access/safety/effectiveness taxonomy. Rapid reviews were performed for each QI to produce evidence summaries for consideration by the panellists. QIs were deemed valid if the median score by the panel was 7–9 without disagreement. Results Of 117 QIs, the expert panel rated 84 (72%) as valid. This included 26 organisational/system QIs across 7 subdomains and 58 clinical QIs within 10 subdomains. Most QIs were process indicators (n=62; 74%) while QIs describing structural elements and desired outcomes were less common (n=13; 15% and n=9; 11%, respectively). Non-exclusively, 18 (21%) QIs addressed access to healthcare, 21 (25%) described safety aspects and 64 (76%) specified elements contributing to effective services and care. QIs on general time intervals, such as response time, were not considered valid by the panel. Conclusion This study demonstrates that with consideration of best available evidence a substantial proportion of QIs scoped and synthesised from the international literature are valid for use in the Australian prehospital care context.
... Phase 1: scoping review This phase has been completed and involved preparatory work in the form of a scoping review. 17 The purpose of the review was to map the attributes of 'quality' in the context of prehospital care and to chart existing international prehospital care QIs. The review employed the Joanna Briggs Institute (JBI) methodology for conducting scoping reviews. ...
... QIs are often selected arbitrarily 46 47 ; however, there appears to be growing interest in finding better ways to measure the quality of prehospital care provided by ambulance services. 17 Measurement using intelligent and meaningful QIs over time is key to understanding variation and ultimately where and how to conduct improvement efforts. 48 The QIs which will be developed in this project provide a mechanism to appraise Australian ambulance services' performance and a framework to direct, monitor and demonstrate quality improvement efforts. ...
... The scoping review has been published. 17 Further findings of the project will be communicated using a comprehensive dissemination strategy. This strategy includes several different forms of dissemination to reach out to individuals and stakeholder groups at the national and international level. ...
Article
Full-text available
Introduction Historically, ambulance services were established to provide rapid transport of patients to hospital. Contemporary prehospital care involves provision of sophisticated ‘mobile healthcare’ to patients across the lifespan presenting with a range of injuries or illnesses of varying acuity. Because of its young age, the paramedicine profession has until recently experienced a lack of research capacity which has led to paucity of a discipline-specific, scientific evidence-base. Therefore, the performance and quality of ambulance services has traditionally been measured using simple, evidence-poor indicators forming a deficient reflection of the true quality of care and providing little direction for quality improvement efforts. This paper reports the study protocol for the development and testing of quality indicators (QIs) for the Australian prehospital care setting. Methods and analysis This project has three phases. In the first phase, preliminary work in the form of a scoping review was conducted which provided an initial list of QIs. In the subsequent phase, these QIs will be developed by aggregating them and by performing related rapid reviews. The summarised evidence will be used to support an expert consensus process aimed at optimising the clarity and evaluating the validity of proposed QIs. Finally, in the third phase those QIs deemed valid will be tested for acceptability, feasibility and reliability using mixed research methods. Evidence-based indicators can facilitate meaningful measurement of the quality of care provided. This forms the first step to identify unwarranted variation and direction for improvement work. This project will develop and test quality indicators for the Australian prehospital care setting. Ethics and dissemination This project has been approved by the University of Adelaide Human Research Ethics Committee. Findings will be disseminated by publications in peer-reviewed journals, presentations at appropriate scientific conferences, as well as posts on social media and on the project’s website.
... After transferring patient to the hospital and hospitalization, all the information about administration data or medical procedures performed for the patient in the hospital are registered in the mission EMR. To perform the whole EMS processes through a comprehensive information system, the existence of special capabilities and key indicators is crucial [5]. Therefore, the Iranian EMS organization and the Statistics and Information Technology Management Center of the Ministry of Health and Medical Education (ITMC-MOHME) determined and developed the main indicators of a capable system to meet EMS requirements. ...
... The indicators are used for the government to guarantee the reliability, responsibility and accessibility of this critical information system. The system is a live operational information system at the level of Transaction Processing Systems (TPS), and uninterrupted activity is inevitable [5]. The first step constituted determining clear objectives for the system. ...
Article
Full-text available
Evaluation of Emergency Medical Services Management Information System (EMSIS) makes it possible to assess the extent to which the objectives of supporting of healthcare delivery services. This paper presents an overview of the regulatory process in prehospital EMS electronic data registration and provides a minimum data set for the purpose of developing such a care system on a national scale. It further offers an evaluation framework for such systems.
... The placement of a PCCD on a patient with a mechanism of injury suggestive of pelvic ring disruption is now commonly regarded to be an indicator of highquality prehospital trauma care [13][14][15]. As such, many ambulance services utilize this quality indicator (QI) in the measurement of their clinical performance [16]. ...
... As the initial part of the larger research project, a scoping review was conducted in accordance with Joanna Briggs Institute (JBI) methodology [16]. The scoping review's purpose was to map the attributes of 'quality' in the context of prehospital care, to chart existing international prehospital care QIs and explore their development processes. ...
Article
Full-text available
Background: Pelvic fractures, especially when unstable, may cause significant haemorrhage. The early application of a pelvic circumferential compression device (PCCD) in patients with suspected pelvic fracture has established itself as best practice. Ambulance services conduct corresponding performance measurement. Quality indicators (QIs) are ideally based on high-quality evidence clearly demonstrating that the desirable effects outweigh the undesirable effects. In the absence of high-quality evidence, best available evidence should be combined with expert consensus. Objectives: The aim of the present study was to identify, appraise and summarize the best available evidence regarding PCCDs for the purpose of informing an expert panel tasked to evaluate the validity of the following QI: A patient with suspected pelvic fracture has a PCCD applied. Methods: A rapid review of four databases was conducted to identify relevant literature published up until 9 June 2020. Systematic reviews, experimental, quasi-experimental and observational analytic studies written in English were included. One author was responsible for study selection and quality appraisal. Data extraction using a priori extraction templates was verified by a second reviewer. Study details and key findings were summarized in tables. Results: A total of 13 studies were assessed to be eligible for inclusion in this rapid review. Of these, three were systematic reviews, one was a randomized clinical trial (crossover design), two were before-after studies, and seven were retrospective cohort studies. The systematic reviews included mostly observational studies and could therefore not be considered as high-level evidence. Overall, the identified evidence is of low quality and suggests that PCCD may provide temporary pelvic ring stabilization and haemorrhage control, although a potential for adverse effects exists. Conclusion: Given the low quality of the best available evidence, this evidence would need to be combined with expert consensus to evaluate the validity of a related quality indicator before its implementation.
... With limited systems of prehospital quality assessment in LMICs [9], quality indicators (QIs) have been based on simple, non-clinical endpoints like response time intervals [10]. The validity of metrics based on non-clinical endpoints as holistic indicators of prehospital care has been challenged as new ways to measure quality have emerged [9,11,12], with capacity assessments guiding priority-setting and resource allocation in LMICs [13]. However, many of these indicators are dependent on high-level policymaker involvement and surveillance infrastructure, difficult in resource-limited settings [8]. ...
Article
Background: WHO recommends training lay first responders (LFRs) as the first step toward formal emergency medical services development, yet no tool exists to evaluate LFR programs. Methods: We developed Prehospital Emergency Trauma Care Assessment Tool (PETCAT), a seven-question survey administered to first-line hospital-based healthcare providers, to independently assess LFR prehospital intervention frequency and quality. PETCAT surveys were administered one month pre-LFR program launch (June 2019) in Makeni, Sierra Leone and again 14 months post-launch (August 2020). Using a difference-in-differences approach, PETCAT was also administered in a control city (Kenema) with no LFR training intervention during the study period at the same intervals to control for secular trends. PETCAT measured change in both the experimental and control locations. Cronbach's alpha, point bi-serial correlation, and inter-rater reliability using Cohen's Kappa assessed PETCAT reliability. Results: PETCAT administration to 90 first-line, hospital-based healthcare providers found baseline prehospital intervention were rare in Makeni and Kenema prior to LFR program launch (1.2/10 vs. 1.8/10). Fourteen months post-LFR program implementation, PETCAT demonstrated prehospital interventions increased in Makeni with LFRs (5.2/10, p < 0.0001) and not in Kenema (1.2/10) by an adjusted difference of + 4.6 points/10 (p < 0.0001) ("never/rarely" to "half the time"), indicating negligible change due to secular trends. PETCAT demonstrated high reliability (Cronbach's α = 0.93, Cohen's K = 0.62). Conclusions: PETCAT measures changes in rates of prehospital care delivery by LFRs in a resource-limited African setting and may serve as a robust tool for independent EMS quality assessment.
... Through a scoping review study, Howard et al. (2018) investigated the characteristics and development methods of the QIs in the field of PEC, who identified 331 QIs by the article review and 15 by the website review [29]. Similarly, there are several published studies [25,[30][31][32][33] that develop indicators for measuring the performance of PEC. ...
Article
Full-text available
s Background Pre-Hospital Emergency Care (PEC) is a fundamental property of prevention of Road Traffic Injuries (RTIs). Thus, this sector requires a system for evaluation and performance improvement. This study aimed to develop quality indicators to measure PEC for RTIs. Methods Following the related literature review, 14 experts were interviewed through semi-structured interviews to identify Quality Measurement Indicators (QMIs). The extracted indicators were then categorized into three domains: structure, performance, and management. Finally, the identified QMIs were confirmed through two rounds of the Delphi technique. Results Using literature review 11 structural, 13 performance, and four managerial indicators (A total of 28 indicators) were identified. Also, four structural, four performance, and three managerial indicators (A total of 11indicators) were extracted from interviews with experts. Two indicators were excluded after two rounds of Delphi’s technics. Finally, 14 structural, 16 performance and, seven managerial indicators (A total of 37indicators) were finalized. Conclusion Due to the importance and high proportion of RTIs compared to other types of injuries, this study set out to design and evaluate the QMIs of PEC delivered for RTIs. The findings of this research contribute to measuring and planning aimed at improving the performance of PEC.
Article
Objective To overcome the lack of larger, population-based studies reporting the prevalence of insertion of PIVCs and IO devices, and to describe the patient-related and service-related characteristics of these devices, inserted by paramedics, in an Australian state ambulance service. Methods A retrospective analysis of the electronic Ambulance Report Form (medical record) and Computer Aided Dispatch system from the 1st July 2016 until 30th June 2017. Results 709,217 events were analysed. Of these, 20.4% involved at least one successful PIVC insertion and 0.07% involved at least one successful IO device insertion; most of the time on first attempt (89% and 86.4% respectively). Most PIVCs were inserted into the right antecubital fossa or dorsum of the right hand while IO devices were inserted into the proximal tibia. Of male patients, 21.4% received PIVCs while 19.5% of female patients received PIVCs. Very low numbers of both male and female patients received IOs (0.1%). Medical, non-traumatic presentations were the most common presentation and received the most insertions of both devices, followed by trauma presentations. Advanced Care Paramedics inserted 84.0% of PIVCs while Critical Care Paramedics inserted 94.4% of IO devices. Time treating and transporting patients generally increased with number of attempts at vascular access undertaken. Conclusions Queensland paramedic practices relating to insertion of PIVCs, and IO devices appears consistent with documented practice internationally. Further study is required to determine whether the antecubital fossa and dorsum of the hand insertions are clinically necessary in this population as areas of flexion and distal extremities are generally to be avoided for PIVC insertion.
Article
Full-text available
Background and Aims Improving the quality of pre-hospital traumatic shock care, especially in low- and middle-income countries, is particularly relevant to reducing the large global burden of disease from injury. What clinical interventions represent high-quality care is an actively evolving field and often dependent on the specific injury pattern. A key component of improving the quality of care is having a consistent way to assess and measure the quality of shock care in the pre-hospital setting. The objective of this study was to develop and validate a chart abstraction instrument to measure the quality of trauma care in a resource-limited, pre-hospital emergency care setting. Methods Traumatic shock was selected as the tracer condition. The pre-hospital quality of traumatic shock care (QTSC) instrument was developed and validated in three phases. A content development phase utilized a rapid literature review and expert consensus to yield the contents of the draft instrument. In the instrument validation phase, the QTSC instrument was created and underwent end user and content validation. A pilot-testing phase collected user feedback and performance characteristics to iteratively refine draft versions into a final instrument. Accuracy and inter- and intra-rater agreement were calculated. Results The final QTSC instrument contains 10 domains of quality, each with specific criteria that determine how the domain is measured and the level of quality of care rendered. The instrument is over 90% accurate and has good inter- and intra-rater reliability when used by trained pre-hospital provider users in South Africa. Pre-hospital provider user feedback indicates the tool is easy to learn and quick to use. Conclusion We created and validated a novel chart abstraction instrument that can reliably and accurately measure the quality of pre-hospital traumatic shock care. We provide a systematic methodology for developing and validating a quality of care tool for resource-limited care settings.
Article
Aim This study aimed to examine the quality of published paramedic scoping reviews against pre-existing frameworks to assess the extent to which they fulfil the requirements of this methodological approach. Subsequently, recommendations will be presented regarding improvements for future paramedic scoping reviews. Methods A scoping review was conducted guided by the PRISMA Extension for Scoping Reviews. A literature search was performed in six electronic databases as well as the grey literature to identify previous scoping reviews that focussed on paramedic or emergency medical service personnel. Relevant data were extracted from included articles and presented in narrative and tabular formats. Results The literature search initially identified 475 articles, of which 20 remained after title/abstract and full-text screening. There was a general increase in the number of studies published over time, the majority of articles (80%) had conducted their scoping review utilising published frameworks, and 75% of first authors were paramedics. Although many areas of these reports comply with published guidelines, there was an overall lack of consistency in the specific information included, the level of detail of that information, and the location of information within the reports. Conclusion All paramedic scoping studies should be reported with the use of a published framework to enable standardisation in the reporting, thus facilitating understanding, reproducibility, and utility. The PRISMA Extension for Scoping Reviews provides a checklist and thorough explanations of each step in the reporting process and is recommended for use with all future paramedic scoping reviews.
Article
Evolution of ambulance service response models has resulted in significant numbers of patients not being conveyed to Emergency Departments. Prior research has attempted to measure patient-safety aspects of non-conveyance with inconclusive results. Several authors have recommended investigation of patient experience as an alternative metric. Understanding patient experience is acknowledged as a core requirement for design and evaluation of changes to healthcare delivery. However, it is unclear to what extent patient experience of non-conveyance is described in academic literature. Aims To map scholarly literature that describes patient experience of non-conveyance and identify knowledge gaps that guide future research. Methods Scoping review guided by the Joanna Briggs Institute (JBI) framework. Results Ten studies of heterogenous methodology were included. Commonly, high levels of satisfaction with paramedic care were reported, yet contributing factors to satisfaction were generally not described. Qualitative studies provided deeper insight into experience. Value was attributed to reassurance and being empowered in the decision-making process. Not having concerns validated by paramedics led to negative experiences. Conclusion There is a scarcity of quality research that has investigated patient experience of non-conveyance following emergency ambulance service response. Methods used by existing research is of low-quality. Patient experience of non-conveyance is not fully known. Keywords Emergency medical services, Non-conveyance, Paramedics, Patient experience, Pre-hospital emergency care
Article
Full-text available
Background There is increasing interest for quality measurement in health care services; pre-hospital emergency medical services (EMS) included. However, attempts of measuring the quality of physician-staffed EMS (P-EMS) are scarce. The aim of this study was to develop a set of quality indicators for international P-EMS to allow quality improvement initiatives. MethodsA four-step modified nominal group technique process (expert panel method) was used. ResultsThe expert panel reached consensus on 26 quality indicators for P-EMS. Fifteen quality indicators measure quality of P-EMS responses (response-specific quality indicators), whereas eleven quality indicators measure quality of P-EMS system structures (system-specific quality indicators). DiscussionWhen measuring quality, the six quality dimensions defined by The Institute of Medicine should be appraised. We argue that this multidimensional approach to quality measurement seems particularly reasonable for services with a highly heterogenic patient population and complex operational contexts, like P-EMS. The quality indicators in this study were developed to represent a broad and comprehensive approach to quality measurement of P-EMS. Conclusions The expert panel successfully developed a set of quality indicators for international P-EMS. The quality indicators should be prospectively tested for feasibility, validity and reliability in clinical datasets. The quality indicators should then allow for adjusted quality measurement across different P-EMS systems.
Article
Full-text available
Objectives Performance indicators are a popular mechanism for measuring the quality of healthcare to facilitate both quality improvement and systems management. Few studies make comparative assessments of different countries’ performance indicator frameworks. This study identifies and compares frameworks and performance indicators used in selected Organisation for Economic Co-operation and Development health systems to measure and report on the performance of healthcare organisations and local health systems. Countries involved are Australia, Canada, Denmark, England, the Netherlands, New Zealand, Scotland and the United States. Methods Identification of comparable international indicators and analyses of their characteristics and of their broader national frameworks and contexts were undertaken. Two dimensions of indicators – that they are nationally consistent (used across the country rather than just regionally) and locally relevant (measured and reported publicly at a local level, for example, a health service) – were deemed important. Results The most commonly used domains in performance frameworks were safety, effectiveness and access. The search found 401 indicators that fulfilled the ‘nationally consistent and locally relevant’ criteria. Of these, 45 indicators are reported in more than one country. Cardiovascular, surgery and mental health were the most frequently reported disease groups. Conclusion These comparative data inform researchers and policymakers internationally when designing health performance frameworks and indicator sets.
Article
Full-text available
Aim of database The aim of the Danish quality database for prehospital emergency medical services (QEMS) is to assess, monitor, and improve the quality of prehospital emergency medical service care in the entire prehospital patient pathway. The aim of this review is to describe the design and the implementation of QEMS. Study population The study population consists of all “112 patient contacts” defined as emergency patients, where the entrance to health care is a 112 call forwarded to one of the five regional emergency medical coordination centers in Denmark since January 1, 2014. Estimated annual number of included “112 patients” is 300,000–350,000. Main variables We defined nine quality indicators and the following variables: time stamps for emergency calls received at one of the five regional emergency medical coordination centers, dispatch of prehospital unit(s), arrival of first prehospital unit, arrival of first supplemental prehospital unit, and mission completion. Finally, professional level and type of the prehospital resource dispatched to an incident and end-of-mission status (mission completed by phone, on scene, or admission to hospital) are registered. Descriptive data Descriptive data included age, region, and Danish Index for Emergency Care including urgency level. Conclusion QEMS is a new database under establishment and is expected to provide the basis for quality improvement in the prehospital setting and in the entire patient care pathway, for example, by providing prehospital data for research and other quality databases.
Article
Full-text available
Systematic reviews should build on a protocol that describes the rationale, hypothesis, and planned methods of the review; few reviews report whether a protocol exists. Detailed, well-described protocols can facilitate the understanding and appraisal of the review methods, as well as the detection of modifications to methods and selective reporting in completed reviews. We describe the development of a reporting guideline, the Preferred Reporting Items for Systematic reviews and Meta-Analyses for Protocols 2015 (PRISMA-P 2015). PRISMA-P consists of a 17-item checklist intended to facilitate the preparation and reporting of a robust protocol for the systematic review. Funders and those commissioning reviews might consider mandating the use of the checklist to facilitate the submission of relevant protocol information in funding applications. Similarly, peer reviewers and editors can use the guidance to gauge the completeness and transparency of a systematic review protocol submitted for publication in a journal or other medium.
Article
The current scoping review seeks to locate, examine and describe international literature on indicators used to measure pre-hospital care quality. Specifically, the review will: Map attributes of definitions or descriptions of ''quality'' in the context of pre-hospital care provided by ambulance services. Chart indicators that have been developed to measure pre-hospital care quality and detail their development processes as well as how the indicators fit into respective measurement frameworks/matrixes. © 2017 Joanna Briggs Institute. Unauthorized reproduction of this article is prohibited.
Article
Objectives: To systematically review the literature on quality indicators (QIs) for evaluating trauma care, identify QIs, map their definitions, and examine the evidence base in support of the QIs. Data Sources: We searched MEDLINE, EMBASE, CINAHL, Cochrane Database of Systematic Reviews, Cochrane Database of Abstracts of Reviews of Effects, and Cochrane Central Register of Controlled Trials from the earliest available date through January 14, 2009. To increase the sensitivity of the search, we also searched the grey literature and select journals by hand, reviewed reference lists to identify additional studies, and contacted experts in the field. Study Selection and Data Extraction: We selected all articles that identified or proposed 1 or more QIs to evaluate the quality of care delivered to patients with major traumatic injuries. Minimum inclusion criteria were a description of 1 or more QIs designed to evaluate patients with major traumatic injuries (defined as multisystem injuries resulting in hospitalization or death) and focused on prehospital care, hospital care, posthospital care, or secondary injury prevention. Data Synthesis: The literature search identified 6869 citations. Review of abstracts led to the retrieval of 538 fulltext articles for assessment, of which 192 articles were selected for review. Of these, 128 (66.7%) articles were original research, predominantly trauma database case series (57 [29.7%]) and cohort studies (55 [28.6%]), whereas 37 (19.3%) were narrative reviews and 8 (4.2%) were guidelines. A total of 1572 QIs in trauma care were identified and classified into 8 categories: non-American College of Surgeons Committee on Trauma (ACS-COT) audit filters (42.0%), ACS-COT audit filters (19.1%), patient safety indicators (13.2%), trauma center/system criteria (10.2%), indicators measuring or benchmarking outcomes of care (7.4%), peer review (5.5%), general audit measures (1.8%), and guideline availability or adherence (0.8%). Measures of prehospital and hospital processes (60.4%) and outcomes (22.8%) were the most common QIs identified. Posthospital and secondary injury prevention QIs accounted for less than 5% of QIs. Conclusions: Many QIs for evaluating the quality of trauma care have been proposed, but the evidence to support these indicators is not strong. Practical recommendations to select QIs to measure the quality of trauma care will require systematic reviews of identified candidate indicators and empirical studies to fill the knowledge gaps for postacute QIs.
Article
Introduction Key performance indicators (KPIs) are used to monitor and evaluate critical areas of clinical and support functions that influence patient outcome. Traditional prehospital emergency care performance monitoring has focused solely on response time metrics. The landscape of emergency care delivery in Ireland is in the process of significant national reconfiguration. The development of KPIs is therefore considered one of the key priorities in prehospital research. Aims The aim of this study was to develop a suite of KPIs for prehospital emergency care in Ireland. Methods A systematic literature review of prehospital care performance measurement was undertaken followed by a three-round Delphi consensus process facilitated by a broad-based multidisciplinary group of panellists. The consensus process was conducted between June 2012 and October 2013. Each candidate indicator on the Delphi survey questionnaire was rated using a 5-point Likert-type rating scale. Agreement was defined as at least 70% of responders rating an indicator as ‘agree’ or ‘strongly agree’ on the rating scale. Data were analysed using descriptive statistics. Sensitivity of the ratings was examined for robustness by bootstrapping the original sample. Results Of the 78 citations identified by the systematic review, 5 relevant publications were used to select candidate indicators for the Delphi round 1 questionnaire. Response rates in Delphi rounds 1 and 2 were 89% and 83%, respectively. Following the consensus development conference, 101 KPIs reached consensus. Based on the Donabedian framework for quality-of-care indicators, 7 of the KPIs which reached agreement were structure KPIs, 74 were process KPIs and 20 were outcome KPIs. The highest ranked indicator was a process KPI (‘Direct transport of ST-elevation myocardial infarction patients to a primary percutaneous intervention (PCI)-capable facility for ECG to PCI time <90 min’). Conclusion Improving the quality of prehospital care requires the development and implementation of performance measurement using scientifically valid and reliable KPIs. Employing a Delphi panel of key multidisciplinary Emergency Medical Service stakeholders, it was feasible to develop a suite of 101 KPIs for performance monitoring of prehospital emergency care in Ireland. This suite of KPIs may contribute to a framework for achieving safer, better care in the prehospital environment.
Article
Indicators to measure the quality of trauma care may be instrumental in benchmarking and improving trauma systems. This retrospective, observational study investigated whether data on three indicators for competencies of Dutch trauma teams (i.e. education, exposure, experience; agreed upon during a prior Delphi procedure) can be retrieved from existing registrations. The validity and distinctive power of these indicators were explored by analysing available data in four regions. Data of all polytrauma patients treated by the Helicopter Emergency Medical Services were collected retrospectively over a 1-year period. During the Delphi procedure, a polytrauma patient was defined as one with a Glasgow Coma Scale of 9 or less or a Paediatric Coma Scale of 9 or less, together with a Revised Trauma Score of 10 or less. Information on education, exposure and experience of the Helicopter Emergency Medical Services physician and nurse were registered for each patient contact. Data on 442 polytrauma patients could be retrieved. Of these, according to the Delphi consensus, 220 were treated by a fully competent team (i.e. both the physician and the nurse fulfilled the three indicators for competency) and 22 patients were treated by a team not fulfilling all three indicators for competency. Across the four regions, patients were treated by teams with significant differences in competencies (P=0.002). The quality indicators of education, exposure and experience of prehospital physicians and nurses can be measured reliably, have a high level of usability and have distinctive power.