ArticlePDF AvailableLiterature Review


Background: Intersectoral costs and benefits (ICBs), i.e. costs and benefits of healthcare interventions outside the healthcare sector, can be a crucial component in economic evaluations from the societal perspective. Pivotal to their estimation is the existence of sound resource-use measurement (RUM) instruments; however, RUM instruments for ICBs in the education or criminal justice sectors have not yet been systematically collated or their psychometric quality assessed. This review aims to fill this gap. Methods: To identify relevant instruments, the Database of Instruments for Resource Use Measurement (DIRUM) was searched. Additionally, a systematic literature review was conducted in seven electronic databases to detect instruments containing ICB items used in economic evaluations. Finally, studies evaluating the psychometric quality of these instruments were searched. Results: Twenty-six unique instruments were included. Most frequently, ICB items measured school absenteeism, tutoring, classroom assistance or contacts with legal representatives, police custody/prison detainment and court appearances, with the highest number of items listed in the Client Service Receipt Inventory/Client Sociodemographic and Service Receipt Inventory/Client Service Receipt Inventory-Children's Version (CSRI/CSSRI/CSRI-C), Studying the Scope of Parental Expenditures (SCOPE) and Self-Harm Intervention, Family Therapy (SHIFT) instruments. ICBs in the education sector were especially relevant for age-related developmental disorders and chronic diseases, while criminal justice resource use seems more important in mental health, including alcohol-related disorders or substance abuse. Evidence on the validity or reliability of ICB items was published for two instruments only. Conclusion: With a heterogeneous variety of ICBs found to be relevant for several disease areas but many ICB instruments applied in one study only (21/26 instruments), setting-up an international task force to, for example, develop an internationally adaptable instrument is recommended.
Health-Related Resource-Use Measurement Instruments
for Intersectoral Costs and Benefits in the Education and Criminal
Justice Sectors
Susanne Mayer
Aggie T. G. Paulus
Agata Łaszewska
Judit Simon
Ruben M. W. A. Drost
Dirk Ruwaard
Silvia M. A. A. Evers
ÓThe Author(s) 2017. This article is an open access publication
Background Intersectoral costs and benefits (ICBs), i.e.
costs and benefits of healthcare interventions outside the
healthcare sector, can be a crucial component in economic
evaluations from the societal perspective. Pivotal to their
estimation is the existence of sound resource-use mea-
surement (RUM) instruments; however, RUM instruments
for ICBs in the education or criminal justice sectors have
not yet been systematically collated or their psychometric
quality assessed. This review aims to fill this gap.
Methods To identify relevant instruments, the Database of
Instruments for Resource Use Measurement (DIRUM) was
searched. Additionally, a systematic literature review was
conducted in seven electronic databases to detect instru-
ments containing ICB items used in economic evaluations.
Finally, studies evaluating the psychometric quality of
these instruments were searched.
Results Twenty-six unique instruments were included.
Most frequently, ICB items measured school absenteeism,
tutoring, classroom assistance or contacts with legal
representatives, police custody/prison detainment and court
appearances, with the highest number of items listed in the
Client Service Receipt Inventory/Client Sociodemographic
and Service Receipt Inventory/Client Service Receipt
Inventory–Children’s Version (CSRI/CSSRI/CSRI-C),
Studying the Scope of Parental Expenditures (SCOPE) and
Self-Harm Intervention, Family Therapy (SHIFT) instru-
ments. ICBs in the education sector were especially rele-
vant for age-related developmental disorders and chronic
diseases, while criminal justice resource use seems more
important in mental health, including alcohol-related dis-
orders or substance abuse. Evidence on the validity or
reliability of ICB items was published for two instruments
Conclusion With a heterogeneous variety of ICBs found to
be relevant for several disease areas but many ICB
instruments applied in one study only (21/26 instruments),
setting-up an international task force to, for example,
develop an internationally adaptable instrument is
Electronic supplementary material The online version of this
article (doi:10.1007/s40273-017-0522-4) contains supplementary
material, which is available to authorized users.
&Susanne Mayer
Department of Health Economics, Center for Public Health,
Medical University of Vienna, Kinderspitalgasse 15/1, 1090
Vienna, Austria
Department of Health Services Research, Care and Public
Health Research Institute (CAPHRI), Faculty of Health,
Medicine and Life Sciences, Maastricht University,
Maastricht, The Netherlands
Trimbos Institute, Netherlands Institute of Mental Health and
Addiction, Utrecht, The Netherlands
DOI 10.1007/s40273-017-0522-4
Key Points for Decision Makers
Health-related interventions often incur costs and
benefits outside the healthcare sector (i.e.
intersectoral costs and benefits [ICBs]) which, if
relevant, should be considered in economic
evaluations and cost-of-illness studies from the
broader societal analytical perspective to arrive at
unbiased conclusions.
Reliable and valid instruments to measure such
resource use in these sectors are pivotal. Based on a
systematic review of existing instruments used in
applied economic evaluations, this paper provides an
overview of generic and disease-specific resource-
use measurement instruments in the education and
criminal justice sectors, including their psychometric
properties. Relevant instruments will be included in
the Database of Instruments for Resource Use
Measurement (DIRUM;
Many instruments identified in this review were
specifically applied in one study only, potentially
implying considerable duplication of work across
studies. Hence, bundling individual efforts could be
a more cost-effective strategy overall, and setting-up
an international task force to support these activities,
e.g. by fostering methodological transparency and
developing an internationally adaptable, harmonized
instrument including relevant ICB items, is thus
1 Introduction
For economic evaluations to be a useful tool to inform
policy makers regarding the allocation of scarce resources,
accurate measurement and valuation of all relevant costs
and benefits related to an intervention is key. Costs and
benefits related to healthcare interventions that occur out-
side the healthcare sector, i.e. intersectoral costs and ben-
efits (ICBs) [1,2], have been shown to be a vital
component in economic evaluations applying a societal
perspective [36]. At the same time, taking a societal
viewpoint in economic evaluations is recommended by
guidelines for the use of economic evaluation in several
countries [7], amounting to a proportion of approximately
two-thirds of the analyzed countries in 2010 [8], and most
recently also including the US [9]. The societal perspective
implies that all benefits and costs that are directly or
indirectly caused by the intervention (if significantly
present) should be included in the economic evaluation,
independently of who incurs them [10]. These costs and
benefits can occur outside the healthcare system, including
the education and criminal justice sectors, and household
and leisure activities, in addition to productivity loss [6].
In recent research, generic methods for valuing ICB unit
costs have been developed and applied in the context of the
Dutch education and criminal justice system [11]. How-
ever, the issue of how to measure such intersectoral
resource use in trial-based economic evaluations and cost-
of-illness studies with standardized resource-use measure-
ment (RUM) instruments has received little systematic
scientific attention to date [12]. This might hinder the
consideration of relevant ICB resource use in economic
evaluations in the first place [9], and consequently lead to
biased results from a societal perspective [13]. It also
potentially creates heterogeneity in the measurement due to
variabilities in considered cost components, which in turn
impairs the comparability of cross-study cost estimates.
In addition, it is unclear to what extent the quality of
existing ICB RUM instruments in terms of their psycho-
metric properties has been evaluated. Sound instruments
are pivotal to adequately capture the impact of healthcare
interventions in other sectors. A review of health-related
RUM instruments that led to the set-up of the Database of
Instruments for Resource Use Measurement (DIRUM) in
2011 [14] showed that limited evidence is available
regarding the validity, and especially reliability, of the
listed instruments [15]. This presumably also applies to
instruments measuring ICBs. One major exception includes
the measurement of lost productivity, for which both a
number of RUM instruments have been developed, vali-
dated, and the quality of their validation evidence critically
appraised in recent publications [16,17]. Similarly,
regarding ICBs in the household and leisure sectors,
instruments to measure informal care have been collected
and/or assessed in previous research [1820]. This is not
yet the case for the education or criminal justice RUM
Against this background, this research focuses on
health-related ICBs linked to resource use in the education
sector (i.e. related to the impairment of educational
achievement of individuals [2,9]) and criminal justice
sector (i.e. related to the costs of crimes [2,9]), and has two
aims. Our main aim was to provide a systematic overview
of the characteristics of current RUM instruments used in
health economic analyses that include ICB items capturing
the impacts on the education and criminal justice sectors.
Second, we sought to determine the existence of psycho-
metric quality assessments of these instruments. This will
not only provide practical guidance for researchers to
identify sound instruments for ICB RUM in specific dis-
ease areas but also help detect disease areas for which such
S. Mayer et al.
measurement in economic evaluations might be crucial. In
addition, it will provide an evidence-base to potentially
pioneer the development of standardized new items to
measure relevant costs and benefits outside the healthcare
2 Methods
2.1 Identification of Intersectoral Cost and Benefit
(ICB) Instruments
2.1.1 Databases
To identify instruments with relevant ICB items, a stepwise
approach was adopted. First, the open-access repository
DIRUM ([14], listing 77 RUM
instruments in July 2016, was hand-searched. DIRUM
contains full texts of health-related RUM instruments, as
well as information and/or references about respective
instrument qualities. DIRUM has a search function allow-
ing the selection of specific ‘items of resources being
measured’ (e.g. criminal justice service, educational, other
non-National Health Service), however, all of the 77 listed
instruments were examined in detail.
Second, as DIRUM is not exhaustive [21], an extensive
systematic literature search was conducted. Herein, the
adopted research approach rests on the assumption that
existing ICB instruments that measure such resource use
are cited and/or listed in economic evaluations and cost-of-
illness studies that were conducted from a societal per-
spective. Seven electronic databases were searched in July
2016, including MEDLINE (PubMed), EMBASE (Ovid),
Social Science Citation Index (SSCI; Web of Science),
PsycINFO, Econlit, Education Resources Information
Centre (ERIC) and CINAHL (EBSCOhost). This covers
two general medical literature databases (MEDLINE,
EMBASE) [22], which, in combination with CINAHL and
PsycINFO, are considered appropriate to identify economic
evaluations [23]. SSCI, Econlit and ERIC were included to
incorporate a social science, education and economic per-
spective, respectively. No date or language restrictions
were applied (electronic supplementary Appendix
Table A1). Both search strategy and database choices were
discussed and agreed with the project team and an infor-
mation scientist.
2.1.2 Inclusion and Exclusion Criteria
Eligibility criteria of the identified studies, framed around
PICOS [24], included all non-institutionalized and institu-
tionalized age groups of the population (P), regardless of
the intervention (I), comparators (C) and outcomes (O), set
up as a trial-based (non-simulation/non-model) full eco-
nomic evaluation (i.e. comparing both costs and outcomes
of the intervention group and at least one comparator) or as
a cost-of-illness study, adopting a societal perspective (i.e.
including costs in both the healthcare sector and other
sectors) as the study design (S) and measuring ICBs in their
analysis, published as a full paper or report. Consequently,
study exclusion criteria included no original research, no
full economic evaluation or cost-of-illness study, based on
model or simulation, and not adopting a societal perspec-
tive. Studies building on nationwide population surveys
were excluded, whereas articles focusing on the description
of RUM instruments were also screened. All studies ful-
filling the inclusion criteria irrespective of language were
reviewed for referenced patient/carer-reported ICB RUM
instruments in the education or criminal justice sectors (and
information on relevant psychometric evidence). Instru-
ments either had to measure education or criminal justice
resource use, or the article had to refer to a specified
(published) instrument that could potentially include such
items. Relevant information about the instrument was
2.2 Retrieval of ICB Instruments
Full texts of instruments identified through DIRUM were
readily available from the DIRUM website [14]. Full texts
of the instruments identified through the database search of
economic evaluations and cost-of-illness studies, if not
included in the identified publication itself, were retrieved
via two channels. A Google (Scholar) search was con-
ducted first. If unsuccessful, the (corresponding) authors of
the articles were then contacted via email, and a full ver-
sion of the utilized instrument (and, in a second step,
information about the psychometric properties) was
requested. No language restrictions were applied in terms
of the included instruments, and translators were used for
extraction. Instruments had to be available as a full version
(for free).
2.3 Identification and Retrieval of Studies Assessing
Instrument Properties
Regarding psychometric properties that assess the quality
of the identified instruments, this review focuses on their
validity and reliability. Validity captures the degree to
which an instrument measures what it intends to measure,
while reliability refers to the ability of an instrument to do
this in a consistent manner [25]. To identify studies
assessing the instruments in this respect, the references of
the studies listed in DIRUM under ‘instrument qualities’
were investigated. For the instruments included based on
the systematic literature review, secondary database
ICB Resource-Use Measurement Instruments
searches were carried out based on the names of the
instruments and other relevant additional information
found in the initially reviewed studies. The same databases
were searched as for the identification of the instruments.
2.4 Screening of Studies and Instruments
The methodology and reporting of the systematic review
were generally set up to be consistent with the Guideline
for Conducting Systematic Literature Reviews in Eco-
nomic Evaluation [24], the methods proposed by the
Cochrane Collaboration [26], and the Preferred Reporting
Items for Systematic Reviews and Meta-Analyses
(PRISMA) statement [27]. The PRISMA checklist is
presented in electronic supplementary Appendix
Table A2. No review protocol was registered. Titles and
abstracts (level 1) were screened by SM. In addition, AŁ
conducted a second independent screening of the titles
and abstracts for 10% of all studies. Differences in
inclusion (level 1) were found in approximately 20% of
the studies, but did not affect the final inclusion of unique
instruments, for which close to no disagreement was
observed. This approach also seems justifiable given that
all studies that did not explicitly contradict the inclusion
criteria on level 1 were assessed further, independently of
actually mentioning a (specific or non-specific) instrument
in the title or abstract. Full-text screening of the articles
(level 2), instrument extraction, and the additional search
for psychometric evidence was independently conducted
by two reviewers (SM, AŁ) and any disagreement
resolved by discussion against the inclusion criteria. The
instrument extraction sheet, which partly follows the
taxonomy [15] designed for the set-up of DIRUM, was
discussed with the project team and piloted on 10% of the
included instruments.
3 Results
Of 3637 unique studies identified in the systematic review
of the literature, 167 full texts were included for assess-
ment of the instruments mentioned, cited or listed in these
studies (Fig. 1). Following the screening of the full
instruments, a total of 55 instruments were identified,
resulting in 26 unique RUM instruments with ICB items in
the education and/or criminal justice sectors after dedu-
plication (Table 1); 8 instruments were retrieved based on
the DIRUM search, 13 instruments were found based on
given references in economic evaluations and cost-of-ill-
ness studies, and 5 instruments came up in both searches.
Of the 26 instruments, five were included following email
correspondence with the study authors as full texts were
not publicly available. Twenty-one of the 26 instruments
were found to be used only once, i.e. in one study, for one
A descriptive overview of the characteristics of the
included instruments is presented in Table 1. With the
exception of the Dutch [28] RUM instrument, all instru-
ments were published in English, mostly referring to the
UK healthcare system [2941], the US [4347], Australia
[48], Canada [49,50], Cambodia [51] and Kenya [52].
Three instruments [42,53,54] were designed for multi-
country studies, and were published in English and several
other European languages. Mostly, instruments were used
for research related to mental health problems or devel-
opmental disorders [28,29,31,38,39,42,43,49], alcohol-
related disorders or substance abuse [30,36,44,50,54],
chronic conditions [32,33,37,45] or other specific dis-
eases [34,35,41,4648,5153]. In total, 14 instruments
refer to children and/or adolescents [28,3135,38,
39,43,46,49,51,55,56], three instruments refer to
adolescents and adults [45,47,53], and nine instruments
refer to the adult population only [29,30,36,42,44,
48,50,52,54]. Although many instruments were designed
for, or used in, multiple administration modes and forms of
recording, 9 [29,4346,48,50,51,54] of the 26 instru-
ments were administered in an interview set-up, including
direct contact with a researcher.
3.1 ICB Instrument Content
All but two instruments not only include ICB-specific
items but also a variety of healthcare utilization RUM
questions. Education-related RUM items are included in 21
instruments, most frequently in instruments designed in the
context of various specific disease areas [34,35,41,
4648,5153], developmental disorders [31,38,39,
43,49] and chronic conditions [32,33,45,55]. Criminal
justice RUM items are included in 13 instruments, most
frequently related to developmental disorders [38,39,
43,49], alcohol-related disorders or substance abuse
[30,36,44,50,54] and mental health in general [29,42].
Seven instruments include items from both the education
and criminal justice sectors. Extracted details on the
instrument content regarding, for example, item wording
are presented in the electronic supplementary Appendix
Table A3.
As shown in Fig. 2, ICB items referring to the education
sector mostly cover time missed at school due to the
specific disease (15 items), followed by extra need for
tutoring activities (six items) and classroom assistance (six
items). Attendance at a special/boarding school is captured
by five items, as is social/school functioning. ICB items
referring to the criminal justice system most commonly
S. Mayer et al.
refer to the use of lawyers or general legal assistance (11
items), police custody or prison detainment (9 items),
appearances in criminal or civil courts (6 items), injuries (5
items), police contacts (5 items), or probation/correction
services (5 items) [Fig. 3].
3.2 Psychometric Properties of ICB Items
The psychometric properties were assessed for seven of the
identified instruments, namely the Child and Adolescent
Services Assessment (CASA) interview [43], Client
Fig. 1 PRISMA flowchart of electronic database search and DIRUM
search. PRISMA Preferred Reporting Items for Systematic Reviews
and Meta-Analyses, ICB intersectoral costs and benefits, DIRUM
Database of Instruments for Resource Use Measurement. Asterisk
refers to the instruments included from the DIRUM search
ICB Resource-Use Measurement Instruments
Table 1 Instrument administration and content
Q name, administration mode Language,
Disease category Target
Person filling in
Recall period
RUM in
Psychometric evidence, pilot
Mental health (general)
Bodden 2008: diary Dutch,
Anxiety disorder Children,
Parents 2 weeks
[28][28]; SLR
and E
Client Service Receipt Inventory
(CSRI): originally interview, paper-
based instrument
English, UK,
into various
Mental health Adults Patient/carer;
3 months 4
[73,74]; validity of
Italian version [75], pilot
[29]; D and
Client Sociodemographic and
Service Receipt Inventory—
European Version (CSSRI—EU):
paper-based instrument
English and
several other
Adults Researcher 3 months 4
reliability of German
version [77] (but no
[42]; D and
Alcohol-related disorders
Alcohol: Evaluating Stepped care in
Older Populations Study (AESOPS)
Instrument: paper-based instrument
English, UK Alcoholism Adults Patient 6 months 44 [30]; D and
Health Service Utilization Inventory
(HSUI), modified: interview
Fetal alcohol
spectrum disorder
Adults Parents 12 months 4
[78][50]; SLR
and E
Parrott 2006: paper-based instrument English, UK Alcoholism Adults Patient 6 months 44 [36 ]; SLR
and E
Sommers 2011: interview English, USA Alcoholism and
substance abuse
Adults Researcher 12 months 444 [44]; SLR
and E
European Addiction Severity Index
(EuropASI): interview
English and
several other
Substance abuse Adults Researcher 1 month/
4Pilot tested; validated in
several countries (e.g.
France, Germany, Czech
Republic) [54], see, for
example, Dutch version
SLR and E
Developmental disorders
Child and Adolescent Services
Assessment (CASA) child
interview/parent interview, version
5: interview/paper-based instrument
English, USA Developmental
problems; mental
3 months 4
validity (for
healthcare RUM only) [43]
[43]; SLR
Client Service Receipt Inventory
Children’s Version (CSRI-C):
originally paper-based instrument
English, UK Developmental,
psychosocial and
learning problems
Children Parent/carer;
12 months; per
44 Pilot tested [81][31]; D
Studying the Scope of Parental
Expenditures (SCOPE): web-based
Children Parent 6 months; per
44Pilot tested [49][49]; SLR
Self-Harm Intervention, Family
Therapy (SHIFT) Parent or Carers
Questionnaire Booklet: postal
English, UK Developmental,
psychosocial and
learning problems
Adolescents Parent 3 months 444 [38]; D
S. Mayer et al.
Table 1 continued
Q name, administration mode Language,
Disease category Target
Person filling in
Recall period
RUM in
Psychometric evidence, pilot
Self-Harm Intervention, Family
Therapy (SHIFT) Young Person
Questionnaire Booklet: postal
English, UK Developmental,
psychosocial and
learning problems
Adolescents Patient 3 months 444 [39]; D
Chronic disease
Client Service Receipt Inventory for
Children with Diabetes (CSRI-CD):
paper-based instrument
English, UK Diabetes Children,
Researcher (with
child or parent and
1 month 44 [32]; D
Client Service Receipt Inventory for
Adolescents with Chronic Pain
(CSRI-Pain): paper-based
English, UK Chronic pain Adolescents Parents 12 months 444Pilot tested [82][33]; SLR
SubCutaneous Insulin: Pumps or
Injections (SCIPI) RUM: paper-
based instrument
English, UK Endocrine and
Researcher 3 months 44 [37,55]; D
Wetterneck 2006: interview English, USA Chronic hair pulling Adolescents,
Researcher 3 months 44 [45]; SLR
Other diseases
¨n 2014: web- or
paper-based survey
Patient Specified
based on
6 months
44 Pilot tested [53][53]; SLR
Cost of Trauma Instrument (COTI):
mailed questionnaire/telephone
Orthopedics and
Adults Patient (parent for
12 months 444 [48]; D and
HUGS V: Hemophilia Costs and
Impact of Disease Study, Version 2:
English, USA Hemophilia Children,
(patient \18 years
of age): version 2
1 month 44 [46]; SLR
and E
Huy 2009: interview English,
Dengue fever Children/
Parent/researcher Fever episode 4
(but no details
given), pilot tested [51]
[51]; SLR
MAGnesium NEbuliser Trial In
Children (MAGNETIC)
questionnaire: postal instrument
English, UK Lungs and airways Children,
Parent 1 month 44 Pilot tested [34][34]; D
North of England and Scotland
Study of Tonsillectomy and Adeno-
tonsillectomy in Children
(NESSTAC) Parent’s
Questionnaire: postal instrument
English, UK Ear, nose and throat Children,
Parent/carer 3 months 44 Pilot tested [35][35]; D
SafetY and Cost Effectiveness of
Adalimumab in Combination with
diary and questionnaire
English, UK Eyes and vision Children,
patient (diary)
3 months
44 [41][56]; D
ICB Resource-Use Measurement Instruments
Service Receipt Inventory (CSRI) [29], Client Sociode-
mographic and Service Receipt Inventory—European
Version (CSSRI—EU) [42], cost diary by Bodden et al.
[28], Work Productivity and Activity Impairment (WPAI)
instrument [47], Health Service Utilization Inventory
(HSUI) [50], and European Addiction Severity Index
(EuropASI) [54]. In case of one instrument [51], validation
was mentioned but no further details were given. In the
economic evaluations citing the included instruments, most
authors provided details on the development of the
instrument, including, for example, the development team
or information on which other instrument the relevant
instrument is based on. Pilot testing of the instrument was
reported for nine instruments.
Psychometric assessment of ICB-related items could be
confirmed for two of these seven instruments; for the
CASA instrument [43], both reliability and validity were
determined. Test–retest reliability was found to be high for
the most intensively used services, including juvenile jus-
tice; however, services used in the child’s natural setting,
for example school services, were found to be reported
with low reliability [57]. The assessment of concurrent
validity (i.e. correlation of service use measured with two
instruments) was restricted to mental health service use
only, and, again, varied for individual services [58]. Con-
struct validity and reproducibility of the general WPAI
Questionnaire was established in earlier research [59]. For
the WPAI Questionnaire plus Classroom Impairment
Questions: Specific Health Problem (WPAI ?CIQ:SHP,
version 2.0) specifically, psychometric evidence is avail-
able for its allergy-specific (AS) first version (the
WPAI ?CIQ:AS), testing its discriminative and evalua-
tive validity [60]. Except for time missed from the class-
room, allergic rhinitis symptoms were found to be well-
correlated for most impairment measures (discriminative
validity). The same conclusion applies to the correlation
analysis on the change in symptoms and change in time
missed from the classroom (evaluative validity).
4 Discussion and Conclusion
Conducting economic evaluations from a broader societal
perspective rather than from a narrow healthcare viewpoint
is increasingly acknowledged as the gold standard [61], and
also manifests in pharmacoeconomic guidelines across
Europe [7] and in the US [9]. To be able to consider health-
related impacts of interventions in the education or crimi-
nal justice sectors in practice, sound instruments measuring
relevant resource use are a prerequisite. This review
identified a total of 26 unique instruments with a variety of
such ICB items. Most frequently, ICB items in the edu-
cation sector measured resource use due to school
Table 1 continued
Q name, administration mode Language,
Disease category Target
Person filling in
Recall period
RUM in
Psychometric evidence, pilot
The Tool to Estimate Patients’
Costs: paper-based questionnaire
English, Kenya Tuberculosis Adults Researcher During
44 Pilot tested [83][52]; D and
Work Productivity and Activity
Impairment Questionnaire plus
Classroom Impairment Questions:
Specific Health Problem Version 2.0
English, USA Adaptable to specific
Patient 1 week 4
[60][47]; SLR
Qquestionnaire, DQ full-text retrieved via search in DIRUM, SLR Q identified via systematic literature review, EQ full-text received following e-mail correspondence with study author(s), ICB intersectoral costs and benefits, RUM
resource-use measurement, DIRUM Database of Instruments for Resource Use Measurement, 4indicates yes (included)
Psychometric evidence (partly) assessed for these sections of the instrument (for details, see column ‘Psychometric evidence’)
S. Mayer et al.
absenteeism, tutoring activities, classroom assistance and
school or social functioning. Resource use related to the
criminal justice sector most commonly captured legal
assistance, police custody or prison detainment, criminal or
civil court appearance, injuries, police contacts or the use
of probation/correction services. Psychometric evidence
was found to be examined for seven instruments that, with
two exceptions [43,47], focused only on healthcare RUM
items, hence generally leaving out ICB items from their
assessment. For the two exceptions, validity (discrimina-
tive and evaluative) was assessed for one instrument
(WPAI ?CIQ:AS [47]), and reliability (test–retest) was
assessed for another (CASA [43]).
From a methodological perspective, compared with
patient-reported outcome measures, methods of RUM within
economic evaluations have been less of a focal point [21].
Specifically, a culture of psychometric validation of RUM
instruments for self-reported measurement is lacking in
health economics [21,62]. In a review of UK Health Tech-
nology Assessment (HTA) program funds trials, only
approximately 30% of the studies were found to report on
validation of their RUM data [63]. This review confirms such
Fig. 2 Education-related ICB items in the instruments. Instrument
references by ICB items: (1) absenteeism from school
[28,32,34,35,39,41,4345,47,51,53,55]; (2) tutoring
[31,33,41,43]; (3) classroom assistance [31,33,38,39,49,56]; (4)
special school/boarding school [31,33,38,39,43,49]; (5) school
functioning [35,45,46]; (6) social functioning [46]; (7) adolescence/
school counsellor [38,39,50]; (8) special school teacher [33,43]; (9)
special needs statement [31,38]; (10) special class [43]; (11) school
dropout [48,52]; (12) other educational services [38,39]
[31,35,4749,51]. As more than one item per topic may be
contained in an instrument, the number of references do not
necessarily add up to the numbers indicated in the figure. ICB
intersectoral costs and benefits
Fig. 3 Criminal justice-related ICB items in the instruments. Instru-
ment references by ICB item: (1) lawyer/legal assistance
[33,36,38,39,49,50]; (2) police custody/prison detainment
[29,30,42,44,50,54]; (3) criminal/civil court appearance
[29,30,42,44,50]; (4) injury [36]; (5) police contact
[29,38,39,42,50]; (6) probation/correction services
[38,39,43,50,54]; (7) aggressiveness/violence [36,54]; (8) traffic
accident [36,44]; (9) psychiatric assessment in custody [29,42]; (10)
property damage [38,48,54]; (11) youth offending team [38,39];
(12) other services [29,42][36,38,44,54]. As more than one item
per topic may be contained in an instrument, the number of references
do not necessarily add up to the numbers indicated in the figure. ICB
intersectoral costs and benefits
ICB Resource-Use Measurement Instruments
lack of psychometric evidence, also specifically for ICB
items. Indeed, establishing validity, in particular for ICB in
the education or criminal justice sectors, might prove even
more challenging than for non-ICB items [64]. For example,
it might prove difficult, in practice, to get access to alterna-
tive data sources (e.g. administrative information) for items
such as the number and/or duration of police contacts, psy-
chiatric assessments, received legal assistance, or the num-
ber of traffic tickets (electronic supplementary Appendix
Table A3) to establish some degree of criterion validity [15].
This is a drawback, especially given that criminal justice
resource use is likely a sensitive topic and potentially rather
prone to, for example, measurement error [36]. For instance,
earlier research suggests that a social desirability bias is more
likely to be present in interview situations with a researcher
than, for example, self-administered questionnaires [64,89].
This issue could thus also be relevant for existing ICB items,
given that an interview-type administration mode was used
for 9 of the 26 identified instruments. Lack of psychometric
assessment was also found with regard to piloting of the
instruments, although pilot-testing is another critical step in
ensuring reliability [63,65]. For 9 of the 26 instruments
included in this review, pilot-testing was reported, which is
comparable to the proportion identified in the UK HTA
review [63].
A recent systematic literature review by Leggett et al.
[12] also analyses RUM instruments. RUM items outside
the healthcare sector, including travel expenses, out-of-
pocket costs and productivity losses, were concluded to be
particularly rare in the 15 identified instruments. Other ICB
items were not discussed in the review, which was gener-
ally restricted to publicly available, validated instruments
designed for adult populations only. Overlap between the
instruments identified by Leggett et al. [12] and this
research is consequently limited to the CSRI instrument
only. The lacking overlap is also likely due to this review’s
indirect search approach (Sect. 2.1), contrasting the com-
mon method of directly identifying validated instruments
through literature search in earlier reviews [12,16,66].
4.1 Strengths and Limitations
This review is the first to give a comprehensive overview
of instruments including ICB items for measurement of
resource use in the education or criminal justice system. It
discloses both a heterogeneous variety of existing ICB
RUM items in these sectors and reveals a lack of psycho-
metric evaluation of these instruments. The latter finding
strongly supports the choice of the adopted indirect search
strategy. Relevant new instruments identified in the course
of this review will be fed into DIRUM, which will help
future researchers identify and retrieve instruments with
ICB items in a more efficient manner. Indeed, the fact that
the majority (21 of 26) of the included instruments were
found to be specifically designed for and used in one study
only points out substantial efficiency potential for
researchers through sharing instruments. Finally, in most
cases, the identified ICB items are part of broader instru-
ments; in this sense, this review also provides an overview
of existing international instruments for healthcare RUM,
albeit not being the initial aim.
Note though that several limitations apply to the instru-
ment identification strategy. Firstly, instruments that gener-
ally contain ICB items, but were used in an economic
evaluation from a healthcare perspective only, were poten-
tially overlooked. However, all instruments that were spec-
ified in the studies were checked for relevant ICB RUM
items, even if inclusion of such elements was not to be
expected from the respective economic evaluation or cost-
of-illness analysis. At the same time, the high number of
studies using non-specified, non-referenced, non-listed
instruments detected in this review is striking. For example,
of all 167 studies included for instrument assessment,
approximately one in five articles (n=31) reported using,
for example, some ‘(adapted) standardized’ instruments or a
general ‘economic’ instrument, and failed to provide more
details. This lack of methodological transparency in instru-
ment reporting, which was also seen in earlier research [64],
hampers not only the potential comparability of these eco-
nomic evaluations but also impairs the quality assessment of
the measurement of the included cost components.
Secondly, this review was restricted to instruments
mentioned in published health-related economic evalua-
tions or cost-of-illness studies, and thus health services
research in general. Looking at other disciplines such as,
for example, education research, economic evaluations
seem to be applied rather sparsely [67]. However, addi-
tional RUM instruments might be found, for example, in
the literature on education program impacts [66]. Future
research might thus want to consider looking into instru-
ments developed in other disciplines. Trial registries could
be an additional channel to identify more recent ICB
measurement instruments.
Thirdly, comparable to the review by Leggett et al. [12],
a quality assessment of the psychometric evidence was not
carried out. This is a limitation since not only the validation
findings themselves are crucial but also the quality of the
methodology applied [16]. However, given the currently
limited evidence published on instrument quality for ICB
items (2 of 26 instruments), at this point such an assess-
ment would be premature. Once more relevant studies have
been conducted, the quality of the validation studies could
be checked, e.g. by applying the COSMIN (COnsensus-
based Standards for the selection of health Measurement
INstruments) checklist [68], as was done by recent research
S. Mayer et al.
4.2 Research and Policy Implications
When choosing an RUM instrument for use in a trial-based
economic evaluation, it is crucial that this instrument
covers the domains that are needed to capture the real-
world economic consequences of an intervention in the
specific disease area [42]. Based on the included instru-
ments, ICB RUM items in the education sector seem par-
ticularly relevant not only for age-related developmental
disorders affecting children and adolescents but also
chronic diseases, including diabetes and chronic pain. In
contrast, criminal justice service resource use seems more
important in the fields of alcoholism and substance abuse,
and developmental disorders, as well as mental health in
general. Future economic analyses, particularly in these
fields, should thus consider measuring ICBs from the
education and criminal justice sectors, respectively.
Against the backdrop of the recently developed ICB clas-
sification scheme [2], the majority of ICB items in the edu-
cation sector are captured by the existing instruments. Most
items are found in the CSRI–Children’s Version (CSRI-C)
[31], Studying the Scope of Parental Expenditures (SCOPE;
based on CSRI) [49] and Self-Harm Intervention, Family
Therapy (SHIFT) instruments [38,39]. Regarding the crimi-
nal justice sector, most ICB items are included in the CSRI
[29], CSSRI [42], SHIFT [38] and the instrument developed
by Sommers et al. [44]. However, compared with the ICB
classification scheme, existing instruments lack ICB items
regarding, for example, child maltreatment, sexual assaults
and crime consequences on victims, which should be taken
into account when developing new items.
Besides validating existing ICB items, future research
could focus on the development of a harmonized new
instrument with a broad variety of relevant ICB elements.
Indeed motivated by the fact that RUM instruments lack
validation, RUM items from the DIRUM database are cur-
rently being reviewed by Thorn et al. [62] with the aim of
developing a standardized RUM instrument for the UK. This
instrument will focus on the health and social care sector and
will exclude ICBs. Therefore, the first step in developing a
new ICB instrument would include a literature search to
identify the main cost-driving elements from relevant eco-
nomic evaluations in a specific disease area [21]. This was
selectively performed in prior research [11,70]; however,
given that the inclusion of ICBs does not seem to have a long
tradition [71], these empirical studies might be missing such
cost elements for this very reason [72]. Thus, collating ICBs
that were mentioned in studies but not necessarily measured
appears more reasonable [2]. The second step in the devel-
opment of a new instrument [21] includes the identification
of existing instruments, which was also one of the main aims
of this review. Following this step, focus groups with
healthcare professionals, experts in the education/criminal
justice sectors and patients to discuss these instruments
should be organized to develop a first version of a new
instrument. Many instruments identified in this review were
specifically developed for use in one study only, which
potentially implies considerable duplication of work hap-
pening across studies. Therefore, bundling individual efforts
could be a more cost-effective strategy overall. Setting-up an
international task force to support these activities by, for
example, reflecting on and exploring different sources of, for
instance, electronic data as a means of validation, and also
developing an internationally adaptable, validated instru-
ment, could be the next step. Such an initiative should take
into account the state of the art of RUM classifications by
detailing the key components of an RUM [64,71]. Following
a structured taxonomy such as the one developed by Ridyard
et al. [64], by providing a description of the data source, who
completes the instrument, the administration mode, and the
methods and medium of recording, clarity and method-
ological transparency could be established in the develop-
ment of such a new instrument.
Data Availability Statement The authors declare that
the data supporting the findings of this study are available
within the article and its supplementary electronic infor-
mation files. Full instruments not directly retrieved from
the literature but following email communication with the
instrument developer may be directly requested from them
based on the information provided (Table 1, ‘Reference
and instrument identification’). Relevant instruments will
be included in DIRUM (
Acknowledgements Many thanks to Gregor Franssen (Information
Scientist at Maastricht University) for expert advice on setting up the
search strategy, and Marianne Tilly for her support with study full-
text retrieval. The authors also gratefully acknowledge the help of
several members of the Department of Health Services Research at
Maastricht University with language translations. Feedback from
Joanna Thorn on an earlier version of the manuscript was greatly
appreciated. We also gratefully acknowledge the valuable input from
participants at the 2017 lowlands Health Economists’ Study Group
(lolaHESG) conference in Rotterdam, The Netherlands. This manu-
script was drafted during Susanne Mayer’s research stay at Maastricht
Author Contributions The initial idea for this study was first
brought up by SE and AP. SM developed the study concept, to which
input was provided by AP, JS, RD, DR and SE. All co-authors were
involved in the discussion of all steps of the review. SM conducted
the literature search, with AŁ as the second screener. SM analyzed the
data and wrote the manuscript, and all authors read and approved the
final version of this article.
Compliance with Ethical Standards
Conflicts of interest Susanne Mayer, Aggie T.G. Paulus, Agata
Łaszewska, Judit Simon, Ruben M.W.A. Drost, Dirk Ruwaard, Silvia
M.A.A. Evers declare that they have no conflicts of interest relevant
to the content of this article.
ICB Resource-Use Measurement Instruments
Funding No funding from external sources was received for this
Open Access This article is distributed under the terms of the
Creative Commons Attribution-NonCommercial 4.0 International
License (, which per-
mits any noncommercial use, distribution, and reproduction in any
medium, provided you give appropriate credit to the original
author(s) and the source, provide a link to the Creative Commons
license, and indicate if changes were made.
1. Weatherly H, Drummond M, Claxton K, et al. Methods for
assessing the cost-effectiveness of public health interventions:
key challenges and recommendations. Health Policy.
2. Drost RM, Paulus AT, Ruwaard D, Evers SM. Inter-sectoral costs
and benefits of mental health prevention: towards a new classi-
fication scheme. J Ment Health Policy Econ. 2013;16(4):179–86.
3. Krol M, Brouwer W, Rutten F. Productivity costs in economic
evaluations: past, present, future. Pharmacoeconomics.
4. Krol M, Papenburg J, van Exel J. Does including informal care in
economic evaluations matter? A systematic review of inclusion
and impact of informal care in cost-effectiveness studies. Phar-
macoeconomics. 2015;33(2):123–35.
5. Byford S, Raftery J. Perspectives in economic evaluation. BMJ.
6. Drost RM, van der Putten IM, Ruwaard D, Evers SM, Paulus AT.
Conceptualizations of the societal perspective within economic
evaluations: systematic review. Int J Technol Assess Health Care
7. Claxton K, Walker S, Palmer S, Sculpher M. Appropriate per-
spectives for health care decisions. Working Papers no 054cherp,
Centre for Health Economics, University of York; 2010.
8. Knies S, Severens JL, Ament AJ, Evers SM. The transferability
of valuing lost productivity across jurisdictions. Differences
between national pharmacoeconomic guidelines. Value Health.
9. Sanders GD, Neumann PJ, Basu A, et al. Recommendations for
conduct, methodological practices, and reporting of cost-effec-
tiveness analyses: second panel on cost-effectiveness in health
and medicine. JAMA. 2016;316(10):1093–103.
10. Gold MR, Siegel JE, Russell LB, Weinstein MC. Cost-effec-
tiveness in health and medicine. New York: Oxford University
Press; 1996.
11. Drost RM, Paulus AT, Ruwaard D, Evers SM. Valuing inter-
sectoral costs and benefits of interventions in the healthcare
sector: methods for obtaining unit prices. Expert Rev Pharma-
coecon Outcomes Res. 2017;17(1):77–84.
12. Leggett LE, Khadaroo RG, Holroyd-Leduc J, et al. Measuring
resource utilization: a systematic review of validated self-re-
ported questionnaires. Medicine. 2016;95(10):e2759.
13. Evers SM, Hiligsmann M, Adarkwah C. Risk of bias in trial-
based economic evaluations: identification of sources and bias-
reducing strategies. Psychol Health. 2015;30(1):52–71.
14. DIRUM, Database of Instruments for Resource Use Measure-
ment. Accessed 28 Jul 2016.
15. Ridyard CH, Dyfrig A, DIRUM Team. Development of a data-
base of instruments for resource-use measurement: purpose,
feasibility, and design. Value Health. 2012;15(5):650–5.
16. Noben CY, Evers SM, Nijhuis FJ, de Rijk AE. Quality appraisal
of generic self-reported instruments measuring health-related
productivity changes: a systematic review. BMC Public Health.
17. Tang K. Estimating productivity costs in health economic eval-
uations: a review of instruments and psychometric evidence.
Pharmacoeconomics. 2015;33(1):31–48.
18. Tanco K, Park JC, Cerana A, et al. A systematic review of
instruments assessing dimensions of distress among caregivers of
adult and pediatric cancer patients. Palliat Supp Care.
19. Hoefman RJ, van Exel J, Brouwer W. How to include informal
care in economic evaluations. Pharmacoeconomics.
20. van Durme T, Macq J, Jeanmart C, Gobert M. Tools for mea-
suring the impact of informal caregiving of the elderly: a litera-
ture review. Int J Nurs Stud. 2012;49(4):490–504.
21. Thorn JC, Coast J, Cohen D, et al. Resource-use measurement
based on patient recall: issues and challenges for economic
evaluation. Appl Health Econ Health Policy. 2013;11(3):155–61.
22. Glanville J, Paisley S. Identifying economic evaluations for
health technology assessment. Int J Technol Assess Health Care.
23. Alton V, Eckerlund I, Norlund A. Health economic evaluations:
how to find them. Int J Technol Assess Health Care.
24. Akers J, Aguiar-Iba
˜ez R, Baba-Akbari Sari A. CRD’s guidance
for undertaking reviews in health care. York: Centre for Reviews
and Dissemination (CRD); 2009.
25. Streiner DL, Norman GR, Cairney J. Health measurement scales:
a practical guide to their development and use. Oxford: Oxford
University Press; 2014.
26. Higgins JP, Green S. Cochrane Handbook for Systematic
Reviews of Interventions. Version 5.1.0 2011. http://handbook. Accessed 21 June 2016.
27. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group.
Preferred reporting items for systematic reviews and meta-anal-
yses: the PRISMA statement. Ann Intern Med.
28. Bodden DH, Dirksen CD, Bogels SM. Societal burden of clini-
cally anxious youth referred for treatment: a cost-of-illness study.
J Abnorm Child Psychol. 2008;36(4):487–97.
29. Beecham J, Knapp M. Costing psychiatric interventions. In:
Thornicroft G, editor. Measuring mental health needs. London:
Gaskell; 2001. p. 200–24.
30. Watson JM, Crosby H, Dale VM, et al. AESOPS: a randomised
controlled trial of the clinical effectiveness and cost-effectiveness
of opportunistic screening and stepped care interventions for
older hazardous alcohol users in primary care. Health Technol
Assess. 2013;17(25):1–158.
31. Client Service Receipt Inventory Children’s Version. http://www. Accessed 1 Aug 2017.
32. CSRI-CD, Client Service Receipt Inventory for Children with
Diabetes. Acces-
sed 1 Aug 2017.
33. CSRI-Pain, Client Service Receipt Inventory for Adolescents
with Chronic Pain—Parent report.
csri/chronic-pain-csri/. Accessed 1 Aug 2017.
34. Powell CV, Kolamunnage-Dona R, Lowe J, et al. MAGNEsium
Trial In Children (MAGNETIC): a randomised, placebo-con-
trolled trial and economic evaluation of nebulised magnesium
sulphate in acute severe asthma in children. Health Technol
Assess. 2013;17(45):1–216.
35. Lock C, Wilson J, Steen N, et al. North of England and Scotland
Study of Tonsillectomy and Adeno-tonsillectomy in Children
(NESSTAC): a pragmatic randomised controlled trial with a
S. Mayer et al.
parallel non-randomised preference study. Health Technol
Assess. 2010;14(13):1–164, iii–iv.
36. Parrott S, Godfrey C, Heather N, Clark J, Ryan T. Cost and
outcome analysis of two alcohol detoxification services. Alcohol
Alcohol. 2006;41(1):84–91.
37. DIRUM: SCIPI RUM: instrument qualities 2016. http:// Accessed 1 Aug 2017.
38. SHIFT, Self-Harm Intervention, Family Therapy (SHIFT), Parent
or Carers Questionnaire Booklet.
Carers%20Questionnaire%20Booklet.pdf. Accessed 1 Aug
39. SHIFT, Self-Harm Intervention, Family Therapy (SHIFT),
Young Person Questionnaire Booklet.
Person%20Questionnaire%20Booklet.pdf. Accessed 1 Aug
40. Barrett B, Byford S. Collecting service use data for economic
evaluation in DSPD populations: development of the Secure
Facilities Service Use Schedule. Br J Psychiatry Suppl.
41. SYCAMORE CSRI Questionnaire.
naire.pdf. Accessed 1 Aug 2017.
42. Chisholm D, Knapp MR, Knudsen HC, et al. Client Socio-De-
mographic and Service Receipt Inventory-European Version:
development of an instrument for international research. EPSI-
LON Study 5. European Psychiatric Services: Inputs Linked to
Outcome Domains and Needs. Br J Psychiatry Suppl.
43. Burns BJ, Angold A, Magruder-Habib K, et al. The child and
adolescent services assessment (CASA) Child Interview Version
5.0. Accessed 1 Aug 2017.
44. Sommers BD, Fargo JD, Lyons MS, Shope JT, Sommers MS.
Societal costs of risky driving: an economic analysis of high-risk
patients visiting an urban emergency department. Traffic Inj Prev.
45. Wetterneck CT, Woods DW, Norberg MM, Begotka AM. The
social and economic impact of trichotillomania: results from two
nonreferred samples. Behav Intervent. 2006;21(2):97–109.
46. Zhou ZY, Koerper MA, Johnson KA, et al. Burden of illness:
direct and indirect costs among persons with hemophilia A in the
United States. J Med Econ. 2015;18(6):457–65.
47. WPAI ?CIQ:AS. Work Productivity and Activity Impairment
Questionnaire plus Classroom Impairment Questions: Allergy
Specific, Version 2.
html. Accessed 1 Aug 2017.
48. Connelly LB, Webber J, Rowell D. Cost of Trauma Instrument
(COTI), 2005.
tools/cost-of-trauma-instrument-coti. Accessed 1 Aug 2017.
49. Genereaux D, van Karnebeek CD, Birch PH. Costs of caring for
children with an intellectual developmental disorder. Disabil
Health J. 2015;8(4):646–51.
50. Browne G, Gafni A, Roberts J, Goldsmith A, Jamieson E.
Approach to the measurement of costs (expenditures) when
evaluating health and social programmes. Hamilton: McMaster
University; 1995.
51. Huy R, Wichmann O, Beatty M, et al. Cost of dengue and other
febrile illnesses to households in rural Cambodia: a prospective
community-based case-control study. BMC Public Health.
52. KNCV Tuberculosis Foundation. The Tool to Estimate Patients’
Costs, 2008.
supplementary/1471-2458-11-43-S1.PDF. Accessed 1 Aug 2017.
53. Aygoren-Pursun E, Bygum A, Beusterien K, et al. Socioeco-
nomic burden of hereditary angioedema: results from the
hereditary angioedema burden of illness study in Europe.
Orphanet J Rare Dis. 2014;9:99.
54. Blacken PH, Pozzi G, Tempesta E, et al. European Addiction
Severity Index (EuropASI) Cost A6: A guide to training and
administering EuropASI interviews 1994. http://www.emcdda. Accessed 1 Aug 2017.
55. SCIPI RUM, SubCutaneous Insulin: Pumps or Injections (SCIPI)
RUM, 2011.
634968699228637202-SCIPI_Questionnaire.pdf. Accessed 1
Aug 2017.
56. Diary SC. Syncamore Treatment Diary.
Diary.pdf. Accessed 1 Aug 2017.
57. Farmer EM, Angold A, Burns BJ, Costello EJ. Reliability of self-
reported service use: test-retest consistency of children’s
responses to the Child and Adolescent Services Assessment
(CASA). J Child Fam Stud. 1994;3(3):307–25.
58. Ascher BH, Farmer EM, Burns BJ, Angold A. The child and
adolescent services assessment (CASA) description and psycho-
metrics. J Emot Behav Disord. 1996;4(1):12–20.
59. Reilly MC, Zbrozek AS, Dukes EM. The validity and repro-
ducibility of a work productivity and activity impairment
instrument. Pharmacoeconomics. 1993;4(5):353–65.
60. Reilly MC, Tanner A, Meltzer EO. Work, classroom and activity
impairment instruments: validation studies in allergic rhinitis.
Clin Drug Investig. 1996;11(5):278–88.
61. Ten Jo
¨nsson B. arguments for a societal perspective in the eco-
nomic evaluation of medical innovations. Eur J Health Econ.
62. Thorn JC, Ridyard CH, Riley R, et al. Identification of items for a
standardised resource-use measure: review of current instru-
ments. Value Health. 2015;18(7):A688.
63. Ridyard CH, Hughes DA. Methods for the collection of resource
use data within clinical trials: a systematic review of studies
funded by the UK Health Technology Assessment program.
Value Health. 2010;13(8):867–72.
64. Ridyard CH, Hughes DA, DIRUM Team. Taxonomy for methods
of resource use measurement. Health Econ. 2015;24(3):372–8.
65. Johnston K, Buxton MJ, Jones DR, Fitzpatrick R. Assessing the
costs of healthcare technologies in clinical trials. Health Technol
Assess. 1999;3(6):1–76.
66. Schellingerhout JM, Verhagen AP, Heymans MW, et al. Mea-
surement properties of disease-specific questionnaires in patients
with neck pain: a systematic review. Qual Life Res.
67. Ladd HF, Edward EB. Handbook of research in education finance
and policy. Abingdon: Routledge; 2012.
68. Terwee CB, Mokkink LB, Knol DL, et al. Rating the method-
ological quality in systematic reviews of studies on measurement
properties: a scoring system for the COSMIN checklist. Qual Life
Res. 2012;21(4):651–7.
69. van Ballegooijen W, Riper H, Cuijpers P, van Oppen P, Smit JH.
Validation of online psychometric instruments for common
mental health disorders: a systematic review. BMC Psychiatry.
70. Goodrich K, Kaambwa B, Al-Janabi H. The inclusion of informal
care in applied economic evaluation: a review. Value Health.
71. Ridyard CH, Hughes DA. Review of resource-use measures in
UK economic evaluations. In: Curtis L, Burns A, editors. Unit
Costs of Health and Social Care 2015. Canterbury: Personal
Social Services Research Unit, University of Kent; 2015.
p. 22–30.
72. Mayer S, Kiss N, Laszewska A, Simon J. Health economic
costing methods and reporting in Austria. Value Health.
ICB Resource-Use Measurement Instruments
73. Patel A, Rendu A, Moran P, et al. A comparison of two methods
of collecting economic data in primary care. Fam Pract.
74. Byford S, Leese M, Knapp M, et al. Comparison of alternative
methods of collection of service use data for the economic
evaluation of health care interventions. Health Econ.
75. Mirandola M, Bisoffi G, Bonizzato P, Amaddeo F. Collecting
psychiatric resources utilisation data to calculate costs of care: a
comparison between a service receipt interview and a case reg-
ister. Soc Psychiatry Psychiatr Epidemiol. 1999;34(10):541–7.
76. Heinrich S, Deister A, Birker T, et al. Accuracy of self-reports of
mental health care utilization and calculated costs compared to
hospital records. Psychiatry Res. 2011;185(1–2):261–8.
77. Kilian R, Roick C, Bernert S, et al. Instruments for the eco-
nomical evaluation of psychiatric service systems: methodologi-
cal foundations of the european standardisation and the German
adaptation [in German]. Psychiatr Prax. 2001;28(Suppl 2):74–8.
78. Browne GB, Arpin K, Corey P, Fitch M, Gafni A. Individual
correlates of health service utilization and the cost of poor
adjustment to chronic illness. Med Care. 1990;28(1):43–58.
79. Hendriks VM, Kaplan CD, van Limbeek J, Geerlings P. The
Addiction Severity Index: reliability and validity in a Dutch
addict population. J Subst Abuse Treat. 1989;6(2):133–41.
80. Dijkgraaf MG, van der Zanden BP, de Borgie CA, et al. Cost
utility analysis of co-prescribed heroin compared with methadone
maintenance treatment in heroin addicts in two randomised trials.
BMJ. 2005;330(7503):1297–300.
81. Knapp M, Scott S, Davies J. The cost of antisocial behaviour in
younger children. Clin Child Psychol Psychiatry. 1999;4(4):
82. Sleed M, Eccleston C, Beecham J, Knapp M, Jordan A. The
economic impact of chronic pain in adolescence: methodological
considerations and a preliminary cost-of-illness study. Pain.
83. Mauch V, Woods N, Kirubi B, et al. Assessing access barriers to
tuberculosis care with the tool to Estimate Patients’ Costs: pilot
results from two districts in Kenya. BMC Public Health.
84. UKATT Research Team. Cost effectiveness of treatment for
alcohol problems: findings of the randomised UK alcohol treat-
ment trial (UKATT). BMJ. 2005;331(7516):544.
85. STEPWICE Research Group. The effectiveness and cost-effec-
tiveness of screening and stepped care interventions for alcohol
use disorders in the primary care setting. Cardiff: Welsh Office of
Research & Development; 2003.
86. Coulton S, Watson J, Bland M, et al. The effectiveness and cost-
effectiveness of opportunistic screening and stepped care inter-
ventions for older hazardous alcohol users in primary care
(AESOPS)—a randomised control trial protocol. BMC Health
Serv Res. 2008;8:129.
87. Stade B, Ali A, Bennett D, et al. The burden of prenatal exposure
to alcohol: revised measurement of cost. Can J Clin Pharmacol.
88. Ramanan AV, Dick AD, Benton D, et al. A randomised con-
trolled trial of the clinical effectiveness, SafetY and cost effec-
tiveness of adalimumab in combination with MethOtRExate for
the treatment of juvenile idiopathic arthritis associated uveitis
(SYCAMORE). Trials. 2014;9:14.
89. Bowling A. Mode of questionnaire administration can have
serious effects on data quality. J Public Health.
S. Mayer et al.

Supplementary resource (1)

... A workgroup consisting of seven health economists (LJ, SE, AP, CD, WH, SN, JT) brainstormed to identify existing RUM aspects in health economics, i.e. all concepts that are part of the complex phenomenon of RUM. Afterwards, the four methodological studies [3,4,20,21] on RUM, familiar with the workgroup, were read full-text by one researcher (LJ) to validate the identified RUM aspects and to complement the list with other relevant RUM aspects. The RUM aspects identified during the expert meeting and the RUM aspects identified in the literature provided the input for the draft framework, developed in the next phase. ...
... First, a structured literature search was conducted. The search scopes of four existing systematic reviews addressing RUM that were identified by the experts in Phase 1 were used as a basis for the current literature search [3,4,20,21]. Databases were chosen based on the scope and topic coverage and, as RUM is a relatively new and underexplored area of research, multiple databases were chosen. ...
... The face-to-face expert meeting and four systematic reviews addressing RUM [3,4,20,21] provided the input for the draft framework. Clustering of the RUM aspects resulted in a framework with six main methodological RUM domains and corresponding aspects (ESM 3): (1) What to measure? ...
Full-text available
Background While the methods for conducting health economics research in general are improving, current guidelines provide limited guidance regarding resource use measurement (RUM). Consequently, a variety of methods exists, yet there is no overview of aspects to consider when deciding on the most appropriate RUM methodology. Therefore, this study aims to (1) identify and categorize existing knowledge regarding aspects of RUM, and (2) develop a framework that provides a comprehensive overview of methodological aspects regarding RUM.Methods Relevant articles were identified by enrolling a search string in six databases and handsearching the DIRUM database. Included articles were descriptively reviewed and served as input for a comprehensive framework. Health economics experts were involved during the process to establish the framework’s face validity.ResultsForty articles were included in the scoping review. The RUM framework consists of four methodological RUM domains: ‘Whom to measure’, addressing whom to ask and whom to measure; ‘How to measure’, addressing the different approaches of measurement; ‘How often to measure’, addressing recall period and measurement patterns; and ‘Additional considerations’, which covers additional aspects that are essential for further refining the methodologies for measurement. Evidence retrieved from the scoping review was categorized according to these domains.Conclusion This study clustered the aspects of RUM methodology in health economics into a comprehensive framework. The results may guide health economists in their decision making regarding the selection of appropriate RUM methods and developing instruments for RUM. Furthermore, policy makers may use these findings to review study results from an evidence-based perspective.
... The inclusion of education and criminal justice ICBs in economic evaluations is supported by several national pharmacoeconomic guidelines [14][15][16]. Furthermore, several studies investigated the identification, measurement, and valuation of education and criminal justice ICBs in health economics research [10,17,18]. Nevertheless, to date, few economic evaluations incorporate these ICBs, even though they might be relevant to the study context [19,20]. ...
... Although an overview of relevant education and criminal justice ICBs does exist [10], it does not provide guidance in terms of which ICBs are the most important to include in economic evaluations. Mayer et al. [18] developed a list of the most common education and criminal justice ICBs, based on the existing health-related resource-use measurement instruments. While this list could help us to gain insight into the importance of these ICBs, it does not provide a ranking. ...
... A mutually exclusive attribute list is essential for obtaining valid outcomes in BWS analysis [23]. First, potential attributes were extracted by one researcher (LB) from earlier research on ICBs [10,18] and resource-use measurement instruments, via a hand search of the Database of Instruments for Resource Use Measurement (DIRUM) [24]. Second, the attributes were compiled and clustered based on similarity using the classification of Drost et al. [10] as a reference. ...
Full-text available
Objectives Mental and behavioural disorders (MBDs) and interventions targeting MBDs lead to costs and cost savings in the healthcare sector, but also in other sectors. The latter are referred to as intersectoral costs and benefits (ICBs). Interventions targeting MBDs often lead to ICBs in the education and criminal justice sectors, yet these are rarely included in economic evaluations. This study aimed to investigate the attitudes held by health economists and health technology assessment experts towards education and criminal justice ICBs in economic evaluations and to quantify the relative importance of these ICBs in the context of MBDs. Methods An online survey containing open-ended questions and two best–worst scaling object case studies was conducted in order to prioritise a list of 20 education ICBs and 20 criminal justice ICBs. Mean relative importance scores for each ICB were generated using hierarchical Bayes analysis. Results Thirty-nine experts completed the survey. The majority of the respondents (68%) reported that ICBs were relevant, but only a few (32%) included them in economic evaluations. The most important education ICBs were “special education school attendance”, “absenteeism from school”, and “reduced school attainment”. The most important criminal justice ICBs were “decreased chance of committing a crime as a consequence/effect of mental health programmes/interventions”, “jail and prison expenditures”, and “long-term pain and suffering of victims/victimisation”. Conclusions This study identified the most important education and criminal justice ICBs for economic evaluations of interventions targeting MBDs and suggests that it could be relevant to include these ICBs in economic evaluations.
... Because inter-sectoral costs and consequences were excluded in economic studies of DPHP for a long time [5,19,28,29], several researchers highlighted the need to consider these costs [5,14,19,32,41]. However, the tools developed were heterogeneous and showed limited evidence on validity and reliability [42]. ...
... Although validated and well-accepted tools for the inclusion of these inter-sectoral costs are lacking, the adoption and consistent application of a societal perspective would stimulate efforts to include costs and effects beyond the health sector [42]. ...
Full-text available
Background We aimed to provide a comprehensive overview of methodological challenges in economic evaluations of disease prevention and health promotion (DPHP)-measures. Methods We conducted an overview of reviews searching MEDLINE, EMBASE, NHS Economic Evaluation Database, Database of Promoting Health Effectiveness Reviews, Cochrane Database of Systematic Reviews (CDSR) and Database of Promoting Health Effectiveness Reviews (DOPHER) (from their inception to October 2021). We included both systematic and scoping reviews of economic evaluations in DPHP addressing following methodological aspects: (i) attribution of effects, (ii) outcomes, (iii) inter-sectoral (accruing to non-health sectors of society) costs and consequences and (iv) equity. Data were extracted according to the associated sub-criteria of the four methodological aspects including study design economic evaluation (e.g. model-based), type/scope of the outcomes (e.g. outcomes beyond health), perspective, cost categories related to non-health sectors of society, and consideration of equity (method of inclusion). Two reviewers independently screened all citations, full-text articles, and extracted data. A narrative synthesis without a meta-analysis or other statistical synthesis methods was conducted. Results The reviewing process resulted in ten systematic and one scoping review summarizing 494 health economic evaluations. A lifelong time horizon was adopted in about 23% of DPHP evaluations, while 64% of trial-based evaluations had a time horizon up to 2 years. Preference-based outcomes (36%) and non-health outcomes (8%) were only applied in a minority of studies. Although the inclusion of inter-sectoral costs (i.e. costs accruing to non-health sectors of society) has increased in recent years, these were often neglected (between 6 and 23% depending on the cost category). Consideration to equity was barely given in economic evaluations, and only addressed in six of the eleven reviews. Conclusions Economic evaluations of DPHP measures give only little attention to the specific methodological challenges related to this area. For future economic DPHP evaluations a tool with structured guidance should be developed. This overview of reviews was not registered and a published protocol does not exist.
... Despite recent methodological developments in identifying, measuring, and valuing education costs [12][13][14][15][16], their inclusion in costing research remains limited [17]. This could be attributed to several reasons. ...
... It is important to note that the costs associated with reduced school attainment become more apparent in the longer term, which makes it more challenging to capture these costs in studies with shorter time horizons of 1 or 2 years. In addition, the most frequently included education cost items, "special education," "absenteeism from school," and "additional educational support (e.g., tutoring, homework help)," correspond with the findings of the recent review of health-related resource-use measurement instruments by Mayer et al. [13]. However, less tangible resource-use items such as "school functioning" and "social functioning" were also not included in the reviewed studies. ...
Full-text available
Background Psychosocial (e.g., anxiety or behavior) problems lead to costs not only in the healthcare sector but also in education and other sectors. As psychosocial problems develop during the critical period of establishing educational trajectories, education costs are particularly relevant in the context of psychosocial problems among children and adolescents.Objectives This study aimed to gain insights into the methods used for the inclusion of education costs in health economics studies and into the proportion of the education costs in relation to the total costs associated with a condition or an intervention.Methods We systematically searched the PubMed, Embase, SSCI, CINAHL, PsycINFO, ERIC, and Econlit databases in August 2019 for economic evaluations of mental health, psychosocial and educational interventions, and cost-of-illness studies of mental, behavioral, and neurodevelopmental disorders conducted from a societal perspective in populations of children and adolescents. An additional search was conducted in February 2021 to update the review.ResultsIn total, 49 articles were included in the analysis. The most common cost items were special education, school absenteeism, and various educational professionals (educational psychologist). A variety of methods were employed for the identification, measurement, and/or valuation of education costs. The proportion of education costs to the total costs of condition/intervention ranged from 0 to 67%, with the mean being 18.5%.DiscussionSince education costs can constitute a significant proportion of the total costs of an intervention or condition, including them in health economics studies might be important in informing optimal resource allocation decisions. Although various methods are available for including education costs in health economics studies, further research is needed to develop evidence-based methods for producing comparable estimates.
... Analysts should attempt to quantify in the impact inventory as many relevant costs and outcomes as possible within time and resources available, unless these are likely to have a negligible effect on the result of the analysis (Hill et al., 2017;Sanders et al., 2016). Because criminal justice service resource use is particularly important in the field of alcoholism (Mayer et al., 2017), it would be advisable for future trial-based economic evaluations of alcohol interventions to consider the routine adoption of resource-use measurement instruments to measure impacts falling on criminal justice system. Lists of resource-use measurement instruments in the criminal justice sector have been recently proposed (Drost et al., 2013;Mayer et al., 2017). ...
... Because criminal justice service resource use is particularly important in the field of alcoholism (Mayer et al., 2017), it would be advisable for future trial-based economic evaluations of alcohol interventions to consider the routine adoption of resource-use measurement instruments to measure impacts falling on criminal justice system. Lists of resource-use measurement instruments in the criminal justice sector have been recently proposed (Drost et al., 2013;Mayer et al., 2017). Unit costs estimates for the criminal justice system are available for the United Kingdom (Brand & Price, 2000;Dubourg et al., 2005;Newton, May, Eames, & Ahmed, 2019). ...
Full-text available
Cost-effectiveness analyses of health care programmes often focus on maximising health and ignore non-health impacts. Assessing the cost-effectiveness of public health interventions from a narrow health care perspective would likely underestimate their full impact, and potentially lead to inefficient decisions about funding. The aim of this study is to provide a practical application of a recently proposed framework for the economic evaluation of public health interventions, evaluating an intervention to reduce alcohol misuse in criminal offenders. This cross-sectoral analysis distinguishes benefits and opportunity costs for different sectors, makes explicit the value judgements required to consider alternative perspectives, and can inform heterogeneous decision makers with different objectives in a transparent manner. Three interventions of increasing intensity are compared: client information leaflet; brief advice; brief lifestyle counselling. Health outcomes are measured in quality-adjusted life-years and criminal justice outcomes in re-convictions. Costs considered include intervention costs, costs to the NHS and costs to the criminal justice system. The results are presented for four different perspectives: ‘narrow’ health care perspective; criminal justice system perspective; ‘full’ health care perspective; and joint ‘full’ health and criminal justice perspective. Conclusions and recommendations differ according to the normative judgement on the appropriate perspective for the evaluation.
... Although there is substantial uncertainty about the exact incremental cost-effectiveness of CTOs compared to non-CTOs, none of the sensitivity analyses changed the conclusion that CTOs are unlikely to be cost-effective. In line with the increasing consideration of so-called inter-sectoral costs and benefits of health care services, programmes and interventions [45], however, it is vital that the findings on the increased informal care, legal procedure burdens, and other indirect costs of CTOs are taken into consideration in future decision-making. ...
Full-text available
PurposeCurrent RCT and meta-analyses have not found any effect of community treatment orders (CTOs) on hospital or social outcomes. Assumed positive impacts of CTOs on quality-of-life outcomes and reduced hospital costs are potentially in conflict with patient autonomy. Therefore, an analysis of the cost and quality-of-life consequences of CTOs was conducted within the OCTET trial.Methods The economic evaluation was carried out comparing patients (n = 328) with psychosis discharged from involuntary hospitalisation either to treatment under a CTO (CTO group) or voluntary status via Section 17 leave (non-CTO group) from the health and social care and broader societal perspectives (including cost implication of informal family care and legal procedures). Differences in costs and outcomes defined as quality-adjusted life years (QALYs) based on the EQ-5D-3L or capability-weighted life years (CWLYs) based on the OxCAP-MH were assessed over 12 months (£, 2012/13 tariffs).ResultsMean total costs from the health and social care perspective [CTO: £35,595 (SD: £44,886); non-CTO: £36,003 (SD: £41,406)] were not statistically significantly different in any of the analyses or cost categories. Mental health hospitalisation costs contributed to more than 85% of annual health and social care costs. Informal care costs were significantly higher in the CTO group, in which there were also significantly more manager hearings and tribunals. No difference in health-related quality of life or capability wellbeing was found between the groups.ConclusionCTOs are unlikely to be cost-effective. No evidence supports the hypothesis that CTOs decrease hospitalisation costs or improve quality of life. Future decisions should consider impacts outside the healthcare sector such as higher informal care costs and legal procedure burden of CTOs.
... Bislang wurden die dafür in der Praxis verwendeten Instrumente jedoch noch nicht systematisch erhoben und analysiert. Auf Basis einer umfassenden systematischen Literaturrecherche hat Dr. in Mayer erstmals einen internationalen Überblick über die in der Praxis verwendeten Messinstrumente geschaffen [1]. Über 25 Messinstrumente konnten so identifiziert werden, die insbesondere die Auswirkungen gesundheitsbezogener Interventionen auf Aspekte wie Schulabwesenheit, Bedarf an Förderungsaktivitäten oder Assistenz im Klassenraum bzw. ...
... For instance, a child with autism spectrum disorder (ASD) may require a home-based special educator or early intervention therapies in early childhood [23], special education during school years [24] and employment support or vocational training during the transition into adulthood [25]. Consequently, compiling and quantifying service usage in monetary units for an economic evaluation can be challenging [26]. While adopting a narrower public health-care system perspective might be an option, failure to incorporate other relevant sources and settings may bias the results when the majority of service usage is outside the healthcare system. ...
Full-text available
Economic evaluation is a tool used to inform decision makers on the efficiency of comparative healthcare interventions and inform resource allocation decisions. There is a growing need for the use of economic evaluations to assess the value of interventions for children with neurodevelopmental disorders (NDDs), a population that has increasing demands for healthcare services. Unfortunately, few evaluations have been conducted to date, perhaps stemming from challenges in applying existing economic evaluation methodologies in this heterogeneous population. Opportunities exist to innovate methods to address key challenges in conducting economic evaluations of interventions for children with NDDs. In this paper, we discuss important considerations and highlight areas for future work. This includes the paucity of appropriate instruments for measuring outcomes meaningful to children with NDDs and their families, difficulties in the measurement of costs due to service utilization in a wide variety of sectors, complexities in the measurement of caregiver and family effects and considerations in estimating long-term productivity costs. Innovation and application of evaluation approaches in these areas will help inform decisions around whether the resources currently spent on interventions for children with NDDs represent good value for money, or whether greater benefits for children could be generated by spending money in other ways.
Full-text available
Zusammenfassung Dem Dreischritt „Status quo – Konsequenzen – Maßnahmen“ folgend werden Daten zum Ausmaß körperlicher Aktivität und deren Einflussfaktoren, mögliche gesellschaftliche Kosten mangelnder körperlicher Aktivität sowie Einflussebenen und Maßnahmen zur Bewegungsförderung in Österreich präsentiert. In Österreich ist der Anteil an Personen, die die Bewegungsempfehlungen nicht erfüllen, hoch und es besteht diesbezüglich klar Verbesserungspotenzial, insbesondere in Bezug auf muskelkräftigende Aktivitäten bei Erwachsenen. Je nach Datenquelle erfüllen von den Erwachsenen 42–50% die Empfehlungen für ausdauerorientierte Bewegung und 18–33% die für muskelkräftigende Aktivitäten. Gleichzeitig ist die österreichische Bevölkerung im internationalen Vergleich aktiver als jene vieler anderer Länder. Große Unterschiede bestehen im Bewegungsverhalten hinsichtlich demografischer, sozioökonomischer und geografischer Determinanten. Die gesellschaftlichen Folgen durch mangelnde körperliche Aktivität sind beträchtlich, sowohl hinsichtlich verlorener Lebensjahre durch vorzeitigen Tod, als auch den Kosten, die der Allgemeinheit entstehen, beispielsweise an Gesundheitskosten und Produktivitätsverlusten. Die für das Jahr 2017 hochgerechnete Summe von 248 Mio. Euro stellt aufgrund methodischer Schwierigkeiten wahrscheinlich eine Unterschätzung dar. Wissenschaftlich abgesicherte Einflussebenen auf das Bewegungsverhalten umfassen im Sinne der lebens(um)weltortientierten Herangehensweise die Lebenswelten Kindergarten und Schule, Betriebe, Gemeinde/Städte, Pflegewohnheime und die Natur.
Full-text available
Background Mental health disorders and their treatments produce significant costs and benefits in both healthcare and non-healthcare sectors. The latter are often referred to as intersectoral costs and benefits (ICBs). Little is known about healthcare-related ICBs in the criminal justice sector and how to include these in health economics research. Objectives The triple aim of this study is (i) to identify healthcare-related ICBs in the criminal justice sector, (ii) to validate the list of healthcare-related ICBs in the criminal justice sector on a European level by sector-specific experts, and (iii) to classify the identified ICBs. Methods A scientific literature search in PubMed and an additional grey literature search, carried out in six European countries, were used to retrieve ICBs. In order to validate the international applicability of the ICBs, a survey was conducted with an international group of experts from the criminal justice sector. The list of criminal justice ICBs was categorized according to the PECUNIA conceptual framework. Results The full-text analysis of forty-five peer-reviewed journal articles and eleven grey literature sources resulted in a draft list of items. Input from the expert survey resulted in a final list of fourteen unique criminal justice ICBs, categorized according to the care atom. Conclusion This study laid further foundations for the inclusion of important societal costs of mental health-related interventions within the criminal justice sector. More research is needed to facilitate the further and increased inclusion of ICBs in health economics research.
Full-text available
Objectives: The aim of this study was to investigate how the societal perspective is conceptualized in economic evaluations and to assess how intersectoral costs and benefits (ICBs), that is, the costs and benefits pertaining to sectors outside the healthcare sector, impact their results. Methods: Based on a search in July 2015 using PubMed, Embase, CINAHL, and PsychINFO, a systematic literature review was performed for economic evaluations which were conducted from a societal perspective. Conceptualizations were assessed in NVivo version 11 using conventional and directed content analysis. Trial-based evaluations in the fields of musculoskeletal and mental disorders were analyzed further, focusing on the way ICBs impact the results of economic evaluations. Results: A total of 107 studies were assessed, of which 74 (69.1 percent) provided conceptualizations of the societal perspective. These varied in types of costs included and in descriptions of cost bearers. Labor productivity costs were included in seventy-two studies (67.3 percent), while only thirty-eight studies (35.5 percent) included other ICBs, most of which entailed informal care and/or social care costs. ICBs within the educational and criminal justice sectors were each included five times. Most of the trial-based evaluations analyzed further ( n = 21 of 28) reported productivity costs. In nine, these took up more than 50 percent of total costs. In several studies, criminal justice and informal care costs were also important. Conclusions: There is great variety in the way the societal perspective is conceptualized and interpreted within economic evaluations. Use of the term “societal perspective” is often related to including merely productivity costs, while other ICBs could be relevant as well.
Full-text available
A variety of methods may be used to obtain costing data. Although administrative data are most commonly used, the data available in these datasets are often limited. An alternative method of obtaining costing is through self-reported questionnaires. Currently, there are no systematic reviews that summarize self-reported resource utilization instruments from the published literature. The aim of the study was to identify validated self-report healthcare resource use instruments and to map their attributes. A systematic review was conducted. The search identified articles using terms like “healthcare utilization” and “questionnaire.” All abstracts and full texts were considered in duplicate. For inclusion, studies had to assess the validity of a self-reported resource use questionnaire, to report original data, include adult populations, and the questionnaire had to be publically available. Data such as type of resource utilization assessed by each questionnaire, and validation findings were extracted from each study. In all, 2343 unique citations were retrieved; 2297 were excluded during abstract review. Forty-six studies were reviewed in full text, and 15 studies were included in this systematic review. Six assessed resource utilization of patients with chronic conditions; 5 assessed mental health service utilization; 3 assessed resource utilization by a general population; and 1 assessed utilization in older populations. The most frequently measured resources included visits to general practitioners and inpatient stays; nonmedical resources were least frequently measured. Self-reported questionnaires on resource utilization had good agreement with administrative data, although, visits to general practitioners, outpatient days, and nurse visits had poorer agreement. Self-reported questionnaires are a valid method of collecting data on healthcare resource utilization.
Full-text available
Background: Online questionnaires for measuring common mental health disorders such as depression and anxiety disorders are increasingly used. The psychometrics of several pen-and-paper questionnaires have been re-examined for online use and new online instruments have been developed and tested for validity as well. This study aims to review and synthesise the literature on this subject and provide a framework for future research. Methods: We searched Medline and PsycINFO for psychometric studies on online instruments for common mental health disorders and extracted the psychometric data. Studies were coded and assessed for quality by independent raters. Results: We included 56 studies on 62 online instruments. For common instruments such as the CES-D, MADRS-S and HADS there is mounting evidence for adequate psychometric properties. Further results are scattered over different instruments and different psychometric characteristics. Few studies included patient populations. Conclusions: We found at least one online measure for each of the included mental health disorders and symptoms. A small number of online questionnaires have been studied thoroughly. This study provides an overview of online instruments to refer to when choosing an instrument for assessing common mental health disorders online, and can structure future psychometric research. Keywords: Internet, Depression, Anxiety, Measurement, Psychometrics, Systematic review
Importance Since publication of the report by the Panel on Cost-Effectiveness in Health and Medicine in 1996, researchers have advanced the methods of cost-effectiveness analysis, and policy makers have experimented with its application. The need to deliver health care efficiently and the importance of using analytic techniques to understand the clinical and economic consequences of strategies to improve health have increased in recent years. Objective To review the state of the field and provide recommendations to improve the quality of cost-effectiveness analyses. The intended audiences include researchers, government policy makers, public health officials, health care administrators, payers, businesses, clinicians, patients, and consumers. Design In 2012, the Second Panel on Cost-Effectiveness in Health and Medicine was formed and included 2 co-chairs, 13 members, and 3 additional members of a leadership group. These members were selected on the basis of their experience in the field to provide broad expertise in the design, conduct, and use of cost-effectiveness analyses. Over the next 3.5 years, the panel developed recommendations by consensus. These recommendations were then reviewed by invited external reviewers and through a public posting process. Findings The concept of a “reference case” and a set of standard methodological practices that all cost-effectiveness analyses should follow to improve quality and comparability are recommended. All cost-effectiveness analyses should report 2 reference case analyses: one based on a health care sector perspective and another based on a societal perspective. The use of an “impact inventory,” which is a structured table that contains consequences (both inside and outside the formal health care sector), intended to clarify the scope and boundaries of the 2 reference case analyses is also recommended. This special communication reviews these recommendations and others concerning the estimation of the consequences of interventions, the valuation of health outcomes, and the reporting of cost-effectiveness analyses. Conclusions and Relevance The Second Panel reviewed the current status of the field of cost-effectiveness analysis and developed a new set of recommendations. Major changes include the recommendation to perform analyses from 2 reference case perspectives and to provide an impact inventory to clarify included consequences.
Objective: Caregivers of cancer patients face intense demands throughout the course of the disease, survivorship, and bereavement. Caregiver burden, needs, satisfaction, quality of life, and other significant areas of caregiving are not monitored regularly in the clinic setting, resulting in a need to address the availability and clinical effectiveness of cancer caregiver distress tools. This review aimed to determine the availability of cancer caregiver instruments, the variation of instruments between different domains of distress, and that between adult and pediatric cancer patient populations. Method: A literature search was conducted using various databases from 1937 to 2013. Original articles on instruments were extracted separately if not included in the original literature search. The instruments were divided into different areas of caregiver distress and into adult versus pediatric populations. Psychometric data were also evaluated. Results: A total of 5,541 articles were reviewed, and 135 articles (2.4%) were accepted based on our inclusion criteria. Some 59 instruments were identified, which fell into the following categories: burden (n = 26, 44%); satisfaction with healthcare delivery (n = 5, 8.5%); needs (n = 14, 23.7%); quality of life (n = 9, 15.3%); and other issues (n = 5, 8.5%). The median number of items was 29 (4-125): 20/59 instruments (33.9%) had ≤20 items; 13 (22%) had ≤20 items and were psychometrically sound, with 12 of these 13 (92.3%) being self-report questionnaires. There were 44 instruments (74.6%) that measured caregiver distress for adult cancer patients and 15 (25.4%) for caregivers of pediatric patients. Significance of results: There is a significant number of cancer caregiver instruments that are self-reported, concise, and psychometrically sound, which makes them attractive for further research into their clinical use, outcomes, and effectiveness.
Conference Paper
Background: Hereditary angioedema (HAE) due to C1 inhibitor deficiency is a rare but serious and potentially life-threatening disease marked by spontaneous, recurrent attacks of swelling. The study objective was to characterize direct and indirect resource utilization associated with HAE from the patient perspective in Europe. Methods: The study was conducted in Spain, Germany, and Denmark to assess the real-world experience of HAE via a cross-sectional survey of HAE patients, including direct and indirect resource utilization during and between attacks for patients and their caregivers over the past 6 months. A regression model examined predictors of medical resource utilization. Results: Overall, 164 patients had an attack in the past 6 months and were included in the analysis. The most significant predictor of medical resource utilization was the severity of the last attack (OR 2.6; p < 0.001). Among patients who sought medical care during the last attack (23%), more than half utilized the emergency department. The last attack prevented patients from their normal activities an average of 4-12 hours. Patient and caregiver absenteeism increased with attack severity and frequency. Among patients who were working or in school (n = 120), 72 provided work/school absenteeism data, resulting in an estimated 20 days missing from work/school on average per year; 51% (n = 84) indicated that HAE has hindered their career/educational advancement. Conclusion: HAE poses a considerable burden on patients and their families in terms of direct medical costs and indirect costs related to lost productivity. This burden is substantial at the time of attacks and in between attacks.