Decision-making criteria among national policymakers in five countries: a discrete choice experiment eliciting relative preferences for equity and efficiency.

Abstract

Worldwide, there is a need for formalization of the priority setting processes in health. Recent research has used the term multicriteria decision analysis for methods that systematically include preferences for both equity and efficiency. The present study compares decision-makers' preferences at the country level for a set of equity and efficiency criteria according to a multicriteria decision analysis framework.
Discrete choice experiments were conducted for Brazil, Cuba, Nepal, Norway, and Uganda. By using standardized methods, we elicited preferences for intervention attributes using a individual choice questionnaire. A multinomial logistic regression was applied to estimate the coefficients for all single-policy criteria, per country. Attributes were assigned to an equity group or to an efficiency group. After testing for scale variance, predicted probabilities for interventions with both types of attributes were compared across countries.
The Norway and Nepal groups showed considerable preferences for efficiency criteria over equity criteria with percent change in respective predicted sum probabilities of [10%, -84%] and [6%, -79%]. Brazil and Uganda also showed preference for the efficiency criteria though less convincingly ([-34%, -93%], [-18%, -63%], respectively). The Cuban group showed the strongest preferences with equity attributes dominating efficiency ([-52%, 213%]).
Group preferences of policymakers show explicit but varying trade-offs of efficiency and equity in these diverse settings. This multicriteria decision analysis approach, using discrete choice experiments, indicates that systematic setting of health priorities is possible across a variety of countries. It may be a valuable tool to guide health reform initiatives.

Full text

Page 1

Health Policy Analysis
Decision-Making Criteria among National Policymakers in Five
Countries: A Discrete Choice Experiment Eliciting Relative
Preferences for Equity and Efficiency
Andrew Mirelman, MPH1,*, Emmanouil Mentzakis, PhD2, Elizabeth Kinter, MHS, PhD1,3, Francesco Paolucci, PhD4,
Richard Fordham, PhD2, Sachiko Ozawa, MHS, PhD1, Marcos Ferraz, MD, PhD5, Rob Baltussen, PhD6,
Louis W. Niessen, MD, Reg PH, PhD1,2
1Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA;2Norwich Medical School, University of East Anglia, Norwich, United Kingdom;3Campbell
Alliance, Boston, MA, USA;4Australian Centre for Economics Research on Health, The Australian National University, Canberra, Australia;5São Paulo Center for
Health Economics, Federal University of São Paulo, São Paolo, Brazil;6Department of Primary and Community Care, Radboud University, Nijmegen Medical Center,
Nijmegen, The Netherlands
A B S T R A C T
Background: Worldwide, there is a need for formalization of the pri-
ority setting processes in health. Recent research has used the term
multicriteria decision analysis for methods that systematically include
preferencesforbothequityandefficiency.Thepresentstudycompares
decision-makers’preferencesatthecountrylevelforasetofequityand
efficiency criteria according to a multicriteria decision analysis
framework. Methods: Discrete choice experiments were conducted
for Brazil, Cuba, Nepal, Norway, and Uganda. By using standardized
methods, we elicited preferences for intervention attributes using a
individual choice questionnaire. A multinomial logistic regression was
applied to estimate the coefficients for all single-policy criteria, per
country. Attributes were assigned to an equity group or to an efficiency
group. After testing for scale variance, predicted probabilities for inter-
ventions with both types of attributes were compared across countries.
Results: The Norway and Nepal groups showed considerable prefer-
ences for efficiency criteria over equity criteria with percent change in
respective predicted sum probabilities of [10%, ?84%] and [6%, ?79%].
Brazil and Uganda also showed preference for the efficiency criteria
though less convincingly ([?34%, ?93%], [?18%, ?63%], respectively).
The Cuban group showed the strongest preferences with equity attri-
butes dominating efficiency ([?52%, 213%]). Conclusions: Group pref-
erences of policymakers show explicit but varying trade-offs of effi-
ciency and equity in these diverse settings. This multicriteria decision
analysis approach, using discrete choice experiments, indicates that
systematic setting of health priorities is possible across a variety of
countries. It may be a valuable tool to guide health reform initiatives.
Keywords: discrete choice experiment, efficiency, equity, priority set-
ting.
Copyright © 2012, International Society for Pharmacoeconomics and
Outcomes Research (ISPOR). Published by Elsevier Inc.
Introduction
Worldwide, there is a need for formalization of health priority
setting processes at both the national and local levels [1–5]. Too
often policy decisions are made without transparency of decision-
making criteria, but with implicit trade-offs. Some countries have
taken steps toward making the process more transparent such as
in the United Kingdom where the coalition government’s program
on “Freedom, Fairness and Responsibility” reconfirmed the lead-
ing role of the National Institute for Health and Clinical Excellence
(NICE) to support value-based decisions in health care [6]. With its
15-year-old advanced health priority setting system, which takes
an efficiency-based, extra-welfarist approach, the United King-
dom is currently reviewing its threshold cost per quality-adjusted
life-year as a reference base for health funding decisions. In low-
and middle-income countries, the decision-making criteria are
less formalized. With the adoption of the Millennium Develop-
mentGoals,developingcountrygovernmentshavemadecommit-
ments to address major health issues affecting their populaces, in
areas of maternal health, child health, and HIV/AIDS. While these
goals have been implemented into the health plans of the minis-
tries of health in many developing countries, progress on the Mil-
lennium Development Goals has been slow. Making national and
local decision-making criteria and values more transparent may
be part of a solution toward achieving better health outcomes in
an effective manner.
Preference-based techniques provide a tool for mapping the
policyenvironmentandhaveapotentialtoleadtoevidence-based
development of policies. Methodologies such as cost-effective
analysis have been increasingly used to provide evidence for
smarter policy decisions. While cost-effective analysis has been
* Address correspondence to: Andrew Mirelman, Johns Hopkins Bloomberg School of Public Health, 615 North Wolfe Street, Room E8132,
Baltimore, MD 21205, USA.
E-mail: amirelma@jhsph.edu.
1098-3015/$36.00 – see front matter Copyright © 2012, International Society for Pharmacoeconomics and Outcomes Research (ISPOR).
Published by Elsevier Inc.
http://dx.doi.org/10.1016/j.jval.2012.04.001
V A L U E I N H E A L T H 1 5 ( 2 0 1 2 ) 5 3 4 – 5 3 9
Available online at www.sciencedirect.com
journal homepage: www.elsevier.com/locate/jval

Page 2

useful in the past decade in institutions such as NICE, this meth-
odology carries limitations [7–11]. While the concept of efficiency
is incorporated into cost-effective analyses, it does not inherently
capture a clinical perspective, such as severity of the disease nor
the societal distribution of disease and disability from the policy
perspective [7–9]. National agencies, such as NICE, have stated
that the formal inclusion of social values in the national policy
process may be applied as a result of local interpretation of rec-
ommendations, which could lead to variations in regional and
district health service delivery [10,11].
In the past decade, systematic techniques have been devel-
oped to define the rational basis of allocating health services. This
work has coalesced into a subfield of health economics with pref-
erence research presenting the methods and feasibility for includ-
ing equity, efficiency, and other criteria in decision making in a
variety of countries and settings [1–3,11–17]. These experiences
may show a substantial and important impact of these criteria on
decision making based on effectiveness and efficiency [12].
The analysis presented here explores what attributes a di-
versity of national-level policymakers in five countries find im-
portant while formulating plans on the extension and distribu-
tion of health care services in environments with constrained
resources. We apply a method to analyze the simultaneous use
of several criteria based on discrete choice experiments (DCEs)
applied in health [14,18,19]. The use of DCEs fits into a broader
framework for using evidence for rational priority setting,
which istermed multicriteria
[12,15,20–25]). MCDA findings can illuminate similarities and
differences across groups of country decision makers for a more
transparent policy process.
Theobjectiveofourpresentstudywastodescribethedecision-
makingprocessusedbypolicymakersapplyingDCEsconductedin
multiple country settings using an MCDA framework and present
a comparative analysis of the results across countries. In doing so,
wegroupthecriteriaresultsinanequityandefficiencycategoryto
estimate the relative importance of these two criteria in country
health priority settings.
decision analysis (MCDA)
A Diversity of Country Health Systems
Our sample of five countries explicitly accounts for the diversity in
economic development, population size, and state of health care
systems across the globe. Countries were selected in which MCDA
had been performed by using similar questionnaire designs.
Among their national health systems, the countries exhibit much
heterogeneity and different levels of sophistication. The countries
also vary by the level of gross domestic product (GDP) per capita
ranging from very low-income countries such as Uganda and
Nepal to the middle-income country Brazil and the high-income
country Norway. A brief overview of each country’s health system
characteristics is provided here.
Nepal is at the lowest end of the scale for GDP and has a na-
tional public health service. The country has taken many steps to
address the Millennium Development Goals, and the governmen-
tal health sector develops both short-term and long-term strategic
plans for the improvement of health indicators. The most recent
long-term health plan for 1997–2017 explicitly focuses on provid-
ing care to its marginalized populations. Uganda receives a large
amount of donor support for its health system including substan-
tial funding from the United States’ President’s Emergency Plan
for AIDS Relief program. Classified as a low-income country by the
World Bank, Uganda is confronted with high levels of HIV/AIDS
and malaria. Despite its significant health problems, under five
mortality rates and maternal mortality rates have been steadily
declining in Uganda for the past few years. A coordinated effort for
health provision in Uganda occurs through the national Health
Sector Strategic Planning initiative that involves hospitals, district
health centers, and village health teams. While the government
and donors provide many essential health services, Uganda is re-
liantonout-of-pocketpaymentsandprivateinsurancecompanies
to pay for services.
Cuba’s national health system is managed by the Cuban gov-
ernment, which assumes fiscal and administrative responsibility
for the funding and delivery of health care to all its citizens. While
one may expect stakeholders in health in Cuba to have a prefer-
ence for equitable distribution of health programs, Cuba also pur-
sues efficiency through one of the most proactive medical training
programs in the world. In Brazil, the provision of health care fol-
lows multiyear plans developed through the country’s Unified
Health System. These 4-year plans focus on issues such as reduc-
ing child mortality, political organization of the health sector, and
consolidation of decentralization of the health care system. The
governmentfocusesonprimarycareprovisionthroughtheFamily
Health Program, through collaboration between the national Min-
istry of Health and individual states. The private sector comprises
the majority of the overall services provided in terms of expendi-
tures.
Norway, at the highest end of the GDP scale, has a health sys-
tem that is completely centralized. The entire population is cov-
ered by the national health plan, and it has a well-established
financing and service provision system. As with other developed
countries, Norway faces a growing issue of how to tailor its health
system to a population that is quickly aging.
Methods
We used a standardized survey with a core set of preference cri-
teria as attributes based on literature reviews and verified by na-
tional focus groups before each DCE took place (12,17,26]. Our
study identified six primary attributes for the present analysis,
which include key criteria used in health decisions. Five attributes
each had two levels and one attribute, the age criteria, had three
levels. The attributes were disease severity (healthy life expec-
tancy less than 2 years), total beneficiaries (reaching more than
10% of the population), age (reaching groups in low, middle, and
upper ages), individual benefits (giving greater than 5 years of full
health), willingness to subsidize (propoor, greater than 70% gov-
ernment subsidization), and cost-effective (cost per life-year less
than gross national product per capita). These attributes are con-
sistent with those used in previous studies, which have been
shown to be important criteria for understanding the preferences
when choices are made for prioritizing interventions [1,12]. The
parameter definitions remained consistent for all five countries.
Nepal data were collected by using slight variations in definitions
for the variables, which were modified to allow for an analysis
with the other countries. All attribute definitions are detailed in
Appendix A as Supplemental Materials found at doi 10.1016/j.
jval.2012.04.001. The research group selected the five countries
(Cuba, Brazil, Uganda, Norway, and Nepal) included in this study
(L.N. and F.P.), allowing for a large range of health systems and
development stages as indicated by GDP and coverage level of the
nationalhealthsystem.WefollowstandardizedDCEmethodology
and have reported individual country studies elsewhere (for a
summary, see Baltussen et al. [1]). Table 1 provides a summary of
the country respondents and settings including the number of
participants and the response rate.
The data collection methodology was standardized across
countries. Preparatory workshops and direct self-administered in-
terviews were carried out together with leading local country re-
searchers and organizers from established research institutions.
There were up to 80 policymakers per country who participated in
this study. A core group for each country was involved in the ad-
aptation and translation of the DCE questionnaire, supervised the
implementation of the survey, and chaired the survey and feed-
back sessions. Respondents took on average 1 hour to complete
535
V A L U E I N H E A L T H 1 5 ( 2 0 1 2 ) 5 3 4 – 5 3 9

Page 3

the questionnaire. All surveys were conducted in person accord-
ing to protocol.Table 1provides a summary of the country respon-
dents and settings including the number of participants and the
response rate. The response rate was high, ranging between 91%
and 100% across countries. In each case, the choice experiment
was carried out inside each country among groups of relevant
policymakers and decision makers involved in national-level de-
cisions. Individuals from countries outside of the sample or with
international positions were not included. The specific attributes
as described in the Appendix were included in a protocolized ex-
perimental design with a high efficiency (99.4%) [1,17,26,27]. To
facilitate the consistency of answers, participants in the surveys
were further given instructions and coaching on how to answer
the questionnaires correctly, both in the group sessions and indi-
vidually. Collected data were pooled for all countries, and a pa-
rameterizedheteroskedasticmultinomiallogitmodelwasfittedto
test for the equality of scale (i.e., variance) across countries [28,29].
Subsequently, we tested for the equality of the estimated param-
eters between models by using a likelihood ratio test [29–31].
To make inferences, we employed the percentage differences
in predicted probabilities [32]. Differences in predicted probabili-
ties take into account the problems with varying levels of the scale
attributes and allow comparisons of the five countries [33]. The
base scenario (from which differences are calculated) is an option
where all attributes take the value of 1 (except for age where upper
age is 1) while the other alternatives simulate the effects of select-
ing interventions based on equity and efficiency.
The attributes were post hoc categorized as belonging to either
the equity or efficiency realm. The equity category includes crite-
ria for fairness, defined as any criterion dealing with the distribu-
tional impact across subpopulations such as age, income level,
and severity of disease [12,34]. The category of efficiency, defined
asthemaximalhealthgainfortheleastcost,includedattributesof
individualhealthimpact,healthimpactbytotalnumberofpeople,
and health impact per cost. Attributes classified for equity were
disease severity, age group, and willingness to subsidize, while
those classified for efficiency were total beneficiaries, individual
benefit, and cost-effectiveness.
By using these common definitions for equity and efficiency,
we introduced three hypothetical interventions: one baseline in-
Table 2 – Model (heteroskedastic MNL) results for six policy criteria, by country.
All BrazilCuba UgandaNorwayNepal
Disease severity
Total beneficiaries
Target age mid
Target age high
Individual benefits
Propoor (WTS)
Cost-effectiveness
Scale
Brazil
Cuba
Uganda
Norway
Chi-square test equality of scales
Log-L
LR test equality of parameters
No. of observations
No. of individuals
0.468‡(0.000)
0.354‡(0.000)
?0.0705 (0.249)
?0.453‡(0.000)
0.451‡(0.000)
0.162‡(0.000)
0.871‡(0.000)
0.501‡(0.000)
0.660‡(0.000)
?0.386‡(0.001)
?0.0928 (0.342)
0.507‡(0.000)
0.0101 (0.926)
1.498‡(0.000)
?0.158 (0.153)
?1.032‡(0.000)
0.810‡(0.000)
0.284* (0.051)
?0.258‡(0.002)
0.604‡(0.000)
0.150* (0.067)
0.698‡(0.000)
0.355†(0.012)
?0.137 (0.408)
?0.719†(0.026)
0.200 (0.222)
0.221 (0.247)
0.444‡(0.002)
0.682‡(0.000)
0.238†(0.028)
?0.243 (0.133)
?0.978‡(0.000)
0.688‡(0.001)
0.203 (0.110)
0.936‡(0.000)
0.538‡(0.000)
0.484‡(0.000)
0.358‡(0.002)
?0.833‡(0.000)
0.579‡(0.000)
0.235‡(0.001)
0.510‡(0.000)
0.158 (0.211)
?15.26 (0.977)
?0.235 (0.184)
0.126 (0.309)
5.34 (0.255)
?2047.7
357.3 (0.000)
7178
225
?484.6
?346.7
?148.9
?270.9
?617.9
2126
73
1144
37
518
17
10242366
32 66
Note. P values in parentheses. LR test calculated as ?2[LLPooled– (LLBrazil? LLCuba? LLUganda? LLNorway? LLNepal)].
LR, likelihood ratio; MNL, multinomial logit; WTS, willingness to subsidize.
* P ? 0.1.
†P ? 0.05.
‡P ? 0.01.
Table 1 – Characteristic country respondents.
Brazil Cuba UgandaNorwayNepal
Number
Respondent type in
health field
733717 32 66
Policymakers
and
professionals
National level
Policymakers and
professionals
Policymakers and health
experts
Policymakers and
professionals
Policymakers and
health experts
Sampling frameNational levelNational and program
level
Senior
National levelNational and program
level
Mid-level
100
Directorate of Health,
Kathmandu
Experience (y)
Response rate (%)
Setting
Junior to senior
91
Center for Health
Economics,
São Paulo
Junior to senior Mid-level to senior
100
Directorate of Health
and Public Health
Offices, Oslo
9795
Cuban Society for
Health
Economics,
Havana
Uganda National Academy
of Sciences, Kampala
Note. Professionals: researchers or medical experts; health experts: medical or public health experts; time period: Nepal 2006; others 2009–2010.
536
V A L U E I N H E A L T H 1 5 ( 2 0 1 2 ) 5 3 4 – 5 3 9

Page 4

tervention (with all attributes equal to 1), a second with all attri-
butes for efficiency equal to 1 (and all equity attributes equal to 0),
and a third intervention with all equity attributes equal to 1 (and
all efficiency attributes equal to 0). These three interventions
(baseline, efficiency, and equity) are shown in Table 3 as S0, S1,
and S2.
By applying these reference interventions, we calculated the
predicted probability of selection for the baseline intervention and
then for the subsequent interventions with qualities of either eq-
uity or efficiency. The difference between the predicted probabil-
ities for the equity-only and efficiency-only interventions and that
of the baseline intervention is taken by simple subtraction and
then the percentage change is calculated with respect to the base-
line predicted probability. Using this approach, the magnitude of
the contribution of the equity and efficiency components is as-
sessed, indicating the implicit willingness to trade-off equity and
efficiency. To ease the interpretation of this trade-off, we present
an equity-efficiency frontier by plotting the extreme values {1,0}
and {0,1}, and fit a logarithmic curve for the frontier [35].
Results
On the basis of the heteroskedastic multinomial logit and the sub-
sequent likelihood ratio test, we find that variance scale is not a
problem in the sample of countries, and we can safely ignore its
effects. The results for one country—China—did not pass this test
and were excluded from the comparison. Table 2 presents the
estimation results by country. The results show that preferences
appeared relatively consistent across the diversity of country
health systems. Significant decision-making criteria (at P ? 0.05 or
below) in Brazil were disease severity, total beneficiaries, middle
age group, individual benefits, and cost-effectiveness. Cuba se-
lected criteria for total beneficiaries, middle age group, individual
benefit, and willingness to subsidize as important. Uganda
showed significant preferences for disease severity, total benefi-
ciaries, upper age group, and cost-effectiveness. All criteria except
middle age group and willingness to subsidize were significant in
Norway, while all criteria weights were significant in Nepal.
Some notable findings are the large positive preferences for
cost-effective interventions in Brazil and Norway, which contrib-
ute to the preference for efficiency in these countries. Cuba
showed large positive preferences for the middle age group and
willingness to subsidize and negative preferences for total benefi-
ciariesandindividualbenefit,allcontributingtothepreferencefor
equity. Nepal gave positive weights for all factors except for the
upper age group criterion, which was negative. The most impor-
tant criteria preferences that this group prefers were disease se-
verity and individual health gains. By the nature of the scaling
method, one cannot directly compare the country results in Table
2acrosscountries;however,wecanmakeconclusionsonthepref-
erences for equity compared with efficiency within each country.
Table 3 shows the predicted probabilities by country when
comparing baseline probability values with the equity-only and
efficiency-only interventions and the percentage differences. The
absolute differences are depicted in Figure 1 and give the marginal
preference levels for interventions with either all-equity or all-
efficiency criteria compared with the baseline. This figure shows
that Cuba exhibits a highly positive preference for equity and is
ready to give up on efficiency when doing so, as shown by the
placement in the southeast quadrant of Figure 1. Among the other
countries, Norway shows a trade-off for efficiency over equity,
shown by the placement in the northwest quadrant. Brazil, Nepal,
Fig. 1 – Country differences in probability of selection for
interventions of efficiency and equity.
Table 3 – Country equity and efficiency preference results.
Intervention Log odds of selection Probability of selection Percentage difference of predicted
probabilities (in reference to the
base intervention) (%)
Brazil
Base
Equity
Efficiency
Cuba
Base
Equity
Efficiency
Uganda
Base
Equity
Efficiency
Norway
Base
Equity
Efficiency
Nepal
Base
Equity
Efficiency
S0
S1
S2
3.08
0.42
2.67
0.56
0.04
0.37
?93
?34
S0
S1
S2
?0.41
0.73
?1.14
0.16
0.51
0.08
213
?52
S0
S1
S2
1.20
0.20
1.00
0.40
0.15
0.33
?63
?18
S0
S1
S2
1.77
?0.09
1.86
0.41
0.06
0.45
?84
10
S0
S1
S2
1.517
?0.06
1.57
0.40
0.08
0.43
?79
6
537
V A L U E I N H E A L T H 1 5 ( 2 0 1 2 ) 5 3 4 – 5 3 9

Page 5

and Uganda all show that the baseline intervention is the most
preferred, being placed in the southwest quadrant; however, the
small distances from the efficiency axis for all three countries
means that efficiency is only slightly preferred over equity.
In Figure 2, the equity and efficiency probabilities of selection
from Table 3 were mapped onto an equity-efficiency frontier [35].
The country scores show that all the countries exhibit stronger
preferences for efficiency over equity, except for Cuba, where the
opposite is true. This is visualized by noting that all the countries
except Cuba are located above the 45° line, where the line repre-
sents an indifferent preference between equity and efficiency. The
curve shows the maximum levels of equity and efficiency that are
achievable given the measured preferences for each country
group.
Discussion
Four of the five countries that we studied demonstrated a prefer-
ence for efficiency over equity. While there can be many reasons
for this, it likely relates to the characteristics, in particular the
attitudes, of the included groups of policymakers and how they
perceived existing levels of efficiency and equity within their ex-
istingnationalhealthsystems.Ourresultssuggestthatinaddition
to the efficiency criteria examined here, further criteria of fairness
seem to play an additional but varying role in health decisions in
these countries.
The past 10 years have shown methodological progress in ad-
dressing these trade-offs in specific policy decisions as exempli-
fied by the role of NICE in the United Kingdom [11,18]. Ongoing
debates are taking place on the relative balance between fairness
and equity, as we reported for Norway [36]. Up to this point, the
importance of achieving efficiency at the national level often over-
rides concerns for equity and other policy imperatives at the re-
gional and local levels. Our study presents insights into the minds
of a diversity of decision makers, presented to formalize the
weighing of criteria in a deliberation process. Further studies of
other countries are required to expand the data set and to analyze
it in terms of income, relevant socioeconomic characteristics, and
geography. As future work is conducted, we recommend that de-
cision-making prioritization approaches present a balance be-
tween equity and efficiency. Balancing on these two characteris-
tics, as we have done, would ensure that an optimal level of equity
and efficiency is achieved when making decisions related to
health provision.
The limitations of our methodology come primarily from the
aggregationofindividualcountryscoresandtheinevitablylimited
sample of countries. These stem from conducting choice experi-
ments in a uniform manner between countries, which may have
varying interpretations of attribute definitions. There are natu-
rally going to be variations in the time, space, and procedure for
conducting this experiment in diverse settings that limit the abil-
ity to draw robust generalizable comparisons. This is inherent to
the nature of this type of study and limits the direct comparability
of the results between countries. There is also evidence that sug-
gests that health decisions may take into account more factors
beyond equity and efficiency [26,36]. Our study focuses on criteria
classified into either equity or efficiency; however, if other opera-
tional factors influence decision making in a significant manner,
then our model will be underspecified. Future work should build
on our approach and measure preferences for other system-level
characteristics that may be identified as important to decision
makers [17,26,37]. Another limitation may be that our method-
ological approach used a post hoc classification of attributes and
changes in the predicted probabilities of simulated interventions
to account for heterogeneity in the data. In future work, explicit
concepts of equity and efficiency can be directly calibrated among
the interview participants at the beginning of the study to ensure
congruence of definitions and alternative preference elicitation
methods such as best-worst scaling method, which can address
scaling problems of both the variance and the levels, should be
attempted [38].
Given our consistent findings across the national groups of
policymakers and their face validity, one could argue in favor of
a more explicit use of MCDA in policymaking, while aiming at a
more transparent process. This may also be the case for older
nationalized health systems such as the United Kingdom, espe-
cially at local commissioning levels, where decisions are taken
with regard to multiple criteria and where MCDA is now under
wider consideration at a national level [11]. When a country
decides to design packages for health interventions or to eval-
uate health system efforts in terms of equity and efficiency, an
MCDA may be a valuable tool within the formal context of a
deliberative process. Explicitly stated preferences may assist
stakeholders at all levels to make difficult decisions by consid-
ering various trade-offs. DCE results provide an insightful point
for further studying and formalizing the rationales of decision
makers and, in addition, may contribute to further development
of a rational policy process.
Our results confirm that there are measurable preference dif-
ferences in relation to equity and efficiency criteria across coun-
tries. While this may seem like an obvious result, it is worth ex-
ploring where commonalities and differences exist. Many global
health initiatives operate across the globe in support of countries
that cannot yet support their own health systems and may apply
the same assumptions and processes across countries. Better un-
derstanding of local preferences with respect to equity and effi-
ciency may improve the acceptability and performance of inter-
national health programs. Monitoring these preferences over
time, perhaps at 5- to 10-year intervals, may also help reflect the
relationships between health, economic growth, and preferences.
Although this will vary to some degree, those with leading roles
in the political system in place in a certain time period will
moderate the outcomes; the explicit approach may provide
greater clarity on the stability and validity of these preferences.
Greater use of MCDA in health priority setting would likely
make national decisions more transparent and perhaps more
rational and allow countries to characterize their efficiency and
equity trade-offs in a manner that is consistent with their level
of development and societal preferences.
Acknowledgments
We thank our colleague researchers and the respondent policy-
makers and experts in the participating countries for their partic-
Fig. 2 – Efficiency-equity country frontier of maximum sum
scores on efficiency (y-axis) and equity (x-axis).
538
V A L U E I N H E A L T H 1 5 ( 2 0 1 2 ) 5 3 4 – 5 3 9

Page 6

ipation in the priority setting workshops and discrete choice ex-
periments.
Source of Funding: The authors have no other financial rela-
tionships to disclose.
Supplemental Materials
Supplementalmaterialaccompanyingthisarticlecanbefoundinthe
online version as a hyperlink at http://dx.doi.org/10.1016/j.jval.2012.
04.001 or, if a hard copy of article, at www.valueinhealthjournal.
com/issues (select volume, issue, and article).
R E F E R E N C E S
[1] Baltussen RM, Youngkong S, Paolucci F, Niessen LW. Multi-criteria
decision analysis to prioritize health interventions: capitalizing on
first experiences. Health Policy 2010;96:262–4.
[2] Donaldson C, Bate A, Brambleby P, Waldner H. Moving forward on
rationing: an economic view. BMJ 2008;337:903–6.
[3] Douw K, Vondeling H, Oortwijn W. Priority setting for horizon
scanning of new health technologies in Denmark: views of health care
stakeholders and health economists. Health Policy 2006;76:334–45.
[4] Rawlins MD, Culyer AJ. National Institute for Clinical Excellence and
its value judgments. BMJ 2004;329:224–7.
[5] US Senate. The Patient Protection and Affordable Care Act. 2010.
Available from: http://democrats.senate.gov/reform/patient-protection-
affordable-care-act-as-passed.pdf. [Accessed May 10, 2011].
[6] HMG Government. Coalition: our programme for government –
freedom, fairness, responsibility. May 2010. Available from: http://
programmeforgovernment.hmg.gov.uk/files/2010/05/coalition-programme.
pdf.[AccessedFebruary24,2011].
[7] Michaels JA. Improving NICE’s social value judgments. BMJ 2006;332:
48–50.
[8] Norheim OF. Clinical priority setting. BMJ 2008;337:903–6.
[9] Ferner RE, Hughes DA, Aronson JK. NICE and new: appraising
innovation. BMJ 2010;340:245–7.
[10] NHS National Institute for Health and Clinical Effectiveness. Social
Value Judgments: Principles for the Development of NICE Guidance
(2nd ed.). NICE – Citizens’ Council, 2008:1–36. Available from:
www.nice.org.uk/media/C18/30/SVJ2PUBLICATION2008.pdf. [Accessed
March 8, 2010].
[11] Wilson E, Sussex J, Macleod C, Fordham RJ. Prioritizing health
technologies in a primary care trust. J Health Serv Res Policy 2007;12:
80–5.
[12] Baeten SA, Baltussen RM, Uyl-de Groot CA, et al. Incorporating equity-
efficiency interactions in cost-effectiveness analysis—three
approaches applied to breast cancer control. Value Health
2010;13:573–9.
[13] Green C. Investigating public preferences on ‘severity of health’ as a
relevant condition for setting healthcare priorities. Soc Sci Med 2009;
68:2247–55.
[14] Green C, Gerard K. Exploring the social value of health-care
interventions: a stated preference discrete choice experiment. Health
Econ 2009;18:951–76.
[15] Jehu-Appiah C, Baltussen RM, Acquah C, et al. Balancing equity and
efficiency in health priorities in Ghana: the use of multicriteria
decision analysis. Value Health 2008;11:1081–7.
[16] Koopmanschap MA, Stolk EA, Koolman X. Dear policy maker: have
you made up your mind? A discrete choice experiment among policy
makers and other health professionals. Int J Technol Assess Health
Care 2010;26:198–204.
[17] Noorani HZ, Husereau DR, Boudreau R, Skidmore B. Priority setting for
health technology assessments: a systematic review of current
practical approaches. Int J Technol Assess Health Care 2007;23:310–5.
[18] Ruta D, Mitton C, Bate A, Donaldson C. Programme budgeting and
marginal analysis: bridging the divide between doctors and managers.
BMJ 2005;330:1501–3.
[19] Ryan M, Gerard K. Using discrete choice experiments to value health
care programmes: current practice and future research reflections.
Appl Health Econ Health Policy 2003;2:55–64.
[20] Baltussen R, Niessen L. Priority setting of health interventions: the
need for multi-criteria decision analysis. Cost Eff Resour Alloc 2006;
4:14.
[21] Baltussen R, Stolk E, Chisholm D, Aikins M. Towards a multi-criteria
approach for priority setting: an application to Ghana. Health Econ
2006;15:689–96.
[22] Baltussen RM, ten Asbroek AHA, Koolman X, et al. Priority setting
using multiple criteria: should a lung health programme be
implemented in Nepal? Health Policy Plan 2007;22:178–85.
[23] Linkov I, Varghese A, Jamil S, et al. Multi-criteria decision analysis: a
framework for structuring remedial decisions at contaminated sites.
In: Linkov I, Ramadan A, eds., Comparative Risk Assessment and
Environmental Decision Making. Dordrecht, The Netherlands:
Springer, 2005.
[24] Mills A, Bennett S, Bloom G, et al. Strengthening Health Systems: The
Role and Promise of Policy and Systems Research. Alliance for Health
Policy and Systems Research, 2004: 130. Available from: www.who.int/
alliance-hpsr/resources/Strengthening_complet.pdf. [Accessed
February 10, 2012].
[25] National Economic Research Associates. DTLR multi-criteria analysis
manual. 2000:1–145. Available from: http://iatools.jrc.ec.europa.eu/
public/IQTool/MCA/DTLR_MCA_manual.pdf. [Accessed January 5,
2012].
[26] Youngkong S, Kapiriri L, Baltussen R. Setting priorities for health
interventions in developing countries: a review of empirical studies.
Trop Med Int Health 2009;14:930–9.
[27] Burgess L. Discrete Choice Experiments [computer software].
Department of Mathematical Sciences, University of Technology,
Sydney, 2007. Available from: http://crsu.science.uts.edu.au/choice/.
[Accessed February 12, 2012].
[28] Hole AR. Small-sample properties of tests for heteroskedasticity in the
conditional logit model. Econ Bull 2006;3:1–14.
[29] Hensher D, Louviere J, Swait J. Combining sources of preference data. J
Econ 1999;89:197–221.
[30] Swait J, Louviere J. The role of the scale parameter in the estimation
and comparison of multinomial logit models. J Market Res 1993;30:
305–14.
[31] DeShazo JR, Fermo G. Designing choice sets for stated preference
methods: the effects of complexity on choice consistency. J Env Econ
Mgmt 2002;44:123–43.
[32] Hall J, Kenny P, King M, et al. Using stated preference discrete choice
modeling to evaluate the introduction of varicella vaccination. Health
Econ 2002;11:457–65.
[33] Lancsar E, Louviere JJ, Flynn TN. Several methods to investigate
relative attribute impact in stated preference experiments. Soc Sci
Med 2007;64:1738–53.
[34] James C, Carrin G, Savedoff W, Hanvoravongchai P. Clarifying
efficiency-equity tradeoffs through explicit criteria, with a focus on
developing countries. Health Care Anal 2005;13:33–51.
[35] Wagstaff A. QALYs and the equity-efficiency trade-off. J Health Econ
1991;10:21–41.
[36] Defechereux T, Paolucci F, Mirelman A, et al. Health care priority
setting in Norway—a multi-criteria decision analysis. BMC Health Serv
Res 2012;12:7.
[37] Kapiriri L, Norheim OF, Martin DK. Fairness and accountability for
reasonableness: do the views of priority setting decision makers differ
across health systems and levels of decision making? Soc Sci Med
2009;68:766–73.
[38] Flynn TN, Louviere JJ, Peters TJ, Coast J. Best-worst scaling: what it can
do for health care research and how to do it. J Health Econ 2007:26:
171–89.
539
V A L U E I N H E A L T H 1 5 ( 2 0 1 2 ) 5 3 4 – 5 3 9