Article

A Critique of the World Health Organization's Evaluation of Health System Performance

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

The World Health Organisation's (WHO) approach to the measurement of health system efficiency is briefly described. Four arguments are then presented. First, equity of finance should not be a criterion for the evaluation of a health system and, more generally, the same objectives and importance weights should not be imposed upon all countries. Secondly, the numerical value of the importance weights do not reflect their true importance in the country rankings. Thirdly, the model for combining the different objectives into a single index of system performance is problematical and alternative models are shown to alter system rankings. The WHO statistical analysis is replicated and used to support the fourth argument which is that, contrary to the author's assertion, their methods cannot separate true inefficiency from random error. The procedure is also subject to omitted variable bias. The econometric model for all countries has very poor predictive power for the subset of OECD countries and it is outperformed by two simpler algorithms. Country rankings based upon the model are correspondingly unreliable. It is concluded that, despite these problems, the study is a landmark in the evolution of system evaluation, but one which requires significant revision.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... However, a number of issues with the cross-country healthcare efficiency rankings literature have come into focus. Critics attacked Evans et al. [7] and Tandon et al. [24] on their choice of methodology, the output measures selected by the World Health Organization (WHO), the weights assigned to output measures, assumptions made by researchers collecting the data, and the choice to include Organization for Economic Cooperation and Development (OECD) countries along with non-OECD countries in the sample ( [10]; [12]; [16,27]). Gearhart [8] also noted that there was little correlation between efficiency rankings in the original Evans et al. [7] and Tandon et al. [24] papers and his paper, where he utilized the same input and output measures, but a different, non-parametric, estimator. ...
... Each of these papers have compelling theoretical reasons for why the individual inputs and outputs should be used, as each impacts health in a different way. However, the authors fail to make a compelling argument as to why these inputs and outputs should be used cohesively, or why they are strict improvements over the efficiency rankings provided in other papers that follow the original WHO studies [7,8,10,12,16,24]. ...
... Unfortunately, these quality-adjusted output measures are not as robust as they need to be when conducting efficiency analysis. Williams [27] and Richardson et al. [16] detailed the quite significant issues with the arbitrary weighting schemes utilized by Evans et al. [7] and Tandon et al. [24] when creating their COMP output measure. Hollingsworth and Wildman [12] noted that these quality output measures are often self-reported measures or utility measures (QALY). ...
Article
Full-text available
Since 2000 several papers have examined the efficiency of healthcare delivery systems worldwide. These papers have extended the literature using drastically different input and output combinations from one another, with little theoretical or empirical support backing these specifications. Issues arise that many of these inputs and outputs are available for a subset of OECD countries each year. Using a common estimator and the different specifications proposed leads to the result that efficiency rankings across papers can diverge quite significantly, with several countries being highly efficient in one specification and highly inefficient in another. Broad input-output measures that are collected annually provide consistent efficiency rankings across specifications, compared to specifications that utilize specific measures collected infrequently. This paper also finds that broad output measures that are not quality-adjusted, such as life expectancy, seem to be a suitable alternative for infrequently collected quality-adjusted output measures, such as disability adjusted life years.
... As well as the published commentaries listed in the references below (e.g. Häkkinen 2000; Jamison and Sandbu 2001;Gravelle et al. 2002;Grignon 2001;Richardson et al. 2002;Hollingsworth and Wildman 2002;etc.), we have also seen a number of as yet unpublished papers that we are unable to cite. ...
... The model of production  The technical consultation on 'Measurement of Efficiency of Health Systems' seemed to be comfortable with the use of a single production function (on the grounds that all countries have access to the same medical technologies). However, several commentators have argued that the health production function may not be identical between nations, suggesting that there is disagreement about whether the use of a single model is appropriate (Richardson et al. 2002;. ...
... The role of income needs to be properly modelled even if estimation turns out to be econometrically inconvenient (because income is highly correlated with both inputs and outputs) (Pedersen 2002  There is evidence of a structural difference between developed and less-developed countries, implying the need for separate modelling (Richardson et al. 2002). ...
... In both reports, the efficiency of healthcare systems was estimated for 191 countries using a frontier estimation approach. The reports culminated in rankings of healthcare systems; rankings that attracted a disproportionate amount of attention from public policy decision-makers. 2 Academic discussion focused on improving estimation methods (Pedersen, 2002;Gravelle et al., 2003;Hollingsworth et al., 2003;Richardson et al., 2003;Greene, 2004;Spinks and Hollingsworth, 2009) and conceptual approaches (Navarro, 2000;Pedersen, 2002). ...
... The sparks that first ignited interest in efficiency estimations at the system level were two reports published by the World Health Organization (WHO) that estimate health-system efficiency for 191 countries between 1993 and 1997 (Evans et al., 2000;Tandon et al., 2001). The initial excitement was quelled by the follow-up literature on the shortcomings in the methodologies (Pedersen, 2002;Gravelle et al., 2003;Hollingsworth and Wildman, 2003;Richardson et al., 2003;Greene, 2004;Spinks and Hollingsworth, 2009) and conceptual approaches (Navarro, 2000;Pedersen, 2002) used in the WHO reports. ...
Article
Full-text available
This paper assesses which policy-relevant characteristics of a healthcare system contribute to health-system efficiency. Health-system efficiency is measured using the stochastic frontier approach. Characteristics of the health system are included as determinants of efficiency. Data from 21 OECD countries from 1970 to 2008 are analysed. Results indicate that broader health-system structures, such as Beveridgian or Bismarckian financing arrangements or gatekeeping, are not significant determinants of efficiency. Significant contributors to efficiency are policy instruments that directly target patient behaviours, such as insurance coverage and cost sharing, and those that directly target physician behaviours, such as physician payment methods. From the perspective of the policymaker, changes in cost-sharing arrangements or physician remuneration are politically easier to implement than changes to the foundational financing structure of the system.
... uent research justified the lower ratio of physicians with a better distribution of the skill mix and increased productivity resulting from a more efficient task delegation (Bloor and Maynard, 2003). Health system performance comparisons and composite indexes have also been subject to heavy criticism, in part due to some methodological fragilities (Richardson et. al, 2003). ...
... As an alternative to constructing composite indices or absolute performance rankings, we start by creating clusters of countries that have similar results in several reference indicators, including those usually associated with demand for healthcare services (mortality and morbidity-based indicators, and utilisation statistics), and with the supply of healthcare services (physical and human resources available). This allows for intra and inter-group local comparisons, avoiding attractive and yet inconclusive global performance rankings of substantially different health systems that have generated discord (Richardson et al., 2003), if not outright criticism (Bronnum-Hansen, 2014). ...
Research
Full-text available
With rising healthcare costs, using health personnel and resources efficiently and effectively is critical. International cross-country and simple worker-to-population ratio comparisons are frequently used for improving the efficiency of health systems, planning of health human resources, and guiding policy changes. These comparisons are made between countries typically of the same continental region. However, if used imprudently, inconsistencies arising from frail comparisons of health systems may outweigh the positive benefits brought by new policy insights. In this work, we propose a different approach to international health system comparisons. We present a methodology to group similar countries in terms of mortality, morbidity, utilization levels, and human and physical resources, which are all factors that influence health gains. Instead of constructing an absolute rank or comparing against the average, the method finds countries that share similar ground, upon which more reliable comparisons can then be conducted, including performance analysis. We apply this methodology using data from WHO’s HFA-DB, and we present some interesting empirical relationships between indicators that may provide new insights into how such information can be used to promote better healthcare planning and policy guidance.
... Subsequent research justified the lower ratio of physicians with a better distribution of the skill mix and increased productivity resulting from a more efficient task delegation (Bloor & Maynard, 2003). Health system performance comparisons and composite indices have also been subject to heavy criticism, in part due to some methodological fragilities (Richardson et al, 2003). ...
... As an alternative to constructing composite indices or absolute performance rankings, we start by creating clusters of countries that have similar results in several reference indicators, including those usually associated with demand for healthcare services (mortality and morbidity-based indicators, and utilisation statistics), and with the supply of healthcare services (physical and human resources available). This allows for intra-and inter-group local comparisons, avoiding attractive and yet inconclusive global performance rankings of substantially different health systems that have generated discord (Richardson et al, 2003), if not outright criticism (Bronnum-Hansen, 2014). ...
Article
With rising healthcare costs, using health personnel and resources efficiently and effectively is critical. International crosscountry and simple worker-to-population ratio comparisons are frequently used for improving the efficiency of health systems, planning of health human resources and guiding policy changes. These comparisons are made between countries typically of the same continental region. However, if used imprudently, inconsistencies arising from frail comparisons of health systems may outweigh the positive benefits brought by new policy insights. In this work, we propose a different approach to international health system comparisons. We present a methodology to group similar countries in terms of mortality, morbidity, utilisation levels, and human and physical resources, which are all factors that influence health gains. Instead of constructing an absolute rank or comparing against the average, the method finds countries that share similar ground, upon which more reliable comparisons can then be conducted, including performance analysis. We apply this methodology using data from the World Health Organization's Health for All database, and we present some interesting empirical relationships between indicators that may provide new insights into how such information can be used to promote better healthcare planning and policy guidance.
... An optimum health information system management is intended to increase the efficiency of health services using fewer resources with the primary goal being minimization of losses in services at the effectiveness level [3] EID health information system in Kenya is relatively new and was initiated during the rapid scale-up of EID program and the driver for its initiation was the need for strong systems of centralized data at the Ministry of Health [7]. This could enable assessment of the national program by monitoring of progress and inform interventions which were required in strengthening the program [8]. Program assessments in turn revealed the need for interventions at the facility level on how to improve care and retention outcomes, including improved data tools for linking & tracking HIVexposed infants [7]. ...
... However, we believe that the most important aspect is to relate information needs to interventions with a focus on how information generated could be used and influence local decisions. Some experiences, for example in Kyrgyzstan and in South Africa, showed improved data quality by giving health workers the basic skills to monitor their own work, leading to a sense of ownership of the generated information [8]. ...
Article
Full-text available
Recent changes to Kenya's early infant diagnosis (EID) testing algorithm have raised hope that the national goal of reducing perinatal HIV transmission rates to less than 5% can be attained. While programmatic efforts to reach this target are underway, obtaining complete and accurate data from clinical sites to track progress presents a major challenge. The purpose of this study was therefore to assess data completeness, accuracy and challenges in relation to routine EID data management in Kisumu East and Kisumu West Districts within Kisumu County, Kenya. Purposive sampling was used to select 23 active health facilities across the two districts disproportionate by partners. From the selected facilities a sampling frame of 192 patients was established and a sample size of 130 patient's data selected. Accuracy and completeness were determined by computing sensitivity, specificity, and predictive values. Infant sex, Infant prophylaxis, Breastfeeding information, Entry point, and Test results were found to have registered replication completeness not significantly different from 99% replication at (p>0.05) as oppose to Sample code, Infant Age, Date of sample collection, and PMTCT prophylaxis were found to be significantly different from the 99% replication (p<0.05). On completeness; Sample code and date of sample collection registered completeness beyond the hypothesized value, implying they were complete as required. The remaining data elements including infant sex, infant age, infant prophylaxis, PMTCT prophylaxis, breastfeeding information and entry point were significantly lower than the hypothesized completeness value (p<0.0001) except for test results that had completeness score equal to 0.99. The study, therefore, concludes that despite the shortage of staffing and other challenges, personnel working in the data management system appear to be dedicated, informed and conscientious. However, this research suggests that there is a suboptimal use of the information for local action in certain areas. This assessment thus serves to enlighten policy-makers on the current state of the EID data management system in Kisumu East and West districts.
... The US placed 37 th out of 191, behind Costa Rica, on overall health system performance. The report has been criticized for its objectives, confounding of social influences with health care system performance, poor data quality, and narrow scope in methodology [8][9][10]. Some of these critics re-estimated efficiency and rankings, using different approaches, which generally led to different rankings [9,11,12]. ...
... The report has been criticized for its objectives, confounding of social influences with health care system performance, poor data quality, and narrow scope in methodology [8][9][10]. Some of these critics re-estimated efficiency and rankings, using different approaches, which generally led to different rankings [9,11,12]. The US tended to rank higher in these newer studies, although none placed the US at the top. ...
Article
Full-text available
Rankings from the World Health Organization (WHO) place the US health care system as one of the least efficient among Organization for Economic Cooperation and Development (OECD) countries. Researchers have questioned this, noting simplistic or inappropriate methodologies, poor measurement choice, and poor control variables. Our objective is to re-visit this question by using newer modeling techniques and a large panel of OECD data. We primarily use the OECD Health Data for 25 OECD countries. We compare results from stochastic frontier analysis (SFA) and fixed effects models. We estimate total life expectancy as well as life expectancy at age 60. We explore a combination of control variables reflecting health care resources, health behaviors, and economic and environmental factors. The US never ranks higher than fifth out of all 36 models, but is also never the very last ranked country though it was close in several models. The SFA estimation approach produces the most consistent lead country, but the remaining countries did not maintain a steady rank. Our study sheds light on the fragility of health system rankings by using a large panel and applying the latest efficiency modeling techniques. The rankings are not robust to different statistical approaches, nor to variable inclusion decisions. Future international comparisons should employ a range of methodologies to generate a more nuanced portrait of health care system efficiency.
... The report sought to measure performance in all WHO Member States, using a series of key indicators. Controversially, it ranked countries in terms of health system performance, generating a heated debate around the selection and comparability of indicators, the areas for performance assessment and the wisdom of measuring performance with poor data availability (Maes, 2000;Reinhardt & Cheng, 2000;Braveman et al., 2001;Walt & Mills, 2001;Nord, 2002;Richardson, Wildman & Robertson, 2003). ...
Article
In this session, WHO together with the UHC2030 Technical Working Group on Health System Assessment and the European Observatory on Health Systems and Policies discuss the main elements of their joint approach to HSPA. The workshop intends to propose a common and practical framework that links descriptive health system analysis with evaluation, focussing on health system outcomes. The framework's approach to uncovering health system challenges and bottlenecks and the sources of variation will be discussed - showcasing how Health System Performance Assessment can be used as a means to uncover sources of health system inefficiencies and inequities. The workshop intends to identify the links between intermediate and ultimate goals and relate these to specific actions and actors within a health system, thus providing policy makers with an actionable tool to assess performance. The session is structured into two parts. In the first part the process of developing the HSPA framework and the framework itself will be introduced. The presentation will provide insights into the joint framework based on health system functions and emphasise the links to the intermediate and final goals of the HSPA framework. This will be followed by four short case studies pertaining to the four functions - governance, financing, resource generation and service delivery - and linking specific assessment areas to intermediate and final goals. The second part is an interactive discussion, to be kicked off by the input of Kanitsorn Sumriddetchkajorn and moderated by Josep Figueras, to jointly identify the strengths and weaknesses in performance of a given health system. Key messages • Health Systems Performance Assessments should focus on what health systems do and how well, rather than on what they are. • Health System Performance Assessment should focus on performance and use their evaluative angle to get to the root causes of not achieving final health system goals.
... This has lead to wide criticism too. 36,37 ...
Article
Full-text available
Various frameworks for measuring health system performance have been proposed and discussed. The scope of using performance indicators are broad, ranging from examining national health system to individual patients at various levels of health system. Development of innovative and easy index is essential to measure multidimensionality of health systems. We used indicators, which also serve as proxy to the set of activities, whose primary goal is to maintain and improve health. We used eleven indicators of MDGs, which represent all dimensions of health to develop index. These indicators are computed with similar methodology that of human development index. We used published data of Nepal for computation of the index for districts of Nepal as an illustration. To validate our finding, we compared the indices of these districts with other development indices of Nepal. An index for each district has been computed from eleven indicators. Then indices are compared with that of human development index, socio-economic and infrastructure development indices and findings has shown the similarity on distribution of districts. Categories of low and high performing districts on health system performance are also having low and high human development, socioeconomic, and infrastructure indices respectively. This methodology of computing index from various indicators could assist policy makers and program managers to prioritize activities based on their performance. Validation of the findings with that of other development indicators show that this can be one of the tools, which can assist on assessing health system performance for policy makers, program managers and others. Keywords: health system performance; health system performance index; human development index; socioeconomic index; infrastructure index.
... 14 The summary WHO indices of health system achievement and performance examined in this analysis have drawn criticism relating to the data on and methods by which they were derived. 34,35 In addition, these indices were derived at an earlier time period than REACH Registry enrolment and may have limited relevance in more contemporary settings. Nevertheless, the observed relationship of these indices to objective disease-specific risk factors in high-risk patients suggests their potential validity for assessing health system performance with respect to effective cardiovascular disease prevention and management. ...
Article
Elevated serum cholesterol accounts for a considerable proportion of cardiovascular disease worldwide. An understanding of the relationship between country-level economic and health system factors and elevated cholesterol may provide insight for prioritization of cardiovascular prevention programs. Using hierarchical models, we examined the relationship between elevated total cholesterol (>200 mg/dL) in 53 570 outpatients from 36 countries, and tertiles of several country-level indices: (1) gross national income, (2) total expenditure on health as percentage of gross domestic product, (3) government expenditure on health as percentage of total expenditure on health, (4) out-of-pocket expenditures as percentage of private expenditure on health, and the World Health Organization indices of (5) Health System Achievement and (6) Performance/Efficiency. Overall, 38% of outpatients had total cholesterol >200 mg/dL (>5.18 mmol/L), and 9.3% of the total variability in elevated cholesterol was at the country level; this proportion was higher for patients with (12.1%) versus without (7.4%) history of hyperlipidemia. Among patients with history of hyperlipidemia, countries in the highest tertile of gross national income or World Health Organization Health System Achievement had lower odds of elevated cholesterol than lower tertiles (P<0.001, for both). Countries in the highest tertile of out-of-pocket health expenditures had higher odds of elevated cholesterol than those in the lowest tertile (P<0.001). No significant associations were found for patients without history of hyperlipidemia. Global variations in the prevalence of elevated cholesterol among patients with history of hyperlipidemia are associated with country-level economic development and health system indices. These results support the need for strengthening efforts toward effective cardiovascular disease prevention and control and may provide insight for health policy setting at the national level.
... Using a framework for the socioeconomic determinants of health, the study concludes that policy makers should be aware of the limitations and uncertainty of using such techniques in the production of health settings. Likewise, Richardson et al. (2001) provide a critical opinion on the choice of the above five objectives (namely, maximizing population health, reducing inequalities in population health, maximizing health system responsiveness, reducing inequalities in responsiveness, and financing health care equitably), in that it gives undue prominence to the equity of financing and, more generally, inappropriately imposes a particular set of values upon all countries (or regional geographic units, say districts here). There is particularly strong case for omitting the equity of financing from the list and replacing it with an index of access, both financial and geographical. ...
Article
This paper attempts a sub-state level analysis of health system efficiency, focusing on West Bengal, a low income Indian state. Using a stochastic frontier model, it provides an idealized yardstick for evaluation. Our results suggest that overall efficiency of the public health delivery system remains low due to considerable disparities across districts. This is owing to differentials in availability and utilization of inputs such as the per capita availability of hospitals, beds, and manpower, and adversely affects life expectancy. Overcoming these factoral disparities may help the deficient districts to improve life expectancy. It may require a considerable increase in medical and public health expenditure in rural areas in the state and especially calls for resource mobilization to improve infrastructure facilities and maintain essential supplies at primary health centers. This could be attempted partly through funds from the National Rural Health Mission (NRHM) and by improving rural sanitation in poorer districts.
... Richardson et al. [23] maintained that effective weights depend on variation in scores across countries as well as the nominal weights. Consequently, if there is no difference in, for example, health inequality then, regardless of the weight of 0.25, it would contribute nothing to the ranking scores. ...
Article
Full-text available
Objective: To improve previous approaches to health system goals valuation. Methods: We reviewed literature on health system performance and previous comparative performance assessments, and combined this with literature on process utility to create a theoretical foundation for health system goals. We used a discrete choice experiment to elicit goal weights. To obtain social justice weights respondents were placed behind a 'veil of ignorance'. To ensure that respondents understood their task, we instructed them in a classroom setting. Results: We identified five health system goals. All five goals significantly affected choice behavior. An equitable distribution of health obtained the highest weight (0.34), followed by average level of health (0.29) and financial fairness (0.24). Both process outcomes (utility derived from the process and its distribution) received much lower weights (0.07 and 0.06, respectively). Conclusions: Our framework adds to that of the World Health Organization. We demonstrated the feasibility of measuring societal valuation of health system goals with a multi-attribute technique based on trade-offs. Our weights placed much greater emphasis on health and health inequality than on process outcomes. Our study improves the methodology of international health system performance comparison and thereby enhances global evidence-based health policy information.
... The WHO and the OECD, for example, have compared and ranked health systems across a range of functions and performance indicators. These exercises have sometimes been controversial but also difficult to achieve because of the complexity of comparing different health systems [5,6,7]. Health service planners and managers are faced with numerous challenges, not least having limited resources with which to provide services at an acceptable level of quality that are equitable and accessible to all. ...
Data
Full-text available
Introduction: There is growing interest in health system performance and recently WHO launched a report on health systems strengthening emphasising the need for close monitoring using system-wide approaches. One recent method is the balanced scorecard system. There is limited application of this method in middle-and low-income countries. This paper applies the concept of balanced scorecard to describe the baseline status of three intervention districts in Zambia. Methodology: The Better Health Outcome through Mentoring and Assessment (BHOMA) project is a randomised step-wedged community intervention that aims to strengthen the health system in three districts in the Republic of Zambia. To assess the baseline status of the participating districts we used a modified balanced scorecard approach following the domains highlighted in the MOH 2011 Strategic Plan. Results: Differences in performance were noted by district and residence. Finance and service delivery domains performed poorly in all study districts. The proportion of the health workers receiving training in the past 12 months was lowest in Kafue (58%) and highest in Luangwa district (77%). Under service capacity, basic equipment and laboratory capacity scores showed major variation, with Kafue and Luangwa having lower scores when compared to Chongwe. The finance domain showed that Kafue and Chongwe had lower scores (44% and 47% respectively). Regression model showed that children's clinical observation scores were negatively correlated with drug availability (coeff 20.40, p = 0.02). Adult clinical observation scores were positively association with adult service satisfaction score (coeff 0.82, p = 0.04) and service readiness (coeff 0.54, p = 0.03). Conclusion: The study applied the balanced scorecard to describe the baseline status of 42 health facilities in three districts of Zambia. Differences in performance were noted by district and residence in most domains with finance and service delivery performing poorly in all study districts. This tool could be valuable in monitoring and evaluation of health systems.
... The WHO and the OECD, for example, have compared and ranked health systems across a range of functions and performance indicators. These exercises have sometimes been controversial but also difficult to achieve because of the complexity of comparing different health systems [5,6,7]. Health service planners and managers are faced with numerous challenges, not least having limited resources with which to provide services at an acceptable level of quality that are equitable and accessible to all. ...
Article
Full-text available
There is growing interest in health system performance and recently WHO launched a report on health systems strengthening emphasising the need for close monitoring using system-wide approaches. One recent method is the balanced scorecard system. There is limited application of this method in middle- and low-income countries. This paper applies the concept of balanced scorecard to describe the baseline status of three intervention districts in Zambia. The Better Health Outcome through Mentoring and Assessment (BHOMA) project is a randomised step-wedged community intervention that aims to strengthen the health system in three districts in the Republic of Zambia. To assess the baseline status of the participating districts we used a modified balanced scorecard approach following the domains highlighted in the MOH 2011 Strategic Plan. Differences in performance were noted by district and residence. Finance and service delivery domains performed poorly in all study districts. The proportion of the health workers receiving training in the past 12 months was lowest in Kafue (58%) and highest in Luangwa district (77%). Under service capacity, basic equipment and laboratory capacity scores showed major variation, with Kafue and Luangwa having lower scores when compared to Chongwe. The finance domain showed that Kafue and Chongwe had lower scores (44% and 47% respectively). Regression model showed that children's clinical observation scores were negatively correlated with drug availability (coeff -0.40, p = 0.02). Adult clinical observation scores were positively association with adult service satisfaction score (coeff 0.82, p = 0.04) and service readiness (coeff 0.54, p = 0.03). The study applied the balanced scorecard to describe the baseline status of 42 health facilities in three districts of Zambia. Differences in performance were noted by district and residence in most domains with finance and service delivery performing poorly in all study districts. This tool could be valuable in monitoring and evaluation of health systems.
... The question of public opinion and putting certain weights to indicators evokes the discussion about country differences in preferences. For instance, Richardson at al. (2003) hold that the WHO importance weights attached to the system objectives have not been validated. ...
... The 191 WHO member states were then ranked based on these performance measures, producing the highly controversial league table of the world's health systems. This approach has been the subject of intensive debate, ranging from the implied values underlying the approach taken (Mulligan et al., 2000;Navarro, 2000;Williams, 2001;Musgrove, 2003;Richardson et al., 2003) to technical considerations of specific aspects (Almeida et al., 2001). Nonetheless, the 2000 report played an indisputable role in raising awareness of the potential impact of international benchmarking of health care systems, and the stream of criticism that it engendered helped bring to light the methodological challenges inherent in conducting and interpreting international comparisons. ...
Article
Full-text available
Purpose – The pupose of this paper is to present an overview of the main conceptual and methodological issues that underlie our current understanding of benchmarking initiatives in the field of health. Design/methodology/approach – The paper is based on a pragmatic review of the literature and policy reports. It outlines some of the major conceptual problems associated with the use of benchmarking indicators and discusses how health policy research and practice is evolving to address the challenges raised, drawing examples from national and international benchmarking initiatives. Findings – Benchmarking has become an intrinsic part of most developed health care systems, yet the impact of benchmarking initiatives on improvements in system performance and their integration within existing policy processes still need to be elucidated. Several methodological challenges remain in the field of benchmarking, many of them related to the selection and quality of indicators used to make comparisons both within and between health care systems. Further research and applications are needed to ensure that benchmarking in health fulfils its objective, namely to further our understanding of where to focus policy efforts in order to improve the performance of health care systems. Originality/value – This paper poses the timely question of whether benchmarking initiatives are in fact guiding health policy towards the improvement of health care system performance. It draws from the policy literature and existing frameworks to offer an outline for the future evaluation of benchmarking initiatives by policy-makers.
... The report aimed to assess the performance and relative ranking of the world's health systems across five dimensions of performance including health and responsiveness (in both levels and distribution) and fairness in financial contributions. The report generated an enormous interest but also attracted criticism on the basis of its scientific merits and associated country-level league tables (Williams, 2001;Gravelle, et al., 2002;Richardson, et al., 2003). ...
Article
This paper explores the use of anchoring vignettes as a means to adjust survey reports of health system performance for differential reporting behaviour using data contained within the World Health Survey (WHS). Survey respondents are asked to rate their experiences of health systems across a number of domains on a five-point categorical scale. Using data provided through a set of vignettes we investigate variations in reporting of interactions with health services across both socio-demographic groups and countries. We show how the method of anchoring vignettes can be used to enhance cross-country comparability of performance. Our results show large differences in the rankings of country performance once adjustment for systematic country-level reporting behaviour has been undertaken compared to a ranking based on raw unadjusted data.
... In the process, it has faced inevitable criticism from various stakeholders. This criticism has unveiled-not for the first time-some misinterpretation of the WHO's mandate, its authority, or the lack thereof, over its member states and a number of organizational and legal instrument constraints that have impacted pandemic preparedness and response 6,[22][23][24] . The WHO has three key roles in addressing health emergencies: coordination, normative guidance and technical steering 25 . ...
Article
Full-text available
Fourteen months into the SARS-CoV-2 pandemic, we identify key lessons in the global and national responses to the pandemic. The World Health Organization has played a pivotal technical, normative and coordinating role, but has been constrained by its lack of authority over sovereign member states. Many governments also mistakenly attempted to manage COVID-19 like influenza, resulting in repeated lockdowns, high excess morbidity and mortality, and poor economic recovery. Despite the incredible speed of the development and approval of effective and safe vaccines, the emergence of new SARS-CoV-2 variants means that all countries will have to rely on a globally coordinated public health effort for several years to defeat this pandemic. Global coordination of public health efforts will be needed to defeat the COVID-19 pandemic and to prepare for future public health emergencies.
... Each dimension is in turn covered by several questionnaireitems , which where evaluated based on a survey of country experts; see Annex 6 in WHO (2000) and Valentine et al. (2000) for a detailed description of the survey and the construction of the responsiveness index. Despite some critique, given for instance by Williams (2001), Navarro (2001) and Richardson et al. (2003), the WHO responsiveness index is the only available proxy of beyond-health output, which can reasonably claim to be a valid, albeit imperfect measure. Moreover, most of the critique raised concerns the quality of the survey data on which the evaluation is based. ...
Article
Political debates on the reform of health systems are stimulated by dissatisfaction of citizens with their health system. To adapt health systems in order to achieve more satisfaction, policy makers must know what citizens expect from it, in particular, what actually determines the citizen's satisfaction. The paper will analyze the topic of satisfaction with health systems under three aspects: What properties and outputs of the health system are most important for citizens satisfaction? What properties of the individuals and the society affect the individual's satisfaction? Moreover, is there a systematic interaction among individual and societal factors in the sense, that societal features determine what is most relevant for individual satisfaction? In particular, does this interaction oper-ate by the mechanism that overall societal development changes the expectations of citizens regarding the health system? At the theoretical level, the paper provides an explanation of why in particular wealth and economic development might change the criteria by which citizens evaluate their health system. The paper empirically analyzes the impact of individ-ual-level attitudes and features located at the health system level on an individual's satisfaction with the health system us-ing Eurobarometer survey data. The findings indicate that of the different types of health system output, the restoration of physiological health is no longer crucial for satisfaction. While not irrelevant, this output is taken for granted. Instead, "beyond-health outputs", like responsiveness, are the main determinants of satisfaction in developed countries.
... The World Health Organization has ranked the performance of its 191 world members based on several indicators including: (i) improving health status and reduced health inequalities, (ii) level of responsiveness to the population's expectations, (iii) inequalities in responsiveness and fairness in financial contributions. 1 Based on these indicators, they concluded that countries with a history of civil conflict or a high prevalence of HIV and AIDS are less efficient in providing healthcare to their populations, and performance increases with higher health expenditure per capita. 2 However, countries may give different weights to these indicators, 3 as people in different cultural and social settings value individual healthcare goals in different ways. For example, some countries may assign greater importance to indicators of health distribution and less importance to health level, so flexibility is needed when weighing the importance of each health indicator. ...
Article
Background Variations in healthcare provision around the world may impact how patients with functional gastrointestinal disorder (FGIDs) are investigated, diagnosed, and treated. However, these differences have not been reviewed.PurposesThe Multinational Working Team of the Rome Foundation, established to make recommendations on the conduct of multinational, cross-cultural research in FGIDs, identified seven key issues that are analyzed herein: (i) coverage afforded by different healthcare systems/providers; (ii) level of the healthcare system where patients with FGIDs are treated; (iii) extent/types of diagnostic procedures typically undertaken to diagnose FGIDs; (iv) physicians’ familiarity with and implementation of the Rome diagnostic criteria in clinical practice; (v) range of medications approved for FGIDs and approval process for new agents; (vi) costs involved in treating FGIDs; and (vii) prevalence and role of complementary/alternative medicine (CAM) for FGIDs. Because it was not feasible to survey all countries around the world, we compared a selected number of countries based on their geographical and ethno-cultural diversity. Thus, we included Italy and South Korea as representative of nations with broad-based coverage of healthcare in the population and India and Mexico as newly industrialized countries where there may be limited provision of healthcare for substantial segments of the population. In light of the paucity of formal publications on these issues, we included additional sources from the medical literature as well as perspectives provided by local experts and the media. Finally, we provide future directions on healthcare issues that should be taken into account and implemented when conducting cross-cultural and multinational research in FGIDs.
... [55][56][57] The WHO justified its method by stating: "the purpose of the first survey was not to describe preferences in a population, but rather empirically derive a set of weights reflecting normative choices." 58 Richardson et al. 59 maintained that effective weights depend on variation in scores across countries as well as the nominal weights. Consequently, if there is no difference in, for example, health inequality then, regardless of the weight of 0.25, it would contribute nothing to the ranking scores. ...
Article
Summary Health technology assessment already informed Dutch policymaking in the early 1980s. Evidence of health economic evaluations is, however, only systematically used in drug reimbursement decision making. Outpatient drugs with an added therapeutic value and expensive specialist drugs require evidence from an economic evaluation. Due to many exemptions, however, the availability of evidence of health economic evaluations remains rather low. Although the Dutch reimbursement agency suggested a cost-effectiveness threshold range depending on the severity of the disease (i.e., €10,000 – 80,000 per Quality Adjusted Life Year), it was never confirmed nor endorsed by the Ministry of Health. It is highly questionable whether health economic evaluations currently play a role in actual Dutch reimbursement decision making. Although the requirements exist in policy procedures, recent cases show that Dutch policymakers experience great difficulties in putting restrictions on reimbursement based on evidence from health economic evaluations. The near future will show whether the need increases to base decisions on societal value for money, and whether Dutch policymakers show the courage to take health economic evaluations seriously.
... This has lead to wide criticism too. 36,37 ...
Article
Full-text available
Various frameworks for measuring health system performance have been proposed and discussed. The scope of using performance indicators are broad, ranging from examining national health system to individual patients at various levels of health system. Development of innovative and easy index is essential to measure multidimensionality of health systems. We used indicators, which also serve as proxy to the set of activities, whose primary goal is to maintain and improve health. We used eleven indicators of MDGs, which represent all dimensions of health to develop index. These indicators are computed with similar methodology that of human development index. We used published data of Nepal for computation of the index for districts of Nepal as an illustration. To validate our finding, we compared the indices of these districts with other development indices of Nepal. An index for each district has been computed from eleven indicators. Then indices are compared with that of human development index, socio-economic and infrastructure development indices and findings has shown the similarity on distribution of districts. Categories of low and high performing districts on health system performance are also having low and high human development, socioeconomic, and infrastructure indices respectively. This methodology of computing index from various indicators could assist policy makers and program managers to prioritize activities based on their performance. Validation of the findings with that of other development indicators show that this can be one of the tools, which can assist on assessing health system performance for policy makers, program managers and others.
... The question of public opinion and putting certain weights to indicators evokes the discussion about country differences in preferences. For instance, Richardson at al. (2003) hold that the WHO importance weights attached to the system objectives have not been validated. ...
... There are different opinions in the literature on which variables should be considered as good health outcomes (Richardson, Wildman, & Robertson, 2003;Williams, 2003). The quantity output variables often included in DEA models are number of patient days, number of minor operations, and number of signifi cant operations (Al-Shammari, 1999); total outpatient visits (all visits to hospital emergency and outpatient facilities), and training full-time equivalents (Nayar & Ozcan, 2008). ...
Article
Full-text available
The focus placed on the efficiency of the healthcare system can vary across the countries. This paper aims to analyse and compare the technical efficiency of medical care for CVD patients across selected OECD countries using the data envelopment analysis (DEA) method according to two models. The first model (TE) incorporates the quantitative outputs that are connected with the quantity of the hospital outcomes (the number of surgical operations and procedures related to disease of the circulatory system per 100,000 inhabitants; hospital discharge rates for in-patients with diseases of the circulatory system). The second model (QE) includes the quality outputs that are connected with the health outcomes (survival rates of patients with diseases of the circulatory system). A number of cardiologists and angiography equipment per 100,000 inhabitants and total healthcare costs of CVD patients per 100,000 inhabitants were considered as inputs in both models. Secondly, we analyse whether endogenous (institutional arrangements) and exogenous (population behaviour, economic determinants) factors are associated with the efficiency of medical care. We utilise Data Envelopment Analysis (DEA) to calculate the efficiency of medical care for CVD patients in selected OECD countries and establish healthcare systems’ rankings according to TE and efficient healthcare delivery for CVD patients. The study found that the technically efficient countries were not as far efficient when the quality measure was used to calculation of efficiency. On the other hand, some of the technically inefficient countries were performing well concerning efficiency based on a quality measure.
... 26 Interest in this approach only began to grow after the WHO annual report for the year 2000 measured the health system performance of 191 countries. 27 This report was much disputed and criticised, [28][29][30] but it resulted in numerous papers on the measurement of cross-country healthcare efficiency. One criticism of the report was that it made no reference to what a country should do to improve its current ranking. ...
Article
Both citizens and policymakers demand the best possible results from a country's healthcare system. It is of utmost importance to accurately and objectively assess the efficiency of a healthcare system and to note the key indicators, where resources are lost, and possibilities for improvement. This paper evaluates the efficiency of health systems in 38 countries, mainly members of the Organization for Economic Co‐operation and Development, using data envelopment analysis (DEA). In the first stage, bootstrapped Ivanovic distance is used to generate weights for the indicators, thus taking into consideration different country's goals, but not to the extent of reducing the possibility of comparison. The analysis shows that human resources are the most important health system resource and countries should pay special attention to developing and employing competent medical workers. The reorganization of human resources and the funds allocated to them could also increase efficiency. The second stage examines environmental indicators to find the causes of inefficiency. No proof is found that any one basic health system funding model produces better health outcomes than the others. Obesity is identified as a major issue.
... For example, the method that used summary health indicators as a proxy for health system performance was criticized on the grounds that health was a function of the whole of society rather than just the health (care) system, and that health outcomes could not be attributed only to the activities of the health system (Arah et al. 2006;Handler et al. 2001;Kaltenthaler et al. 2004;Navarro 2001;Rosen 2001). Studies that simply added up indicators by giving them equal weight were also consistently criticized (Richardson et al. 2003;Wibulpolprasert and Tangcharoensathien 2001). The use of conceptual frameworks in performance assessment was generally applauded for acknowledging the complexity of health systems, but the way each framework was operationalized was often heavily criticized (Bhargava 2001;Blendon et al. 2001;Deber 2004;Mulligan et al. 2000;Wagstaff 2002;Wibulpolprasert and Tangcharoensathien 2001). ...
Article
Introduction: Since the release of the World Health Report in 2000, health system performance ranking studies have garnered significant health policy attention. However, this literature has produced variable results. The objective of this study was to synthesize the research and analyze the ranked performance of Canada's health system on the international stage. Method: We conducted a scoping review exploring Canada's place in ranked health system performance among its peer Organisation for Economic Co-operation and Development countries. Arksey and O'Malley's five-stage scoping review framework was adopted, yielding 48 academic and grey literature articles. A literature extraction tool was developed to gather information on themes that emerged from the literature. Synthesis: Although various methodologies were used to rank health system performance internationally, results generally suggested that Canada has been a middle-of-the-pack performer in overall health system performance for the last 15 years. Canada's overall rankings were 7/191, 11/24, 10/11, 10/17, "Promising" and "B" grade across different studies. According to past literature, Canada performed well in areas of efficiency, productivity, attaining health system goals, years of life lived with disability and stroke mortality. By contrast, Canada performed poorly in areas related to disability-adjusted life expectancy, potential years of life lost, obesity in adults and children, diabetes, female lung cancer and infant mortality. Conclusion: As countries introduce health system reforms aimed at improving the health of populations, international comparisons are useful to inform cross-country learning in health and social policy. While ranking systems do have shortcomings, they can serve to shine a spotlight on Canada's health system strengths and weaknesses to better inform the health policy agenda.
... For example, the method that used summary health indicators as a proxy for health system performance was criticized on the grounds that health was a function of the whole of society rather than just the health (care) system, and that health outcomes could not be attributed only to the activities of the health system (Arah et al. 2006;Handler et al. 2001;Kaltenthaler et al. 2004;Navarro 2001;Rosen 2001). Studies that simply added up indicators by giving them equal weight were also consistently criticized (Richardson et al. 2003;Wibulpolprasert and Tangcharoensathien 2001). The use of conceptual frameworks in performance assessment was generally applauded for acknowledging the complexity of health systems, but the way each framework was operationalized was often heavily criticized (Bhargava 2001;Blendon et al. 2001;Deber 2004;Mulligan et al. 2000;Wagstaff 2002;Wibulpolprasert and Tangcharoensathien 2001). ...
Article
Abstract Introduction: Since the release of the World Health Report in 2000, health system performance ranking studies have garnered significant health policy attention. However, this literature has produced variable results. The objective of this study was to synthesize the research and analyze the ranked performance of Canada's health system on the international stage. Method: We conducted a scoping review exploring Canada's place in ranked health system performance among its peer Organisation for Economic Co-operation and Development countries. Arksey and O'Malley's five-stage scoping review framework was adopted, yielding 48 academic and grey literature articles. A literature extraction tool was developed to gather information on themes that emerged from the literature. Synthesis: Although various methodologies were used to rank health system performance internationally, results generally suggested that Canada has been a middle-of-the-pack performer in overall health system performance for the last 15 years. Canada's overall rankings were 7/191, 11/24, 10/11, 10/17, “Promising” and “B” grade across different studies. According to past literature, Canada performed well in areas of efficiency, productivity, attaining health system goals, years of life lived with disability and stroke mortality. By contrast, Canada performed poorly in areas related to disability-adjusted life expectancy, potential years of life lost, obesity in adults and children, diabetes, female lung cancer and infant mortality. Conclusion: As countries introduce health system reforms aimed at improving the health of populations, international comparisons are useful to inform cross-country learning in health and social policy. While ranking systems do have shortcomings, they can serve to shine a spotlight on Canada's health system strengths and weaknesses to better inform the health policy agenda.
... International comparison methodologies using ranking systems have received considerable criticism within the literature. 11,[96][97][98][99] Simple ranking systems can unintentionally oversimplify complex phenomena, be fraught with normative assumptions and fail to adequately acknowledge the important differences between countries' health systems, population demographics, and social and economic realities. 98,100,101 Further, ranking systems use ordinal scaling (the ordering of values), which can falsely imply a meaningful difference between country performance in the absolute values. ...
Technical Report
Full-text available
International comparisons can facilitate cross-country learning to improve health system performance. This study helps paint a picture of Canada’s health system performance relative to that of 17 other high-income countries by comparing trends in mortality over a 50-year period (1960 to 2010). Data from the Organisation for Economic Co-operation and Development (OECD) was used to study potential years of life lost (PYLL). PYLL is a measure of premature mortality that provides an estimate of the additional time a person would have lived had he or she not died prematurely (before age 70). This study addressed the following questions: How did Canada perform on overall PYLL relative to OECD comparator countries between 1960 and 2010? This study demonstrated that, like all other countries included in the analysis, Canada made substantial improvements in PYLL, experiencing a reduction of 6,000 absolute potential years of life lost per 100,000 population between 1960 and 2010. However, relative to other peer countries over the 50-year period, Canada maintained a middle-of-the-pack performance. This finding suggests that although Canada is making progress in absolute terms, it is keeping pace with the international median in relative terms. How did Canadian men and women perform on overall PYLL relative to men and women in OECD comparator countries? In terms of overall PYLL, Canadian men and women showed similar patterns of middle-of-the-pack performance in earlier decades; more recently, though, Canada’s performance for men and women moved in different directions. From the 1990s to 2010, Canada’s men were consistently ahead of the international median relative to men in the comparator countries. Conversely, Canada’s women lost ground from the 1990s and in 2010 lagged behind the international median relative to other women. There continued to be a sex gap in Canada for PYLL overall. In 2010 in Canada, men lost almost twice as many potential years of life as women. This sex gap was predominant for certain causes of death. In 2010, absolute PYLL rates for deaths due to external causes were higher among Canadian men than Canadian women, with a male-to-female ratio of almost 3:1. A similar pattern existed for PYLL rates for ischemic heart disease, with a male-to-female ratio of 4:1. How did Canada perform on PYLL for main causes of death relative to OECD comparator countries? Canada’s relative PYLL performance stratified by specific causes of death can help shed light on areas where Canada did well over the 50 years studied, and areas where Canada has room for improvement compared with other countries. For PYLL among men due to cancer, Canada performed ahead of the international median. PYLL for cancer among Canadian women tells a different story: on this front, Canada performed behind the international median. For PYLL due to external causes of death (e.g., traffic accidents, falls, accidental poisonings, intentional self-harm), Canada had a middle-of-the-pack performance for both sexes, losing some ground in the last decade of analysis. For PYLL for men and women due to ischemic heart disease, Canada performed behind the international median across the 50-year period. And finally, for PYLL due to stroke, Canadian men and women fared well; on this front, Canada consistently performed well ahead of the international median for both sexes. These findings suggest that Canada has room for improvement on ischemic heart disease and deaths due to external causes for both sexes, and for cancers in women. Ischemic heart disease, cancers and external causes are large contributors to all-cause PYLL at the population level, so improvements in these areas have the potential to improve Canada’s overall performance on the international stage. Canada has much to learn from all peer countries as we strive to achieve optimal health status for the population. However, international comparisons are complex. Further policy analysis must take into account a range of factors, such as health and health care systems, as well as the broader social and economic drivers that influence country-level health system performance and population health outcomes.
... International comparison methodologies using ranking systems have received considerable criticism within the literature. 11,[96][97][98][99] Simple ranking systems can unintentionally oversimplify complex phenomena, be fraught with normative assumptions and fail to adequately acknowledge the important differences between countries' health systems, population demographics, and social and economic realities. 98,100,101 Further, ranking systems use ordinal scaling (the ordering of values), which can falsely imply a meaningful difference between country performance in the absolute values. ...
Article
The challenges facing European health systems have changed little over 30 years but the responses to them have. Policy ideas that emerged in some countries spread to others; however, the way policies were implemented and the impact they have had has been shaped by specific national contexts. Comparative policy analysis has evolved in response to this, moving away from simple classifications of health systems and crude rankings to studies that try and understand more deeply what works, where and why. For policymakers interested in how other countries have dealt with common challenges, it is important that they avoid the naïve transplantation of policy solutions but understand the need to translate policies to fit the institutional context of a particular country. Policies that cross borders will necessarily be shaped by the social and political institutions of a country. These dimensions should not be ignored in comparative research. The next decade will require health systems to deliver improved care for people with complex needs while at the same time delivering greater value. Policymakers will benefit from looking backwards as well as to their neighbours in order to develop appropriate policy solutions.
Article
The New Zealand public health sector has used DEA since 1997 to identify efficient expenditure levels to set prices for hospital services at the DRG level. Given the size of the expenditure (NZ$ 2.6 billion), considerable robustness was required for the results and sophistication of the models/process. While the model development and application appeared to be successful, politics overturned the results in the short run. In the longer term, the results have been shown to be reasonably robust and have become a base-line reference for future developments. As such, this paper reports a relatively successful transfer of theory into practice.
Article
Despite the increasingly global nature of health care, much of the research about journal rankings and directions for future research in health care management is from a United States based viewpoint. There is a lack of information about influential journals and trends for health care management research from a global perspective. This exploratory study gathered the opinions of health care management researchers from 17 countries regarding which journals are considered most influential, popular research topics and areas needing more attention from the research community. An online survey was sent to individuals in high-income Organisation for Economic Co-operation and Development countries who were identified through author relationships, academic institution websites, editorial boards of international journals, and academic and practitioner associations in the countries of interest. Results indicate that journal rankings vary substantially from prior published studies evaluating health care management journals and international ranking lists, and the list of influential journals includes a much more diverse array of publications. Respondents also indicated a diverse number of topics for current and future research, highlighting the global complexity of the field. The implications of this study are valuable to scholars evaluating outlets for disseminating research, and highlighting areas for collaborative research in health care management globally.
Article
Publicly funded health system reforms increasingly require the evaluation of competing programmes. However, programmes are made of multi-dimensional attributes of value (where value refers to latent expectations of health system improvement). This paper identifies the design, implementation and validation of a methodology to elicit health system values to guide health care priority setting. The exercise suggests that the proposed methodology is suitable for eliciting and validating health system values, and its findings show that pursuing health gain alone does not fully capture the dimensions of health system value. More specifically, we identify a list of health system values (elicited by both potential and actual users) and classify them in terms of process-related values (e.g., shorter waiting lists, greater choice, etc.) and improvements in health system equity in addition to value derived from health gain.
Article
This paper examines cross-state health care efficiency rankings using modern non-parametric estimators. Cross-state efficiency rankings are robust to minor modifications in the input–output combinations used for estimation. This paper finds that there is no clear relationship between health care efficiency rankings and per capita health care expenditures in that state in the models used for this paper, even though this is a key variable that policymakers target. It also finds that Massachusetts, in one dataset, has shown significant productivity improvement from 2005 to 2008, the time period during which its health care reform was launched. In a second dataset, from 2002 to 2007, productivity regressed in Massachusetts. This may hint that efficiency gains from structural health care reform can outweigh population behavioral inefficiencies from using the ER as a source of primary care with insurance coverage expansion. I also find that states that chose to expand Medicaid were less efficient, on average, than states that did not choose to expand Medicaid. Simple variable comparisons suggest that this is an artifact of the data and political decision making, rather than people migrating for Medicaid or productive inefficiency.
Article
Objective Interest in comparative quality measurement and evaluation has grown considerably over the past two decades due to several factors, such as recognition of widespread variation in clinical practice, the increased availability of evidence on medical effectiveness, and increasing concern about the cost and quality of healthcare. Interest in ensuring that healthcare is safe has grown, particularly since 2001 when the Institute of Medicine (IOM) included patient safety as another quality dimension.This interest in safety is not limited by international borders or by classifications of “more developed” or “less developed” countries. International initiatives at the highest levels of government such as the Organization for Economic Cooperation and Development's (OECD) Health Care Quality Indicators Project and its Patient Safety Working Group, as well as the World Alliance for Patient Safety, highlight the need for international agreement to increase learning on patient safety. However, little is being done to improve the availability and comparability of data/indicators on patient safety within and between countries.
Article
Healthcare institutions face significant challenges in implementing quality initiatives such as performance measurement. In the wake of multiple studies emphasising deficiencies in performance measurement in healthcare, measuring the performance of healthcare services to improve quality takes a great attention in the literature. Understanding the relationships between performance indicators is a first step for this. Effectiveness, which is one of the performance indicators in healthcare, reflects the effect of health interventions on health outcomes. Accessibility of services and utilisation are the two main dimensions of this concept. These indicators have the potential for improving outcomes, performance, and quality in health. This study explores the relationship between effectiveness and health outcome indicators as they relate to the development level and geographic region of 81 provinces in Turkey using a path analytic model. The numbers of hospitals and physicians are used as indicators of accessibility of healthcare services, while the average length of stay and number of surgical operations are used as indicators of utilisation. Life expectancy and general satisfaction from healthcare services are determined as outcome measures. According to the final path model, a strong relationship exists between accessibility indicators and health outcomes. A strong relationship was also found between life expectancy and general satisfaction with healthcare services, which are the objective and subjective outcome measures in healthcare, respectively. These results help our understanding of the relationship among key performance measures to improve health systems performance and quality.
Article
Full-text available
The aim of the paper is presentation of construction of 2 measures (indicators) of health care system effectiveness and, also, the results of studies in this area conducted in OECD countries. We’ve also elaborated the matrix of correlation coefficients for specific socio-economic factors. Basing on this, we’ve evaluated the relations between the health care expenditure and selected indicators of health status. We’ve used data coming from the WHO, Bloomberg and Eurostat. We’ve hypothesized that expenditure on health correlate negatively with PYLL. Potential Years of Life Lost (PYLL) estimate of the average years a person would have lived if he or she had not died prematurely. Existing of this correlation should result in incorporation of this factor into the health care system’s efficiency measure to assure the proper diagnosis of the efficiency. Results, obtained during research, especially Pearson correlation coefficient seem to proof the validity of assumption, that the diagnosis of the health system’s overall effectiveness should contain a component of Potential Years of Life Lost (PYLL). JEL codes: I14, I15, P46, R51.
Article
Full-text available
Budget experiments can provide additional guidance to health system reform requiring the identification of a subset of programs and services that accrue the highest social value to 'communities'. Such experiments simulate a realistic budget resource allocation assessment among competitive programs, and position citizens as decision makers responsible for making 'collective sacrifices'. This paper explores the use of a participatory budget experiment (with 88 participants clustered in social groups) to model public health care reform, drawing from a set of realistic scenarios for potential health care users. We measure preferences by employing a contingent ranking alongside a budget allocation exercise (termed 'willingness to assign') before and after program cost information is revealed. Evidence suggests that the budget experiment method tested is cognitively feasible and incentive compatible. The main downside is the existence of ex-ante "cost estimation" bias. Additionally, we find that participants appeared to underestimate the net social gain of redistributive programs. Relative social value estimates can serve as a guide to aid priority setting at a health system level.
Book
New York. London. Paris. Although these cities have similar sociodemographic characteristics, including income inequalities and ethnic diversity, they have vastly different health systems and services. This book compares the three and considers lessons that can be applied to current and future debates about urban health care. Highlighting the importance of a national policy for city health systems, the authors use well-established indicators and comparable data sources to shed light on urban health policy and practice. Their detailed comparison of the three city health systems and the national policy regimes in which they function provides information about access to health care in the developed world's largest cities. The authors first review the current literature on comparative analysis of health systems and offer a brief overview of the public health infrastructure in each city. Later chapters illustrate how timely and appropriate disease prevention, primary care, and specialty health care services can help cities control such problems as premature mortality and heart disease. In providing empirical comparisons of access to care in these three health systems, the authors refute inaccurate claims about health care outside of the United States. © 2010 by The Johns Hopkins University Press. All rights reserved.
Article
This paper re-examines analyses of cross-country healthcare efficiency using modern, non-parametric estimators and Malmquist indices to determine productivity changes over the panel. This paper finds that cross-country heterogeneity leads to different efficiency rankings than previously thought, and that the hyperbolic order-α estimator leads to more robust efficiency scores when looking across different output measures, only looking at the more homogeneous OECD countries. It finds that the United States, if excluding the percent of healthcare expenditures that are publicly financed, is one of the more inefficient healthcare delivery systems in the world. What are commonly thought of as well-run healthcare systems (Austria and France) are either inefficient themselves or have variation in their efficiency rankings, showcasing difficulties in using other countries' healthcare systems as models for reform. It also finds that there has been productivity regression in all countries except the United States. These highlight the difficulties in cross-country efficiency comparisons.
Chapter
This chapter presents the experiences of transforming the hospital sector in 22 countries of Central and Eastern Europe and former Soviet Republics: Albania, Armenia, Azerbaijan, Bulgaria, Belarus, the Czech Republic, Estonia, Georgia, Hungary, Kazakhstan, Kyrgyzstan, Latvia, Lithuania, Moldova, Poland, Romania, Russian Federation, Slovakia, Tajikistan, Turkmenistan, Ukraine and Uzbekistan. Using a qualitative approach, the country-level case studies produce detailed and internationally comparable descriptions of changes that took place in the area of hospital governance. Next, the individual reform paths are generalised in order to develop and substantiate a model of transforming hospital governance. Going beyond the public-private delineation, the model identifies five unique stages of prevailing forms of hospital governance: (1) the integrated Semashko model, (2) decentralised hospital management, (3) devolved hospital ownership and (4) corporatised and (5) privatised hospitals. Each stage corresponds to a distinct distribution of decision powers, financial risks and residual claims between the sector participants. These characterisations can be interpreted as efficiency factors associated with decentralisation. Recognising the importance of the above processes leads to an extended typology of post-communist health care systems. Accounting for both the dominant financing arrangements and the governance setup improves our knowledge of transition, compared to common classifications based exclusively on the introduction of social health insurance. The chapter concludes with a reflection on good practices and common mistakes found in this region-wide process.
Article
Full-text available
The lack of an active neighbourhood living environment can impact community health to a great extent. One such impact manifests in walkability, a measure of urban design in connecting places and facilitating physical activity. Although a low level of walkability is generally considered to be a risk factor for childhood obesity, this association has not been established in obesity research. To further examine this association, we conducted a literature search on PubMed, Web of Science and Scopus for articles published until 31 December 2018. The included literature examined the association between measures of walkability (e.g., walkability score and walkability index) and weight‐related behaviours and/or outcomes among children aged under 18 years. A total of 13 studies conducted in seven countries were identified, including 12 cross‐sectional studies and one longitudinal study. The sample size ranged from 98 to 37 460, with a mean of 4971 ± 10 618, and the age of samples ranged from 2 to 18. Eight studies reported that a higher level of walkability was associated with active lifestyles and healthy weight status, which was not supported by five studies. In addition to reviewing the state‐of‐the‐art of applications of walkability indices in childhood obesity studies, this study also provides guidance on when and how to use walkability indices in future obesity‐related research.
Article
Well-functioning health systems are of critical importance to the achievement of both national policy objectives and international policy commitments. Financing systems should be the servants of policy - the choice of financing mechanisms should be informed by the type of health systems that will meet overall policy objectives. This article analyzes the main types of health systems and financing systems across the world, exploring their policy objectives and performance. The article also emphasizes the importance of culture and history to health system evolution. Finally, we consider additional local and international factors that cause or constrain change in health system development.
Conference Paper
The aim of this study was to evaluate the health system in Europe, including a vision of health care in the 21st century, human resource management in health care, doctrinal approaches, the basic tasks of public health, and the main health problems in Europe. The review of the state of health care in our country has been provided. The special emphasis was on the issue of human resource management ie. principles, techniques and problems of management of human resources in health. The activities and processes of importance to business management in the health sector were emphasized as well. The paper points to the openness towards European integration in the areas of health and education, the need for financial, organizational and strategic reconstruction of the health system and education, a stronger connection between educational sector and the health sector and the labor market, with the condition that patient is always in the focus of the health care policy. The second part presents a model for the organization and management of the health system of most vulnerable population: paediatric, age from birth to 18 years, in Bosnia and Herzegovina, during period from 2003 to 2016, which has resulted in health brand, competitive in the Balkan countries as well as on the European market labor, a brand that respects the principles of good medical practice and evidence-based medicine. When institutions invest wisely in human resources: eduction, training its staff, the result is a satisfied and motivated staff, which provides high-quality health services in the country and the region. This institution fully fulfills its mission, contribute to society, providing an excellent healthcare services to population, institution which is recognized and acknowledged in this region by the medical experts.
Article
This article aims to identify how the economies that do not necessarily prioritise social rights in their social policy arrangements fare in achieving various healthcare objectives. The big five of East Asian countries – China, Japan, South Korea, Taiwan, Singapore plus Hong Kong – are considered as such cases. It first highlights a wide range of variations in their healthcare offerings. It then shows that, contrary to the common belief, they constitute a surprisingly high level of redistributive elements in them. Deviating from their overall welfare regime characteristics, each healthcare system presents a unique combination of policy objectives in social, medical, economic and political terms, raising a question of the utility of social rights as a central conceptual lens to understand the world of welfare capitalism.
Thesis
Full-text available
Discussion of the current overall ranking of health systems by clustering these in their way of financing and equity.
Article
Full-text available
Background: Health Care provision in terms of prevention, detection and treatment is primarily dependent on the quality of the hosting Health System. In its health report 2000, the WHO's attempt to assess and rank health systems' quality Worldwide was heavily criticized. We propose a novel framework for health system performance and ranking using three indicators for three domains; general health system performance, clinical outcome of treatment applied to the main causes of death and health system sustainability domains. Methods: Each domain was rated as "A - high", "B - intermediate" or "C - poor" according to the aggregate score values of its three indicators. Hence the highest rank a health system can achieve is "AAA" and the lowest is "CCC". If there is a need to define a "numerical rank" to further differentiate health systems with similar rating from one another, the total health expenditure per capita per year was used as an additional "number 10" indicator to achieve that level of differentiation. The framework was applied to Health Systems serving most of the World population including China, India, Brazil, USA, Russia, Germany, Japan, UK, France, Singapore and Switzerland. Data pertinent to each indicator was captured from published reports in peer-reviewed journals and/or from official websites. A Delphi survey was conducted for data not available online. Results: Among the 11 health systems tested, no one scored AAA, Switzerland, France, Germany and Japan scored AAB, Singapore scored ABB, UK scored BBB, USA, Russia and China scored BBC, Brazil scored BCC while India scored CCC. Total health expenditure per capita per year lead to ranking Switzerland first followed by France, Germany, and Japan. Conclusion: This novel ranking system is a practical and an applicable tool that test health system performance and sustainability. It can be utilized to guide all organizations, people and actions whose primary intent is to promote, restore or maintain health to achieve their targets. An International Health System Ranking database that will be hosted by the Institute of Global, Health, Faculty of Medicine, University of Geneva, Switzerland.
Book
Full-text available
The Disability Adjusted Life Year (DALY) is a measure of population health that was developed, as part of a World Bank initiated study, to inform health strategy development, priority setting, resource allocation and research, and to measure the global burden of disease. The innovative feature of the DALY was the combination of information on morbidity and mortality within a single index. Since its development in 1992, the rate at which it has been adopted by governments, multilateral agencies and researchers has been staggering. The enthusiasm with which the measure has been taken-up perhaps reflects a desire on the part of health policy makers to embrace an “evidence-based” approach to health policy. Although the DALY has been heavily criticised in some quarters, it has survived. There are a number of reasons why this may be: • It is a good measure of population health; • The enormous political will to see it succeed; and • The lack of empirical data to challenge the validity of the DALY; • In addition, many of the criticisms of the DALY have been about the implicit and explicit social values, which are hard to argue on technical grounds. This report details the findings of an empirical investigation of some of the technical and social assumptions on which the DALY is based.
Article
Full-text available
Models of Autoregressive Conditional Heteroscedasticity (ARCH) and their generalizations are widely used in applied econometric research, especially for analysis of financial markets. We bring to our reader’s attention a consul-tation on this topic prepared from the book of Marno Verbeek “A Guide to Modern Econometrics” appearing soon in the Publishing House “Nauchnaya Kniga” Note: this is not the textbook "A Guide to Modern Econometrics", which is copyright owned by John Wiley and Sons.
Article
Full-text available
In this paper we consider the efficient instrumental variables estimation of a panel data model with heterogeneity in slopes as well as intercepts. Using a panel of U.S. airlines, we apply our methodology to a frontier production function with cross-sectional and temporal variation in levels of technical efficiency. Our approach allows us to estimate time-varying efficiency levels for individual firms without invoking strong distributional assumptions for technical inefficiency or random noise. We do so by including in the production function a flexible function of time whose parameterization depends on the firm. We also generalize the results of Hausman and Taylor (1981) to exploit assumptions about the uncorrelatedness of certain exogenous variables with the temporal pattern of the firm's technical inefficiency. Our empirical analysis of the airline industry over two periods of regulation yields believable evidence on the pattern of changes in efficiency across regulatory environments.
Article
Full-text available
In its latest World Health Report, The World Health Organization (WHO) argues that a key dimension of a health system's performance is the fairness of its financing system. The report discusses how policymakers can improve this aspect of performance, proposes an index of fairness, discusses how it should be put into operation, and presents a league table of countries, ranked by fairness with which their health services are financed. The author shows that the WHO index cannot discriminate between health financing systems that are regressive, and those that are progressive - andcannot discriminate between horizontal inequity, and progressiveness, or regressiveness. The index cannot tell policymakers whether it deviates from 1 (complete fairness) because households with similar incomes spend different amounts on health care (horizontal inequity), or because households with different incomes spend different proportions of their income on health care (vertical inequity, given the WHO's interpretation of the ability-to-pay principle) - although the two have different policy implications. With the WHO's index, progressiveness, and regressiveness are both treated as unfair. This makes no sense, because policymakers who may be strongly averse to regressive payments (which worsen income distribution) may in the name of fairness be quite receptive to progressive payments (requiring that the better-off, who may be willing to spend proportionately more on health care, are required to pay proportionately more). The author compares the WHO index with an alternative, and more illuminating approach developed in the income redistribution literature in the early 1990s, and used in the late 1990s, to study the fairness of various OECD health care financing systems. He illustrates the differences between the approaches with an empirical comparison, using data on out-of-pocket payments for health services in Vietnam for 1993 and 1998. This analysis is of some interest in its own right, given the large share of health spending from out-of-pocket payments in Vietnam, and the changes in fees, and drug prices over the 1990s.
Article
Full-text available
There has been increasing interest in measuring the productive performance of health care services, since the mid-1980s. This paper reviews this literature and, in particular, the concept and measurement of efficiency and productivity. Concerning measurement, we focus on the use of Data Envelopment Analysis (DEA), a technique particularly appropriate when multiple outputs are produced from multiple inputs. Applications to hospitals and to the wider context of general health care are reviewed and the empirical evidence from both the USA and Europe (EU) is that public rather than private provision is more efficient.
Article
Full-text available
obert blendon and colleagues compare results of a single survey ques- tion on satisfaction with "the way health care runs" in seventeen countries with the World Health Organization's (WHO's) systematic effort to measure levels of responsiveness, overall health system goal achievement, and the efficiency of health system resource use in 191 countries. The comparison, we believe, is useful because it highlights some fundamen- tal limitations to comparing satisfaction over time and across populations and the impor- tance of developing survey instruments that address some of these limitations. Unfortu- nately, in the course of their analysis, Blendon and colleagues misrepresent the WHO ap- proach to measuring health system perform- ance and make a number of incorrect claims. In this response we highlight some of these and direct the interested reader to other sup- porting documentation. 1 Comparing Apples With Oranges Two out of three of Blendon's comparisons seem to be mixing apples with oranges. Satis- faction with the way health care runs in a country is not conceptually comparable with overall health system performance or attain- ment and only partly comparable to respon- siveness. First, WHO defines the health sys- tem broadly to include all actors, institutions, and resources that undertake actions whose primary intent is to improve health. This broad definition includes medical care but also includes efforts to decrease tobacco con- sumption or to reduce highway fatalities through better car design. We suspect that those who responded to the Blendon question were more likely reflecting their satisfaction with personal medical care and not the broader health system. Second, health system performance as measured by WHO is the efficiency with which health system resources are used to achieve socially valued outcomes. It is a value- for-money concept in which the achieve- ments of a health system are compared to what would be possible given the level of spending on the health system and other non-health system factors. The survey ques- tion used by Blendon does not ask respon- dents about efficiency, so why is the compari- son meaningful? Third, even taking into account the differ- ence between the health system and the medi- cal care system, satisfaction with the way health care runs does not seem comparable with the WHO measure of overall health sys- tem goal attainment. This measure captures the levels of health, health inequalities, sys- tem responsiveness, and who bears the bur- den of financing the health system. Satisfac- tion with the way health care runs is unlikely to adequately reflect mortality rates (the re-
Article
This paper presents the results of a ten-country comparative study of health care financing systems and their progressivity characteristics. It distinguishes between the tax-financed systems of Denmark, Portugal and the U.K., the social insurance systems of France, the Netherlands and Spain, and the predominantly private systems of Switzerland and the U.S. It concludes that tax-financed systems tend to be proportional or mildly progressive, that social insurance systems are regressive and that private systems are even more regressive. Out-of-pocket payments are in most countries an especially regressive means of raising health care revenues.
Article
The World Health Organization (WHO) ranked health systems in 191 countries based on measures developed by public health experts. This paper compares the WHO rankings for seventeen industrialized countries with the perceptions of their citizens. The results show little relationship between WHO rankings and the satisfaction of the citizens who experience these health systems. The health systems of some top WHO performers are rated poorly by their citizens, including the low-income and elderly. The two rated most highly by the public rank at the bottom of the WHO ratings. These findings suggest that both public and expert views should be considered in international rankings.
Article
The World Health Report 2000 focuses on the performance of health-care systems around the globe. The report uses efficiency measurement techniques to create a league table of health-care systems, highlighting good and bad performers. Efficiency is measured using panel data methods. This paper suggests that the WHO's estimation procedure is too narrow and that contextual information is hidden by the use of one method. This paper uses and validates a range of parametric and non-parametric empirical methods to measure efficiency using the WHO data. The rankings obtained are compared to the WHO league table and we demonstrate that there are trends and movements of interest within the league tables. We recommend that the WHO broaden its range of techniques in order to reveal this hidden information.
Stata Manuals Release 7. Stata Corporation
  • Statacorp
StataCorp. 2001, Stata Manuals Release 7. Stata Corporation.
  • Von Winterfeldt
  • D Edwards
Von Winterfeldt, D. and Edwards, W. 1993, Decision analysis and Behavioural Research, Cambridge University Press.
  • S C Khumbraker
Khumbraker, S.C. and Lovell, 1990, C.A.K. Stochastic Frontier Analysis, Cambridge, Cambridge University Press.
Decision Analysis and Behavioural Research
  • D Von Winterfeldt
  • W Edwards
von Winterfeldt D, Edwards W. Decision Analysis and Behavioural Research. Cambridge University Press: Cambridge, MA, 1986.
Comparing the efficiency of national health systems: econometric analysis should be handled with care
  • H Gravelle
  • R Jacobs
  • A M Jones
  • A Street
Gravelle H, Jacobs R, Jones AM, Street A. Comparing the efficiency of national health systems: econometric analysis should be handled with care. Mimeo University of York, 2002.
Stochastic Frontier Analysis
  • S C Khumbraker
  • Cak Lovell
Khumbraker SC, Lovell CAK. Stochastic Frontier Analysis. Cambridge University Press: Cambridge, MA, 1990.
Measuring overall health system performance for 191 countries. GPE Discussion Paper Series: No. 30, World Health Organisation
  • A Tandon
  • Cjl Murray
  • J A Lauer
  • D B Evans
Tandon A, Murray CJL, Lauer JA, Evans DB. Measuring overall health system performance for 191 countries. GPE Discussion Paper Series: No. 30, World Health Organisation: Geneva, Switzerland, 2000.
New developments in the estimation of stochastic frontier models with panel data
  • W Greene
Greene W. New developments in the estimation of stochastic frontier models with panel data. Remarks prepared for the 7th European Workshop on Efficiency and Productivity Analysis, University of Oviedo: Spain, September 2001.