ArticlePDF Available

Abstract and Figures

This paper describes and discusses the development and use of health technology assessment (HTA) in five Central and Eastern European countries (CEE): Poland, the Czech Republic, Hungary, Romania and Bulgaria. It provides a general snapshot of HTA policies in the selected CEE countries to date by focusing on country case-studies based on document analysis and expert opinion. It offers an overview of similarities and differences between the individual CEE countries and discusses in detail the role of HTA by assessing its formalization and institutionalization, standardization of methodology, the use of HTA in practice and the degree of professionalization of HTA in the region. It finds that HTA has been to some extent implemented in all five countries studied, with methodologies in accordance with international standards, but that challenges remain when it comes to the role of HTA in health care decision-making as well as to human resource capacities of the countries. This paper suggests that coming years will show whether CEE countries develop adequate national analytical capacity to assess and appraise technologies in the context of local need and affordability, instead of using HTA as a mere administrative procedure to fulfill (inter)national requirements. Finally, suggestions are provided to strengthen HTA in CEE countries through cooperation, mutual learning, a common accreditation of HTA bodies and increased network building among CEE HTA experts.
Content may be subject to copyright.
Health technology assessment in Poland, the Czech Republic,
Hungary, Romania and Bulgaria
´csi Alexandru M. Rotar
Maciej Niewada Olga Lo
´Fanni Rencz
Guenka Petrova Imre Boncz Niek S. Klazinga
Received: 6 February 2014 / Accepted: 31 March 2014 / Published online: 16 May 2014
Springer-Verlag Berlin Heidelberg 2014
Abstract This paper describes and discusses the devel-
opment and use of health technology assessment (HTA) in
five Central and Eastern European countries (CEE):
Poland, the Czech Republic, Hungary, Romania and Bul-
garia. It provides a general snapshot of HTA policies in the
selected CEE countries to date by focusing on country
case-studies based on document analysis and expert opin-
ion. It offers an overview of similarities and differences
between the individual CEE countries and discusses in
detail the role of HTA by assessing its formalization and
institutionalization, standardization of methodology, the
use of HTA in practice and the degree of professionaliza-
tion of HTA in the region. It finds that HTA has been to
some extent implemented in all five countries studied, with
methodologies in accordance with international standards,
but that challenges remain when it comes to the role of
HTA in health care decision-making as well as to human
resource capacities of the countries. This paper suggests
that coming years will show whether CEE countries
develop adequate national analytical capacity to assess and
appraise technologies in the context of local need and
affordability, instead of using HTA as a mere administra-
tive procedure to fulfill (inter)national requirements.
Finally, suggestions are provided to strengthen HTA in
CEE countries through cooperation, mutual learning, a
common accreditation of HTA bodies and increased net-
work building among CEE HTA experts.
Keywords Health technology assessment Bulgaria
Czech Republic Hungary Poland Romania
JEL Classification I 180
Introduction: health technology assessment in Central
and Eastern Europe
Countries of Central and Eastern Europe (CEE) share a
common past but have, since 1989, taken different routes in
the organization and financing of their health care systems.
All have undertaken various reforms in order to improve
their health systems’ performance. Health technology
assessment (HTA) as a tool for informing decision-making
on value for money of publicly reimbursed health tech-
nologies and their conscious introduction and use has been
L. Gula
´csi (&)F. Rencz
Department of Health Economics, Corvinus University of
Budapest, F}
´r 8., 1093 Budapest, Hungary
A. M. Rotar N. S. Klazinga
Department of Social Medicine, University of Amsterdam,
Meibergdreef 9, 22660, 1100 DD Amsterdam, The Netherlands
M. Niewada
Department of Experimental and Clinical Pharmacology,
Medical University of Warsaw, _
Zwirki i Wigury 61, Warsaw,
M. Niewada
HealthQuest Consulting Company, 63 Mickiewicza Street,
01-625 Warsaw, Poland
O. Lo
Department of Public Policy, Central European University, 1051
Budapest, Na
´dor utca 9, Budapest, Hungary
G. Petrova
Department of Social Pharmacy and Pharmacoeconomics,
Faculty of Pharmacy, Medical University, Sofia, Bulgaria
I. Boncz
Institute for Health Insurance, Faculty of Health Sciences,
University of Pe
´cs, Ma
´ria u. 5-7., 7621 Pecs, Hungary
Eur J Health Econ (2014) 15 (Suppl 1):S13–S25
DOI 10.1007/s10198-014-0590-8
one possible avenue to increase efficiency of health sys-
tems, one that many CEE countries have considered and to
some extent implemented. Compared to a decade ago, there
has been a significant increase of activity related to HTA
for decision-making purposes in Central and Eastern Eur-
ope. In this paper we evaluate the developments in the field
of HTA to date in five CEE countries (Poland, the Czech
Republic, Hungary, Romania and Bulgaria), with a focus
on its institutionalization, standardization of methodology,
use of HTA in practice and capacity-building.
Materials for this paper were collected through document
analysis and pooling of country expertise. Experts from all
countries under study were involved in the systematic dis-
cussion of the situation in their country, based on a common
set of questions. Country descriptions were further vali-
dated and refined through consultation with other CEE HTA
experts and policy-makers as well as through discussions
based on draft texts amongst the authors.
This article starts by describing the context of health
care spending in the selected countries. Next, the HTA
situation in the five countries is described, comparing the
institutionalization, standardization and professionalization
of HTA, as well as its use by decision-makers. Finally, an
assessment of current issues with HTA in CEE is presented
and suggestions are put forward for further progress of
HTA in the region.
HTA in context: expenditure on health care in CEE
Economic situations, as well as spending on health care,
vary among individual CEE countries (see Table 1). Sim-
ilarly, pharmaceutical expenditure per capita and its growth
rate (2003–2011) for these countries differ significantly
(Fig. 1). There are countries such as Hungary with high per
capita pharmaceutical expenditure and very high, some-
times double digit, yearly growth rates. Yearly growth rate
was very high, for instance, in Romania (19.2 % from 2007
to 2008), although the starting level of per capita phar-
maceutical expenditure was very low. Both per capita
expenditure and its growth rate were stable in the Czech
Republic in this period. The yearly growth rate was
between 1.9 and 8.2 % in Poland, although Poland started
from a low spending level and in 2011 its drug budget was
still much lower compared to other CEE countries except
Romania. Bulgaria is difficult to analyze due to lack of
Without suggesting any straightforward relationship
between health care expenditure in CEE countries and the
use of HTA in pricing and reimbursement decision-making,
it is important to keep the diverse context in mind as we turn
to a qualitative overview of the role of HTA in Poland, the
Czech Republic, Hungary, Romania and Bulgaria.
Table 1 GDP and expenditure on health care in CEE countries, 2011
Country GDP per capita
(current US$)
Total health expenditure per
capita (current US$)
Total health
expenditure (% of
Public health
expenditure (% of
Private health
expenditure (% of GDP)
Bulgaria 7,287 522 7.3 4 3.2
Czech Republic 20,580 1,507 7.4 6.2 1.2
Hungary 13,909 1,085 7.7 5 2.7
Poland 13,382 899 6.7 4.8 1.9
Romania 8,539 500 5.8 4.7 1.2
Source The World Bank DataBank, available: 16/11/2013
Fig. 1 Changes in pharmaceutical expenditures in US$ PPP per
capita in CEE countries between 2003 and 2011 (or nearest year).
Miscellaneous: pharmaceutical expenditures cannot be separated and
include medical non-durables. Sources: OECD Statistics Database,
Eurostat Statistics Database, available: 10/11/2013. http://epp.euro
S14 L. Gula
´csi et al.
The Agency for Polish Health Technology Assessment
(AHTAPol) was established in 2005 and published its first
HTA guidelines in 2007. The current version of HTA
guidelines was published in 2010 [1]. Additionally, the
Minister of Health issued in April 2012 an official state-
ment on the minimum requirements for HTA reports sup-
porting reimbursement applications, setting of the official
sales price or increasing the official sales price of a drug, a
special purpose dietary supplement or a medical device [2].
Both the statement and HTA guidelines specify how to
prepare the HTA report that is submitted to the Ministry of
Health (MoH) and then transferred to AHTAPol, where the
processes of assessment and appraisal take place and a
statement by the expert Transparency Council (until 2011
known as the Consultative Council) is issued, followed by
a final recommendation from the President of AHTAPol.
There is a standardized format for recommendations that
currently covers: statement on public financing, justifica-
tion, objective, health problem, description of technology,
alternative technology, efficacy, safety, relation of cost to
health effects, impact on payer budget, recommendation
from HTA institutions from other countries, course of
preparing the recommendation. Since 2009 AHTAPol has
been an independent legal entity with its own budget,
operating at the national level under supervision of the
MoH. Manufacturers are obliged to pay a fee (25,000)
after every submitted reimbursement application. The
AHTAPol team consists of around 60 qualified employees
and the annual budget is about 650 000. AHTAPol
assesses and appraises all medical technologies, drugs,
devices, and other services (i.e. screenings or other health
orientated programs funded through local authorities’
budgets) that are claiming public funding. The role of
AHTAPol covers the assessment and appraisal of HTA
reports including scoping (definition of the decision prob-
lem), systematic review of clinical findings, economic
evaluation, and budget impact analysis. An important issue
is the cost-effectiveness threshold of 3 9GDP per capita/
QALY (quality-adjusted life-year) that has been published
in the Reimbursement Act and affects all medical tech-
nologies claiming public funding [3].
Statements of the Transparency Council, the President’s
recommendations and meeting proceedings are available
on the AHTAPol website ( The
majority of reports are submitted by the pharmaceutical
industry and prepared by consulting companies.
Between 1 January 2007 and 31 January 2014, 543
reimbursement recommendations were made: 516 on drugs
and 27 on non-drug medical technologies.
There have been three reviews and evaluations of
AHTAPol recommendations for drug therapies published
so far [46]. Kolasa et al. evaluated the recommendations
for drug therapies issued between 2007 and 2009 and
assessed the impact for policy-making [4]. Altogether 151
recommendations of drug therapies were evaluated: the
number of positive and negative recommendations was 88
and 63, respectively. The reasons for negative recommen-
dations were: insufficient clinical data (32 cases), poor
efficacy or safety (19 cases), unacceptable cost-effective-
ness/cost-utility ratio (9 cases), an unacceptable budget
impact (2 cases) and risk of off label use (1). From the 88
positive recommendations, 33 were classified as for use
with major restrictions, 40 with minor restrictions and 15
without restrictions. A comparison of 67 recommendations
issued in 2008 in Poland with the Scottish Medicinal
Consortium’s decisions [5] showed that among clinical
reasons, inappropriate comparators were the most frequent
cause of negative recommendations and rejections in
Scotland; however, in Poland safety concern was one of the
most often cited reason for rejection.
Another evaluation of the published AHTAPoL recom-
mendations was performed and published by Niewada et al.
[6]. All 344 recommendations completed before 7 October
2011 and available on the AHTAPoL website were ana-
lyzed: 218 positive (62.8 %) and 126 negative (37.2 %)
recommendations. Clinical efficacy, impact of hard end-
points, safety, cost-effectiveness, and formal issues were
explicitly discussed by the Consultative Council in 238
(69.2 %), 169 (49.1 %), 155 (45.1 %), 140 (40.7 %) and 47
(13.7 %) recommendations, respectively. Altogether, 106
(30.8 %) recommendations included cost/QALY and 193
(56.1 %) budget impact estimates. Negative recommenda-
tions (n=126) were made due to unsatisfying and unfa-
vorable results, most important arguments were: clinical
efficacy (68 recommendations, 54 %), impact on hard
endpoints (48, 38.1 %), safety (57, 45.2 %), cost-effec-
tiveness (56, 44.4 %), budget impact (17, 13.5 %) and other
formal issues (61, 17.7 %). No clear relationship was
observed between cost-effectiveness and budget impact and
positive or negative recommendations, while clinical
aspects seem to be more important than economic ones.
Clinical efficacy and safety profile were found to contribute
most to the final recommendations. No empirical threshold
value for cost-effectiveness and budget impact analyses that
would separate positive and negative recommendations
could be identified. Clinical efficacy and safety profile were
found to contribute most to the final recommendations.
Czech Republic
The Czech Republic does not have a formal HTA body in
the sense of an independent agency (such as AHTAPol in
Poland) or a unit with the MoH (such as TAHD in
Health technology assessment S15
Hungary). The creation of an HTA agency was on the
agenda of the Ministry of Health in 2013 [7] and several
organizational setups were considered throughout
2012–2013 [8,9]. The new minister of health (in office
since January 2014) has so far not mentioned HTA as a
policy priority and the fate of a future HTA body is now
On the other hand, the State Institute for Drug Control
[10,11], which has been responsible since 2008 for pricing
and reimbursement (P&R) decisions in the Czech Repub-
lic, has recently developed a certain interest in HTA,
although the institute does not claim to do HTA per se but
focuses rather on further developing and formalizing the
use of pharmacoeconomics. SU
´KL’s future initiatives in
the field could be affected by the dismissal of the institute’s
director in mid February 2014—the new minister of health
expressed dissatisfaction with the institute’s work on
pricing and reimbursement and put forward that SU
should be ‘‘more flexible and under greater control of the
state’’ [12].
P&R decisions are made by SU
´KL in a joint procedure.
The application dossier, mandatory for all new pharma-
ceuticals in order to be covered by public health insurance,
must include a pharmacoeconomic analysis of cost-effec-
tiveness and budget impact analysis in addition to clinical
information and other elements required by law. Organi-
zational, social and other issues considered by multidisci-
plinary HTA analysis are not taken into account by the
Cost-effectiveness and budget impact analyses are typ-
ically carried out by the marketing authorization holder or
consultancies; SU
´KL enters only at the appraisal stage—
which is, as the institute is not an HTA body with advisory
functions but a regulator for P&R, identical with the
decision-making phase. There is no separation of the
appraisal/decision stages within SU
´KL from an organiza-
tional point of view. SU
´KL’s decision-making style has
been described as a ‘‘bureaucratic process’’ by some
observers [13], with an emphasis on legal and formal
transparency of procedures. SU
´KL’s P&R staff has mainly
a legal or pharmacy and medical background, with little
formal academic training in HTA, as there are no master’s
or doctoral study programs in the Czech Republic spe-
cialized primarily in HTA, and only a few which cover
health economics, mainly at post-master level. SU
published, in October 2012, official guidelines for budget
impact analysis and in February 2013 for pharmacoeco-
nomic analysis, as well as checklists for both [10,11].
Cost-utility analysis with QALY (or LYG) is preferred by
the institute; only permanently reimbursed products can be
in principle considered comparators.
The institute’s decisions are publicly available on the
internet and contain an overview of the dossiers as well as
related reasoning. Typically, a decision would provide a
detailed overview of the procedural and legal aspects as
well as brief summaries of evidence provided by the
applicants and used by SU
´KL for appraisal. This evidence
generally includes clinical and sometimes health economic
publications in the Czech language or in English.
Throughout the process, third parties (especially profes-
sional associations of specialists) can provide statements to
inform the decision. Marketing authorization holders have
the right to appeal to the Ministry of Health if they do not
agree with a decision.
Unlike in Poland, there is no legally binding official
financing threshold of cost-effectiveness requirement;
´KL is merely required by Law 48/1997 Article 39b to take
into account the drug’s cost-effectiveness and budget
impact. However, the institute did, in 2013, repeatedly
mention in its decisions a ‘‘generally accepted willingness-
to-pay threshold’’, set at the WHO-recommended 3 times
GDP/QALY [14]. It is at this point unclear how strict the
institute will be in denying reimbursement to drugs that fail
to pass under the threshold. Drugs applying for reimburse-
ment under the special category of ‘‘highly innovative
medicinal products’’ are in any case exempt from having to
prove their cost-effectiveness. Article 40 of Ordinance
376/2011 defines highly innovative medicinal products
(Vysoce inovativnı´le´c
ˇı´pravek, VILP) by their clinical
characteristics in considerable detail. In short, VILPs are
products for severe diseases which either reduce adverse
effects compared to existing treatment, offer clinical benefits
for diseases without known effective therapy, or for which
there is a lack of cost-effectiveness or real-life clinical out-
comes data if available data points to benefits of the product
compared to existing treatment. Regarding the temporary
nature of reimbursement of VILPs, see also the April 2012
Opinion of the Ministry of Health on reimbursement of
highly innovative products, which opens the possibility of
longer reimbursement than the 3 years’ maximum [15].
Drugs approved for reimbursement in the past have been
known to surpass the threshold. This is especially true for
orphan drugs [16]. The weight of economic considerations in
´KL’s decisions is unknown; to the best of our knowledge
no study of its decisions has been done to this date.
We observe in the Czech Republic a hybrid situation: on
the one hand, we see a body which is highly active, for-
malized and transparent in appraising cost-effectiveness.
Moreover, this body has regulatory capacity and its
appraisals of cost-effectiveness analyses are immediately
translated into decisions—which is quite rare also in
countries with longer traditions in HTA [17]. On the other
hand, the body does not show significant interest in other
aspects of HTA. For the moment, cost-effectiveness and
budget impact analysis seem sufficient for decision-making
S16 L. Gula
´csi et al.
The Office of Health Technology Assessment (OHTA) was
established in 2004. OHTA, as an assessment and appraisal
unit of the MoH has the task of providing an organizational
framework for HTA that serves as the basis for the subsidy
approval decisions of the National Health Insurance Fund
Administration (NHIFA). OHTA performs assessments of
drugs (since 2004) and medical devices (since 2007). In
2012, OHTA was integrated into the National Institute for
Quality and Organisational Development in Healthcare and
Medicines, and was re-named as Technology Appraisal
Head Department (TAHD). TAHD carries out assessment,
a formal procedure including the evaluation of the sub-
mitted economic dossier which is a legally required part of
each company submission. In 2002, the Ministry of Health
released guidelines for conducting health economic anal-
yses which determine the methodological issues of health
economic evaluations. The current version of the guideline
was issued in 2013 [18]. In this guideline, technologies
claiming for public funding are declared as cost-effective
under the threshold of 29GDP per capita/QALY, and
proclaimed not cost-effective if the ICER is higher than 39
GDP per capita/QALY.
Between 2004 and 2010 altogether 997 company sub-
missions were received by OHTA and evaluated by its staff
and a further 250 were received and evaluated after 2011
until the end of 2013 [19]. Companies have to pay a con-
tribution fee which is, for pharmaceuticals, under the nor-
mal procedure 1.5 million HUF (5,068) per submission.
Details on TAHD assessments, aspects of decision making
and recommendations are not publicly available. The final
reimbursement decisions made by NHIFA can be found on
the NHIFA website. In 2004, 20 % of the submissions
contained a health economic analysis, while in 2010 this
rate was more than 80 % [19]. OHTA/TAHD published
one systematic literature review (as partial HTA) about
drug eluted stents [20]. Itemized funding refers to a case-
based reimbursement of new medical technology when not
hospitals but NHIFA buys the medical devices or medi-
cations (e.g. biological drugs) and high-value medical
interventions (e.g. PET, CT) from manufacturers, finances
them item by item according to protocol, and distributes
them to hospitals for the treatment of selected patients. A
reimbursement priority score card was introduced by a
ministerial decree in 2010 for the evaluation of new hos-
pital medical technologies financed through the DRG
scheme or itemized funding [21].
Between 2010 and September 2013, 14 company sub-
missions were assessed and appraisal decisions were made.
Technologies included drug pumps, test strips, joint pros-
thesis, monochromatic polarized light, laboratory assays or
a valve replacement system [22]. Recommendations and
Table 2 Reimbursement priorities for hospital medical technologies
in Hungary
Priorities Maximum number
of points
I. Priorities of the health care system 20
I.1. National public health programs 6
I. 2. Health policy priorities 7
I. 3. Total health gain 7
II. Severity of the disease 15
II.1. Acute life-threatening disease 13–15
II.2. Chronic life-threatening disease 10–12
II.3. Acute non-life-threatening disease 8–9
II.2. Chronic non-life-threatening disease 6–7
III. Equity 15
III.1. Size of the target patient population 8
III.2. Accessibility 7
IV. Cost-effectiveness, quality of life 30
IV.1. ICER 15
IV.2. Health gain per patient 15
V. Budget impact 10
VI. Opinions from Hungary and abroad 10
VI.1. Professionals College in Hungary 3
VI.2. International experiences 3
VI.3. Available level of evidence 4
Total 100
Source 28/2010. (12/05/2010) Decree of the Ministry of Health in
I.1. National public health programs procedures gain points which pro-
mote one of the following public health actions: I.1.1. National Action
Plan for Child Health, I.1.2. National Action Against Cancer, I.1.3.
Hungarian National Cardiovascular Program, I.1.4. National Mental
Health Program
I.2. Health policy priorities I.2.1. Procedures that improve efficiency of
the health care system, I.2.2. Procedures that reduce or substitute the
length of hospital stay, I.2.3. Telemedicine: use of telecommunication in
health care service, I.2.4. Minimally invasive or non-invasive procedures,
I.2.5. Procedures that promote rehabilitation, I.2.6. Treatments that affect
the etiology of the disease not symptomatic treatments, I.2.7. Preventive
I.3. Total health gain considering QALYs, DALYs or life-years gained, a
procedure with high societal QALY or life years gained or low DALY
receives more points
II. Severity of the disease acute life-threatening diseases gain more points
and chronic non-life-threatening diseases fewer points
III.1. Size of the target patient population the smaller patient population
gains more points. III.2. Accessibility: procedures gain more points which
are available across the whole country
IV.1. ICER incremental cost-effectiveness ratio, IV.2. Health gain per
patient: considering QALYs, DALYs or life-years gained per patient, a
procedure with high societal QALY or life years gained or low DALY
receives more
V. Budget impact procedures gain more points that result in smaller out-
flows or larger savings in the National Health Insurance Fund
VI.1–2. The College of Professionals in Hungary; International experi-
ences professional opinions and international experiences must be con-
sidered, VI.3. Available level of evidence according to the hierarchy of
evidence, the highest evidence gains 4 points and the levels beneath count
0.5 point less per level
Health technology assessment S17
results of these appraisals are not publicly available. In this
priority scoring system, the maximum achievable score is
100 (Table 2). Scoring is done by NHIFA, technologies
reaching 60 points become potential candidates to be
financed through DRG. Technologies are to be financed if
they receive at least 60 points and reach at least 40 % of
achievable points of all the six criteria.
Pharmaceutical expenditures were growing rapidly over
the past decade in Romania, until the financial crisis of
2008 (Fig. 1). This persistent growth became one of the
concerns of the external creditors of the Romanian gov-
ernment, the International Monetary Fund (IMF) and the
World Bank (WB). Following their suggestions, the gov-
ernment expressed its engagement in initiating develop-
ment of HTA by the end of 2011 as a cost containment
mechanism [23]. As a result, the Romanian government,
financed by the WB, contracted as consultants NICE
International (UK) to provide recommendations on how to
reform the health care system. Among others (e.g. the
revision of the basic benefit package), the advice was to
create a de facto HTA process in order to increase the
transparency and efficiency of decision-making [24,25].
On 24 April 2012, the Romanian government made the
first step in embedding HTA in health care governance.
The first phase was to create a legal framework, followed
by the development of a methodology and a submission
process only for new drugs. The legal framework for HTA
was created through Government Decision 351/2012,
which was an amendment of a previous Government
Decision 144/2010, regarding the organization and func-
tion of the MoH [26]. As a result of this legislation, an
HTA unit was set up within the MoH in late 2012. The
mandate of the HTA unit, introduced by the legislation, is
broad: HTA can be applied to all existing medical tech-
nologies such as pharmaceuticals, medical devices, health
policies and public health. According to an HTA guideline,
published in 2013 by the MoH [27], the assessment of
innovative drugs is made using a 6-item scoring chart
Table 3 Technology
assessment criteria Romania
SMR (Service Me
´dical Rendu –
therapeutic value) index
classifies the importance of an
intervention such as major (I),
important (II), moderate (III),
weak (IV), and insufficient to
justify a reimbursement (V).
HAS Haute Autorite
´de Sante
NICE National Institute of
Clinical Excellence, SMC
Scottish Medicines Consortium,
AWMSG All Wales Medicines
Strategy Group
No. Criteria Points
A Results of HTA evaluation HAS, France
A3 Therapeutic value is insufficient 0
B Results of HTA evaluation NICE, SMC, AWMSG, United Kingdom
B1 Approved reimbursement without restriction 1
B2 Approved reimbursement with restriction 0.5
B3 Not reimbursed 0
C Reimbursement status in EU
C1 Reimbursed in minimum 16 and maximum 24 EU countries 2
C2 Reimbursed in minimum 11 and maximum 15 EU countries 1.5
C3 Reimbursed in minimum 6 and maximum 10 EU countries 1
C4 Reimbursed in minimum 1 and maximum 5 EU countries 0.5
D Relative efficacy
D1 Superior relative efficacy vs comparator/active comparator or placebo 2
D2 Non-inferior relative efficacy vs comparator/active comparator or placebo 1
D3 Lower relative efficacy vs comparator/active comparator or placebo 0
E Relative safety
E1 Lower side effects vs comparator/active comparator or placebo 2
E2 Similar/equal side effects vs comparator/active comparator or placebo 1
E3 More side effects vs comparator/active comparator or placebo 0
F Patient reported outcomes
F1 Superior PRO vs comparator/active comparator or placebo 2
F2 Similar/equal PRO vs comparator/active comparator or placebo 1
F3 Lower PRO vs comparator/active comparator or placebo 0
S18 L. Gula
´csi et al.
where the maximum number of points is 10, and for a
positive reimbursement recommendation, the pharmaceu-
tical product has to score at least 6 (Table 3).
Based on Table 3we can conclude that the decision on
reimbursement is reached by assessing two distinct types of
Reimbursement recommendations given by HAS,
France, and 3 HTA bodies in the UK (NICE/SMC/
AWMSG), and reimbursement status in other EU
member states;
Clinical profile of the intervention: relative efficacy,
relative safety and patient reported outcome (PRO)
In the scoring scheme, all items have equal weights and
none of them represent a knock-out criterion. Apart from the
documents referring to the 6 scoring criteria, budget impact
data is required in the reimbursement dossier. However, this
has the role only of informing the decision-maker on the
potential total expenses of a given technology and does not
influence the final scoring. Supporting documentation has to
be submitted according to a required structure which is
critically assessed by means of a checklist. Both the structure
and the checklist are adapted from the tools developed by the
Canadian think-tank EVIDEM [29].
The supporting documentation has to be submitted to
MoH and received by the HTA unit and the Specialty
Committees. The HTA unit reviews all the documentation,
reports, the reimbursement status of the given drugs in EU
countries, the clinical efficacy, safety, PRO data and ana-
lyzes the budget impact. Specialty Committees review only
clinical efficacy, safety and PRO. The final scoring consists
of an average of the grading given both by the HTA unit
and the Specialty Committees.
Biosimilars are assessed slightly differently. For a
positive recommendation they need to be accepted already
for reimbursement in a certain number of EU countries, out
of the number of countries where the product is marketed.
The maximum price for which they can apply is set by law
to a maximum 80 % of the original INN (International
Nonproprietary Name) price.
By August 2013 the MoH published a list of 167 dos-
siers received [30]. According to law, all these applications
were supposed to be assessed and followed by a final
recommendation in a maximum of 55–60 days after the
day of application. In reality, this timeline was more than
doubled. On 15 November 2013, the MoH started to pub-
lish its appraisals with the commitment for the rest to come
in the following weeks [31]. By late December, the reviews
of the HTA unit and the Specialty Committees for all 167
dossiers received by August were published. This was
followed shortly by a report of the National Committee,
summarizing the appraisals, which also included the deci-
sion of the final reimbursement recommendation and the
need for prescription guidelines and restrictions [32].
Additionally, a new element was included in this docu-
ment: ‘‘conditional reimbursement’’ for 12 months. In this
timeframe, the manufacturer should collect and submit data
from health economics analysis and budget impact. How-
ever, there are no guidelines or specific requirements as to
how to satisfy this requirement.
The pricing and reimbursement processes are controlled by
the National Pricing and Reimbursement Council (NPRC)
which is responsible for the inclusion and exclusion of
pharmaceutical products on the Positive Drugs List (PDL),
as well as its amendments. The NPRC, responsible for
HTA assessment and appraisal, was established in April
2013 by the Council of Ministers and has its own budget, as
well as nearly 40 employees. NPRC decisions are based on
legislative requirements of the Law for Medicine [33], the
Health Insurance Act [34] and related regulations [35].
Overall inclusion in PDL takes at least 60 days. Adapted or
locally prepared pharmacoeconomic analysis as well as
budget impact analysis must be part of the company sub-
mission. No HTA guidelines have been published yet.
Company submissions received by NPRC are assessed by
external experts in pharmacoeconomics, appointed by the
minister of health.
Current requirements for gaining reimbursement are:
(a) a registered price in Bulgaria, (b) a positive reim-
bursement decision in at least in 5 EU countries (c) favor-
able results from pharmacoeconomic analysis submitted
with the application.
Only medicinal products included in the PDL can be
reimbursed by public funds. Once a product has a mar-
keting authorization it must have its price registered, for
OTC products, or regulated, for prescription medicines.
Pharmaceutical products for retail sale are subject to
maximum prices registration. The maximum price of a
prescription product (referred to as ‘‘approved ceiling
price’’) is subject to regulation and approval by the NPRC.
To obtain approval, the manufacturer or holder of the
marketing authorization must submit to the NPRC an
application detailing the elements included in the ceiling
price. The application form is available on the site of the
NPRC. The level of payment for medicinal products with
the same INN and the same formulation reimbursed by
NHIF is determined by the abovementioned HTA criteria.
Clinical efficacy, safety data and results from health eco-
nomics analysis are taken into account. Submissions are
evaluated by the Pricing and Reimbursement Committee:
Health technology assessment S19
Table 4 Comparison of HTA in 5 CEE countries
Poland Czech Republic Hungary Romania Bulgaria
1. Formalization and institutionalization
Legal enforcement of
2004 No 2004 2013 2013
AHTAPoL No HTA body;
Human resource
capacity (HTA)
60 People No HTA
12–14 People 2 People 1–2 People
Technologies assessed Pharmaceuticals, medical devices and all
other medical services claiming public
Pharmaceuticals Pharmaceuticals, medical
devices, hospital medical
Pharmaceuticals, medical devices, medical imaging
technologies, and all other medical services claiming
public funds
2. Standardization
Official HTA guideline
´KL TAHD (MoH) HTA unit (MoH) NPRC
Local data requirements Yes Yes Yes Yes Yes
Criteria for positive
Efficacy, safety, ICER less than 39GDP/
capita, BIA and risk of off-label use
ICER &39
Efficacy, safety, ICER less than
2–39GDP/capita, BIA
HAS, NICE/SMC/AWMSG recommendation,
reimbursement status in EU, relative efficacy and safety,
Expert opinion
Public health priorities Yes Yes Yes Yes Yes
Public health priorities
linked to decision
No No Yes for hospital medical
No No
3. Execution
Application fee Yes Yes Yes No No
National/regional HTA
National National National National National
Number of decisions 870 NA
997 (2004–2010) and 250
between 2010 and 2013
167 271
Number of decisions on
742 NA
NA 167 271
Number of positive
decisions on drugs
547 NA
NA 130 NA
Published appraisals 870 NA No 167 No
4. Further professionalization
Shortage in trained
Yes Yes Yes Yes Yes
Academic educational
Yes Yes Yes No No
Professionals responsible for HTA
´KL&s decisions are mandatory for any change of P&R for all pharmaceuticals
S20 L. Gula
´csi et al.
the decision is based on experts’ opinion. Since its estab-
lishment, the NPRC has assessed 271 medicines included
in the PDL; detailed reports are not publicly available.
In all five CEE countries studied, HTA activities have been
developed and have become internalized in the decision-
making processes on technologies over the past decade.
However, there are important similarities and differences.
Table 4summarizes the main characteristics. It groups the
characteristics by level of formalization and institutionali-
zation (legislation, official institutes with HTA tasks and
their embedding in the health care system), by nature of the
standardization (HTA guidelines, standard methodology,
national criteria for decision-making), by execution
(number and types of decisions made) and by profession-
alization (capacity building).
Formalization and institutionalization
HTA has been embedded in the law of four countries, in
Hungary and Poland in 2005 and in Romania and Bulgaria
in 2013. The Czech Republic has no legal embedding of
HTA but CEA and BIA are mandatory requirements. Each
of the countries studied has an HTA body: one (AHTAPol)
is a legally independent organization with its own budget
and a staff of 60 professionals, another (SU
´KL) has no
separate HTA capacity, while the other three are relatively
small units with 2–4 HTA professionals within the health
care ministry or national insurance institute. Only the
Polish and Hungarian HTA bodies are members of INA-
HTA, although representatives from all five countries
participate in the current EUnetHTA Joint Action 2 as well
as in the European HTA Network established in 2013 by
Directive 2011/24/EU on cross-border health care.
In all five countries HTA bodies play a role in the
decision-making process, although their importance and
competences vary. Stages of assessment and appraisal (in
the sense of evaluation and recommendation, respectively,
as understood in the UK context by NICE) [36] are more or
less intertwined, with most bodies producing recommen-
dations (appraisals) based on a review of company sub-
missions rather than in-depth assessments; the procedure
seems to have more of a technical administrative nature.
Only reports provided by the Polish AHTAPol and the
Hungarian TAHD contain de novo analyses. Final deci-
sions are typically made by the ministry of health, with the
exception of the Czech Republic where SU
´KL’s decision is
binding unless appealed against.
Pharmaceuticals and medical devices are the most fre-
quently assessed technologies. In most of the countries all
medical services claiming public funds are subject to HTA,
in some countries management and policy tools are inclu-
ded as well. The status of vaccines is somewhat different:
they are considered as any other drug and the normal rules
of HTA assessment and appraisal apply in Bulgaria and
Poland, while in Hungary and Romania HTA assessment
and appraisal is not required.
HTA guidelines are provided to industry as a guide to
create company submissions and are comparable in the
CEE countries. Guidelines are also very similar to the ones
in other EU countries, methodology is standardized and
there is no important difference between old EU member
states and CEE countries.
Guidelines require information about clinical efficacy
and safety of the new medical technologies, systematic
reviews, meta-analysis (mainly drugs), epidemiology of the
given disease, disease burden, results from health eco-
nomics analysis and patient reported outcomes. Local data
are required to be used in industry economic dossiers for
submissions. However, there is limited experience in most
of the countries in analyzing published RCTs or other
results (patient level study data from trials is not required in
CEE countries). There is a shortage of input data to local
health economics analysis (costs, unit costs, heath status,
QoL). Only a limited number of registries that can be used
as a local data-sources are maintained; in several cases
insurance databases are used as quasi-registries.
Results from health economics and HTA from other
countries, especially England (NICE) are used, however,
HTA guidelines do not provide methodological support on
how to adapt and transfer these results to CEE jurisdictions.
Transferability of HTA results between countries implies
the question as to what kind of data on effectiveness and
costs can be transferred from one country to another. Up to
now, data from clinical trials have been widely used in
many countries without national/local participation in the
trial. Hence findings on efficacy in trials might in practice
have different levels of effectiveness in different countries
depending on the functioning of the health care system.
Epidemiological data like incidence or mortality might also
be different among countries; however, their transfer is
accepted in many cases. Cost data seem to be the most
country specific issue, which means that applicants must
use country specific cost data in the submission. However,
due to the large number of submissions this is not a realistic
requirement either. Available data have to be used; if the
data is from a similar country in terms of economic
development and overall funding levels in health care, as
for instance from Poland to Hungary, this might be easier;
if data comes from NICE or another Western European
Health technology assessment S21
agency, a sophisticated methodology is needed to transfer
data in a valid way to the local context.
These transferability issues are all the more pressing
where there is no link between public health priorities and
reimbursement decision-making. Although four out of the
five countries state that their country has specific public
health priorities, it is far from clear how this influences the
decision-making process for reimbursement of
There are differences in the criteria for positive rec-
ommendations in CEE countries related to safety issues,
implementation of financing thresholds and the importance
of reimbursement status in other countries. Safety issues
are important and have a greater influence on reimburse-
ment decisions in Poland than perhaps in other countries in
Europe [5]. The quality of clinical trial evidence, used by
the Food and Drug Administration (FDA) varied widely
across indications, the effectiveness and safety of newly
approved therapeutic agents might not be well understood
[36]. A similar review has not been published yet about the
clinical trial evidence used by the EMA; however, the
findings might be generalizable regarding EMA. Due to the
shortage of registries, the capacity to control safety prob-
lems in the CEE might be weaker than in developed
countries. As with effectiveness and costs, it is unclear
whether safety findings from other countries can be safely
transferred to the situation in CEE countries. The financing
threshold is also an important issue: a 39GDP/capita
threshold was implemented officially in Poland and in
Hungary, and tends to be used in the Czech Republic,
Romania and Bulgaria. Providing all drugs under the
threshold and none above might be an attractive decision-
making approach due to its simplicity. However, despite
the existing academic consensus, thresholds alone are not
sufficient for assessing the interventions’ value for money,
and a series of other important factors have to be taken into
consideration [3739]. It is unclear how flexible or rigid the
approach of CEE decision-makers will be with regards to
Reimbursement decisions from other countries, mainly
from NICE, are taken into consideration in all CEE
countries studied. Officially, Romania declared in its HTA
guidelines that reimbursement decisions are based on
decisions from four HTA agencies (UK and France) and
reimbursement practice in other EU member states. On the
one hand this might be a good strategy if the main aim is to
avoid major mistakes in reimbursement decisions; on the
other hand, if the drug is cost-effective and reimbursed (for
whatever reason) in France and in the UK it does not imply
that this drug is cost-effective in Romania and, what is
even more important, if the given drug is cost-effective in
Romania it does not mean that this drug is really needed
given the national public health priorities or, if needed, is
necessarily fundable from public sources. In countries like
England or France the final coverage decisions are the
results of lengthy negotiations with industry and are not
necessarily applicable in other countries with different
economic and health care contexts.
If the aim is to maximize value in health care in CEE
countries there is another topic that needs attention: drugs
already under the reimbursement scheme, the ‘‘old drugs’’.
Copies of products which were patented before 1988–2000
are still available on CEE markets and represent an
important share in the turnover of domestic manufacturers.
Some products have been deleted from the list of available
drugs, usually at the request of the manufacturers. A
number of these drugs were never evaluated and their
effectiveness and cost-effectiveness are unknown.
According to some studies from Hungary, even the clinical
efficacies of some of these drugs are clearly lacking or
questionable, yet these drugs are still reimbursed with
significant budget impact [40]. The economic evaluation of
old drugs presents a major challenge for HTA bodies in
CEE countries. Many of these drugs were never interna-
tionally marketed or are no longer marketed in other EU
countries. As a consequence, no good quality clinical evi-
dence is available on the efficacy of these drugs, or the
available evidence is 20–40 years old. It is also unlikely
that clinical trials will ever be conducted for these drugs by
either pharmaceutical manufacturers or governments [41].
In summary, standardization of HTA in the countries
studied seems very much in line with the international
methodological approaches towards HTA. However, the
tendency to build HTA on effectiveness, safety and cost
data and even norms that stem from more advanced HTA
agencies in Europe has its limitations, and transferability
should be assessed carefully. Furthermore, linkage to
national public health needs and assessment of ‘‘old drugs’
needs more attention.
In all five countries a large number of company submis-
sions are regularly assessed and many appraisal decisions
are made. This large number refers to various administra-
tive procedures: one drug can be assessed for more than
one indication and also already reimbursed drugs can apply
for reimbursement for new indications. An evaluation of
the functioning of this process in practice has only be
performed and published in Poland. Poland also seems the
most transparent in the publication of the details of
appraisal reports in the public domain.
In the Czech Republic, SU
´KL does not carry out
assessments, only appraisal decision-making. The process
is relatively transparent (decisions are available online and
include reasoning) and there are official guidelines. In the
S22 L. Gula
´csi et al.
other countries processes focus on the assessment and
appraisal of dossiers submitted by industry and although
the methodology has been standardized, transparency on
the actual execution of the assessment and appraisal can
still be improved.
In all countries the focus in the execution seems to be on
the administrative procedures of assessment and appraisal
and only limited resources seem to be available to assess
transferability of effectiveness, safety and cost data from
elsewhere and/or de novo analyses with local data. Hence
the danger lurks that HTA in CEE countries remains
reduced to a merely technical administrative process based
on a rationale that has not been fine-tuned for the national
Although all five countries have academics and civil ser-
vants who are knowledgeable and have been trained in the
field of HTA, the overall capacity is still limited. Three of
the five countries (Poland, Czech Republic and Hungary)
have started training schemes to increase the number of
HTA professionals. Whether the human resource capacity
in a given country is sufficient or not is difficult to judge.
Professionals with some knowledge in HTA are employed
by MoHs, consultancy agencies, pharmaceutical compa-
nies, and academic institutions. Professionals are hired
from other countries as well. From a comparative per-
spective, though, Poland is clearly best equipped in terms
of human resources whose number seems to be propor-
tional to fulfill the mission of AHTAPoL.
However, more important than training and headcounts
of professionals as such is the question of whether HTA in
CEE countries will develop in scope and depth in the
direction of the performance of national analytical studies
based on local data and the national context that reflect a
reliable and valid approach towards value creation. This
will depend on whether HTA is merely an administrative
procedure, checking submissions from industry against a
set of criteria, or includes de novo execution of analyses
grounded in local context. In addition, the future of HTA
will depend on how seriously policy-makers take HTA
conclusions and how evidence-based policy processes in
health care overall take place. A negative scenario is an
HTA practice fuelled by industry with little counterbal-
ancing power by health care administrators and policy-
makers, resulting in a bureaucratic decision-making pro-
cess copying results from elsewhere that have not been
validated for the local context. A positive scenario is fur-
ther professionalization of both government HTA staff and
awareness of policy-makers, resulting in transparent deci-
sion-making processes through which evidence from
industry is weighed systematically against societal values
and priorities furnished by national studies based on local
Health technology assessment has, over the past decade,
been developed and implemented in Poland, Hungary,
Romania, Bulgaria, and to some extent also in the Czech
Republic. These five CEE countries have formal require-
ments for HTA and HTA institutes, although organiza-
tional embedding, size and importance for decision-making
differ. Standards for HTA are largely modeled after inter-
national examples. However, after the first phase of for-
malization, institutionalization and standardization, HTA
in CEE countries now seems to be at crossroads. It can
remain a technical administrative exercise to assess and
appraise technologies using effectiveness, safety and cost
data which comes from elsewhere and whose transfer-
ability can be questioned. It can also develop further to a
more robust form of HTA where local data serve increas-
ingly as input for analysis and where decisions are
grounded in national priorities and values.
To achieve the latter, the following actions can be
considered. First, local data on effectiveness, safety and
costs could be shared amongst CEE countries. It can be
expected that these data will increasingly become avail-
able and, given the similarities between CEE countries in
economic development and health care systems, sharing
of the data would enhance transferability compared with
the present situation. Second, a common training of HTA
staff might ensure that stress is put on insights and
methodologies which are presently needed in order to
bring HTA in the region to the next phase. Third, a
mutual assessment and recognition of HTA bodies
through certification or accreditation might be considered.
Of course, this depends on whether countries agree on
developing their HTA to a more mature level, but an
international form of assessment and recognition, as exists
in other areas such as the accreditation of accreditation
organizations for hospitals as run through the Interna-
tional Society for Quality in Health Care, is worth con-
sidering. Lastly, it is important that HTA experts in CEE
countries form their own community and exchange their
knowledge and experience, which has accumulated over
the past years by active participation in international HTA
initiatives, including notably the EUnetHTA and EU-
ROREGIO II initiatives: out of the 40 partners of the
current EUnetHTA Joint Action 2 (2012–2015), 11 are
from CEE countries. These are health ministries, national
schools of public health, quality and accreditation insti-
tutes, and an HTA agency (AHTAPol). Strengthening of
the network of HTA experts in CEE countries should
Health technology assessment S23
facilitate knowledge sharing and promote a more rigorous
and robust approach to HTA in the region.
Acknowledgments The authors would like to thank Prof. Tomasz
Pasierski, Head of the Cardiology and Cardiovascular Diseases Unit
at the Specialist Hospital in Miedzylesie, Department of Bioethics in
the Medical University of Warsaw and Head of the Transparency
Board of AHTAPol for his invaluable comments on drug safety
perception in reimbursement decision-making in Poland and to Dr.
´zsef Gajda
´csi, Deputy Director General, Dr. Pe
´ter Varga, Head of
Analysis, Medical Expertise and Controlling, National Health Insur-
ance Fund Administration, Hungary and Gabriella Jo
´na, Head of
TAHD, Hungary for their helpful comments and contribution.
Proofreading of the manuscript was supported by the TA
4.2.2./B-10/1-2010-0023 project.
1. Order No 1/2010 of President of AHTAPol dated 4 January 2010
on guidelines on health technology assessment. English—http://
HTA_eng_MS_29062009.pdf (02-02-2014)
2. Regulation of the Minister of Health of 2 April 2012 on the
minimum requirements to be satisfied by analyses accounted for
in the applications for reimbursement and setting the official sales
price and for increasing the official sales price of a drug, a special
purpose dietary supplement, a medical device, which do not have
a reimbursed counterpart in a given indication. http://www.aotm.
mum_requirements_03042012_eng.pdf (02-02-2014)
3. Act on the Reimbursement of Drugs, Food Products for Special
Dietary Purposes and Medical Devices with Public Funds 12 May
2011, Journal of Laws 2011 No 122, Poz. 696
4. Kolasa, K., Schubert, S., Manca, A., Hermanowski, T.: A review
of health technology assessment (HTA) recommendations for
drug therapies issued between 2007 and 2009 and their impact on
policymaking processes in Poland. Health Policy 102, 145–151
5. Kolasa, K., Wasiak, R.: Health technology assessment in Poland
and Scotland: comparison of process and decisions. Int. J. Tech.
Assess. Health Care 27(1), 70–76 (2011)
6. Niewada, M., Polkowska, M., Jakubczyk, M., Golicki, D.: What
influences recommendations issued by the agency for health
technology assessment in Poland? A glimpse into decision
makers’ preferences, Value in Health Regional Issues 2, 267–272
7. C
ˇTK. Heger dal vla
ˇ14 opatr
´ke zleps
´ve zdra-
votnictvi (2012)
8. Vepr
ˇek, P.: Reforma zdravotnictvı
´a role HTA C
´. PowerPoint presentation (2012)
9. Vepr
ˇek, P. HTA v C
´republice (2013)
10. Sta
ˇad pro kontrolu le
ˇiv. SP-CAU-027—W Postup pro
´dopadu do rozpoc
11. Sta
ˇad pro kontrolu le
ˇiv. 01-A-SP-CAU-028 Postup pro
12. Ministerstvo zdravotnictvı
ˇR. Ministr Ne
ˇek odvolal s
´ho u
´stavu pro kontrolu le
ˇiv. Press Release. http://www.
ustavu-pro-kontrolu-leciv-_8725_1.html (2014)
13. Bo
¨hm, K., Landwehr, C.: The Europeanization of health care
coverage decisions: EU-regulation, policy learning and coopera-
tion in decision-making. J. Eur. Integr. 36(1), 17–35 (2014).
14. Dolezal, T.: Hranice ochoty platitt ve sve
ˇv podmı
´ch C
ˇR. In
´z cyklu ,,Dialogem k reforme
,,Hranice ochoty platit—proc
ˇ, jak, kdy?’’ Prague, 22 May 2013
15. Vrubel, F.: Stanovisko k u
´m vysoce inovativnı
´ch pr
˚., Pub. L. No. MZDR10517/2012.
tivnich-lecivych-pripravku_6116_1954_3.html (2012)
16. Prochazkova, M., Kubickova, P., Mazelova, J., Hambalek, J.,
Heislerova, M.: PHP130—pricing and reimbursement system of
orphan drugs in the Czech Republic. Value Health 16(7), A474
(2013). doi:10.1016/j.jval.2013.08.880
17. Allen, N., Pichler, F., Wang, T., Patel, S., Salek, S.: Development
of archetypes for non-ranking classification and comparison of
European National Health Technology Assessment systems.
Health Policy 113(3), 305–312 (2013). doi:10.1016/j.healthpol.
18. MoH 2013 Az Emberi Erıforra
´sok Miniszte
´riuma szakmai ir-
´nyelve, az ege
´gtani elemze
´sek ke
´hez, 2013.
¨K. 3. sza
´m EMMI ko
´ny 2 (hata
´lyos: 2013.03.01)
19. Jo
´na G.: A GYEMSZI Technolo
´ly tap-
´l. Corvinus Health Policy Series Conference
2011.10.03. Budapest
20. Jo
´na, G., Kozma, P.O., Na
´dudvari, N., Ka
´ti, K.: Gyo
´stentek alkalmaza
´sa percutan coronaria intervencio
A szakirodalom szisztematikus a
´se, ESKI 2006 Budapest
21. 28/2010. (12/05/2010) Decree of the Ministry of Health in
22. Endrei, D., Molics, B., A
´goston, I.: Multicriteria decision analysis
in the reimbursement of new medical technologies: real word
experiences from Hungary, Value in Health. doi:10.1016/j.jval.
24. Lopert, R., Ruiz, F., Chalkidou, K.: Applying rapid ‘de-facto’
HTA in resource-limited settings: experience from Romania.
Health Policy 112, 202–208 (2013)
25. Ruiz, F., Lopert, R., Chalkidou, K.: Technical assistance in
reviewing the content and listing processes for the Romanian
basic package of health services and technologies, NICE Inter-
national, January 2012.
27. MoH
pdf and MoH 2013 August
12749 (2013)
33. Law for medicines in human medicine. Governmental Gazette N
31, April 13 2007; last amended Gov. gazette 1, January 3, 2014
34. National Health Insurance Act. Governmental Gazette 70, June
19, 1998, last amended Gov. Gazette 1, January 3, 2014
35. Ministry Council Decree 97/19.04.2013, Governmental Gazette
36. NICE: Developing NICE multiple technology appraisals (MTAs),
oping_nice_multiple_technology_appraisals.jsp, Retrieved on
7/3/2014 (2013)
S24 L. Gula
´csi et al.
37. Downing, N.S., Aminawung, J.A., Shah, N.D., Krumholz, H.M.,
Ross, J.S.: Clinical trial evidence supporting FDA approval of
novel therapeutic agents, 2005–2012. JAMA 311(4), 368–377
(2014). doi:10.1001/jama.2013.282034
38. Cleemput, I., Neyt, M., Thiry, N., Laet, C., Leys, M.: Using
threshold values for cost per quality-adjusted life year gained in
health care decisions. Int. J. Technol. Assess. Health Care 27,
71–76 (2011)
39. McCabe, C., Claxton, K., Culyer, A.J.: The NICE cost-effec-
tiveness threshold. What it is and what that means. Pharmaco-
economics 26(9), 733–744 (2008)
40. Gula
´csi, L., Pe
´ntek, M., Brodszky, V.: Az ege
¨gyi tech-
´k finanszı
´sa; tudjuk-e mit veszu
¨nk a millia
´ly 2, 3–27 (2011)
41. Gula
´csi, L., Boncz, I., Drummond, M.: Issues for countries
considering introducing the ‘fourth hurdle’; the case of Hungary.
Int. J. Technol. Assess. Health Care 20(3), 337–341 (2004)
Health technology assessment S25
... Literature (Friedberg et al. 1999, Gulácsi et al. 2014 shows that HTA has an essential role in the public fi nance of healthcare concerning cost savings. However, HTA is a frequent topic of international studies; there have only been a few studies in the Czech Republic seeking solutions for such a phenomenon (Gulácsi et al. 2014). ...
... Literature (Friedberg et al. 1999, Gulácsi et al. 2014 shows that HTA has an essential role in the public fi nance of healthcare concerning cost savings. However, HTA is a frequent topic of international studies; there have only been a few studies in the Czech Republic seeking solutions for such a phenomenon (Gulácsi et al. 2014). We attempt to fi ll this gap in the research and show the progress of HTA in the Czech Republic in the fi eld of medical devices. ...
... Th e Agency for Polish Health Technology Assessment (AHTAPol) was established in 2005 and published its fi rst HTA guidelines in 2007, the Offi ce of Health Technology Assessment (OHTA) was established in 2004 in Hungary and was renamed the Technology Appraisal Head Department (TAHD) in 2012, and an HTA unit was set up within the Ministry of Health in Romania in 2012. Reimbursement decisions from other countries, mainly from the National Institute for Health and Care Excellence (NICE), are considered in these countries (Gulácsi et al. 2014). ...
Full-text available
Health technology assessment (HTA) has become the systematic evaluation of health technology’s properties and effects that inform decision-makers. The implementation and expansion of HTA can contribute to slowing down burgeoning healthcare costs. In the Czech Republic, elements of HTA are quite standardly used in pharmacoeconomics, but questions arise on the use of HTA of medical devices. The theoretical framework developed is followed by a case study of the Czech Republic to assess whether the use of HTA of medical devices in the Czech Republic is implemented. This study uses publicly available resources, mainly public health acts and public notices related to HTA. We examined the institutionalisation of HTA for medical devices (HTA applied only at a selected area of medical devices) in the Czech Republic and compared Czech’s HTA principles of medical devices to the HTA Core Model. It was found that the HTA process used for medical devices is very limited in the Czech Republic. Our data show that HTA was officially established, but in reality, the medical devices have not been assessed following HTA principles.
... The context and structure of HTA systems reflect health system priorities and underpin a country's history, culture, values and preferences. Therefore, HTA is a concept with many facets and may differ in its focus and method, its governance and role, scope and remit, the assessment method employed and its impact on coverage decisions [6][7][8][9][10][11][12]. Taking into consideration these variations, it is important to study the different HTA parameters that can influence the way HTA systems are set up, operate and are integrated within national policies. ...
... Assessment and appraisal are the two different facets of HTA [20,24,30]. Assessment refers to a process of collecting, reviewing and synthesising clinical and economic evidence to support funding decisions [8,16]. Appraisal uses the same clinical and economic evidence but interprets it in the context of the healthcare system in question and takes into account factors that may be of relevance in that context [8,16]. ...
... Assessment refers to a process of collecting, reviewing and synthesising clinical and economic evidence to support funding decisions [8,16]. Appraisal uses the same clinical and economic evidence but interprets it in the context of the healthcare system in question and takes into account factors that may be of relevance in that context [8,16]. These contextual factors are known as social value judgements and can be both explicitly recognised, such as the end-of-life criteria in England and severity in France, or implicit, for example, the possible burden on patients' activities of daily living or the impact on family and carers [31]. ...
Full-text available
Health technology assessment (HTA) systems across countries vary in the way they are set up, according to their role and based on how funding decisions are reached. Our objective was to study the characteristics of these systems and their likely impact on the funding of technologies undergoing HTA. Based on a literature review, we created a conceptual framework that captures key operating features of HTA systems. We used this framework to map current HTA activities across 32 countries in the European Union, the UK, Canada and Australia. Evidence was collected through a systematic search of competent authority websites and grey literature sources. Primary data collection through expert consultation validated our findings and further complemented the analysis. Sixty-three HTA bodies were identified. Most have a national scope (76%), are independent (73%), have an advisory role (52%), evaluate pharmaceuticals predominantly or exclusively (76%), assess health technologies based on their clinical and cost-effectiveness (73%) and involve various stakeholders as members of the HTA committee (94%) and/or through external consultation (76%). The majority of HTA outcomes are not legally binding (81%). Although all study countries implement HTA, the way it fits into decision-making, negotiation processes, and coverage and funding decisions differs significantly across countries. HTA is a dynamic and transformative process and there is a need for transparency to investigate whether evidence-based information influences coverage decisions.
... HB-HTA provides context-specific and methodologically sound analysis [9]. HB-HTA should include the integration of the following principles: providing synthetic information for hospital decision makers; aiming to define leadership and partnerships; establishing a strategy for HB-HTA models, including HB-HTA units; and targeting the economic aspect in order to allocate adequate resources that ensure the proper operation of the HB-HTA model within a health system [2,10,11]. ...
... Information about the environment in which the HB-HTA operates comprises not only the HTA framework and the role of the HTA agency, but also the financing scheme and reimbursement of new medical technologies [9,[28][29][30]. Many countries have different organizational frameworks of HB-HTA, so there is no uniform approach. ...
Full-text available
This paper is aimed at exploring the role of the HB-HTA ecosystem as an important pathway for popularizing the implementation of innovations in healthcare organizations. The scientific debate has largely been focused on the rising importance of HB-HTA and the principles guiding the process. Solutions implemented by individual countries differ, which may be rooted in historical, cultural, and institutional differences. Our understanding of the impact of individual countries’ healthcare systems on HB-HTA solutions and infrastructure still lacks a basis in interpretative studies. A conceptual framework is proposed to assess the aptness of the HB-HTA model designed for hospitals operating in a country or region, focused on the concepts of adaptiveness and responsiveness to features of the healthcare system present there. A tool is proposed for investigating factors that are likely to assist the successful implementation of the HB-HTA ecosystem. A dynamic SWOT analysis on the case of the HB-HTA model designed for Poland provides interesting insights into the building of the conceptual framework. The results of this study help explain how to create an HB-HTA model that is best adapted to the regional or national healthcare system, including potential risks and opportunities.
... As of late 2021, Austria has used a reimbursement system consisting of three tiers subsequent to price setting since 2005 [29]. In Czechia, the State Institute for Drug Control has been responsible for pricing and reimbursement decisions since 2008 and manages reimbursement the same way [30]. In Italy, AIFA (Agenzia Italiana del Farmaco) has assumed the role of HTA agency and negotiated prices on behalf of the Italian NHS (National Health Service) in a decentralised system since 2003 [31]. ...
Full-text available
In response to rapidly rising pharmaceutical costs, many countries have introduced health technology assessment (HTA) as a ‘fourth hurdle’. We evaluated the causal effect of HTA based regulation on access to pharmaceuticals by using the introduction of Germany’s HTA system (AMNOG) in 2011. We obtained launch data on pharmaceuticals for 30 European countries from the IQVIA (formerly IMS) database. Using difference-in-difference models, we estimated the effect of AMNOG on launch delay, the ranking order of launch delays, and the availability of pharmaceuticals. We then compared the results for Germany to Austria, Czechia, Italy, Portugal, and the UK. Across all six countries, launch delays decreased from the pre-AMNOG period (25.01 months) to the post-AMNOG period (14.34 months). However, the introduction of AMNOG consistently reduced the magnitude of the decrease in launch delay in Germany compared to the comparator countries (staggered DiD: + 4.31 months, p = 0.05). Our logit results indicate that the availability of pharmaceuticals in Germany increased as a result of AMNOG (staggered logit: + 5.78%, p = 0.009). We provide evidence on the trade-off between regulation and access. This can help policymakers make better-informed decisions to strike the right balance between cost savings achieved through HTA based regulation and access to pharmaceuticals.
... В систематическом обзоре, опубликованном в 2018 г., были выявлены 13 стран, для которых опубликованы данные по использованию этого порога при принятии решения о возмещении затрат на те или иные вмешательства [33]. В странах Восточной Европы, таких как Чехия и Польша, этот порог, согласно рекомендациям ВОЗ, соответствует утроенной величине ВВП на душу населения [34,35]. Вероятно, в РФ в настоящее время можно использовать аналогичный подход. ...
Full-text available
Relevance . Patients with diabetes mellitus have an increased risk of developing various infections, including those of the lower respiratory tract. Immunization with anti-pneumococcal vaccines reduces pneumonia-related hospitalizations and deaths. Aim . Evaluate the cost-effectiveness of vaccination against pneumococcal infection in 40and 65-year-old patients with type 2 diabetes mellitus (DM2). Material and methods . The analysis was carried out from the perspective of the healthcare system. The Markov model based on Russian epidemiological data, taking into account the results of foreign studies, was used. Vaccination schedules with 1 dose of 13-valent pneumococcal conjugate vaccine (PCV13) followed by 1 dose of pneumococcal 23-valent polysaccharide vaccine (PPV23) and vaccination with only 1 dose of PCV13 were evaluated. The time horizon of the study is 5 years. Costs and life expectancy were discounted by 3.5% per year. Results . Vaccination of 65-year-old patients with type 2 diabetes is characterized by extremely high cost-effectiveness (the incremental cost effectiveness ratio – ICER – for PCV13+PPV23 vaccination is 189.27 thousand rubles/QALY, and PCV13 vaccination entails a cost reduction of 371.92 rubles per 1 vaccinated). When vaccinating 40-year-old patients, the ICER for PCV13 + PPV23 vaccination is 491.31 thousand rubles/QALY, and for PCV13 – 55.31 thousand rubles/QALY. Conclusion . Vaccination against pneumococcal disease in 40and 65-year-old patients with DM2 reduces the associated morbidity and mortality and is highly cost-effective. Compared to PCV13 vaccination alone, vaccination with PCV13 followed by the introduction of PPV23 provides an increase in the number of prevented cases of the disease and the deaths caused by it, but at the same time requires additional costs.
Objectives The reimbursement process for innovative health technologies in Hungary lacks any formalized assessment of clinical added benefit (CAB). The aim of this research is to present the development, retrospective testing, and implementation of a local assessment framework for determining the CAB of cancer treatments at the Department of Health Technology Assessment of the National Institute of Pharmacy and Nutrition in Hungary. Methods The assessment framework was drafted after screening existing methods and a retrospective comparison of local reimbursement dossiers to that of German and French methods. The Magnitude of Clinical Benefit Scale of the European Society for Medical Oncology was chosen to rate the extent of CAB in oncology, as part of a conclusion complemented by the assessment of endpoint relevance and the quality of evidence. Several rounds of retrospective assessments have been conducted involving all clinical assessors, iterated with semistructured discussions to consolidate divergence between assessors. External stakeholders were consulted to provide feedback on the framework. Results Retrospective assessments resulted in average more than 75 percent concordance between assessors on each element of the conclusion. Input from ten stakeholders was also incorporated; stakeholders were generally supportive, and they mostly commented on the concept, the elements of the framework, and its implementation. Conclusions The procedure is suitable for routine use in the decision-making process to describe the CAB of antineoplastic technologies in Hungary. Further extension of the framework is required to cover more disease areas for structured and comparable conclusions on CAB of innovative health technologies.
Generalized forms of invasive meningococcal desease (IMD) are very dangerous because they have a high mortality rate. The aim of the work was to assess the cost-effectiveness of meningococcal vaccination of infants with the 4-valent MenACWY-D conjugate vaccine in the Russian Federation. Material and methods . Cost-effectiveness analysis based on epidemiological data for the Russian Federation was carried out by a modeling method with a horizon of 80 years from the position of the healthcare system and taking into account the social perspective. Vaccination costs were calculated on the basis of the registered price of the vaccine, including VAT, the costs of GFMI therapy and patient rehabilitation – based on the compulsory medical insurance tariffs in St. Petersburg for 2021. Indirect costs due to temporary disability of patients’ parents, disability of patients and premature death were estimated by the lost GDP. In the base case, costs were discounted by 3,5% per year, life expectancy – by 1,5% per year. Results . Given the assumptions made, the average lifetime burden of a child’s illness is 17,556 million rubles. (no discounting). In the base case, the incremental cost per LYG from the payer perspective – 7,821 million rubles, and from the social perspective – 3,328 million rubles. Incremental cost per QALY – 5,350 million rubles and 2,277 million rubles, respectively. The most important factors that have a significant impact on the cost-effectiveness of vaccination are the incidence of IMD, the price of the vaccine, and the value of the discounting of costs and life expectancy. Conclusions . Given the assumptions made, meningococcal vaccination of infants with 4-valent meningococcal vaccine can be considered as a viable option.
Objectives There has been a growing interest in the use of EQ-5D health outcomes measures in Latin America and the Caribbean. Population norms data provide a benchmark against which clinicians, researchers, and policy makers can compare the health status of patient, treatment, or demographic groups. This study aimed to provide EQ-5D-5L population norms for Belize. Methods The EQ-5D-5L questionnaire was included in a national survey in Belize in 2014. The survey also captured key demographic variables. EQ-5D-5L health states, EQ-5D visual analog scale (EQ VAS) scores, and EQ-5D-5L index values (based on the Trinidad and Tobago value set) were obtained for key demographic groups in Belize. Results A representative sample of 2078 respondents completed the survey. The mean index value, EQ VAS score, and ceiling level for Belize were 0.947, 82.6, and 67.8%, respectively. Similar to other Caribbean countries, Belizeans self-reported relatively high EQ VAS scores and ceiling levels compared with non-Caribbean regions. Men reported generally higher health status than women, health status declined as age rises, and the dimensions with the highest burden were pain/discomfort and mobility. Conclusions This study provides researchers and practitioners in Belize with tools to use EQ-5D-5L. Users can apply the EQ VAS scores and EQ-5D-5L states presented herein as reference values. Until an EQ-5D-5L value set is created for Belize, the Trinidad and Tobago index values can be applied to Belizean-reported EQ-5D-5L states, which can then be compared with the index values presented in this study.
Background Efficiency and transparency of pricing and reimbursement (P&R) rules and procedures as well as their implementation in South-eastern Europe (SEE) lag substantially behind Western European practice. Nevertheless, P&R systems in SEE are rarely critically assessed, warranting a detailed and wider-encompassing exploration.Objective Our study provides a comparative assessment of P&R processes for patent-protected medicines in ten SEE countries—EU member states: Croatia, Slovenia, Hungary, Romania and Bulgaria; and non-EU countries: Albania, Montenegro, Serbia, North Maceodina, Bosnia and Herzegovina. P&R systems are compared and evaluated through a research framework that focuses on: (1) public financing of patent-protected medicines, (2) definition of benefit packages, (3) requirements for the submission of reimbursement dossiers, (4) assessment and appraisal processes, (5) reimbursement decision making, (6) processes that occur post reimbursement, and (7) pricing. The study aims to contribute to the discussion on improving the efficiency and quality of P&R of patent-protected medicines in the region.Methods We conducted a non-systematic literature review of published literature, as well as policy briefs and reports on healthcare systems in the SEE region along with legal documents framing the P&R procedures in local languages. The information gathered from these various sources was then discussed and clarified through structured telephone interviews with relevant national experts from each SEE country, mainly current and former senior officials and/or executives of the funding and assessment/ appraisal bodies (total of 20 interviews conducted in late 2019).ResultsCapacity building through sharing knowledge and information on successful reforms across borders is an opportunity for SEE countries to further develop their P&R policies and increase (equitable) access to patent-protected medicines (especially expensive medicines), increasing affordability and containing costs. Simple yet robust and systematic decision-making frameworks that rely on international health technology assessment (HTA) procedures and are based on the pursuit of transparency seem to be the most cost-effective approach to strengthening P&R systems in SEE.Conclusions Further reforms aiming to develop transparent and robust national decision-making frameworks (including oversight) and build institutional HTA-related and decision-making capacity are awaited in most of SEE countries, especially the non-EU members. In non-EU SEE countries, these efforts could increase access to patent-protected medicines, which is—at the moment—very limited. The EU-member SEE countries operate more developed P&R systems but could further benefit from developing their procedures, oversight and value-for-money assessment toolbox and capacity, hence further improving the transparency and efficiency of procedures that regulate access to patent-protected medicines.
Full-text available
The National Institute for Health and Clinical Excellence (NICE) has been using a cost-effectiveness threshold range between £20 000 and £30 000 for over 7 years. What the cost-effectiveness threshold represents, what the appropriate level is for NICE to use, and what the other factors are that NICE should consider have all been the subject of much discussion. In this article, we briefly review these questions, provide a critical assessment of NICE’s utilization of the incremental cost-effectiveness ratio (ICER) threshold to inform its guidance, and suggest ways in which NICE’s utilization of the ICER threshold could be developed to promote the efficient use of health service resources. We conclude that it is feasible and probably desirable to operate an explicit single threshold rather than the current range; the threshold should be seen as a threshold at which ‘other’ criteria beyond the ICER itself are taken into account; interventions with a large budgetary impact may need to be subject to a lower threshold as they are likely to displace more than the marginal activities; reimbursement at the threshold transfers the full value of an innovation to the manufacturer. Positive decisions above the threshold on the grounds of innovation reduce population health; the value of the threshold should be reconsidered regularly to ensure that it captures the impact of changes in efficiency and budget over time; the use of equity weights to sustain a positive recommendation when the ICER is above the threshold requires knowledge of the equity characteristics of those patients who bear the opportunity cost. Given the barriers to obtaining this knowledge and knowledge about the characteristics of typical beneficiaries of UK NHS care, caution is warranted before accepting claims from special pleaders; uncertainty in the evidence base should not be used to justify a positive recommendation when the ICER is above the threshold. The development of a programme of disinvestment guidance would enable NICE and the NHS to be more confident that the net health benefit of the Technology Appraisal Programme is positive.
Full-text available
Many patients and physicians assume that the safety and effectiveness of newly approved therapeutic agents is well understood; however, the strength of the clinical trial evidence supporting approval decisions by the US Food and Drug Administration (FDA) has not been evaluated. To characterize pivotal efficacy trials (clinical trials that serve as the basis of FDA approval) for newly approved novel therapeutic agents. Cross-sectional analysis using publicly available FDA documents for all novel therapeutic agents approved between 2005 and 2012. Pivotal efficacy trials were classified according to the following design features: randomization, blinding, comparator, and trial end point. Surrogate outcomes were defined as any end point using a biomarker expected to predict clinical benefit. The number of patients, trial duration, and trial completion rates were also determined. Between 2005 and 2012, the FDA approved 188 novel therapeutic agents for 206 indications on the basis of 448 pivotal efficacy trials. The median number of pivotal trials per indication was 2 (interquartile range, 1-2.5), although 74 indications (36.8%) were approved on the basis of a single pivotal trial. Nearly all trials were randomized (89.3% [95% CI, 86.4%-92.2%]), double-blinded (79.5% [95% CI, 75.7%-83.2%]), and used either an active or placebo comparator (87.1% [95% CI, 83.9%-90.2%]). The median number of patients enrolled per indication among all pivotal trials was 760 (interquartile range, 270-1550). At least 1 pivotal trial with a duration of 6 months or greater supported the approval of 68 indications (33.8% [95% CI, 27.2%-40.4%]). Pivotal trials using surrogate end points as their primary outcome formed the exclusive basis of approval for 91 indications (45.3% [95% CI, 38.3%-52.2%]), clinical outcomes for 67 (33.3% [95% CI, 26.8%-39.9%]), and clinical scales for 36 (17.9% [95% CI, 12.6%-23.3%]). Trial features differed by therapeutic and indication characteristics, such as therapeutic area, expected length of treatment, orphan status, and accelerated approval. The quality of clinical trial evidence used by the FDA as the basis for recent approvals of novel therapeutic agents varied widely across indications. This variation has important implications for patients and physicians as they make decisions about the use of newly approved therapeutic agents.
The paper presents two cases of Europeanization in health policy – an area that has so far been viewed as hardly affected by European integration. We show that even in the less likely case of coverage decision-making, some traces of Europeanization can be found. This is possible because the Commission has a strong interest in further integration in this field and all other relevant actors have motives to at least engage in cooperation. Our first case deals with the EU’s transparency directive and shows that this has forced member states to establish formal decision-making procedures, but did not result in a harmonization of decision-making processes and institutions, which is why the Commission has fostered cooperation and networking. The second case looks at the Europeanization of health technology assessment, demonstrating how cooperation and policy learning take place and how the Commission has successfully promoted the emergence of a new policy field.
Objective This study aimed to evaluate the factors that are associated with positive (supporting public funding) and negative recommendations of the Agency for Health Technology Assessment in Poland. Methods Two independent analysts reviewed all the recommendations publicly available online before October 7, 2011. For each recommendation, predefined decision rationales, that is, clinical efficacy, safety, cost-effectiveness, and formal aspects, were sought, either advocating or discouraging the public financing. In the analysis, we used descriptive statistics and a logistic regression model so as to identify the association between predefined criteria and the recommendation being positive. Results We identified 344 recommendations—218 positive (62.8%) and 126 negative (37.2%). Negative recommendations were better justified and also the comments were less ambiguous in accordance with the recommendation (except for clinical efficacy). In general, the specified criteria supported the decision (either positive or negative) in 209 (60.8%), 107 (31.1%), 124 (36.0%), 96 (27.9%), and 61 (17.7%) recommendations, respectively, and ran contrary to the actual decision in the remaining ones. Threshold values for either cost-effectiveness or budget impact distinguishing positive from negative recommendations could not be specified. The following parameters reached statistical significance in logistic regression: clinical efficacy (both explicitly positive and explicitly negative evaluations impacted in opposite directions), lack of impact on hard end points, unfavorable safety profile, cost-effectiveness results, and formal shortcomings (all reduced the probability of a positive recommendation). Conclusions Decision making of the Agency for Health Technology Assessment in Poland is multicriterial, and its results cannot be easily decomposed into simple associations or easily predicted. Still, efficacy and safety seem to contribute most to final recommendations.
European countries are increasingly utilising health technology assessment (HTA) to inform reimbursement decision-making. However, the current European HTA environment is very diverse, and projects are already underway to initiate a more efficient and aligned HTA practice within Europe. This study aims to identify a non-ranking method for classifying the diversity of European HTA agencies process and the organisational architecture of the national regulatory review to reimbursement systems. Using a previously developed mapping methodology, this research created process maps to describe national processes for regulatory review to reimbursement for 33 European jurisdictions. These process maps enabled the creation of 2 HTA taxonomic sets. The confluence of the two taxonomic sets was subsequently cross-referenced to identify 10 HTA archetype groups. HTA is a young, rapidly evolving field and it can be argued that optimal practices for performing HTA are yet to emerge. Therefore, a non-ranking classification approach could objectively characterise and compare the diversity observed in the current European HTA environment.