ArticlePDF Available

Cost-effectiveness of artificial intelligence aided vessel occlusion detection in acute stroke: an early health technology assessment

Authors:
  • Jeroen Bosch Hospital / Radboud University Medical Centre (Radboudumc)

Abstract and Figures

Background Limited evidence is available on the clinical impact of artificial intelligence (AI) in radiology. Early health technology assessment (HTA) is a methodology to assess the potential value of an innovation at an early stage. We use early HTA to evaluate the potential value of AI software in radiology. As a use-case, we evaluate the cost-effectiveness of AI software aiding the detection of intracranial large vessel occlusions (LVO) in stroke in comparison to standard care. We used a Markov based model from a societal perspective of the United Kingdom predominantly using stroke registry data complemented with pooled outcome data from large, randomized trials. Different scenarios were explored by varying missed diagnoses of LVOs, AI costs and AI performance. Other input parameters were varied to demonstrate model robustness. Results were reported in expected incremental costs (IC) and effects (IE) expressed in quality adjusted life years (QALYs). Results Applying the base case assumptions (6% missed diagnoses of LVOs by clinicians, $40 per AI analysis, 50% reduction of missed LVOs by AI), resulted in cost-savings and incremental QALYs over the projected lifetime (IC: − $156, − 0.23%; IE: + 0.01 QALYs, + 0.07%) per suspected ischemic stroke patient. For each yearly cohort of patients in the UK this translates to a total cost saving of $11 million. Conclusions AI tools for LVO detection in emergency care have the potential to improve healthcare outcomes and save costs. We demonstrate how early HTA may be applied for the evaluation of clinically applied AI software for radiology.
Content may be subject to copyright.
vanLeeuwenetal. Insights Imaging (2021) 12:133
https://doi.org/10.1186/s13244-021-01077-4
ORIGINAL ARTICLE
Cost-eectiveness ofarticial intelligence
aided vessel occlusion detection inacute stroke:
anearly health technology assessment
Kicky G. van Leeuwen1* , Frederick J. A. Meijer1, Steven Schalekamp1, Matthieu J. C. M. Rutten1,2,
Ewoud J. van Dijk3, Bram van Ginneken1, Tim M. Govers4 and Maarten de Rooij1
Abstract
Background: Limited evidence is available on the clinical impact of artificial intelligence (AI) in radiology. Early
health technology assessment (HTA) is a methodology to assess the potential value of an innovation at an early stage.
We use early HTA to evaluate the potential value of AI software in radiology. As a use-case, we evaluate the cost-
effectiveness of AI software aiding the detection of intracranial large vessel occlusions (LVO) in stroke in comparison
to standard care. We used a Markov based model from a societal perspective of the United Kingdom predominantly
using stroke registry data complemented with pooled outcome data from large, randomized trials. Different scenarios
were explored by varying missed diagnoses of LVOs, AI costs and AI performance. Other input parameters were varied
to demonstrate model robustness. Results were reported in expected incremental costs (IC) and effects (IE) expressed
in quality adjusted life years (QALYs).
Results: Applying the base case assumptions (6% missed diagnoses of LVOs by clinicians, $40 per AI analysis, 50%
reduction of missed LVOs by AI), resulted in cost-savings and incremental QALYs over the projected lifetime (IC:
$156, 0.23%; IE: + 0.01 QALYs, + 0.07%) per suspected ischemic stroke patient. For each yearly cohort of patients
in the UK this translates to a total cost saving of $11 million.
Conclusions: AI tools for LVO detection in emergency care have the potential to improve healthcare outcomes
and save costs. We demonstrate how early HTA may be applied for the evaluation of clinically applied AI software for
radiology.
Keywords: Stroke, Artificial intelligence, Cost–benefit analysis, Computed tomography angiography, Endovascular
procedures
© The Author(s) 2021. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which
permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the
original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or
other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line
to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory
regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this
licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.
Key points
• Early health technology assessment can be used to
assess impact of AI.
e use of AI for large vessel occlusion detection may
be cost-effective.
Increased health and costs savings are expected over
the projected lifetime.
Financial investments and benefits are allocated dif-
ferently, challenging adoption.
Background
Currently, there are more than one hundred CE-certified
artificial intelligence (AI) software products for radiol-
ogy on the market, addressing a wide range of applica-
tions. Vendors often make broad claims on improving
Open Access
Insights into Imaging
*Correspondence: Kicky.vanLeeuwen@radboudumc.nl
1 Department of Medical Imaging, Radboud University Medical Center,
P.O. Box 9101, 6500 HB Nijmegen, The Netherlands
Full list of author information is available at the end of the article
Page 2 of 9
vanLeeuwenetal. Insights Imaging (2021) 12:133
healthcare and increasing efficiency, however evidence
on its impact on healthcare is generally lacking [1]. For
detection of acute ischemic stroke specifically, six regula-
tory cleared (CE and/or FDA) AI products are commer-
cially available that detect large vessel occlusions (LVOs)
on computed tomography angiography (CTA). e main
aim of these products is to improve the diagnostic accu-
racy of LVOs, while fast and accurate diagnosis is crucial
to provide appropriate treatment, especially when intra-
arterial thrombectomy (IAT) is possible, which overall
results in improved patient outcome [24].
ere is some evidence on the stand-alone diagnostic
accuracy of LVO detection software in peer-reviewed
publications (n = 2) and FDA submissions (n = 3) indicat-
ing radiologist-level performance regarding sensitivity
[59]. However, limited research has yet been published
on the added value of these algorithms on the diagno-
sis, treatment decisions, patient outcomes and societal
outcomes [10]. It therefore remains unknown to what
extend the use of AI software for LVO detection impacts
the diagnostic accuracy and, consequently can impact
healthcare outcomes and overall costs.
Early health technology assessment (HTA) is a meth-
odology to assess the potential value of an innovation in
an early stage before it has been implemented [11, 12].
e aim of early HTA is to provide insight in the poten-
tial value of new technology to inform about further
development (is it worthwhile to further develop the
technology and perform research on the technology),
positioning of the technology within the treatment path-
way (for instance should it be an addition to the current
pathway or replace something else), required specifica-
tions (minimal needed effectiveness, pricing) and future
research (what outcomes should be included in future
clinical research). Outcomes provide guidance in allocat-
ing healthcare resources in an efficient way promoting
value-based healthcare. We hypothesize that early HTA
is also a useful method to assess the potential impact of
AI applications on healthcare outcomes and costs. ere-
fore, the aim of this study, was to use early HTA to evalu-
ate the potential cost-effectiveness of using an AI tool in
ischemic stroke for intracranial LVO detection on com-
puted tomography angiography (CTA) in comparison to
standard of care.
Methods
Strategies
To demonstrate the potential value of AI-aided LVO
detection, two strategies were compared with regards
to costs and effects. e first strategy comprised cur-
rent standard of care. Patients receive head CTA with or
without CT perfusion (CTP) when suspected of ischemic
stroke based on symptoms and exclusion of other causes
as demonstrated by non-contrast CT. e images are
evaluated by a radiologist and/or neurologist on duty
after which IAT follows if the patient is deemed eligible
according to current guidelines [13].
e second strategy was defined as a theoretical strat-
egy in which AI is used as an aid for LVO detection on
CTA. For both strategies, only vessel occlusions in the
proximal anterior circulation (ICA, A1, M1, M2) were
regarded as relevant for patient selection to IAT as in
concordance with the recommendations in the current
stroke guidelines [13]. In this strategy, we assume that AI
software is capable of increasing the diagnostic sensitiv-
ity, especially for the detection of M2 occlusions, without
a decrease in specificity. False positives of the AI software
are expected to be neutralized by the judgement of the
reader in order to prevent overtreatment.
Some vendors claim that, besides providing a more
accurate diagnosis, the use of AI may lead to shorter time
to treatment, especially when it enables to bypass the
radiologist [10]. As most currently available commercial
products focus on triage and interactive decision sup-
port, we only assessed the claim that the use of AI could
provide a more accurate diagnosis, i.e. reduce the num-
ber of missed LVOs.
Model structure
We used a Markov model to demonstrate the costs and
health outcomes of the two strategies. e decision tree
represents the acute phase (first 90days) for both strate-
gies (Fig.1). e modified Rankin Score (mRS), describ-
ing the level of disability, at 90days was the initial state
for the subsequent Markov model to model outcomes
over a lifetime horizon. After each yearly cycle, over a
total of 70 cycles, patients could remain in their current
health state, have a recurrent stroke, or die according to
mortality probabilities of the general population [14]. e
analysis was conducted from a societal perspective in the
context of the United Kingdom. Modelling was done in
Microsoft Excel 16.
Population
We based our cohort (n = 71,840) predominantly on
stroke registry data from the UK [15]. e initial cohort,
on which the AI software would be applied, consisted of
patients that received CTA in the diagnostic work-up of
acute stroke. We excluded late presenters of stroke (last
seen well > 4h) as available mRS data was based on early
presenters. Of the patients, 30.6% had a LVO, of whom
43.8% were eligible for IAT treatment [15].
Little is known about the percentage of missed LVOs
in standard of care, but estimates ranging between 6 and
20% have been reported in literature [16, 17]. We there-
fore test a range of this variable in our model and zoom
Page 3 of 9
vanLeeuwenetal. Insights Imaging (2021) 12:133
in to the conservative scenario where 6% of LVOs are
missed. is number was based on an observer study
with three neuroradiologists using CTAs and CTPs for
LVO detection [16].
Other variables needed to complete the model were
compiled from previously performed large stroke stud-
ies. Population age was set at 66years based on the MR
Clean trial [3]. mRS outcomes at 90 days were based
on pooled data from large randomized trials (SWIFT
PRIME, MR CLEAN, REVASCAT, EXTEND-IA,
ESCAPE) [2]. e yearly average probability for recurrent
stroke after the acute phase (first 90days) was 2.84% [18].
Recurrent stroke in the acute phase was included in the
mRS outcome at 90days. In patients who suffered from a
recurrent stroke worse outcome with higher mRS states
were possible. mRS state distribution in patients with a
recurrent stroke were assumed to be equal to a group
of patients who did not receive IAT. Table1 provides an
overview of these model inputs.
Costs
Costs were derived from a previous study (Table2) [19].
Short term costs (< 90days) consisted of costs for treat-
ment, hospitalization and management of adverse events.
Long term costs were attributed to the different mRS
states, based on the OXVASC study, including costs of
personal social services, such as nursing and residential
care costs [20].
All costs were inflated to the level of 2019 according
to the Consumer Price Index from the Office of National
Statistics of the UK and were discounted at 4% annually
[21, 22]. We report all costs in US dollars to ease inter-
pretation by applying the 2019 average exchange rate
(£1 = $1.283) [23].
Health outcome
e health outcomes have been described in quality
adjusted life years (QALYs) with 0 meaning death and
1 being in perfect health. e mRS scores were utility
weighted based on the MR Clean trial (Table3) [26]. Util-
ities were discounted at a rate of 1.5% annually [21].
Analysis
In multiple-scenario analyses we assessed the differences
in health outcomes and costs at different algorithm per-
formance values (percentage of LVOs detected by the
AI tool that would have otherwise been missed), costs
of the AI tool (US dollar per analysis) and percentage of
missed LVOs in standard of care. A base case analysis
was performed using a combination of assumptions for
these parameters. e sensitivity of AI tools validated
stand-alone for detection of LVOs have been reported
to be in the range of 87.8–97.9% [58]. However, for this
analysis it is relevant to know how much the diagnostic
accuracy improves when a radiologist uses the AI tool as
a detection aid. As there is no evidence available on the
percentage of reduced missed LVOs through the use of
Large vessel
occlusion
No or other vessel occlusion
Patients suspected of
stroke receiving CTA
IAT eligible
Not IAT eligible
Occlusion
detected
Occlusion
not detected
mRS 0
mRS 1
mRS 2
mRS 3
mRS 4
mRS 5
Death
mRS 0
mRS 1
mRS 2
mRS 3
mRS 4
mRS 5
Death
mRS 0
mRS 1
mRS 2
mRS 3
mRS 4
mRS 5
Death
Fig. 1 Decision tree applicable to the standard of care strategy and the AI tool strategy. In the AI tool strategy the ratio of occlusions (not) detected
was altered. CTA, computed tomography angiography, IAT, intra-arterial thrombectomy
Page 4 of 9
vanLeeuwenetal. Insights Imaging (2021) 12:133
commercial AI products, we varied this number around
an assumed base case performance of 50%. e price of
the AI per case was set at $40 for the base case scenario.
As previously described the base case assumption of
missed LVOs in standard of care was set at 6%.
e reference value for one QALY was £20,000
($25,662) [27]. We performed multiple one-way sensi-
tivity analyses in which we varied the parameters that
were not included in the scenario analysis to evaluate the
robustness of the model.
is was an observational study for which the Institu-
tional Review Board approval was waived. e model is
made available on https:// www. AIfor Radio logy. com with
the possibility to adapt the variables [28].
Results
Applying AI for LVO detection has the potential to save
costs and increase patient outcomes. Table4 shows the
outcomes in different scenarios regarding the percent-
age of missed LVOs in usual care and the percentage of
missed LVOs that are detected by the innovation. No
costs for the innovation were included in these analyses.
erewith, the potential cost savings represent the maxi-
mum costs for the innovation at which the innovation is
dominant over usual care, i.e., results in both health gain
and cost savings. For example, with 1% of current missed
LVOs and 25% of detection with the innovation the maxi-
mum costs per AI analysis could be $16 for the innova-
tion to be dominant.
Figure 2 illustrates the cost saving per patient when
varying the costs per AI analysis and the percentage
reduction of missed LVOs by the AI tool, assuming a cur-
rent missed LVO rate of 6%. Here, it becomes apparent
at what cost or performance the AI tool will be cost sav-
ing. e filled green bullet demonstrates the base case
scenario (6% missed diagnoses, $40 per AI analysis, 50%
reduction of missed LVOs by AI). Table5 zooms in on
the results of the base case scenario. Here, the model pre-
dicted that the AI strategy results in both cost reduction
and improved patient outcome compared to the standard
of care. For the projected lifetime per ischemic stroke
patient, the incremental costs and incremental efficacy
were $156 ( 0.23%) and + 0.0095 QALYs (+ 0.07%)
respectively. Using the reference value of $25,662 per
QALY, 0.0095 QALY would translate to $244. For each
yearly cohort of patients in the UK this translates to a
total cost saving of $11 million and QALY gain of 682
($17.5 million).
Within a ninety-day window, the intervention scenario
led to incremental costs (IC: + $60) due to the increased
Table 1 Model input—clinical parameters
LVO, large vessel occlusion; CTA, computed tomography angiography; IAT, intra-arterial thrombectomy ; mRS, modied Rankin Scale
Model Input Base case Reference
LVO missed standard care 6% Becks et al. [16]
Reduction of missed LVO through AI detection (varied) 50%
LVO of patients suspected of stroke receiving CTA 30.6% McMeekin et al. [15]
IAT eligible of LVO population 43.8% McMeekin et al. [15]
Population age 66 MR Clean Trial [3]
mRS after LVO treated with IAT Aronsson et al. [2]
mRS 0 11%
mRS 1 18%
mRS 2 20%
mRS 3 17%
mRS 4 16%
mRS 5 4%
Death 14%
mRS after LVO treated without IAT Aronsson et al. [2]
mRS 0 5%
mRS 1 8%
mRS 2 11%
mRS 3 17%
mRS 4 27%
mRS 5 12%
Death 20%
Probability of recurrent stroke 2.84% Pennlert et al. [18]
Page 5 of 9
vanLeeuwenetal. Insights Imaging (2021) 12:133
utilization of the more expensive IAT treatment. Negli-
gible healthcare improvement (IE: + 0.0001 QALY) was
observed. A cost reduction and larger QALY gain are
expected for the lifetime duration after the acute phase
(IC: $216, IE: + 0.0094 QALY ).
Figure 3 shows the results of the one-way sensitivity
analysis taking the base case scenario as its starting point.
In all cases the incremental costs remained negative and
the incremental efficacy positive. e long-term stroke
costs was the parameter causing most variation in the
results followed by the starting age of the patient popu-
lation. Sensitivity analyses of the mRS probabilities and
utility values are reported in the Additional file 1. e
analyses did not include extra costs for possible increased
reading time for false positive cases as this only resulted
in neglectable extra costs $0.07 for each percentage
point of false positives—as shown in the Additional file2.
Discussion
We applied early health technology assessment (HTA)
to demonstrate the potential cost-effectiveness of an AI
tool. Current AI research focuses mainly on the perfor-
mance of algorithms (the means) instead of assessing the
impact on healthcare outcomes and costs, especially in
the early stage of development.
In this study, we demonstrate with the use of early
HTA that AI software for the aided detection of intracra-
nial LVO in ischemic stroke has the potential to improve
healthcare outcomes and save overall costs. e sensitiv-
ity analyses demonstrate the robustness to variation of
model parameters and assumptions. e improved LVO
detection leads to better clinical outcome on both the
Table 2 Model input—costs
* Cost of the average treatment was adjusted for the estimated frequency of the treatment according to dierent studies. For IAT eligible patients, we assumed 85%
to receive both IAT and intravenous thrombolysis (IVT), 10% to receive IAT only, and 5% to receive IV T and going for IAT but who appeared revascularised during
angiography [3, 15, 24]. For non-IAT-eligible patients we assumed 40% to receive IVT based on previous study and our local medical center data[25]
IAT, intra-arterial thrombectomy; mRS, modied R ankin Scale Costs are rounded to the nearest integer
Costs Base case Reference
Cost algorithm per analysis (varied) $40
Treatment costs*
Average costs IAT eligible $11,728 Lobotesis et al. [19], Berkhemer et al. [3], McMeekin
et al. [15], Mulder et al. [24]
Average costs non-IAT eligible $1004 Lobotesis et al. [19], Bandettini di Poggio et al. [25]
Acute stroke costs (< 90 days) Lobotesis et al. [19]
mRS 0 $4350
mRS 1 $5117
mRS 2 $5885
mRS 3 $22,695
mRS 4 $30,704
mRS 5 $36,468
mRS 6 (death) $4603
Long term stroke costs (annually) Lobotesis et al. [19]
mRS 0 $3936
mRS 1 $4631
mRS 2 $5325
mRS 3 $18,944
mRS 4 $25,631
mRS 5 $41,621
Discount costs 4% Guideline for economic evaluations in healthcare [21]
Table 3 Model input—utility values
mRS, modied Rankin Scale
Utility Dijkland et al. [26]
mRS 0 0.95
mRS 1 0.93
mRS 2 0.83
mRS 3 0.62
mRS 4 0.42
mRS 5 0.11
Discount utilities 1.5% Guideline for economic
evaluations in health-
care[21]
Page 6 of 9
vanLeeuwenetal. Insights Imaging (2021) 12:133
Table 4 Potential change in costs and QALYs when applying AI tool as opposed to usual care
Costs and QALYs are per patient receiving CTA with indication stroke, when applying AI tool as opposed to usual care with varying ratio of missed LVOs in current care
and varying rates of reduction in missed LVOs due to innovation. No costs for the AI innovation were included in this analysis
QALY, quality-adjusted life-year
Costs are rounded to the nearest integer
% LVOs missed usual
care % of missed LVOs detected by innovation
1% 25% 50% 75% 100%
1% 1$
+ 0.00003 QALY
16$
+ 0.0008 QALY
33$
+ 0.0016 QALY
49$
+ 0.0024 QALY
65$
+ 0.0032 QALY
3% 2$
+ 0.0001 QALY
49$
+ 0.0024 QALY
98$
+ 0.0048 QALY
147$
+ 0.0071 QALY
196$
+ 0.0095 QALY
6% 4$
+ 0.0002 QALY
98$
+ 0.0048 QALY
196$
+ 0.0095 QALY
294$
+ 0.0143 QALY
392$
+ 0.0190 QALY
10% 7$
+ 0.0003 QALY
163$
+ 0.0791 QALY
327$
+ 0.0158 QALY
490$
+ 0.0237 QALY
654$
+ 0.0317 QALY
20% 13$
+ 0.0006 QALY
327$
+ 0.0158 QALY
654$
+ 0.0317 QALY
981$
+ 0.0475 QALY
1,307$
+ 0.0633 QALY
Fig. 2 Scenario analysis demonstrating incremental costs. Incremental costs at varying prices for the AI tool per analysis ($0–$200) and varying
percentage of reduction of missed large vessel occlusion diagnoses (0–100%). Green circles demonstrate a cost reduction whereas red circles
signify an increase in costs. The size of the circle is related to the height of the incremental costs
Table 5 Results of base case analysis* as the difference between usual care and applying the AI tool
* Base case parameters: missed LVOs, 6%; costs per analysis, $40; reduc tion of missed LVOs, 50%
QALY, quality-adjusted life-year
Costs are rounded to the nearest integer
Incremental costs ($) Incremental ecacy (QALYs)
Population Patient % of usual care Population Patient % of usual care
Acute phase (< 90 days) $4,295,152 $60 0.60 9.45 0.0001 0.06
Rest of life phase (> 90 days) $15,510,277 $216 0.36 672.82 0.0094 0.07
Total $11,215,125 $156 0.23 682.27 0.0095 0.07
Page 7 of 9
vanLeeuwenetal. Insights Imaging (2021) 12:133
short term as well as long term resulting in reduced over-
all healthcare consumption.
Early HTA analyses are not meant to provide a firm
‘go’ or ‘no-go’ recommendation for the development or
purchasing of an innovation, but provide insights in the
direction to head regarding development, implementa-
tion and reimbursement [11]. For example in this study,
the analyses show that cost benefits are obtained in the
long term, while the costs for the software are short term
and are usually covered by the radiology department or
hospital. is observation could contribute to the debate
on the investments, financial accountability and reim-
bursement for the clinical use of AI technology.
e difficulty with doing an early HTA is that not all
data is yet known, hence it is called ‘early’. erefore, we
used data from previous studies and made some assump-
tions to complete the model. e sensitivity analysis
described, demonstrates the effects when varying the
parameters and provides context on the bias and confi-
dence. e model is made publicly available to allow for
personalization of the model and the results by adapting
the parameter values.
One of the main assumptions was the percentage of
missed LVOs. is was assumed to be 6% for the base
case scenario, but may vary depending on the clini-
cal context [16, 17]. Because of the limited evidence, we
based this rate on a study with two neuroradiologists and
one neuroradiology resident as observers. However, in
many clinical settings there might not be a specialized
physician within reach. Some centers have limited expo-
sure to stroke patients and during out-of-office hours the
studies are generally first evaluated by less experienced
observers (residents). We therefore consider 6% to be a
conservative estimate. With higher percentages of missed
LVOs, the benefits of the use of the AI software increase.
Also, we made an assumption on the capability of the
AI to reduce missed LVOs in the base case scenario as
there is limited evidence of the added value of AI tools in
LVO detection. e results of this study only hold, pro-
vided that AI tools have a positive impact on the diagnos-
tic accuracy of LVOs. erefore, we have demonstrated
the effects for an AI tool with the capability to reduce the
percentage of missed LVOs in the range of 0% to 100%. It
should be noted that this percentage cannot be directly
translated to the sensitivity of an AI-algorithm applied
stand-alone (ranging between 87.8% and 97.9% [58]), as
we assume that the cases that were missed by a physician
are also more likely to be missed by an algorithm (e.g. M2
occlusions). For that reason we went for a conservative
50% reduction of missed diagnoses in the base case sce-
nario and we advise to not use sensitivity measures pro-
vided by a vendor directly as the input to this model.
ere are some points of note to our analysis. First, in
this economic evaluation, we considered a pay-per-use
business model and assume costs per analysis. When
vendors employ a software license model, the proposed
evaluation method can still be applied. By dividing the
yearly license fee by the expected population, one obtains
an estimate of the per-analysis costs.
Second, we used data from early-presenters as the
healthcare outcome data was available for this group
only. Currently the indication for IAT is being expanded
Fig. 3 Results of one-way sensitivity analysis. The effect of varying several parameters is shown. In the left diagram the impact on the costs is
demonstrated and in the right diagram the impact on the QALYs. Light gray bars represent lower bounds, dark gray bars upper bounds. The axis
intersects at the base case results of $156 and 0.0095 QALY. LVO, large vessel occlusion; IAT, intra-arterial thrombectomy; QALY, quality-adjusted
life-year
Page 8 of 9
vanLeeuwenetal. Insights Imaging (2021) 12:133
to also include late presenters [13]. e majority of the
UK cohort (86%) was made up of early presenters. We
hypothesize that cost savings and health gain remain pos-
itive, but will be slightly reduced when including the late
presenters in the model. Other AI tools aimed at assess-
ing eligibility for IAT, such as CTP analysis, collateral
status assessment, or ASPECT scoring may have more
impact on the late presenting population, but are beyond
the scope of this study.
Lastly, we did not elaborately take into account the
interaction between the physician and the AI software.
We expect limited effects of false positives when present
at an acceptable rate. We assumed that false positive pre-
dictions of the AI software would be neutralized by the
reader and would not lead to unnecessary invasive pro-
cedures (diagnostic subtraction angiogram). False posi-
tives may contribute to a slightly longer reading time,
while true positives may speed up the reading process. As
shown in the sensitivity analyses the slightly extra reading
time resulted in neglectable extra costs.
In this work, we compared standard of care with use of
an AI tool to increase diagnostic efficacy in LVO detec-
tion in ischemic stroke. Depending on the center and the
way that stroke care is organized, there may be alterna-
tive strategies or a combination of strategies to reach the
same goal that we did not consider, such as the use of
advanced imaging techniques including CTP, additional
training of the physicians interpreting the CTA scans,
a dedicated (neuro)radiologist on call outside of office
hours, or even other types of AI tools. In further cost-
effectiveness studies these alternatives could be explored
to determine if implementing an AI tool for LVO detec-
tion is the most cost-efficient solution.
Conclusion
We demonstrate the potential cost-effectiveness of com-
puter aided LVO detection in ischemic stroke by per-
forming an early health technology assessment. Such
analysis can be used to indicate the potential efficacy of
an AI tool in an early phase to guide development and
implementation. e most important next step is to vali-
date the outcomes of the early health technology assess-
ment in clinical practice. With increasing number of AI
tools implemented in the clinic, it is important to assess
the impact of AI tools on our healthcare system. Real-life
outcome measures should be used to gain insights into
how to apply AI tools in a sensible and safe way. is is a
prerequisite to prove the claim that AI is making health-
care better and more affordable.
Abbreviations
AI: Artificial intelligence; CTA : Computed tomography angiography; CTP:
Computed tomography perfusion; HTA: Health technology assessment; IAT:
Intra-arterial thrombectomy; IC: Incremental costs; IE: Incremental effects; IVT:
Intravenous thrombolysis; LVO: Large vessel occlusion; QALY: Quality adjusted
life year.
Supplementary Information
The online version contains supplementary material available at https:// doi.
org/ 10. 1186/ s13244- 021- 01077-4.
Additional le1. Sensitivity analysis of mRS distributions and utility
values.
Additional le2. Costs false positives.
Authors’ contributions
TMG, MdR, KGvL collected the data and created the model. FJAM and EJvD
provided local data and expert opinion when assumptions had to be made.
KGvL wrote the manuscript with major support from MdR and TMG. BvG
supervised the project. All authors contributed to the draft manuscript by
providing their critical feedback. All authors read and approved the final
manuscript.
Funding
The authors state that this work has not received any funding.
Availability of data and materials
The model is made available on on https:// www. AIfor Radio logy. com with the
possibility to adapt the variables.
Declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
KGvL, FJAM, SS, MJCMR, EJvD, TMG, MdR declare that they have no competing
interests. BvG is co-founder of and receives royalties from Thirona and receives
royalties from Delft Imaging and Mevis Medical Solutions. These disclosures
are unrelated to the submitted work.
Author details
1 Department of Medical Imaging, Radboud University Medical Center, P.O.
Box 9101, 6500 HB Nijmegen, The Netherlands. 2 Department of Radiology,
Jeroen Bosch Hospital, ‘s-Hertogenbosch, The Netherlands. 3 Department
of Neurology, Donders Institute for Brain, Cognition and Behaviour, Centre
for Neuroscience, Radboud University Medical Center, Nijmegen, The Nether-
lands. 4 Department of Operating Rooms, Radboud University Medical Center,
Nijmegen, The Netherlands.
Received: 20 April 2021 Accepted: 23 August 2021
References
1. van Leeuwen KG, Schalekamp S, Rutten MJCM, van Ginneken B, de Rooij
M (2021) Artificial intelligence in Radiology: 100 commercially available
products and their scientific evidence. Eur Radiol. https:// doi. org/ 10.
1007/ s00330- 021- 07892-z
2. Aronsson M, Persson J, Blomstrand C, Wester P, Levin L-Å (2016) Cost-
effectiveness of endovascular thrombectomy in patients with acute
ischemic stroke. Neurology 86:1053–1059. https:// doi. org/ 10. 1212/ WNL.
00000 00000 002439
3. Berkhemer OA, Fransen PSS, Beumer D et al (2015) A randomized trial of
intraarterial treatment for acute ischemic stroke. N Engl J Med 372:11–20.
https:// doi. org/ 10. 1056/ NEJMo a1411 587
Page 9 of 9
vanLeeuwenetal. Insights Imaging (2021) 12:133
4. Goyal M, Menon BK, van Zwam WH et al (2016) Endovascular thrombec-
tomy after large-vessel ischaemic stroke: a meta-analysis of individual
patient data from five randomised trials. Lancet 387:1723–1731. https://
doi. org/ 10. 1016/ S0140- 6736(16) 00163-X
5. (2019) Aidoc Medical, Ltd.- BriefCase LVO - 510(k). U.S. Food & Drug
Administration
6. (2018) Viz.Al, Inc. - ContaCT - De Novo. U.S. Food & Drug Administration
7. (2020) Avicenna.ai - CINA - 510(k). U.S. Food & Drug Administration
8. Amukotuwa SA, Straka M, Smith H et al (2019) Automated detection of
intracranial large vessel occlusions on computed tomography angiog-
raphy. Stroke 50:2790–2798. https:// doi. org/ 10. 1161/ STROK EAHA. 119.
026259
9. Amukotuwa SA, Straka M, Dehkharghani S, Bammer R (2019) Fast
automatic detection of large vessel occlusions on CT angiography. Stroke
50:3431–3438. https:// doi. org/ 10. 1161/ STROK EAHA. 119. 027076
10. Hassan AE, Ringheanu VM, Rabah RR, Preston L, Tekle WG, Qureshi AI
(2020) Early experience utilizing artificial intelligence shows significant
reduction in transfer times and length of stay in a hub and spoke model.
Interv Neuroradiol. https:// doi. org/ 10. 1177/ 15910 19920 953055
11. Grutters JPC, Govers T, Nijboer J, Tummers M, van der Wilt GJ, Rovers MM
(2019) Problems and promises of health technologies: the role of early
health economic modeling. Int J Health Policy Manag 8:575–582. https://
doi. org/ 10. 15171/ ijhpm. 2019. 36
12. Ijzerman MJ, Steuten LMG (2011) Early assessment of medical technolo-
gies to inform product development and market access. Appl Health
Econ Health Policy 9:331–347. https:// doi. org/ 10. 2165/ 11593 380- 00000
0000- 00000
13. Powers WJ, Rabinstein AA, Ackerson T et al (2019) Guidelines for the Early
Management of Patients With Acute Ischemic Stroke: 2019 Update to the
2018 Guidelines for the Early Management of Acute Ischemic Stroke: A
Guideline for Healthcare Professionals From the American Heart Associa-
tion/American Stroke Association. Stroke 50:e344–e418. https:// doi. org/
10. 1161/ STR. 00000 00000 000211
14. (2017) Mortality probabilities. Statistics Netherlands’ database (CBS)
15. McMeekin P, White P, James MA, Price CI, Flynn D, Ford GA (2017) Estimat-
ing the number of UK stroke patients eligible for endovascular thrombec-
tomy. Eur Stroke J 2:319–326. https:// doi. org/ 10. 1177/ 23969 87317 733343
16. Becks MJ, Manniesing R, Vister J et al (2019) Brain CT perfusion improves
intracranial vessel occlusion detection on CT angiography. AJNR Am J
Neuroradiol 46:124–129. https:// doi. org/ 10. 1016/j. neurad. 2018. 03. 003
17. Fasen BACM, Heijboer RJJ, Hulsmans F-JH, Kwee RM (2020) CT angiog-
raphy in evaluating large-vessel occlusion in acute anterior circulation
ischemic stroke: factors associated with diagnostic error in clinical
practice. AJNR Am J Neuroradiol 41:607–611. https:// doi. org/ 10. 3174/
ajnr. A6469
18. Pennlert J, Eriksson M, Carlberg B, Wiklund PG (2014) Long-term risk and
predictors of recurrent stroke beyond the acute phase. Stroke 45:1839–
1841. https:// doi. org/ 10. 1161/ STROK EAHA. 114. 005060
19. Lobotesis K, Veltkamp R, Carpenter IH, Claxton LM, Saver JL, Hodgson R
(2016) Cost-effectiveness of stent-retriever thrombectomy in combina-
tion with IV t-PA compared with IV t-PA alone for acute ischemic stroke in
the UK. J Med Econ 19:785–794. https:// doi. org/ 10. 1080/ 13696 998. 2016.
11748 68
20. Luengo-Fernandez R, Yiin GSC, Gray AM, Rothwell PM (2013) Population-
based study of acute- and long-term care costs after stroke in patients
with AF. Int J Stroke 8:308–314. https:// doi. org/ 10. 1111/j. 1747- 4949. 2012.
00812.x
21. (2016) Guideline for economic evaluations in healthcare. National Health
Care Institute, The Netherlands *
22. (2019) Consumer Price Index UK. Office of National Statistics of the UK
23. (2019) British Pound (GBP) to US Dollar (USD) exchange rate history
24. Mulder MJHL, Jansen IGH, Goldhoorn R-JB et al (2018) Time to endo-
vascular treatment and outcome in acute ischemic stroke. Circulation
138:232–240. https:// doi. org/ 10. 1161/ CIRCU LATIO NAHA. 117. 032600
25. Bandettini-di-Poggio M, Finocchi C, Brizzo F et al (2019) Management of
acute ischemic stroke, thrombolysis rate, and predictors of clinical out-
come. Neurol Sci 40:319–326. https:// doi. org/ 10. 1007/ s10072- 018- 3644-3
26. Dijkland SA, Voormolen DC, Venema E et al (2018) Utility-weighted modi-
fied rankin scale as primary outcome in stroke trials. Stroke 49:965–971.
https:// doi. org/ 10. 1161/ STROK EAHA. 117. 020194
27. (2012) Methods for the development of NICE public health guidance
(third edition). National Institute for Health and Clinical Excellence, United
Kingdom
28. Diagnostic Imaging Analysis Group (2020) AI for Radiology. Radboud
university medical center. https:// www. AIfor Radio logy. com. Accessed 15
Jan 2021
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in pub-
lished maps and institutional affiliations.
... Importantly, CTA utilization in stroke patients has significantly increased over the years, resulting in a considerable increase of workload for radiologists [5,6]. The use of artificial intelligence (AI) software could improve accuracy in detecting occlusions and may improve workflow efficiency for radiologists [7,8]. To date, however, there is limited evidence of the added clinical value of AI software in LVO detection [7]. ...
... The use of artificial intelligence (AI) software could improve accuracy in detecting occlusions and may improve workflow efficiency for radiologists [7,8]. To date, however, there is limited evidence of the added clinical value of AI software in LVO detection [7]. The objective of our study was to evaluate the diagnostic performance of AI software in diagnosing intracranial arterial occlusions in the proximal anterior circulation at CTA and to compare it to manual reading performed in clinical practice. ...
... Although the current AI software cannot replace radiologists and senior radiology residents, our results show that it may be of added value in clinical practice by reducing the number of missed occlusions (Tables 1 and 2). AI software for intracranial arterial occlusion detection has the potential to improve patient outcome and save costs [7]. First of all, the software should have a high sensitivity, because missing a treatable occlusion may result in poorer outcome [1,2,11]. ...
Full-text available
Article
Purpose To evaluate the diagnostic performance of AI software in diagnosing intracranial arterial occlusions in the proximal anterior circulation at CT angiography (CTA) and to compare it to manual reading performed in clinical practice. Methods Patients with acute ischemic stroke underwent CTA to detect arterial occlusion in the proximal anterior circulation. Retrospective review of CTA scans by two neuroradiologists served as reference standard. Sensitivity and specificity of AI software (StrokeViewer) were compared to those of manual reading using the McNemar test. The proportions of correctly detected occlusions in the distal internal carotid artery and/or M1 segment of the middle cerebral artery (large vessel occlusion [LVO]) and in the M2 segment of the middle cerebral artery (medium vessel occlusion [MeVO]) were calculated. Results Of the 474 patients, 75 (15.8%) had an arterial occlusion in the proximal anterior circulation according to the reference standard. Sensitivity of StrokeViewer software was not significantly different compared to that of manual reading (77.3% vs. 78.7%, P = 1.000). Specificity of StrokeViewer software was significantly lower than that of manual reading (88.5% vs. 100%, P < 0.001). StrokeViewer software correctly identified 40 of 42 LVOs (95.2%) and 18 of 33 MeVOs (54.5%). StrokeViewer software detected 8 of 16 (50%) intracranial arterial occlusions which were missed by manual reading. Conclusion The current AI software detected intracranial arterial occlusion with moderate sensitivity and fairly high specificity. The AI software may detect additional occlusions which are missed by manual reading. As such, the use of AI software may be of value in clinical stroke care.
... The AI tools can help expedite this workflow and be cost-effective. A recent cost-effectiveness study of automated LVO detection-applied base case scenario (6% missed diagnosis, $40 per AI analysis, and 50% reduction of missed LVOs by AI) showed substantial cost savings and increased qualityadjusted life-years for patients with acute ischemic stroke over their projected lifetime (23). ...
Full-text available
Article
Brain imaging is necessary for understanding disease symptoms, including stroke. However, frequent imaging procedures encounter practical limitations. Estimating the brain information (e.g., lesions) without imaging sessions is beneficial for this scenario. Prospective estimating variables are non-imaging data collected from standard tests. Therefore, the current study aims to examine the variable feasibility for modelling lesion locations. Heterogeneous variables were employed in the multivariate logistic regression. Furthermore, patients were categorized (i.e., unsupervised clustering through k-means method) by the charasteristics of lesion occurrence (i.e., ratio between the lesioned and total regions) and sparsity (i.e., density measure of lesion occurrences across regions). Considering those charasteristics in models improved estimation performances. Lesions (116 regions in Automated Anatomical Labeling) were adequately predicted (sensitivity: 80.0–87.5% in median). We confirmed that the usability of models was extendable to different resolution levels in the brain region of interest (e.g., lobes, hemispheres). Patients’ charateristics (i.e., occurrence and sparsity) might also be explained by the non-imaging data as well. Advantages of the current approach can be experienced by any patients (i.e., with or without imaging sessions) in any clinical facilities (i.e., with or without imaging instrumentation).
Full-text available
Article
Objectives Map the current landscape of commercially available artificial intelligence (AI) software for radiology and review the availability of their scientific evidence.Methods We created an online overview of CE-marked AI software products for clinical radiology based on vendor-supplied product specifications (www.aiforradiology.com). Characteristics such as modality, subspeciality, main task, regulatory information, deployment, and pricing model were retrieved. We conducted an extensive literature search on the available scientific evidence of these products. Articles were classified according to a hierarchical model of efficacy.ResultsThe overview included 100 CE-marked AI products from 54 different vendors. For 64/100 products, there was no peer-reviewed evidence of its efficacy. We observed a large heterogeneity in deployment methods, pricing models, and regulatory classes. The evidence of the remaining 36/100 products comprised 237 papers that predominantly (65%) focused on diagnostic accuracy (efficacy level 2). From the 100 products, 18 had evidence that regarded level 3 or higher, validating the (potential) impact on diagnostic thinking, patient outcome, or costs. Half of the available evidence (116/237) were independent and not (co-)funded or (co-)authored by the vendor.Conclusions Even though the commercial supply of AI software in radiology already holds 100 CE-marked products, we conclude that the sector is still in its infancy. For 64/100 products, peer-reviewed evidence on its efficacy is lacking. Only 18/100 AI products have demonstrated (potential) clinical impact.Key Points • Artificial intelligence in radiology is still in its infancy even though already 100 CE-marked AI products are commercially available. • Only 36 out of 100 products have peer-reviewed evidence of which most studies demonstrate lower levels of efficacy. • There is a wide variety in deployment strategies, pricing models, and CE marking class of AI products for radiology.
Full-text available
Article
Background and Purpose— The purpose of these guidelines is to provide an up-to-date comprehensive set of recommendations in a single document for clinicians caring for adult patients with acute arterial ischemic stroke. The intended audiences are prehospital care providers, physicians, allied health professionals, and hospital administrators. These guidelines supersede the 2013 Acute Ischemic Stroke (AIS) Guidelines and are an update of the 2018 AIS Guidelines. Methods— Members of the writing group were appointed by the American Heart Association (AHA) Stroke Council’s Scientific Statements Oversight Committee, representing various areas of medical expertise. Members were not allowed to participate in discussions or to vote on topics relevant to their relations with industry. An update of the 2013 AIS Guidelines was originally published in January 2018. This guideline was approved by the AHA Science Advisory and Coordinating Committee and the AHA Executive Committee. In April 2018, a revision to these guidelines, deleting some recommendations, was published online by the AHA. The writing group was asked review the original document and revise if appropriate. In June 2018, the writing group submitted a document with minor changes and with inclusion of important newly published randomized controlled trials with >100 participants and clinical outcomes at least 90 days after AIS. The document was sent to 14 peer reviewers. The writing group evaluated the peer reviewers’ comments and revised when appropriate. The current final document was approved by all members of the writing group except when relationships with industry precluded members from voting and by the governing bodies of the AHA. These guidelines use the American College of Cardiology/AHA 2015 Class of Recommendations and Level of Evidence and the new AHA guidelines format. Results— These guidelines detail prehospital care, urgent and emergency evaluation and treatment with intravenous and intra-arterial therapies, and in-hospital management, including secondary prevention measures that are appropriately instituted within the first 2 weeks. The guidelines support the overarching concept of stroke systems of care in both the prehospital and hospital settings. Conclusions— These guidelines provide general recommendations based on the currently available evidence to guide clinicians caring for adult patients with acute arterial ischemic stroke. In many instances, however, only limited data exist demonstrating the urgent need for continued research on treatment of acute ischemic stroke.
Full-text available
Article
Background: To assess whether early health economic modeling helps to distinguish those healthcare innovations that are potentially cost-effective from those that are not potentially cost-effective. We will also study what information is retrieved from the health economic models to inform further development, research and implementation decisions. Methods: We performed secondary analyses on an existing database of 32 health economic modeling assessments of 30 innovations, performed by our group. First, we explored whether the assessments could distinguish innovations with potential cost-effectiveness from innovations without potential cost-effectiveness. Second, we explored which recommendations were made regarding development, implementation and further research of the innovation. Results: Of the 30 innovations, 1 (3%) was an idea that was not yet being developed and 14 (47%) were under development. Eight (27%) innovations had finished development, and another 7 (23%) innovations were on the market. Although all assessments showed that the innovation had the potential to become cost-effective, due to improved patient outcomes, cost savings or both, differences were found in the magnitude of the potential benefits, and the likelihood of reaching this potential. The assessments informed how the innovation could be further developed or positioned to maximize its cost-effectiveness, and informed further research. Conclusion: The early health economic assessments provided insight in the potential cost-effectiveness of an innovation in its intended context, and the associated uncertainty. None of the assessments resulted in a firm 'no-go' recommendation, but recommendations could be provided on further research and development in order to maximize value for money.
Article
Background The use of compliant dual lumen balloon microcatheters (CDLB) for the endovascular treatment of vascular malformations, wide neck aneurysms, and intracranial angioplasty (for vasospasm) is well documented. Navigation of 4 mm or larger CDLB within tortuous and small distal intracranial vessels can be challenging. Recently, the lower profile Scepter Mini balloon microcatheter (SMB) has been approved for use, with potential for improved intracranial navigation. Objective Discuss operative experience of Scepter Mini (Microvention, Aliso Viejo, CA). Methods We describe our initial experience with the SMB in a series of nine patients. Results The balloon microcatheter was used for delivery of liquid embolic in six patients (Case 1, 2, 6–9), adjunct support for delivery or positioning of the Woven Endobridge (WEB) device in two (Case 3,4), and gentle post-deployment repositioning of a WEB device in the last one (Case 5). We were able to successfully navigate the SMB over a 0.008 “micro wire to the target lesion in all the patients. We experienced initial difficulty with injecting liquid embolic in Case 2. We postulate that the SMB was in a tortuous segment of a dural vessel in this patient, and that it kinked on inflation with occlusion of the liquid embolic delivery lumen; this was overcome with slightly proximal repositioning and reinflation of the SMB. Conclusion Our initial experience shows that the SMB has potential to be useful in endovascular neurosurgical procedures requiring balloon assistance within smaller diameter blood vessels.
Article
Background Recently approved artificial intelligence (AI) software utilizes AI powered large vessel occlusion (LVO) detection technology which automatically identifies suspected LVO through CT angiogram (CTA) imaging and alerts on-call stroke teams. We performed this analysis to determine if utilization of AI software and workflow platform can reduce the transfer time (time interval between CTA at a primary stroke center (PSC) to door-in at a comprehensive stroke center (CSC)). Methods We compared the transfer time for all LVO transfer patients from a single spoke PSC to our CSC prior to and after incorporating AI Software (Viz.ai LVO). Using a prospectively collected stroke database at a CSC, demographics, mRS at discharge, mortality rate at discharge, length of stay (LOS) in hospital and neurological-ICU were examined. Results There were a total of 43 patients during the study period (median age 72.0 ± 12.54 yrs., 51.16% women). Analysis of 28 patients from the pre-AI software (median age 73.5 ± 12.28 yrs., 46.4% women), and 15 patients from the post-AI software (median age 70.0 ± 13.29 yrs., 60.00% women). Following implementation of AI software, median CTA time at PSC to door-in at CSC was significantly reduced by an average of 22.5 min. (132.5 min versus 110 min; p = 0.0470). Conclusions The incorporation of AI software was associated with an improvement in transfer times for LVO patients as well as a reduction in the overall hospital LOS and LOS in the neurological-ICU. More extensive studies are warranted to expand on the ability of AI technology to improve transfer times and outcomes for LVO patients.
Article
Background and purpose: It is currently not completely clear how well radiologists perform in evaluating large-vessel occlusion on CTA in acute ischemic stroke. The purpose of this study was to investigate potential factors associated with diagnostic error. Materials and methods: Five hundred twenty consecutive patients with a clinical diagnosis of acute ischemic stroke (49.4% men; mean age, 72 years) who underwent CTA to evaluate large-vessel occlusion of the proximal anterior circulation were included. CTA scans were retrospectively reviewed by a consensus panel of 2 neuroradiologists. Logistic regression analysis was performed to investigate the association between several variables and missed large-vessel occlusion at the initial CTA interpretation. Results: The prevalence of large-vessel occlusion was 16% (84/520 patients); 20% (17/84) of large-vessel occlusions were missed at the initial CTA evaluation. In multivariate analysis, non-neuroradiologists were more likely to miss large-vessel occlusion compared with neuroradiologists (OR = 5.62; 95% CI, 1.06-29.85; P = .04), and occlusions of the M2 segment were more likely to be missed compared with occlusions of the distal internal carotid artery and/or M1 segment (OR = 5.69; 95% CI, 1.44-22.57; P = .01). There were no calcified emboli in initially correctly identified large-vessel occlusions. However, calcified emboli were present in 4 of 17 (24%) initially missed or misinterpreted large-vessel occlusions. Conclusions: Several factors may have an association with missing a large-vessel occlusion on CTA, including the CTA interpreter (non-neuroradiologists versus neuroradiologists), large-vessel occlusion location (M2 segment versus the distal internal carotid artery and/or M1 segment), and large-vessel occlusion caused by calcified emboli. Awareness of these factors may improve the accuracy in interpreting CTA and eventually improve stroke outcome.
Article
Background and Purpose— Accurate and rapid detection of anterior circulation large vessel occlusion (LVO) is of paramount importance in patients with acute stroke due to the potentially rapid infarction of at-risk tissue and the limited therapeutic window for endovascular clot retrieval. Hence, the optimal threshold of a new, fully automated software-based approach for LVO detection was determined, and its diagnostic performance evaluated in a large cohort study. Methods— For this retrospective study, data were pooled from: 2 stroke trials, DEFUSE 2 (n=62; 07/08–09/11) and DEFUSE 3 (n=213; 05/17–05/18); a cohort of endovascular clot retrieval candidates (n=82; August 2, 2014–August 30, 2015) and normals (n=111; June 6, 2017–January 28, 2019) from a single quaternary center; and code stroke patients (n=501; January 1, 2017–December 31, 2018) from a single regional hospital. All CTAs were assessed by the automated algorithm. Consensus reads by 2 neuroradiologists served as the reference standard. ROC analysis was used to assess diagnostic performance of the algorithm for detection of (1) anterior circulation LVOs involving the intracranial internal carotid artery or M1 segment middle cerebral artery (M1-MCA); (2) anterior circulation LVOs and proximal M2 segment MCA (M2-MCA) occlusions; and (3) individual segment occlusions. Results— CTAs from 926 patients (median age 70 years, interquartile range: 58-80; 422 females) were analyzed. Three hundred ninety-five patients had an anterior circulation LVO or M2-MCA occlusion (National Institutes of Health Stroke Scale 14 [median], interquartile range: 9–19). Sensitivity and specificity were 97% and 74%, respectively, for LVO detection, and 95% and 79%, respectively, when M2 occlusions were included. On analysis by occlusion site, sensitivities were 90% (M2-MCA), 97% (M1-MCA), and 97% (intracranial internal carotid artery) with corresponding area-under-the-ROC-curves of 0.874 (M2), 0.962 (M1), and 0.997 (intracranial internal carotid artery). Conclusions— Intracranial anterior circulation LVOs and proximal M2 occlusions can be rapidly and reliably detected by an automated detection tool, which may facilitate intra- and inter-instutional workflows and emergent imaging triage in the care of patients with stroke.
Article
Background and Purpose— Endovascular thrombectomy is highly effective in acute ischemic stroke patients with an anterior circulation large vessel occlusion (LVO), decreasing morbidity and mortality. Accurate and prompt identification of LVOs is imperative because these patients have large volumes of tissue that are at risk of infarction without timely reperfusion, and the treatment window is limited to 24 hours. We assessed the accuracy and speed of a commercially available fully automated LVO-detection tool in a cohort of patients presenting to a regional hospital with suspected stroke. Methods— Consecutive patients who underwent multimodal computed tomography with thin-slice computed tomography angiography between January 1, 2017 and December 31, 2018 for suspected acute ischemic stroke within 24 hours of onset were retrospectively identified. The multimodal computed tomographies were assessed by 2 neuroradiologists in consensus for the presence of an intracranial anterior circulation LVO or M2-segment middle cerebral artery occlusion (the reference standard). The patients’ computed tomography angiographies were then processed using an automated LVO-detection algorithm (RAPID CTA). Receiver-operating characteristic analysis was used to determine sensitivity, specificity, and negative predictive value of the algorithm for detection of (1) an LVO and (2) either an LVO or M2-segment middle cerebral artery occlusion. Results— CTAs from 477 patients were analyzed (271 men and 206 women; median age, 71; IQR, 60–80). Median processing time was 158 seconds (IQR, 150–167 seconds). Seventy-eight patients had an anterior circulation LVO, and 28 had an isolated M2-segment middle cerebral artery occlusion. The sensitivity, negative predictive value, and specificity were 0.94, 0.98, and 0.76, respectively for detection of an intracranial LVO and 0.92, 0.97, and 0.81, respectively for detection of either an intracranial LVO or M2-segment middle cerebral artery occlusion. Conclusions— The fully automated algorithm had very high sensitivity and negative predictive value for LVO detection with fast processing times, suggesting that it can be used in the emergent setting as a screening tool to alert radiologists and expedite formal diagnosis.
Article
Background and aims Monitoring the quality of acute ischemic stroke (AIS) management is increasingly important since patient outcome could be improved with better access to evidence-based treatments. In this scenario, the aim of our study was to identify thrombolysis rate, reasons for undertreatment, and factors associated with better outcome. Methods From January to December 2016, individuals diagnosed with AIS at the Policlinic San Martino Hospital in Genoa, Italy, were prospectively included. Severity of stroke, site of occlusion, rate and time related in-hospital management of systemic thrombolysis, and mechanical thrombectomy were recorded. Safety and clinical outcomes were compared between different subgroups. Results Of 459 AIS patients (57.3% females, mean age 78.1), 111 received i.v. thrombolysis (24.4%) and 50 received mechanical thrombectomy (10.9%). Apart from arrival behind the therapeutic window, which was the first limitation to thrombolysis, the main reason of undertreatment was minor stroke or stroke in rapid improvement. Baseline NIHSS ≥ 8 was associated with unfavorable clinical outcome (mRS > 2) (OR 20.1; 95% CI, 1.1–387.4, p = 0.047). Age older than 80 years (OR 5.0; 95% CI, 1.4–64.1, p = 0.01), baseline NIHSS ≥ 7 (OR 20.1; 95% CI, 1.1–387.4, p = 0.047), and symptomatic intracranial hemorrhage (OR 22.9; 95% CI, 2.0–254.2, p = 0.01) proved independently associated with mortality. Conclusions i.v. thrombolysis and mechanical thrombectomy rate was higher than that of previous reports. Minor stroke or stroke in rapid improvement was a major reason for exclusion from thrombolysis of eligible patients. Higher NIHSS proved an independent predictor of unfavorable clinical outcome and death. Strategies to avoid in-hospital delays need to be enforced.
Article
Background and purpose: To evaluate whether brain CT perfusion (CTP) aids in the detection of intracranial vessel occlusion on CT angiography (CTA) in acute ischemic stroke. Materials and methods: Medical-ethical committee of our hospital was obtained and informed consent was waived. Patients suspected of acute ischemic stroke who underwent non-contrast CT(NCCT), CTA and whole-brain CTP in our center in the year 2015 were included. Three observers with different levels of experience evaluated the imaging data of 110 patients for the presence or absence of intracranial arterial vessel occlusion with two strategies. In the first strategy, only NCCT and CTA were available. In the second strategy, CTP maps were provided in addition to NCCT and CTA. Receiver-operating-characteristic (ROC) analysis was used for the evaluation of diagnostic accuracy. Results: Overall, a brain perfusion deficit was scored present in 87-89% of the patients with an intracranial vessel occlusion, more frequently observed in the anterior than in the posterior circulation. Performance of intracranial vessel occlusion detection on CTA was significantly improved with the availability of CTP maps as compared to the first strategy (p=0.023), due to improved detection of distal and posterior circulation vessel occlusions (p-values of 0.032 and 0.003 respectively). No added value of CTP was found for intracranial proximal vessel occlusion detection, with already high accuracy based on NCCT and CTA alone. Conclusion: The performance of intracranial vessel occlusion detection on CTA was improved with the availability of brain CT perfusion maps due to the improved detection of distal and posterior circulation vessel occlusions.