Farah Magrabi

• BE PhD
• Professor at Macquarie University

BACKGROUND Artificial intelligence (AI) has significant potential for improving healthcare delivery through enhanced diagnostics, streamlined operations, and predictive analytics. However, healthcare organisations face substantial challenges in implementing AI safely and responsibly. This is due to regulatory complexity, ethical considerations, and...

Health care is changing rapidly. Hospitals are, and will remain, an essential setting to deliver it. We discuss how to maximise the benefits of hospitals in the future in different geographic and health system settings, highlighting a series of cross‐cutting issues. We do this by exploring the evolving roles of hospitals and the main factors that w...

Background Health systems underwent substantial changes to respond to COVID-19. Learning from the successes and failures of health system COVID-19 responses may help us understand how future health service responses can be designed to be both effective and sustainable. This study aims to identify the role that innovation played in crafting health s...

Background The COVID-19 pandemic disrupted health systems around the globe. Lessons from health systems responses to these challenges may help design effective and sustainable health system responses for future challenges. This study aimed to 1/ identify the broad types of health system challenges faced during the pandemic and 2/ develop a typology...

Introduction At least 10% of hospital admissions in high-income countries, including Australia, are associated with patient safety incidents, which contribute to patient harm (‘adverse events’). When a patient is seriously harmed, an investigation or review is undertaken to reduce the risk of further incidents occurring. Despite 20 years of investi...

Computerised decision support (CDS) tools enabled by artificial intelligence (AI) seek to enhance accuracy and efficiency of clinician decision-making at the point of care. Statistical models developed using machine learning (ML) underpin most current tools. However, despite thousands of models and hundreds of regulator-approved tools international...

Objective To support a diverse sample of Australians to make recommendations about the use of artificial intelligence (AI) technology in health care. Study design Citizens’ jury, deliberating the question: “Under which circumstances, if any, should artificial intelligence be used in Australian health systems to detect or diagnose disease?” Settin...

With growing use of machine learning (ML)-enabled medical devices by clinicians and consumers safety events involving these systems are emerging. Current analysis of safety events heavily relies on retrospective review by experts, which is time consuming and cost ineffective. This study develops automated text classifiers and evaluates their potent...

We assessed the safety of a new clinical decision support system (CDSS) for nurses on Australia’s national consumer helpline. Accuracy and safety of triage advice was assessed by testing the CDSS using 78 standardised patient vignettes (48 published and 30 proprietary). Testing was undertaken in two cycles using the CDSS vendor’s online evaluation...

Real-world performance of machine learning (ML) models is crucial for safely and effectively embedding them into clinical decision support (CDS) systems. We examined evidence about the performance of contemporary ML-based CDS in clinical settings. A systematic search of four bibliographic databases identified 32 studies over a 5-year period. The CD...

Clinical simulation is a useful method for evaluating AI-enabled clinical decision support (CDS). Simulation studies permit patient- and risk-free evaluation and far greater experimental control than is possible with clinical studies. The effect of CDS assisted and unassisted patient scenarios on meaningful downstream decisions and actions within t...

Aims and objectives: To examine the nature and use of automation in contemporary clinical information systems by reviewing studies reporting the implementation and evaluation of artificial intelligence (AI) technologies in healthcare settings. Method: PubMed/MEDLINE, Web of Science, EMBASE, the tables of contents of major informatics journals, and...

Background Management of major hemorrhage frequently requires massive transfusion (MT) support, which should be delivered effectively and efficiently. We have previously developed a clinical decision support system (CDS) for MT using a multicenter multidisciplinary user‐centered design study. Here we examine its impact when administering a MT. Stu...

The question of whether the time has come to hang up the stethoscope is bound up in the promises of artificial intelligence (AI), promises that have so far proven difficult to deliver, perhaps because of the mismatch between the technical capability of AI and its use in real-world clinical settings. This perspective argues that it is time to move a...

Objective: This study aims to summarize the research literature evaluating machine learning (ML)-based clinical decision support (CDS) systems in healthcare settings. Materials and methods: We conducted a review in accordance with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta Analyses extension for Scoping Review). Fo...

Despite the renewed interest in Artificial Intelligence-based clinical decision support systems (AI-CDS), there is still a lack of empirical evidence supporting their effectiveness. This underscores the need for rigorous and continuous evaluation and monitoring of processes and outcomes associated with the introduction of health information technol...

Objective: To examine the real-world safety problems involving machine learning (ML)-enabled medical devices. Materials and methods: We analyzed 266 safety events involving approved ML medical devices reported to the US FDA's MAUDE program between 2015 and October 2021. Events were reviewed against an existing framework for safety problems with...

Background and objective: Early identification of patients at risk of deterioration can prevent life-threatening adverse events and shorten length of stay. Although there are numerous models applied to predict patient clinical deterioration, most are based on vital signs and have methodological shortcomings that are not able to provide accurate es...

Background: Managing critical bleeding with massive transfusion (MT) requires a multidisciplinary team, often physically separated, to perform several simultaneous tasks at short notice. This places a significant cognitive load on team members, who must maintain situational awareness in rapidly changing scenarios. Similar resuscitation scenarios h...

UNSTRUCTURED There is an urgent need to incorporate theory-informed health information technology evaluation frameworks into existing and emerging guidelines for the evaluation of Artificial Intelligence. Such frameworks can help developers, implementers, and strategic decision makers to build on existing experience and the existing empirical evide...

Given the requirement to minimize the risks and maximize the benefits of technology applications in health care provision, there is an urgent need to incorporate theory-informed health IT (HIT) evaluation frameworks into existing and emerging guidelines for the evaluation of artificial intelligence (AI). Such frameworks can help developers, impleme...

The COVID-19 pandemic serves as a clarion call to ensure health systems are better prepared to meet future emergencies. Digital Health could play a significant role in preparing health systems to bend and stretch their resources and cope with various shocks by facilitating tasks such as disease monitoring and care delivery. However, the health syst...

The effects of human-induced climate change on our planet are already largely irreversible for many centuries¹ and may remain so for at least 1000 years.² If emissions continue to grow, their effects will trigger multiple critical tipping points and event cascades that will amplify climate effects in unpredicted ways.³ Just in 2022, we have seen fl...

Objective: To summarize the research literature evaluating automated methods for early detection of safety problems with health information technology (HIT). Materials and methods: We searched bibliographic databases including MEDLINE, ACM Digital, Embase, CINAHL Complete, PsycINFO, and Web of Science from January 2010 to June 2021 for studies e...

Despite its exponential growth, artificial intelligence (AI) in healthcare faces various challenges. One of them is a lack of explainability of AI medical devices, which arguably leads to insufficient trust in AI technologies, quality, and accountability and liability issues. The aim of this paper is to examine whether, why and to what extent AI ex...

Background: Current procedures for effective personal protective equipment (PPE) usage rely on the availability of trained observers or 'buddies' who, during the COVID-19 pandemic, are not always available. The application of artificial intelligence (AI) has the potential to overcome this limitation by assisting in complex task analysis. To date,...

Objective Climate change poses a major threat to the operation of global health systems, triggering large scale health events, and disrupting normal system operation. Digital health may have a role in the management of such challenges and in greenhouse gas emission reduction. This scoping review explores recent work on digital health responses and...

Objectives: Patient portals are increasingly implemented to improve patient involvement and engagement. We here seek to provide an overview of ways to mitigate existing concerns that these technologies increase inequity and bias and do not reach those who could benefit most from them. Methods: Based on the current literature, we review the limitati...

This article provides a critical comparative analysis of the substantive and procedural values and ethical concepts articulated in guidelines for allocating scarce resources in the COVID-19 pandemic. We identified 21 local and national guidelines written in English, Spanish, German and French; applicable to specific and identifiable jurisdictions;...

The use of computerized decision support systems (DSS) in nursing practice is increasing. However, research about who uses DSS, where are they implemented, and how they are linked with standards of nursing is limited. This paper presents evidence on users and settings of DSS implementation, along with specific nursing standards of practice that are...

Objective The study sought to summarize research literature on nursing decision support systems (DSSs); understand which steps of the nursing care process (NCP) are supported by DSSs, and analyze effects of automated information processing on decision making, care delivery, and patient outcomes. Materials and Methods We conducted a systematic revi...

Background As a major public health crisis, the novel coronavirus disease 2019 (COVID-19) pandemic demonstrates the urgent need for safe, effective, and evidence-based implementations of digital health. The urgency stems from the frequent tendency to focus attention on seemingly high promising digital health interventions despite being poorly valid...

Objectives: To highlight the role of technology assessment in the management of the COVID-19 pandemic. Method: An overview of existing research and evaluation approaches along with expert perspectives drawn from the International Medical Informatics Association (IMIA) Working Group on Technology Assessment and Quality Development in Health Informat...

Objective To examine how and to what extent medical devices using machine learning (ML) support clinician decision making. Methods We searched for medical devices that were (1) approved by the US Food and Drug Administration (FDA) up till February 2020; (2) intended for use by clinicians; (3) in clinical tasks or decisions and (4) used ML. Descrip...

Healthcare 4.0 (H4.0) adapts principles and applications from the Industry 4.0 movement to healthcare, enabling real-time customization of care to patients and professionals. As such, H4.0 can potentially support resilient performance in healthcare systems, which refers to their adaptive capacity to cope with complexity. This paper explores the imp...

Background: While selecting predictive tools for implementation in clinical practice or for recommendation in clinical guidelines, clinicians and health care professionals are challenged with an overwhelming number of tools. Many of these tools have never been implemented or evaluated for comparative effectiveness. To overcome this challenge, the a...

Objective: The study sought to evaluate the feasibility of using Unified Medical Language System (UMLS) semantic features for automated identification of reports about patient safety incidents by type and severity. Materials and methods: Binary support vector machine (SVM) classifier ensembles were trained and validated using balanced datasets o...

Background: When selecting predictive tools, clinicians are challenged with an overwhelming and ever-growing number, most of which have never been implemented or evaluated for comparative effectiveness. To overcome this challenge, the authors developed an evidence-based framework for grading and assessment of predictive tools (GRASP). The objective...

We identify and describe nine key, short-term, challenges to help healthcare organizations, health information technology developers, researchers, policymakers, and funders focus their efforts on health information technology–related patient safety. Categorized according to the stage of the health information technology lifecycle where they appear,...

Background: When selecting predictive tools, clinicians are challenged with an overwhelming and ever-growing number, most of which have never been implemented or evaluated for comparative effectiveness. The authors developed an evidence-based framework for grading and assessment of predictive tools (GRASP). The objective of this study is to update...

Background: Conversational agents (CAs) are systems that mimic human conversations using text or spoken language. Their widely used examples include voice-activated systems such as Apple Siri, Google Assistant, Amazon Alexa, and Microsoft Cortana. The use of CAs in health care has been on the rise, but concerns about their potential safety risks o...

Objective: To evaluate the feasibility of a convolutional neural network (CNN) with word embedding to identify the type and severity of patient safety incident reports. Materials and methods: A CNN with word embedding was applied to identify 10 incident types and 4 severity levels. Model training and validation used data sets (n_type = 2860, n_s...

This commentary examines the problem of assuring, or establishing, justified confidence in the clinical quality, safety and security of health apps. The overall objective is to raise awareness about this often neglected topic, and to highlight the need for standards and oversight. We begin by considering the inherent complexity of formalising proce...

Background: Clinical predictive tools quantify contributions of relevant patient characteristics to derive likelihood of diseases or predict clinical outcomes. When selecting predictive tools for implementation at clinical practice or for recommendation in clinical guidelines, clinicians are challenged with an overwhelming and ever-growing number...

Objective: To summarize the research literature about safety concerns with consumer-facing health apps and their consequences. Materials and methods: We searched bibliographic databases including PubMed, Web of Science, Scopus, and Cochrane libraries from January 2013 to May 2019 for articles about health apps. Descriptive information about safe...

BACKGROUND Conversational agents (CAs) are systems that mimic human conversations using text or spoken language. Their widely-used examples include voice-activated systems like Apple Siri, Google Assistant, Amazon Alexa, or Microsoft Cortana. The use of CAs in healthcare has been on the rise, but concerns about their potential safety risks often re...

BACKGROUND When selecting predictive tools, for implementation in clinical practice or for recommendation in clinical guidelines, clinicians and healthcare professionals are challenged with an overwhelming number of tools. Many of these tools have never been implemented or evaluated for comparative effectiveness. To overcome this challenge, the aut...

Background While selecting predictive tools for implementation in clinical practice or for recommendation in clinical guidelines, clinicians and health care professionals are challenged with an overwhelming number of tools. Many of these tools have never been implemented or evaluated for comparative effectiveness. To overcome this challenge, the au...

Abstract: Background: When selecting predictive tools, for implementation in their clinical practice or for recommendation in clinical guidelines, clinicians are challenged with an overwhelming and ever-growing number of tools. Many of these have never been implemented or evaluated for comparative effectiveness. To overcome this challenge, the auth...

Background: When selecting predictive tools, clinicians and healthcare professionals are challenged with an overwhelming number of tools, most of which have never been evaluated for comparative effectiveness. To overcome this challenge, the authors developed and validated an evidence-based framework for grading and assessment of predictive tools (G...

Background: Clinical predictive tools quantify contributions of relevant patient characteristics to derive likelihood of diseases or predict clinical outcomes. When selecting a predictive tool, for implementation at clinical practice or for recommendation in clinical guidelines, clinicians are challenged with an overwhelming and ever growing number...

Background: Recent advances in natural language processing and artificial intelligence have led to widespread adoption of speech recognition technologies. In consumer health applications, speech recognition is usually applied to support interactions with conversational agents for data collection, decision support, and patient monitoring. However,...

BACKGROUND Recent advances in natural language processing and artificial intelligence have led to widespread adoption of speech recognition technologies. In consumer health applications, speech recognition is usually applied to support interactions with conversational agents for data collection, decision support, and patient monitoring. However, li...

Background The contribution of usability flaws to patient safety issues is acknowledged but not well-investigated. Free-text descriptions of incident reports may provide useful data to identify the connection between health information technology (HIT) usability flaws and patient safety. Objectives This article examines the feasibility of using inc...

Objectives: This paper draws attention to: i) key considerations for evaluating artificial intelligence (AI) enabled clinical decision support; and ii) challenges and practical implications of AI design, development, selection, use, and ongoing surveillance. Method: A narrative review of existing research and evaluation approaches along with expert...

Objective Clinicians using clinical decision support (CDS) to prescribe medications have an obligation to ensure that prescriptions are safe. One option is to verify the safety of prescriptions if there is uncertainty, for example, by using drug references. Supervisory control experiments in aviation and process control have associated errors, with...

Objective: Our objective was to review the characteristics, current applications, and evaluation measures of conversational agents with unconstrained natural language input capabilities used for health-related purposes. Methods: We searched PubMed, Embase, CINAHL, PsycInfo, and ACM Digital using a predefined search strategy. Studies were include...

Objective: Determine the relationship between cognitive load (CL) and automation bias (AB). Background: Clinical decision support (CDS) for electronic prescribing can improve safety but introduces the risk of AB, where reliance on CDS replaces vigilance in information seeking and processing. We hypothesized high CL generated by high task complex...

Background: Failure in the timely follow-up of test results has been widely documented, contributing to delayed medical care. Yet, the impact of delay in reviewing test results on hospital length of stay (LOS) has not been studied. We examine the relationship between laboratory tests review time and hospital LOS. Methods: A retrospective cohort...

Objectives: The paper draws attention to: i) key considerations involving the confidentiality, privacy, and security of shared data; and ii) the requirements needed to build collaborative arrangements encompassing all stakeholders with the goal of ensuring safe, secure, and quality use of shared data. Method: A narrative review of existing research...

Objective: Many research fields, including psychology and basic medical sciences, struggle with poor reproducibility of reported studies. Biomedical and health informatics is unlikely to be immune to these challenges. This paper explores replication in informatics and the unique challenges the discipline faces. Methods: Narrative review of recen...

Objective To conduct a replication study to validate previously identified significant risks and inefficiencies associated with the use of speech recognition (SR) for documentation within an electronic health record (EHR) system. Methods Thirty-five emergency department clinicians undertook randomly allocated clinical documentation tasks using keyb...

Objective To conduct a usability study exploring the value of using speech recognition (SR) for clinical documentation tasks within an electronic health record (EHR) system. Methods Thirty-five emergency department clinicians completed a system usability scale (SUS) questionnaire. The study was undertaken after participants undertook randomly allo...

Conversational interfaces and speech recognition capabilities are being increasingly used to create more natural and intuitive user interaction with digital technology. While voice-activated technologies have been used to support clinical documentation, their use for reporting patient safety incidents has not been previously investigated. The purpo...

Conversational interfaces and speech recognition capabilities are being increasingly used to create more natural and intuitive user interaction with digital technology. While voice-activated technologies have been used to support clinical documentation, their use for reporting patient safety incidents has not been previously investigated. The purpo...

Objective: To compare the efficiency and safety of using speech recognition (SR) assisted clinical documentation within an electronic health record (EHR) system with use of keyboard and mouse (KBM). Methods: Thirty-five emergency department clinicians undertook randomly allocated clinical documentation tasks using KBM or SR on a commercial EHR s...

Objectives: To set the scientific context and then suggest principles for an evidence-based approach to secondary uses of clinical data, covering both evaluation of the secondary uses of data and evaluation of health systems and services based upon secondary uses of data. Method: Working Group review of selected literature and policy approaches. Re...

Background Approximately 10% of admissions to acute-care hospitals are associated with an adverse event. Analysis of incident reports helps to understand how and why incidents occur and can inform policy and practice for safer care. Unfortunately our capacity to monitor and respond to incident reports in a timely manner is limited by the sheer volu...

Objectives: To set the scientific context and then suggest principles for an evidence-based approach to secondary uses of clinical data, covering both evaluation of the secondary uses of data and evaluation of health systems and services based upon secondary uses of data. Method: Working Group review of selected literature and policy approaches....

Background Clinical decision support (CDS) in e-prescribing can improve safety by alerting potential errors, but introduces new sources of risk. Automation bias (AB) occurs when users over-rely on CDS, reducing vigilance in information seeking and processing. Evidence of AB has been found in other clinical tasks, but has not yet been tested with e-...

Automated identification provides an efficient way to categorize patient safety incidents. Previous studies have focused on identifying single incident types relating to a specific patient safety problem, e.g., clinical handover. In reality, there are multiple types of incidents reflecting the breadth of patient safety problems and a single report...

Systematic health IT evaluation studies are needed to ensure system quality and safety and to provide the basis for evidence-based health informatics. Well-trained health informatics specialists are required to guarantee that health IT evaluation studies are conducted in accordance with robust standards. Also, policy makers and managers need to app...

The use of health information technology (IT) is increasing around the world. However, as complex IT systems are implemented, new types of errors are introduced. These can disrupt workflow and care delivery, and even lead to patient harm. The purpose of this paper is to examine the patterns and causes of IT system downtime in a hospital setting. We...

Automated identification provides an efficient way to categorize patient safety incidents. Previous studies have focused on identifying single incident types relating to a specific patient safety problem, e.g., clinical handover. In reality, there are multiple types of incidents reflecting the breadth of patient safety problems and a single report...

The use of health information technology (IT) is increasing around the world. However, as complex IT systems are implemented, new types of errors are introduced. These can disrupt workflow and care delivery, and even lead to patient harm. The purpose of this paper is to examine the patterns and causes of IT system downtime in a hospital setting. We...

Systematic health IT evaluation studies are needed to ensure system quality and safety and to provide the basis for evidence-based health informatics. Well-trained health informatics specialists are required to guarantee that health IT evaluation studies are conducted in accordance with robust standards. Also, policy makers and managers need to app...

Objective: To systematically review studies reporting problems with information technology (IT) in health care and their effects on care delivery and patient outcomes. Materials and methods: We searched bibliographic databases including Scopus, PubMed, and Science Citation Index Expanded from January 2004 to December 2015 for studies reporting p...

Background and objectives: With growing use of IT by healthcare professionals and patients, the opportunity for any unintended effects of technology to disrupt care health processes and outcomes is intensified. The objectives of this position paper by the IMIA Working Group (WG) on Technology Assessment and Quality Development are to highlight how...

Objectives To demonstrate and promote the importance of applying a scientific process to health IT design and implementation, and of basing this on research principles and techniques. Methods A review by international experts linked to the IMIA Working Group on Technology Assessment and Quality Development. Results Four approaches are presented,...

Alongside all its benefits, the widespread adoption of health care information technology (HIT) is coupled with emerging risks to patient safety. Evidence about patient harms associated with HIT is mounting. Data breach and cyber-crime can also pose risks. Problems with the design, implementation and use of systems give rise to clinical errors. As...