No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Acquisition of Cardiac Point-of-Care Ultrasound Images with Deep Learning: A Randomized Trial for Educational Outcomes with Novices
... Potential solutions include machine-assisted acquisition technologies, training of ancillary medical staff to acquire basic images for remote interpretation, and artificial intelligence-augmented models to aid non-sonographers. 1,4,5 Clinicians can also address this gap through the expanded use of point-of-care ultrasound (POCUS) to make expedient decisions on patient triage, risk stratification, or the need for more extensive diagnostic testing. [6][7][8] Deep learning (DL) algorithms have demonstrated remarkable capability at both aiding in image acquisition and interpretation of ultrasound images. ...
... [6][7][8] Deep learning (DL) algorithms have demonstrated remarkable capability at both aiding in image acquisition and interpretation of ultrasound images. 1,5,9,10 These algorithms can be deployed on ultra-portable devices connected to smartphones or tablets, enabling diagnostic assessments in settings with traditional access barriers. 6,7 DL may aid ancillary medical staff in acquiring ultrasound images that can be interpreted by a radiologist or provide real-time diagnostic augmentation for clinical providers at the patient's bedside. ...
... 16 Despite evidence that nonsonographers can acquire diagnostic-quality images with DL assistance 12,13 , randomized studies comparing educational outcomes and the development of competency remain limited. 5 Such investigations are crucial when considering widespread implementation of DL software in healthcare, particularly in remote or underserved areas. ...
Background
Deep learning (DL) programs can aid in the acquisition of echocardiograms by medical professionals not previously trained in sonography, potentially addressing access issues in underserved communities. This study evaluates whether DL-enabled devices improve limited echocardiogram acquisition by novice clinicians not trained on sonography.
Methods
In this single-center randomized controlled trial (2023-2024), internal medicine residents (N=38) without sonography training received a personal ultrasound device with (N=19) or without (N=19) DL capability for two weeks while caring for patients on a hospital ward. Participants were allowed to use the devices at their discretion for patient-related care. The DL software provided real-time guidance for probe placement and image quality assessment. The primary outcome was time to acquire a five-view limited echocardiogram. Measurements occurred at randomization and after two weeks, with all scans performed on the same standardized patient. Secondary outcomes included image quality using the modified Rapid Assessment for Competency in Echocardiography (RACE) scale and participant attitudes.
Results
At baseline, both groups had comparable scan times and image quality scores. At follow-up, the DL group demonstrated significantly faster total scan times (152 seconds [IQR 115-195] vs. 266 seconds [IQR 206-324]; p<0.001; Cohen's D 1.7) and better image quality with higher RACE scores (15 [IQR 10-18] vs. 11 [IQR 7-13.5]; p=0.034; Cohen's D 0.84). Trust in the AI features did not differ between the groups post-intervention.
Conclusions
Ultrasound machines with DL features may improve image acquisition times and image quality by novices not trained in sonography. These findings suggest DL algorithms could help address critical gaps in image acquisition by healthcare professionals.
... While US AI-based measurements have been extensively explored in recent years for assessing critical cardiac US measurements, such as Ejection Fraction [13,14], VTI [15], IVC overload [16], and RV function [17], the application of these tools is constrained as the majority of AI-based tools necessitate the presence of a skilled POCUS operator during the tool's operation. On the other hand, the impact of AI in assisting novice operators with real-time adjustments of the US probe for image acquisition has been the subject of a relatively minor number of studies [18,19], resulting in less data in this research area. ...
... A similar study with different real-time AI guidance tool demonstrated that nurses without ultrasonography experience were able to obtain diagnostic echocardiographic studies using AI real-time on-screen [18]. Another trial showed that internal medicine residents carrying a POCUS device with AI guidance functionality for two weeks could also obtain superior A4C views [19]. These trials and ours highlight the potential of real-time AI guidance tools on cardiac US performance among novice operators. ...
... This result was expected by our team, as we initially thought that handling the US probe and understanding AI instructions would require a longer duration for each view. Contrasting results of accelerated scanning time with a different application of real-time AI guidance was reported by another team assessing its influence on novice operators [19]. It is worth noting that their measurement was limited to acquiring the A4C view and was taken after two weeks of scanning experience with the AI guidance. ...
Introduction
Artificial Intelligence (AI) modules might simplify the complexities of cardiac ultrasound (US) training by offering real-time, step-by-step guidance on probe manipulation for high-quality diagnostic imaging. This study investigates real-time AI-based guidance tool in facilitating cardiac US training and its impact on novice users’ proficiency.
Methods
This independent, prospective randomized controlled trial enrolled participants who completed a six-hour cardiac US course, followed by a designated cardiac US proficiency exam. Both groups received in-person guided training using the same devices, with the AI-enhanced group receiving additional real-time AI feedback on probe navigation and image quality during both training and testing, while the non-AI group relied solely on the instructor’s guidance.
Results
Data were collected from 44 participants: 21 in the AI-enhanced group and 23 in the non-AI group. Improvement was observed in the assessment of the AI-enhanced group compared to the non-AI in acquiring the Apical-4-chamber and the Apical-5- chamber views [mean 88% (± SD 10%) vs. mean 76% (± SD 17%), respectively; p = 0.016]. On the other hand, a slower time to complete the echocardiography exam was observed by the AI-enhanced group [mean 401 s (± SD 51) vs. 348 s (± SD 81) respectively; p = 0.038].
Discussion
The addition of real-time, AI-based feedback demonstrated benefits in the cardiac POCUS teaching process for the more challenging echocardiography four- and five- chamber views. It also has the potential to surpass challenges related to in-person POCUS training. Additional studies are required to explore the long-term effect of this training approach.
Clinical trial number
Not applicable.
... While US AI-based measurements have been extensively explored in recent years for assessing critical cardiac US measurements, such as Ejection Fraction (13,14), VTI (15), IVC overload(16), and RV function (17), the application of these tools is constrained as the majority of AI-based tools necessitate the presence of a skilled POCUS operator during the tool's operation. On the other hand, the impact of AI in assisting novice operators with realtime adjustments of the US probe for image acquisition has been the subject of a relatively minor number of studies (18,19), resulting in less data in this research area. ...
... A similar study with different real-time AI guidance tool demonstrated that nurses without ultrasonography experience were able to obtain diagnostic echocardiographic studies using AI real-time on-screen(18). Another trial showed that internal medicine residents carrying a POCUS device with AI guidance functionality for two weeks could also obtain superior A4C views (19). These trials and ours highlight the potential of real-time AI guidance tools on cardiac US performance among novice operators. ...
... This result was expected by our team, as we initially thought that handling the US probe and understanding AI instructions would require a longer duration for each view. Contrasting results of accelerated scanning time with a different application of real-time AI guidance was reported by another team assessing its in uence on novice operators (19). It is worth noting that their measurement was limited to acquiring the A4C view and was taken after two weeks of scanning experience with the AI guidance. ...
Introduction : Artificial Intelligence (AI) modules might simplify the complexities of cardiac ultrasound (US) training by offering real-time, step-by-step guidance on probe manipulation for high-quality diagnostic imaging. This study investigates real-time AI-based guidance tool in facilitating cardiac US training and its impact on novice users' proficiency. Methods : This independent, prospective randomized controlled trial enrolled participants who completed a six-hour cardiac US course, followed by a designated cardiac US proficiency exam. Both groups received in-person guided training using the same devices, with the AI-enhanced group receiving additional real-time AI feedback on probe navigation and image quality during both training and testing, while the non-AI group relied solely on the instructor’s guidance. Results: Data were collected from 44 participants: 21 in the AI-enhanced group and 23 in the non-AI group. Improvement was observed in the assessment of the AI-enhanced group compared to the non-AI in acquiring the Apical-4-chamber and the Apical-5- chamber views [mean 88% (±SD 10%) vs. mean 76% (±SD 17%), respectively; p=0.016]. On the other hand, a slower time to complete the echocardiography exam was observed by the AI-enhanced group [mean 401sec (±SD 51) vs. 348sec (±SD 81) respectively; p=0.038]. Discussion : The addition of real-time, AI-based feedback demonstrated benefits in the cardiac POCUS teaching process for the more challenging echocardiography views. It also has the potential to surpass challenges related to in-personPOCUS training. Additional studies are required to explore the long-term effect of this training approach.
... Feedback on image optimisation helps the user become skilful over time, and the feedback helps in a setting where POCUS experts are not available for consultation. [12] In a single-centre study by Baum et al., [13] a randomised investigation showed that novice users with POCUS devices equipped with AI functionality had a shorter apical 4-chamber acquisition time and higher image-quality scores. ...
Artificial intelligence (AI) was once considered avant-garde. However, AI permeates every industry today, impacting work and home lives in many ways. While AI-driven diagnostic and therapeutic applications already exist in medicine, a chasm remains between the potential of AI and its clinical applications. This article reviews the status of AI-powered ultrasound (US) applications in anaesthesiology and perioperative medicine. A literature search was performed for studies examining AI applications in perioperative US. AI applications for echocardiography and regional anaesthesia are the most robust and well-developed. While applications are available for lung imaging and vascular access, AI programs for airway and gastric US imaging solutions have yet to be available. Legal and ethical challenges associated with AI applications need to be addressed and resolved over time. AI applications are beneficial in the context of education and training. While low-resource settings may benefit from AI, the financial burden is a considerable limiting factor.
... While this study demonstrated PCPs' proficiency in acquiring, interpreting, and diagnosing various pulmonary scenarios immediately following the course, it's reasonable to expect that the cohorts' proficiency may change over time in accordance with the time and effort spent practicing and learning LUS [38,39]. Artificial intelligence (AI) and telemedicine solutions might serve as a solution to enhance image acquisition and interpretation precision [40][41][42][43][44][45]. For example, several studies presented that AI can be utilized to count the number of B-Lines, reflecting pulmonary edema [42,46,47]. ...
Background
Point-of-care ultrasound is rapidly gaining traction in clinical practice, including primary care. Yet, logistical challenges and geographical isolation hinder skill acquisition. Concurrently, an evidentiary gap exists concerning such guidance's effectiveness and optimal implementation in these settings.
Methods
We developed a lung point-of-care ultrasound (POCUS) curriculum for primary care physicians in a rural, medically underserved region of the south of Israel. The course included recorded lectures, pre-course assessments, hands-on training, post-workshop lectures, and individual practice. To evaluate our course, we measured learning outcomes and physicians’ proficiency in different lung POCUS domains using hands-on technique assessment and gathered feedback on the course with a multi-modal perception approach: an original written pre- and post-perception and usage questionnaire.
Results
Fifty primary care physicians (PCPs) showed significant improvement in hands-on skills, increasing from 6 to 76% proficiency (p < 0.001), and in identifying normal versus abnormal views, improving from 54 to 74% accuracy (p < 0.001). Ten weeks after training, primary care physicians reported greater comfort using lung ultrasound, rising from 10 to 54% (p < 0.001), and improved grasp of its potential and limits, increasing from 27.5% to 84% (p < 0.001). Weekly usage increased from none to 50%, and the number of primary care physicians not using at all decreased from 72 to 26% (p < 0.001).
Conclusions
A two-day focused in-person and remote self-learning lung-POCUS training significantly improved primary care physicians' lung ultrasound skills, comfort, and implementation.
... For instance, Kosmos portable system utilizes AI to guide image capture, grade quality of image in real-time, and measure parameters such as EF [77]. This system has been shown to improve image quality and accuracy of identifying reduced LV systolic function and reduce scanning time [78]. ...
Purpose of Review
This review discusses the current limitations of assessing the right ventricle (RV) using Point-of-Care Ultrasound, explains the challenges in describing the complexity of the RV, and provides guidance on how to use Point-of-Care Ultrasound data in clinical practice.
Recent Findings
Assessing the RV requires assessments from multiple views and parameters due to its complex shape. Combined use of multiple ultrasound-derived measures along with clinical information from other sources is recommended. Artificial intelligence is increasingly used in Point-of-Care Ultrasound and will help improve validity and consistency of measurements. Future studies are needed to examine the impact of the Point-of-Care Ultrasound exam on clinical outcomes.
Summary
Multiple challenges exist in RV assessment. Future studies and guidance are required to investigate the feasibility of Point-of-Care Ultrasound in specific clinical settings, training and credentialing of examiners, and practice guidance on parameters to be used and reported.
The medical profession is at a crossroads with technology and innovation disrupting traditional care delivery, posing challenges for training future and current workforces. Traditional memorization-based medical education may become impractical due to information overload, exemplified by the rapid growth of research. AI's ability to democratize knowledge and reshape clinical workflows will be crucial in this context. AI-powered platforms have improved medical students' assessment scores by providing real-time feedback, personalized study resources, and virtual patient simulations. These tools enhance clinical reasoning, diagnostic skills, and practical knowledge, while automated assessments offer timely, objective evaluations. Despite these benefits, a lack of understanding of AI among learners and educators, and disparities in access to AI tools, hinder its full potential. This chapter explores AI's transformative impact on medical education, addressing its applications, challenges, and integration.
Point-of-care ultrasound (POCUS) has been developed as a critical tool for diagnostic patient evaluation and clinical management. Its transcendence into anesthesiology necessitates appropriate and effective educational strategies to assist in the development of anesthesia POCUS learners. Several professional societies, including the American Society of Anesthesiologists (ASA), American Society of Regional Anesthesia (ASRA), and Accreditation Council for Graduate Medical Education (ACGME) for anesthesiology have established minimum training standards for POCUS education for anesthesiologists, residents, and fellows.1,4 The article at hand aims to summarize and provide insight into the various educational modalities utilized in POCUS training, incorporate these strategies in the established “Indication, Acquisition, Interpretation, and Medical decision-making” (I-AIM) framework, and include recommendations on the minimum number of POCUS exams to aid in achieving competency.
Background:
More primary care providers (PCPs) have begun to embrace the use of point-of-care ultrasound (POCUS), but little is known about how PCPs are currently using POCUS and what barriers exist. In this prospective study, the largest systematic survey of POCUS use among PCPs, we assessed the current use, barriers to use, program management, and training needs for POCUS in primary care.
Methods:
We conducted a prospective observational study of all VA Medical Centers (VAMCs) between June 2019 and March 2020 using a web-based survey sent to all VAMC Chiefs of Staff and Chiefs of primary care clinics (PCCs).
Results:
Chiefs of PCCs at 105 VAMCs completed the survey (82% response rate). Only 13% of PCCs currently use POCUS, and the most common applications used were bladder and musculoskeletal ultrasound. Desire for POCUS training exceeded current use, but lack of trained providers (78%), ultrasound equipment (66%), and funding for training (41%) were common barriers. Program infrastructure to support POCUS use was uncommon, and only 9% of VAMCs had local policies related to POCUS. Most PCC chiefs (64%) would support POCUS training.
Conclusions:
Current use of POCUS in primary care is low despite the recent growth of POCUS training in Internal Medicine residency programs. Investment in POCUS training and program infrastructure is needed to expand POCUS use in primary care and ensure adequate supervision of trainees.
When a patient presents to the ED, clinicians often turn to medical imaging to better understand their condition. Traditionally, imaging is collected from the patient and interpreted by a radiologist remotely. However, scanning devices are increasingly equipped with analytical software that can provide quantitative assessments at the patient’s bedside. These assessments often rely on machine learning algorithms as a means of interpreting medical images.
Background
Artificial intelligence (AI) needs to be accepted and understood by physicians and medical students, but few have systematically assessed their attitudes. We investigated clinical AI acceptance among physicians and medical students around the world to provide implementation guidance.
Materials and methods
We conducted a two-stage study, involving a foundational systematic review of physician and medical student acceptance of clinical AI. This enabled us to design a suitable web-based questionnaire which was then distributed among practitioners and trainees around the world.
Results
Sixty studies were included in this systematic review, and 758 respondents from 39 countries completed the online questionnaire. Five (62.50%) of eight studies reported 65% or higher awareness regarding the application of clinical AI. Although, only 10–30% had actually used AI and 26 (74.28%) of 35 studies suggested there was a lack of AI knowledge. Our questionnaire uncovered 38% awareness rate and 20% utility rate of clinical AI, although 53% lacked basic knowledge of clinical AI. Forty-five studies mentioned attitudes toward clinical AI, and over 60% from 38 (84.44%) studies were positive about AI, although they were also concerned about the potential for unpredictable, incorrect results. Seventy-seven percent were optimistic about the prospect of clinical AI. The support rate for the statement that AI could replace physicians ranged from 6 to 78% across 40 studies which mentioned this topic. Five studies recommended that efforts should be made to increase collaboration. Our questionnaire showed 68% disagreed that AI would become a surrogate physician, but believed it should assist in clinical decision-making. Participants with different identities, experience and from different countries hold similar but subtly different attitudes.
Conclusion
Most physicians and medical students appear aware of the increasing application of clinical AI, but lack practical experience and related knowledge. Overall, participants have positive but reserved attitudes about AI. In spite of the mixed opinions around clinical AI becoming a surrogate physician, there was a consensus that collaborations between the two should be strengthened. Further education should be conducted to alleviate anxieties associated with change and adopting new technologies.
Background
Artificial intelligence (AI) has affected our day-to-day in a great extent. Healthcare industry is one of the mainstream fields among those and produced a noticeable change in treatment and education. Medical students must comprehend well why AI technologies mediate and frame their decisions on medical issues. Formalizing of instruction on AI concepts can facilitate learners to grasp AI outcomes in association with their sensory perceptions and thinking in the dynamic and ambiguous reality of daily medical practice. The purpose of this study is to provide consensus on the competencies required by medical graduates to be ready for artificial intelligence technologies and possible applications in medicine and reporting the results.
Materials and methods
A three-round e-Delphi survey was conducted between February 2020 and November 2020. The Delphi panel accorporated experts from different backgrounds; (i) healthcare professionals/ academicians; (ii) computer and data science professionals/ academics; (iii) law and ethics professionals/ academics; and (iv) medical students. Round 1 in the Delphi survey began with exploratory open-ended questions. Responses received in the first round evaluated and refined to a 27-item questionnaire which then sent to the experts to be rated using a 7-point Likert type scale (1: Strongly Disagree—7: Strongly Agree). Similar to the second round, the participants repeated their assessments in the third round by using the second-round analysis. The agreement level and strength of the consensus was decided based on third phase results. Median scores was used to calculate the agreement level and the interquartile range (IQR) was used for determining the strength of the consensus.
Results
Among 128 invitees, a total of 94 agreed to become members of the expert panel. Of them 75 (79.8%) completed the Round 1 questionnaire, 69/75 (92.0%) completed the Round 2 and 60/69 (87.0%) responded to the Round 3. There was a strong agreement on the 23 items and weak agreement on the 4 items.
Conclusions
This study has provided a consensus list of the competencies required by the medical graduates to be ready for AI implications that would bring new perspectives to medical education curricula. The unique feature of the current research is providing a guiding role in integrating AI into curriculum processes, syllabus content and training of medical students.
Background:
Point-of-care ultrasound (POCUS) can reduce procedural complications and improve the diagnostic accuracy of hospitalists. Currently, it is unknown how many practicing hospitalists use POCUS, which applications are used most often, and what barriers to POCUS use exist.
Objective:
This study aimed to characterize current POCUS use, training needs, and barriers to use among hospital medicine groups (HMGs).
Design, setting, and participants:
A prospective observational study of all Veterans Affairs (VA) medical centers was conducted between August 2019 and March 2020 using a web-based survey sent to all chiefs of HMGs. These data were compared to a similar survey conducted in 2015.
Result:
Chiefs from 117 HMGs were surveyed, with a 90% response rate. There was ongoing POCUS use in 64% of HMGs. From 2015 to 2020, procedural POCUS use decreased by 19%, but diagnostic POCUS use increased for cardiac (8%), pulmonary (7%), and abdominal (8%) applications. The most common barrier to POCUS use was lack of training (89%), and only 34% of HMGs had access to POCUS training. Access to ultrasound equipment was the least common barrier (57%). The proportion of HMGs with ≥1 ultrasound machine increased from 29% to 71% from 2015 to 2020. An average of 3.6 ultrasound devices per HMG was available, and 45% were handheld devices.
Conclusion:
From 2015 to 2020, diagnostic POCUS use increased, while procedural use decreased among hospitalists in the VA system. Lack of POCUS training is currently the most common barrier to POCUS use among hospitalists.
Objective:
Respiratory symptoms are among the most common chief complaints of pediatric patients in the emergency department (ED). Point-of-care ultrasound (POCUS) outperforms conventional chest X-ray and is user-dependent, which can be challenging to novice ultrasound (US) users. We introduce a novel concept using artificial intelligence (AI)-enhanced pleural sweep to generate complete panoramic views of the lungs, and then assess its accuracy among novice learners (NLs) to identify pneumonia.
Methods:
Previously healthy 0- to 17-year-old patients presenting to a pediatric ED with cardiopulmonary chief complaint were recruited. NLs received a 1-hour training on traditional lung POCUS and the AI-assisted software. Two POCUS-trained experts interpreted the images, which served as the criterion standard. Both expert and learner groups were blinded to each other's interpretation, patient data, and outcomes. Kappa was used to determine agreement between POCUS expert interpretations.
Results:
Seven NLs, with limited to no prior POCUS experience, completed examinations on 32 patients. The average patient age was 5.53 years (±1.07). The median scan time of 7 minutes (minimum-maximum 3-43; interquartile 8). Three (8.8%) patients were diagnosed with pneumonia by criterion standard. Sensitivity, specificity, and accuracy for NLs AI-augmented interpretation were 66.7% (confidence interval [CI] 9.4-99.1%), 96.5% (CI 82.2-99.9%), and 93.7% (CI 79.1-99.2%). The average image quality rating was 2.94 (±0.16) out of 5 across all lung fields. Interrater reliability between expert sonographers was high with a kappa coefficient of 0.8.
Conclusion:
This study shows that AI-augmented lung US for diagnosing pneumonia has the potential to increase accuracy and efficiency.
Importance:
Artificial intelligence (AI) has been applied to analysis of medical imaging in recent years, but AI to guide the acquisition of ultrasonography images is a novel area of investigation. A novel deep-learning (DL) algorithm, trained on more than 5 million examples of the outcome of ultrasonographic probe movement on image quality, can provide real-time prescriptive guidance for novice operators to obtain limited diagnostic transthoracic echocardiographic images.
Objective:
To test whether novice users could obtain 10-view transthoracic echocardiographic studies of diagnostic quality using this DL-based software.
Design, setting, and participants:
This prospective, multicenter diagnostic study was conducted in 2 academic hospitals. A cohort of 8 nurses who had not previously conducted echocardiograms was recruited and trained with AI. Each nurse scanned 30 patients aged at least 18 years who were scheduled to undergo a clinically indicated echocardiogram at Northwestern Memorial Hospital or Minneapolis Heart Institute between March and May 2019. These scans were compared with those of sonographers using the same echocardiographic hardware but without AI guidance.
Interventions:
Each patient underwent paired limited echocardiograms: one from a nurse without prior echocardiography experience using the DL algorithm and the other from a sonographer without the DL algorithm. Five level 3-trained echocardiographers independently and blindly evaluated each acquisition.
Main outcomes and measures:
Four primary end points were sequentially assessed: qualitative judgement about left ventricular size and function, right ventricular size, and the presence of a pericardial effusion. Secondary end points included 6 other clinical parameters and comparison of scans by nurses vs sonographers.
Results:
A total of 240 patients (mean [SD] age, 61 [16] years old; 139 men [57.9%]; 79 [32.9%] with body mass indexes >30) completed the study. Eight nurses each scanned 30 patients using the DL algorithm, producing studies judged to be of diagnostic quality for left ventricular size, function, and pericardial effusion in 237 of 240 cases (98.8%) and right ventricular size in 222 of 240 cases (92.5%). For the secondary end points, nurse and sonographer scans were not significantly different for most parameters.
Conclusions and relevance:
This DL algorithm allows novices without experience in ultrasonography to obtain diagnostic transthoracic echocardiographic studies for evaluation of left ventricular size and function, right ventricular size, and presence of a nontrivial pericardial effusion, expanding the reach of echocardiography to clinical settings in which immediate interrogation of anatomy and cardiac function is needed and settings with limited resources.
We present the novel use of a deep learning–derived technology trained on the skilled hand movements of cardiac sonographers that guides novice users to acquire high-quality bedside cardiac ultrasound images. We illustrate its use at the point of care through a series of patient encounters in the COVID-19 intensive care unit. (Level of Difficulty: Beginner.)
Objective:
To assess usability and usefulness of a machine learning-based order recommender system applied to simulated clinical cases.
Materials and methods:
43 physicians entered orders for 5 simulated clinical cases using a clinical order entry interface with or without access to a previously developed automated order recommender system. Cases were randomly allocated to the recommender system in a 3:2 ratio. A panel of clinicians scored whether the orders placed were clinically appropriate. Our primary outcome included the difference in clinical appropriateness scores. Secondary outcomes included total number of orders, case time, and survey responses.
Results:
Clinical appropriateness scores per order were comparable for cases randomized to the order recommender system (mean difference -0.11 order per score, 95% CI: [-0.41, 0.20]). Physicians using the recommender placed more orders (median 16 vs 15 orders, incidence rate ratio 1.09, 95%CI: [1.01-1.17]). Case times were comparable with the recommender system. Order suggestions generated from the recommender system were more likely to match physician needs than standard manual search options. Physicians used recommender suggestions in 98% of available cases. Approximately 95% of participants agreed the system would be useful for their workflows.
Discussion:
User testing with a simulated electronic medical record interface can assess the value of machine learning and clinical decision support tools for clinician usability and acceptance before live deployments.
Conclusions:
Clinicians can use and accept machine learned clinical order recommendations integrated into an electronic order entry interface in a simulated setting. The clinical appropriateness of orders entered was comparable even when supported by automated recommendations.
Purpose of Review
With the unprecedented advancement of data aggregation and deep learning algorithms, artificial intelligence (AI) and machine learning (ML) are poised to transform the practice of medicine. The field of orthopedics, in particular, is uniquely suited to harness the power of big data, and in doing so provide critical insight into elevating the many facets of care provided by orthopedic surgeons. The purpose of this review is to critically evaluate the recent and novel literature regarding ML in the field of orthopedics and to address its potential impact on the future of musculoskeletal care.
Recent Findings
Recent literature demonstrates that the incorporation of ML into orthopedics has the potential to elevate patient care through alternative patient-specific payment models, rapidly analyze imaging modalities, and remotely monitor patients.
Summary
Just as the business of medicine was once considered outside the domain of the orthopedic surgeon, we report evidence that demonstrates these emerging applications of AI warrant ownership, leverage, and application by the orthopedic surgeon to better serve their patients and deliver optimal, value-based care.
Ultrasound (US) has become one of the most commonly performed imaging modalities in clinical practice. It is a rapidly evolving technology with certain advantages and with unique challenges that include low imaging quality and high variability. From the perspective of image analysis, it is essential to develop advanced automatic US
image analysis methods to assist in US diagnosis and/or to make such assessment more objective and accurate. Deep learning has recently emerged as the leading machine learning tool in various research fields, and especially in general imaging analysis and computer vision. Deep learning also shows huge potential for various automatic US image analysis tasks. This review first briefly introduces several popular deep learning architectures, and then summarizes and thoroughly discusses their applications in various specific tasks in US image analysis, such as classification, detection, and segmentation. Finally, the open challenges and potential trends of the future application of deep learning in medical US image analysis are discussed.
Ultrasound guidance is used increasingly to perform the following 6 bedside procedures that are core competencies of hospitalists: abdominal paracentesis, arterial catheter placement, arthrocentesis, central venous catheter placement, lumbar puncture, and thoracentesis. Yet most hospitalists have not been certified to perform these procedures, whether using ultrasound guidance or not, by specialty boards or other institutions extramural to their own hospitals. Instead, hospital privileging committees often ask hospitalist group leaders to make ad hoc intramural certification assessments as part of credentialing. Given variation in training and experience, such assessments are not straightforward "sign offs." We thus convened a panel of experts to conduct a systematic review to provide recommendations for credentialing hospitalist physicians in ultrasound guidance of these 6 bedside procedures. Pathways for initial and ongoing credentialing are proposed. A guiding principle of both is that certification assessments for basic competence are best made through direct observation of performance on actual patients.
Background:
The rapid adoption of point-of-care ultrasound (POCUS) has created a need to develop assessment tools to ensure that learners can competently use these technologies. In this study, the authors developed and tested a rating scale to assess the quality of point-of-care thoracic ultrasound studies performed by novices. In Phase 1, the Assessment of Competency in Thoracic Sonography (ACTS) scale was developed based on structured interviews with subject matter experts. The tool was then piloted on a small series of ultrasound studies in Phase 2. In Phase 3 the tool was applied to a sample of 150 POCUS studies performed by ten learners; performance was then assessed by two independent raters.
Results:
Evidence for the content validity of the ACTS scale was provided by a consensus exercise wherein experts agreed on the general principles and specific items that make up the scale. The tool demonstrated reasonable inter-rater reliability despite minimal requirements for evaluator training and displayed evidence of good internal structure, with related scale items correlating well with each other. Analysis of the aggregate learning curves suggested a rapid early improvement in learner performance with slower improvement after approximately 25-30 studies.
Conclusions:
The ACTS scale provides a straightforward means to assess learner performance. Our results support the conclusion that the tool is an effective means of making valid judgments regarding competency in point-of-care thoracic ultrasound, and that the majority of learner improvement occurs during their first 25-30 practice studies.
POCUS is performed by the treating clinician at the bedside, with immediate interpretation and clinical integration of the imaging results. This review discusses POCUS technology, clinical applications, and the complementarity of POCUS and consultative ultrasonography in primary imaging specialties.
Description:
The American College of Physicians (ACP) developed this guideline to provide clinical recommendations on the appropriate use of point-of-care ultrasonography (POCUS) in patients with acute dyspnea in emergency department (ED) or inpatient settings to improve the diagnostic, treatment, and health outcomes of those with suspected congestive heart failure, pneumonia, pulmonary embolism, pleural effusion, or pneumothorax.
Methods:
The ACP Clinical Guidelines Committee based this guideline on a systematic review on the benefits, harms, and diagnostic test accuracy of POCUS; patient values and preferences; and costs of POCUS. The systematic review evaluated health outcomes, diagnostic timeliness, treatment decisions, and test accuracy. The critical health, diagnostic, and treatment outcomes evaluated were in-hospital mortality, time to diagnosis, and time to treatment. The important outcomes evaluated were intensive care unit admissions, correctness of diagnosis, disease-specific outcomes, hospital readmissions, length of hospital stay, and quality of life. The critical test accuracy outcomes included false-positive results for suspected pneumonia, pneumothorax, and pulmonary embolism and false-negative results for suspected congestive heart failure, pneumonia, pneumothorax, and pulmonary embolism. Important test accuracy outcomes included false-positive results for suspected congestive heart failure and false-negative and false-positive results for suspected pleural effusion. This guideline was developed using the GRADE (Grading of Recommendations Assessment, Development and Evaluation) method.
Target audience and patient population:
The target audience is all clinicians, and the target patient population is adult patients with acute dyspnea in ED or inpatient settings.
Recommendation:
ACP suggests that clinicians may use point-of-care ultrasonography in addition to the standard diagnostic pathway when there is diagnostic uncertainty in patients with acute dyspnea in emergency department or inpatient settings (conditional recommendation; low-certainty evidence).
BACKGROUND: Little is known about how to effectively train residents with point-of-care ultrasonography (POCUS) despite increasing usage. OBJECTIVE: This study aimed to assess whether handheld ultrasound devices (HUDs), alongside a year-long lecture series, improved trainee image interpretation skills with POCUS. METHODS: Internal medicine intern physicians (N = 149) at a single academic institution from 2016 to 2018 participated in the study. The 2017 interns (n = 47) were randomized 1:1 to receive personal HUDs (n = 24) for patient care vs no-HUDs (n = 23). All 2017 interns received a repeated lecture series regarding cardiac, thoracic, and abdominal POCUS. Interns were assessed on their ability to interpret POCUS images of normal/abnormal findings. The primary outcome was the difference in end-of-the-year assessment scores between interns randomized to receive HUDs vs not. Secondary outcomes included trainee scores after repeating lectures and confidence with POCUS. Intern scores were also compared with historical (2016, N = 50) and contemporaneous (2018, N = 52) controls who received no lectures. RESULTS: Interns randomized to HUDs did not have significantly higher image interpretation scores (median HUD score: 0.84 vs no-HUD score: 0.84; P = .86). However, HUD interns felt more confident in their abilities. The 2017 cohort had higher scores (median 0.84), compared with the 2016 historical control (median 0.71; P = .001) and 2018 contemporaneous control (median 0.48; P < .001). Assessment scores improved after first-time exposure to the lecture series, while repeated lectures did not improve scores. CONCLUSIONS: Despite feeling more confident, personalized HUDs did not improve interns’ POCUS-related knowledge or interpretive ability. Repeated lecture exposure without further opportunities for deliberate practice may not be beneficial for mastering POCUS.
Background
There is insufficient knowledge about how personal access to handheld ultrasound devices (HUDs) improves trainee learning with point-of-care ultrasound (POCUS).
Objective
To assess whether HUDs, alongside a yearlong lecture series, improved trainee POCUS usage and ability to acquire images.
Methods
Internal medicine intern physicians (n = 47) at a single institution from 2017 to 2018 were randomized 1:1 to receive personal HUDs (n = 24) for patient care/self-directed learning vs no-HUDs (n = 23). All interns received a repeated lecture series on cardiac, thoracic, and abdominal POCUS. Main outcome measures included self-reported HUD usage rates and post-intervention assessment scores using the Rapid Assessment of Competency in Echocardiography (RACE) scale between HUD and no-HUD groups.
Results
HUD interns reported performing POCUS assessments on patients a mean 6.8 (SD 2.2) times per week vs 6.4 (SD 2.9) times per week in non-HUD arm (P = .66). There was no relationship between the number of self-reported examinations per week and a trainee's post-intervention RACE score (rho = 0.022, P = .95). HUD interns did not have significantly higher post-intervention RACE scores (median HUD score 17.0 vs no-HUD score 17.8; P = .72). Trainee confidence with cardiac POCUS did not correlate with RACE scores.
Conclusions
Personal HUDs without direct supervision did not increase the amount of POCUS usage or improve interns' acquisition abilities. Interns who reported performing more examinations per week did not have higher RACE scores. Improved HUD access and lectures without additional feedback may not improve POCUS mastery.
Recent advances in ultrasound technology have made ultrasound equipment more versatile, portable and accessible than ever. Modern hand-held, ultra-portable ultrasound devices have been developed by multiple companies and are contributing to make bedside ultrasound evaluation a practice available to all physicians. The significance of making point-of-care ultrasound a common practice that all physicians can eventually use in the evaluation of their patients is changing the way medicine is practiced, allowing physicians to quickly obtain valuable information to complement the traditional physical exam. Despite the proven benefits of using bedside ultrasound imaging as a part of the patient evaluation and for procedure guidance, adoption of this technology is still not widespread among anesthesiology clinicians, nor is there uniform teaching of ultrasound skills to anesthesia residents and faculty. Among obstacles that have been identified as precluding achievement of the goal of widespread utilization of POCUS among anesthesia professionals and trainees is the availability of equipment for all physicians when it is needed and lack of instructor supervision for trainees who desire to use ultrasound but do not always have an instructor knowledgeable in POCUS with them when an US exam is warranted. We analyze the characteristics, advantages and limitations of available ultra-portable, hand-held ultrasound devices, with a focus on the Butterfly IQ pocket-probe, which is available at our institution, and how some of its features, like the capacity to emulate multiple transducers and its cloud-sharing and teleguidance technology, may contribute to increase the availability and utilization of POCUS by anesthesia clinicians.
Clinical decision support tools that automatically disseminate patterns of clinical orders have the potential to improve patient care by reducing errors of omission and streamlining physician workflows. However, it is unknown if physicians will accept such tools or how their behavior will be affected. In this randomized controlled study, we exposed 34 licensed physicians to a clinical order entry interface and five simulated emergency cases, with randomized availability of a previously developed clinical order recommender system. With the recommender available, physicians spent similar time per case (6.7 minutes), but placed more total orders (17.1 vs. 15.8). The recommender demonstrated superior recall (59% vs 41%) and precision (25% vs 17%) compared to manual search results, and was positively received by physicians recognizing workflow benefits. Further studies must assess the potential clinical impact towards a future where electronic health records automatically anticipate clinical needs.
Point-of-care ultrasonography (POCUS) has the potential to transform healthcare delivery through its diagnostic expediency. Trainee competency with POCUS is now mandated for emergency medicine through the Accreditation Council for Graduate Medical Education (ACGME), and its use is expanding into other medical specialties, including internal medicine. However, a key question remains: how does one define "competency" with this emerging technology? As our trainees become more acquainted with POCUS, it is vital to develop validated methodology for defining and measuring competency amongst inexperienced users. As a framework, the assessment of competency should include evaluations that assess the acquisition and application of POCUS-related knowledge, demonstration of technical skill (e.g., proper probe selection, positioning, and image optimization), and effective integration into routine clinical practice. These assessments can be performed across a variety of settings, including web-based applications, simulators, standardized patients, and real clinical encounters. Several validated assessments regarding POCUS competency have recently been developed, including the Rapid Assessment of Competency in Echocardiography (RACE) or the Assessment of Competency in Thoracic Sonography (ACTS). However, these assessments focus mainly on technical skill and do not expand upon other areas of this framework, which represents a growing need. In this review, we explore the different methodologies for evaluating competency with POCUS as well as discuss current progress in the field of measuring trainee knowledge and technical skill.
Background:
Little is known about how to effectively train residents with point-of-care ultrasonography (POCUS) despite increasing usage.
Objective:
This study aimed to assess whether handheld ultrasound devices (HUDs), alongside a year-long lecture series, improved trainee image interpretation skills with POCUS.
Methods:
Internal medicine intern physicians (N = 149) at a single academic institution from 2016 to 2018 participated in the study. The 2017 interns (n = 47) were randomized 1:1 to receive personal HUDs (n = 24) for patient care vs no-HUDs (n = 23). All 2017 interns received a repeated lecture series regarding cardiac, thoracic, and abdominal POCUS. Interns were assessed on their ability to interpret POCUS images of normal/abnormal findings. The primary outcome was the difference in end-of-the-year assessment scores between interns randomized to receive HUDs vs not. Secondary outcomes included trainee scores after repeating lectures and confidence with POCUS. Intern scores were also compared with historical (2016, N = 50) and contemporaneous (2018, N = 52) controls who received no lectures.
Results:
Interns randomized to HUDs did not have significantly higher image interpretation scores (median HUD score: 0.84 vs no-HUD score: 0.84; P = .86). However, HUD interns felt more confident in their abilities. The 2017 cohort had higher scores (median 0.84), compared with the 2016 historical control (median 0.71; P = .001) and 2018 contemporaneous control (median 0.48; P < .001). Assessment scores improved after first-time exposure to the lecture series, while repeated lectures did not improve scores.
Conclusions:
Despite feeling more confident, personalized HUDs did not improve interns' POCUS-related knowledge or interpretive ability. Repeated lecture exposure without further opportunities for deliberate practice may not be beneficial for mastering POCUS.
Advances in technology are increasingly changing relationships between consumers and providers of products and services in every business. The democratization of recognized expertise that accompanies the use of information technology can be a positive force for improving access, cost, and equity, but also can challenge the role and status of traditional experts, including marginalizing them. Consumers no longer need a professional taxi driver to tell them the fastest way to the airport and can book flights without the help of a travel agent.
Recent applications of artificial intelligence (AI) and deep learning (DL) in health care include enhanced diagnostic imaging modalities to support clinical decisions and improve patients’ outcomes. Focused on using automated DL‐based systems to improve point‐of‐care ultrasound (POCUS), we look at DL‐based automation as a key field in expanding and improving POCUS applications in various clinical settings. A promising additional value would be the ability to automate training model selections for teaching POCUS to medical trainees and novice sonologists. The diversity of POCUS applications and ultrasound equipment, each requiring specialized AI models and domain expertise, limits the use of DL as a generic solution. In this article, we highlight the most advanced potential applications of AI in POCUS tailored to high‐yield models in automated image interpretations, with the premise of improving the accuracy and efficacy of POCUS scans.
Technology has the potential to both distract and reconnect providers with their patients. The widespread adoption of electronic medical records in recent years pulls physicians away from time at the bedside. However, when used in conjunction with patients, technology has the potential to bring patients and physicians together. The increasing use of point-of-care ultrasound by physicians is changing the bedside encounter by allowing for real-time diagnosis with the treating physician. It is a powerful example of the way technology can be a force for refocusing on the bedside encounter.
Purpose:
Optimal instruction and assessment of critical care ultrasound (CCUS) skills requires an assessment tool to measure learner competency and changes over time. In this study, a previously published tool was used to monitor the development of critical care echocardiography (CCE) competencies, the attainment of performance plateaus, and the extent to which previous experience influenced learning.
Materials and methods:
A group of experts used the Rapid Assessment of Competency in Echocardiography (RACE) scale to rate a large pool of CCE studies performed by novices in a longitudinal design. A total of 380 studies performed by twelve learners were assessed; each study was independently rated by two experts.
Results:
Learners demonstrated improvement in mean RACE scores over time, with peak performance occurring early in training and a performance plateau thereafter. Learners with little experience received the greatest benefit from training, with an average performance plateau reached at the twentieth study.
Conclusions:
Supporting earlier results, the RACE scale provided a straightforward means to assess learner performance with minimal requirements for evaluator training. The results of the present study suggest that novices experience the greatest gains in competency during their first twenty practice studies, a threshold which should serve to guide training initiatives.
Objectives:
Increased use of point-of-care ultrasound (US) requires the development of assessment tools that measure the competency of learners. In this study, we developed and tested a tool to assess the quality of point-of-care cardiac US studies performed by novices.
Methods:
In phase 1, the Rapid Assessment of Competency in Echocardiography (RACE) scale was developed on the basis of structured interviews with subject matter experts; the tool was then piloted on a small series of US studies in phase 2. In phase 3, the tool was applied to a sample of 154 point-of-care US studies performed by 12 learners; each study was independently rated by 2 experts, with quantitative analysis subsequently performed.
Results:
Evidence of the content validity of the RACE scale was supported by a consensus exercise, wherein experts agreed on the assessment dimensions and specific items that made up the RACE scale. The tool showed good inter-rater reliability. An analysis of inter-item correlations provided support for the internal structure of the scale, and the tool was able to discriminate between learners early in their point-of-care US learning and those who were more advanced in their training.
Conclusions:
The RACE scale provides a straightforward means to assess learner performance with minimal requirements for evaluator training. Our results support the conclusion that the tool is an effective means of making valid judgments regarding competency in point-of-care cardiac US.
Ultrasound image interpretation and education relies on obtaining a high-quality ultrasound image; however, no literature exists to date attempting to define a high-quality ultrasound image. The purpose of this study was to design and perform a pilot reliability study of the Brightness Mode Quality Ultrasound Imaging Examination Technique (B-QUIET) method for ultrasound quality image assessment.
A single sonologist performed a Trinity hypotensive ultrasound protocol on 3 participants of varying body types. Each participant's ultrasound examination was repeated in 4 locations; static clinic location, mobile ambulance, airplane, and helicopter. Images were reviewed by a sonographer, radiologist, and emergency medicine physician using the B-QUIET method and underwent statistical analysis using generalizability theory for reliability of the assessments using the tool.
The B-QUIET method showed high reliability of most subscale items. Approximately two-thirds of the reviewed images had complete inter-rater reliability on 90% of the items. There was relatively low inter-rater reliability for the Identification/ Orientation subscale items. The inter-rater reliability κ value was calculated as 0.676 overall for the method.
The need for a standardized method to evaluate the quality of an ultrasound image is well documented. The B-QUIET method represents the first attempt to quantify the sonographer component of ultrasound images. Further reliability and validation studies of this method will be needed; however, it represents a tool for standardized ultrasound interpretation, ultrasound training, and institutional quality assessment.
The state of ultrasound education in U.S. medical schools: results of a national survey
- Bahner