No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Challenges and Opportunities to Improve the Clinician Experience Reviewing Electronic Progress Notes
Abstract
Background High-quality clinical notes are essential to effective clinical communication. However, electronic clinical notes are often long, difficult to review, and contain information that is potentially extraneous or out of date. Additionally, many clinicians write electronic clinical notes using customized templates, resulting in notes with significant variability in structure. There is a need to understand better how clinicians review electronic notes and how note structure variability may impact clinicians' note-reviewing experiences.
Objective This article aims to understand how physicians review electronic clinical notes and what impact section order has on note-reviewing patterns.
Materials and Methods We conducted an experiment utilizing an electronic health record (EHR) system prototype containing four anonymized patient cases, each composed of nine progress notes that were presented with note sections organized in different orders to different subjects (i.e., Subjective, Objective, Assessment, and Plan, Assessment, Plan, Subjective, and Objective, Subjective, Assessment, Objective, and Plan, and Mixed). Participants, who were mid-level residents and fellows, reviewed the cases and provided a brief summary after reviewing each case. Time-related data were collected and analyzed using descriptive statistics. Surveys were administered and interviews regarding experiences reviewing notes were collected and analyzed qualitatively.
Results Qualitatively, participants reported challenges related to reviewing electronic clinical notes. Experimentally, time spent reviewing notes varied based on the note section organization. Consistency in note section organization improved performance (e.g., less scrolling and searching) compared with Mixed section organization when reviewing progress notes.
Discussion Clinicians face significant challenges reviewing electronic clinical notes. Our findings support minimizing extraneous information in notes, removing information that can be found in other parts of the EHR, and standardizing the display and order of note sections to improve clinicians' note review experience.
Conclusion Our findings support the need to improve EHR note design and presentation to support optimal note review patterns for clinicians.
... Generally, the chaplain transmits this spiritual assessment information within the electronic health record (EHR) as either structured documentation (Burkhart et al., 2021), as a narrative clinical note (Hultman et al., 2019), or as a combination (Woggon et al., 2022). The progress note is the most typical clinical documentation tool for interprofessional clinicians when caring for hospitalized patients or those seen in an outpatient care setting (Hultman et al., 2019). ...
... Generally, the chaplain transmits this spiritual assessment information within the electronic health record (EHR) as either structured documentation (Burkhart et al., 2021), as a narrative clinical note (Hultman et al., 2019), or as a combination (Woggon et al., 2022). The progress note is the most typical clinical documentation tool for interprofessional clinicians when caring for hospitalized patients or those seen in an outpatient care setting (Hultman et al., 2019). Progress note documentation for palliative care professionals, including chaplains, is a necessarily routine component of clinical work to describe assessments made, care provided, and outcomes observed (Rosenbloom et al., 2011). ...
... This is a communication concern, especially when it is recognized that chaplains make significant spiritual and relational connections (Lee et al., 2017) and interpretively, hear valuable data through this care. The spiritual assessment found within a palliative care chaplain's clinical note is crucial for communicating the needs assessed and the plan detailed (Hultman et al., 2019). Because a spiritual assessment is contained within a clinical note, an assessment and the note are distinct but necessary for one another. ...
... Physicians and other clinicians create clinical notes through direct text entry (typing), dictation, copy-and-paste from other notes, and automated text insertion from structured information held elsewhere in the EHR. Multiple problems plague the current use of clinical notes in the EHR [18], including non-grammatical forms, colloquialisms, acronyms [19], non-standard terminology, slang [20], jargon [21], [22], euphemisms, misspellings [23], and ambiguous language [20], [21], [23]- [25]. Studies have shown that as many as 20% of text is abbreviations [26], which range from unapproved and non-standard to cryptic and dangerous (e.g., MS for "mental status", "morphine sulfate," or "multiple sclerosis") [27]- [32]. ...
... Studies have shown that as many as 20% of text is abbreviations [26], which range from unapproved and non-standard to cryptic and dangerous (e.g., MS for "mental status", "morphine sulfate," or "multiple sclerosis") [27]- [32]. Notes may be poorly organized and lack proper headers for important sections such as the history, examination, impression, and plan [10], [18]. Note-writing skills vary by clinician, with some adept while others struggle [33]. ...
Clinician notes are a rich source of patient information but often contain inconsistencies due to varied writing styles, colloquialisms, abbreviations, medical jargon, grammatical errors, and non-standard formatting. These inconsistencies hinder the extraction of meaningful data from electronic health records (EHRs), posing challenges for quality improvement, population health, precision medicine, decision support, and research. We present a large language model approach to standardizing a corpus of 1,618 clinical notes. Standardization corrected an average of grammatical errors, spelling errors, converted non-standard terms to standard terminology, and expanded abbreviations and acronyms per note. Additionally, notes were re-organized into canonical sections with standardized headings. This process prepared notes for key concept extraction, mapping to medical ontologies, and conversion to interoperable data formats such as FHIR. Expert review of randomly sampled notes found no significant data loss after standardization. This proof-of-concept study demonstrates that standardization of clinical notes can improve their readability, consistency, and usability, while also facilitating their conversion into interoperable data formats.
... Some claim that poor note quality is one of the unintended consequences of nationwide electronic health record (EHR) adoption [3][4][5][6]. The introduction of electronic workflows has allowed the flow of raw uncontextualized clinical data into provider notes [5,[7][8][9]. Over time, and with the change in the culture of clinical practice, many electronic notes have become full of noise, and clinically significant signals are now becoming harder to find [1,3,5,9,10]. ...
... Over time, and with the change in the culture of clinical practice, many electronic notes have become full of noise, and clinically significant signals are now becoming harder to find [1,3,5,9,10]. There is an urgent need to develop and implement innovative approaches in electronic clinical documentation that improve note quality and reduce unnecessary bloating [5,8,9]. ...
Background:
The introduction of electronic workflows has allowed for the flow of raw un-contextualized clinical data into medical documentation. As a result, many electronic notes have become replete of "noise" and deplete of clinically significant "signals". There is an urgent need to develop and implement innovative approaches in electronic clinical documentation that improve note quality and reduce unnecessary bloating.
Objective:
To describe the development and impact of a novel set of templates designed to change the flow of information in medical documentation.
Methods:
This is a multi-hospital nonrandomized prospective improvement study conducted on the Inpatient General Internal Medicine Service across three hospital campuses at the New York University (NYU) Langone Health System. A group of physician leaders representing each campus met biweekly for six months. The output of these meetings included 1) a conceptualization of the note bloat problem as a dysfunction in information flow 2) a set of guiding principles for organizational documentation improvement 3) the design and build of novel electronic templates that reduced the flow of extraneous information into provider notes by providing link outs to best practice data visualizations and 4) a documentation improvement curriculum for inpatient medicine providers. Prior to go-live, pragmatic usability testing was performed with the new progress note template, and the overall user experience measured using the System Usability Scale (SUS). Primary outcomes measures after go-live include template utilization rate and note length in characters.
Results:
In usability testing amongst 22 medicine providers, the new progress note template averaged a usability score of 90.6/100 on the System Usability Scale. 77% of providers strongly agreed that the new template was easy to use. 68% strongly agreed that they would like to use the template frequently. In the three months after template implementation, General Internal Medicine providers wrote 65% of all inpatient notes with the new templates. During this period of time the organization saw a 46%, 47%, and 32% reduction in note length for general medicine progress notes, consults, and H&Ps, respectively, when compared to a baseline measurement period prior to interventions.
Conclusions:
A bundled intervention that included deployment of novel templates for inpatient general medicine providers significantly reduced average note length on the clinical service. Templates designed to reduce the flow of extraneous information into provider notes performed well during usability testing, and these templates were rapidly adopted across all hospital campuses. Further research is needed to assess the impact of novel templates on note quality, provider efficiency and patient outcomes.
... Gathering these data togetherreferred to as 'foraging' in the vernaculartakes substantial time and effort. Often a step known as 'scrubbing' helps to identify key information before outpatient visits to streamline care; observational studies of providers before, during, and after visits demonstrate a substantial (up to an average of 12 minutes per patient) 18,19 amount of time gathering information to understand what to ask; how to act on what has been learned; and assure themselves that these actions were complete and appropriate. Prioritization of these data is crucial: the volume of information available far outstrips the ability of an individual to consume and process it 20 , leading to substantial errors. ...
Importance. Data, information and knowledge in health care has expanded exponentially over the last 50 years, leading to significant challenges with information overload and complex, fragmented care plans. Generative AI has the potential to facilitate summarization and integration of knowledge and wisdom to through rapid integration of data and information to enable efficient care planning.
Objective. Our objective was to understand the value of AI generated summarization through short synopses at the care transition from hospital to first outpatient visit.
Design. Using a de-identified data set of recently hospitalized patients with multiple chronic illnesses, we used the data-information-knowledge-wisdom framework to train clinicians and an open-source generative AI Large Language Model system to produce summarized patient assessments after hospitalizations. Both sets of synopses were judged blinded in random order by clinician judges.
Participants. De-identified patients had multiple chronic conditions and a recent hospitalization. Raters were physicians at various levels of training.
Main outcome. Accuracy, succinctness, synthesis and usefulness of synopses using a standardized scale with scores > 80% indicating success.
Results. AI and clinicians summarized 80 patients with 10% overlap. In blinded trials, AI synopses were rated as useful 75% of the time versus 76% for human generated ones. AI had lower succinctness ratings for the Data synopsis task (55-67%) versus human (84-86%). For accuracy and synthesis, AI had near equal or better scores in other domains (AI: 72%-79%, humans: 68%-84%), with best scores from AI in Wisdom. Interrater agreement was moderate, indicating different preferences for synopsis content, and did not vary between AI and human-created synopses.
Discussion. AI-created synopses that were nearly equivalent to human-created ones; they were slightly longer and did not always synthesize individual data elements compared to humans. Given their rapid introduction into clinical care, our framework and protocol for evaluation of these tools provides strong benchmarking capabilities for developers and implementers.
... Focusing on the performance of AUROC alone can lead to unreliable models as it is less informative if the classes are highly imbalanced [19,20], which is often observed in sepsis studies [21][22][23]. Recently, a few studies also reported the use of clinician notes as input for training models along with other medical data [24,25], however, incorporating textual notes comes with its own challenges [26]. Few sepsis algorithms have been rigorously reviewed in terms of patient outcomes [27,28]. ...
Prediction of sepsis using machine-learning approaches has recently gained traction. However, the lack of translation of these algorithms into clinical routine remains a major issue. Existing early sepsis detection methods are either based on the older definition of sepsis or do not accurately detect sepsis leading to the high frequency of false-positive alarms. This results in a well-known issue of clinicians’ “alarm fatigue”, leading to decreased responsiveness and identification, ultimately resulting in delayed clinical intervention. Hence, there is a fundamental, unmet need for a clinical decision system capable of accurate and timely sepsis diagnosis, running at the point of need. In this work, SepsisAI–a deep-learning algorithm based on long short-term memory (LSTM) networks was developed to predict the early onset of hospital-acquired sepsis in real-time for patients admitted to the ICU. The models are trained and validated with data from the PhysioNet Challenge, consisting of 40,336 patient data files from two healthcare systems: Beth Israel Deaconess Medical Center and Emory University Hospital. In the short term, the algorithm tracks frequently measured vital signs, sparsely available lab parameters, demographic features, and certain derived features for making predictions. A real-time alert system, which monitors the trajectory of the predictions, is developed on top of the deep-learning framework to minimize false alarms. On a balanced test dataset, the model achieves an AUROC, AUPRC, sensitivity, and specificity of 0.95, 0.96, 88.19%, and 96.75%, respectively at the patient level. In terms of lookahead time, the model issues a warning at a median of 6 hours (IQR 6 to 20 hours) and raises an alert at a median of 4 hours (IQR 2 to 5 hours) ahead of sepsis onset. Most importantly, the model achieves a false-alarm ratio of 3.18% for alerts, which is significantly less than other sepsis alarm systems. Additionally, on a disease prevalence-based test set, the algorithm reported similar outcomes with AUROC and AUPRC of 0.94 and 0.87, respectively, with sensitivity, and specificity of 97.05%, and 96.75%, respectively. The proposed algorithm might serve as a clinical decision support system to assist clinicians in the accurate and timely diagnosis of sepsis. With exceptionally high specificity and low false-alarm rate, this algorithm also helps mitigate the well-known issue of clinician alert fatigue arising from currently proposed sepsis alarm systems. Consequently, the algorithm partially addresses the challenges of successfully integrating machine-learning algorithms into routine clinical care.
... However, note-taking in the EHR, while necessary, suffers from note bloat and information overload, causing challenges for end-users (Liu et al., 2022). The cognitive burden on providers using EHR documents remains high and only increases in our digital era (Furlow, 2020;Hultman et al., 2019). Summarization tasks in clinical natural language processing (NLP) are a promising approach to overcome note bloat and help extract relevant, active diagnoses to help mitigate diagnostic errors. ...
The BioNLP Workshop 2023 initiated the launch of a shared task on Problem List Summarization (ProbSum) in January 2023. The aim of this shared task is to attract future research efforts in building NLP models for real-world diagnostic decision support applications, where a system generating relevant and accurate diagnoses will augment the healthcare providers' decision-making process and improve the quality of care for patients. The goal for participants is to develop models that generated a list of diagnoses and problems using input from the daily care notes collected from the hospitalization of critically ill patients. Eight teams submitted their final systems to the shared task leaderboard. In this paper, we describe the tasks, datasets, evaluation metrics, and baseline systems. Additionally, the techniques and results of the evaluation of the different approaches tried by the participating teams are summarized.
... However, note-taking in the EHR, while necessary, suffers from note bloat and information overload, causing challenges for end-users (Liu et al., 2022). The cognitive burden on providers using EHR documents remains high and only increases in our digital era (Furlow, 2020;Hultman et al., 2019). Summarization tasks in clinical natural language processing (NLP) are a promising approach to overcome note bloat and help extract relevant, active diagnoses to help mitigate diagnostic errors. ...
The BioNLP Workshop 2023 initiated the launch of a shared task on Problem List Summarization (ProbSum) in January 2023. The aim of this shared task is to attract future research efforts in building NLP models for real-world diagnostic decision support applications, where a system generating relevant and accurate diagnoses will augment the healthcare providers decision-making process and improve the quality of care for patients. The goal for participants is to develop models that generated a list of diagnoses and problems using input from the daily care notes collected from the hospitalization of critically ill patients. Eight teams submitted their final systems to the shared task leaderboard. In this paper, we describe the tasks, datasets, evaluation metrics, and baseline systems. Additionally, the techniques and results of the evaluation of the different approaches tried by the participating teams are summarized.
... However, EHRs also serve as a billing tool and unnecessary information is copied and pasted into the note contributing to note bloat [3,4]. Information and cognitive overload subsequently occur and contribute to missed diagnoses and medical errors [5][6][7]. The National Academy of Medicine (NAM, formerly known as Institute of Medicine), showed that medical errors are the sixth leading cause for deaths [8], and diagnostic error is one of the more frequent types of medical errors [9]. ...
The meaningful use of electronic health records (EHR) continues to progress in the digital era with clinical decision support systems augmented by artificial intelligence. A priority in improving provider experience is to overcome information overload and reduce the cognitive burden so fewer medical errors and cognitive biases are introduced during patient care. One major type of medical error is diagnostic error due to systematic or predictable errors in judgement that rely on heuristics. The potential for clinical natural language processing (cNLP) to model diagnostic reasoning in humans with forward reasoning from data to diagnosis and potentially reduce cognitive burden and medical error has not been investigated. Existing tasks to advance the science in cNLP have largely focused on information extraction and named entity recognition through classification tasks. We introduce a novel suite of tasks coined as Diagnostic Reasoning Benchmarks, Dr.Bench, as a new benchmark for developing and evaluating cNLP models with clinical diagnostic reasoning ability. The suite includes six tasks from ten publicly available datasets addressing clinical text understanding, medical knowledge reasoning, and diagnosis generation. DR.BENCH is the first clinical suite of tasks designed to be a natural language generation framework to evaluate pre-trained language models for diagnostic reasoning. The goal of DR. BENCH is to advance the science in cNLP to support downstream applications in computerized diagnostic decision support and improve the efficiency and accuracy of healthcare providers during patient care. We fine-tune and evaluate the state-of-the-art generative models on DR.BENCH. Experiments show that with domain adaptation pre-training on medical knowledge, the model demonstrated opportunities for improvement when evaluated in DR. BENCH. We share DR. BENCH as a publicly available GitLab repository with a systematic approach to load and evaluate models for the cNLP community. We also discuss the carbon footprint produced during the experiments and encourage future work on DR.BENCH to report the carbon footprint.
... However, EHRs also serve as a billing tool and unnecessary information is copied and pasted into the note contributing to note bloat 3,4 . Information and cognitive overload subsequently occur and contribute to missed diagnoses and medical errors 5,6 . The National Academy of Medicine (NAM, formerly known as Institute of Medicine), showed that medical errors are the sixth leading cause for deaths 7 , and diagnostic error is one of the more frequent types of medical errors 8 . ...
The meaningful use of electronic health records (EHR) continues to progress in the digital era with clinical decision support systems augmented by artificial intelligence. A priority in improving provider experience is to overcome information overload and reduce the cognitive burden so fewer medical errors and cognitive biases are introduced during patient care. One major type of medical error is diagnostic error due to systematic or predictable errors in judgment that rely on heuristics. The potential for clinical natural language processing (cNLP) to model diagnostic reasoning in humans with forward reasoning from data to diagnosis and potentially reduce the cognitive burden and medical error has not been investigated. Existing tasks to advance the science in cNLP have largely focused on information extraction and named entity recognition through classification tasks. We introduce a novel suite of tasks coined as Diagnostic Reasoning Benchmarks, DR.BENCH, as a new benchmark for developing and evaluating cNLP models with clinical diagnostic reasoning ability. The suite includes six tasks from ten publicly available datasets addressing clinical text understanding, medical knowledge reasoning, and diagnosis generation. DR.BENCH is the first clinical suite of tasks designed to be a natural language generation framework to evaluate pre-trained language models. Experiments with state-of-the-art pre-trained generative language models using large general domain models and models that were continually trained on a medical corpus demonstrate opportunities for improvement when evaluated in DR. BENCH. We share DR. BENCH as a publicly available GitLab repository with a systematic approach to load and evaluate models for the cNLP community.
... Many of the EHR-based benchmarks are time-insensitive, such as discharge summaries (Mullenbach et al., 2021;Uzuner et al., 2008), radiology reports (Abacha et al., 2021;Peng et al., 2018). They also have a strong focus on modeling clinical language instead of potentials for clinical applications with a practitioner-derived focus (Hultman et al., 2019). Recent advances in large scale language modeling enables pre-training on massive corpora and fine-tuning for in-domain tasks, such as transfer learning for BERT (Devlin et al., 2019;He et al., 2020) with ClinicalBERT (Alsentzer et al., 2019;Hao et al., 2020). ...
Applying methods in natural language processing on electronic health records (EHR) data is a growing field. Existing corpus and annotation focus on modeling textual features and relation prediction. However, there is a paucity of annotated corpus built to model clinical diagnostic thinking, a process involving text understanding, domain knowledge abstraction and reasoning. This work introduces a hierarchical annotation schema with three stages to address clinical text understanding, clinical reasoning, and summarization. We created an annotated corpus based on an extensive collection of publicly available daily progress notes, a type of EHR documentation that is collected in time series in a problem-oriented format. The conventional format for a progress note follows a Subjective, Objective, Assessment and Plan heading (SOAP). We also define a new suite of tasks, Progress Note Understanding, with three tasks utilizing the three annotation stages. The novel suite of tasks was designed to train and evaluate future NLP models for clinical text understanding, clinical knowledge representation, inference, and summarization.
... Many of the EHR-based benchmarks are time-insensitive, such as discharge summaries (Mullenbach et al., 2021;Uzuner et al., 2008), radiology reports (Abacha et al., 2021;Peng et al., 2018). They also have a strong focus on modeling clinical language instead of potentials for clinical applications with a practitioner-derived focus (Hultman et al., 2019). Recent advances in large scale language modeling enables pre-training on massive corpora and fine-tuning for in-domain tasks, such as transfer learning for BERT (Devlin et al., 2019;He et al., 2020) with ClinicalBERT (Alsentzer et al., 2019;Hao et al., 2020). ...
Applying methods in natural language processing on electronic health records (EHR) data is a growing field. Existing corpus and annotation focus on modeling textual features and relation prediction. However, there is a paucity of annotated corpus built to model clinical diagnostic thinking, a process involving text understanding, domain knowledge abstraction and reasoning. This work introduces a hierarchical annotation schema with three stages to address clinical text understanding, clinical reasoning, and summarization. We created an annotated corpus based on an extensive collection of publicly available daily progress notes, a type of EHR documentation that is collected in time series in a problem-oriented format. The conventional format for a progress note follows a Subjective, Objective, Assessment and Plan heading (SOAP). We also define a new suite of tasks, Progress Note Understanding, with three tasks utilizing the three annotation stages. The novel suite of tasks was designed to train and evaluate future NLP models for clinical text understanding, clinical knowledge representation, inference, and summarization.
... Therefore, APSO (Assessment, Plan, Subjective, Objective) format was introduced supported by studies that this format shortened the time needed because the Assessment and Plan are readily viewed at the top of the note. 11 12 Lin et al introduced the APSO format to 13 outpatient clinics at a large academic medical center, and reported that about 80% of clinicians felt APSO was faster and easier to find data. Although the APSO format was felt to be more user friendly, there was no difference detected in the actual time providers took to answer clinically related questions from APSO versus SOAP notes. ...
A physician’s progress note is an essential piece of documentation regarding key events and the daily status of patients during their hospital stay. It serves not only as a communication tool between care team members, but also chronicles clinical status and pertinent updates to their medical care. Despite the importance of these documents, little literature exists on how to help residents to improve the quality of their daily progress notes. A narrative literature review of English language literature was performed and summated to provide recommendations on how to write an inpatient progress note more accurately and efficiently. In addition, the authors will also introduce a method to build a personal template with the goal of extracting relevant data automatically to reduce clicks for an inpatient progress note in the electronic medical record system.
... Many of the EHR-based benchmarks are time-insensitive, such as discharge summaries (Mullenbach et al., 2021;Uzuner et al., 2008), radiology reports (Abacha et al., 2021;Peng et al., 2018). They also have a strong focus on modeling clinical language instead of potentials for clinical applications with a practitioner-derived focus (Hultman et al., 2019). Recent advances in large scale language modeling enables pre-training on massive corpora and fine-tuning for in-domain tasks, such as transfer learning for BERT (Devlin et al., 2019;He et al., 2020) with ClinicalBERT (Alsentzer et al., 2019;Hao et al., 2020). ...
Applying methods in natural language processing on electronic health records (EHR) data is a growing field. Existing corpus and annotation focus on modeling textual features and relation prediction. However, there is a paucity of annotated corpus built to model clinical diagnostic thinking, a process involving text understanding, domain knowledge abstraction and reasoning. This work introduces a hierarchical annotation schema with three stages to address clinical text understanding, clinical reasoning, and summarization. We created an annotated corpus based on an extensive collection of publicly available daily progress notes, a type of EHR documentation that is collected in time series in a problem-oriented format. The conventional format for a progress note follows a Subjective, Objective, Assessment and Plan heading (SOAP). We also define a new suite of tasks, Progress Note Understanding, with three tasks utilizing the three annotation stages. The novel suite of tasks was designed to train and evaluate future NLP models for clinical text understanding, clinical knowledge representation, inference, and summarization.
... For example, simply having clinical notes organized consistently can minimize the time clinicians spend reviewing them. 47 Interoperability of EHR systems (e.g., data collected from different health care institutions) is also an important factor to consider when looking at designing an EHR search function. If the search function can mine both data collected within the native-EHR system and data collected in another system, it may be useful to differentiate between those different data types when provided search results to users. ...
Objective:
Although vast amounts of patient information are captured in electronic health records (EHRs), effective clinical use of this information is challenging due to inadequate and inefficient access to it at the point of care. The purpose of this study was to conduct a scoping review of the literature on the use of EHR search functions within a single patient's record in clinical settings to characterize the current state of research on the topic and identify areas for future study.
Methods:
We conducted a literature search of four databases to identify articles on within-EHR search functions or the use of EHR search function in the context of clinical tasks. After reviewing titles and abstracts and performing a full-text review of selected articles, we included 17 articles in the analysis. We qualitatively identified themes in those articles and synthesized the literature for each theme.
Results:
Based on the 17 articles analyzed, we delineated four themes: (1) how clinicians use search functions, (2) impact of search functions on clinical workflow, (3) weaknesses of current search functions, and (4) advanced search features. Our review found that search functions generally facilitate patient information retrieval by clinicians and are positively received by users. However, existing search functions have weaknesses, such as yielding false negatives and false positives, which can decrease trust in the results, and requiring a high cognitive load to perform an inclusive search of a patient's record.
Conclusion:
Despite the widespread adoption of EHRs, only a limited number of articles describe the use of EHR search functions in a clinical setting, despite evidence that they benefit clinician workflow and productivity. Some of the weaknesses of current search functions may be addressed by enhancing EHR search functions with collaborative filtering.
... When reviewing EHRs, physicians have complained about "note bloat" or too much information, accompanied by requirements for excessive scrolling. 4 The increasing use of templated notes has extended concerns that the templates do not always aid clinical decision-making and care delivery. 5 In this issue, Epstein et al. reported a randomized clinical simulation process to compare the perceived note quality of notes using a new note template to notes using a "standard" template. ...
... 31 It appears that the flexibility in note organization creates additional burden for clinicians reviewing notes. 32,33 Most clinicians actually preferred to review the assessment and plan sections first. ...
Clinicians face competing pressures of being clinically productive while using imperfect electronic health record (EHR) systems and maximizing face-to-face time with patients. EHR use is increasingly associated with clinician burnout and underscores the need for interventions to improve clinicians’ experiences. With an aim of addressing this need, we share evidence-based informatics approaches, pragmatic next steps, and future research directions to improve 3 of the highest contributors to EHR burden: (1) documentation, (2) chart review, and (3) inbox tasks. These approaches leverage speech recognition technologies, natural language processing, artificial intelligence, and redesign of EHR workflow and user interfaces. We also offer a perspective on how EHR vendors, healthcare system leaders, and policymakers all play an integral role while sharing responsibility in helping make evidence-based sociotechnical solutions available and easy to use.
... ICU notes are often long and difficult to review. Clinicians face remarkable challenges reviewing clinical notes [27]. Therefore, a tool that can make accurate predictions on the basis of clinical notes has a major positive influence on how clinicians make informed decisions and intervene in advance when required. ...
Hospital readmission shortly after discharge is threatening to plague the quality of inpatient care. Readmission is a severe episode that leads to increased medical care costs. Federal regulations and early readmission penalties have created an incentive for healthcare facilities to reduce their readmission rates by predicting patients at a high risk of readmission. Scientists have developed prediction models by using rule-based assessment scores and traditional statistical methods, and most have focused on structured patient records. Recently, a few researchers utilized unstructured clinical notes. However, they achieved moderate prediction accuracy by making predictions of a single diagnosis subpopulation via extensive feature engineering. This study proposes the use of machine learning to learn deep representation of patient notes for the identification of high-risk readmission in a hospital-wide population. We describe and train several predictive models (standard machine learning and neural network), to which several setups have not been applied. Results show that complex deep learning models significantly outperform ( P < 0.001) conventionally applied simple models in terms of discrimination ability. We also demonstrate a simple feature evaluation using a standard model, which allows the determination of potential clinical conditions/procedures for targeting. Unlike modeling using structured patient information with considerable variability in structure when different templates or databases are adopted, this study shows that the machine learning approach can be applied to prognosticate readmission with clinical free text in various healthcare settings. Using minimum feature engineering, the trained models perform comparably well or better than other predictive models established in previous literature.
... It is not reasonable or practical to expect a physician or clinician to copy and move pages and pages of care summaries as an appropriate solution to improving interoperability. Such an approach inhibits addressing specific questions and contributes to substantial note bloat and information overload (128). Physicians need data presented in a way that allows them to interpret the important elements and apply medical judgment to the patient at hand, communicate and educate patients on their health, and engage in shared medical decision making (119). ...
The American College of Physicians (ACP) has long advocated for universal access to high-quality health care in the United States. Yet, it is essential that the U.S. health system goes beyond ensuring coverage, efficient delivery systems, and affordability. Fundamental restructuring of payment policies and delivery systems is required to achieve a health care system that puts patients' interests first and supports physicians and their care teams to deliver high-value, patient- and family-centered care. The ACP calls for reform of U.S. payment, delivery, and information technology systems to achieve this vision. The ACP's recommendations include increased investment in primary care; alignment of financial incentives to achieve better patient outcomes, lower costs, reduce inequities in health care, and facilitate team-based care; freeing patients and physicians of inefficient administrative and billing tasks and documentation requirements; and development of health information technologies that enhance the patient-physician relationship.
The majority of palliative care patients and their loved ones desire for their spiritual care needs and preferences to be assessed and addressed. Most clinicians also desire to provide some form of spiritual care to patients and those within the patient’s inner circle of support. Barriers persist, however, to the delivery of this spiritual care. This chapter will examine the evidence base concerning professional, personal, and institutional barriers as well as explore ways of overcoming these barriers. The professional barriers discussed include confusion regarding whose job it is to provide spiritual care, communication concerns about spiritual care within the interdisciplinary team, the need for spiritual care training, and existing structural challenges. From a clinician’s stated discomfort to the possible fear of offending patients, personal barriers are also considered. The institutional barriers of time and money are also explored from the lens of both the chaplain and non-chaplains. This chapter concludes with suggestions for overcoming spiritual care barriers. Three possible solutions include expanding the knowledge and evidence base for spiritual care, optimising the use of telehealth, and promoting professional development. As spiritual barriers are overcome, the healing benefits received will strengthen whole person care for those living with serious illness.
Objective
Leverage electronic health record (EHR) audit logs to develop a machine learning (ML) model that predicts which notes a clinician wants to review when seeing oncology patients.
Materials and Methods
We trained logistic regression models using note metadata and a Term Frequency Inverse Document Frequency (TF-IDF) text representation. We evaluated performance with precision, recall, F1, AUC, and a clinical qualitative assessment.
Results
The metadata only model achieved an AUC 0.930 and the metadata and TF-IDF model an AUC 0.937. Qualitative assessment revealed a need for better text representation and to further customize predictions for the user.
Discussion
Our model effectively surfaces the top 10 notes a clinician wants to review when seeing an oncology patient. Further studies can characterize different types of clinician users and better tailor the task for different care settings.
Conclusion
EHR audit logs can provide important relevance data for training ML models that assist with note-writing in the oncology setting.
Background and Objective Clinical documentation is essential for conveying medical decision-making, communication between providers and patients, and capturing quality, billing, and regulatory measures during emergency department (ED) visits. Growing evidence suggests the benefits of note template standardization; however, variations in documentation practices are common. The primary objective of this study is to measure the utilization and coding performance of a standardized ED note template implemented across a nine-hospital health system.
Methods This was a retrospective study before and after the implementation of a standardized ED note template. A multi-disciplinary group consensus was built around standardized note elements, provider note workflows within the electronic health record (EHR), and how to incorporate newly required medical decision-making elements. The primary outcomes measured included the proportion of ED visits using standardized note templates, and the distribution of billing codes in the 6 months before and after implementation.
Results In the preimplementation period, a total of six legacy ED note templates were being used across nine EDs, with the most used template accounting for approximately 36% of ED visits. Marked variations in documentation elements were noted across six legacy templates. After the implementation, 82% of ED visits system-wide used a single standardized note template. Following implementation, we observed a 1% increase in the proportion of ED visits coded as highest acuity and an unchanged proportion coded as second highest acuity.
Conclusion We observed a greater than twofold increase in the use of a standardized ED note template across a nine-hospital health system in anticipation of the new 2023 coding guidelines. The development and utilization of a standardized note template format relied heavily on multi-disciplinary stakeholder engagement to inform design that worked for varied documentation practices within the EHR. After the implementation of a standardized note template, we observed better-than-anticipated coding performance.
Background and objectives:
Despite use of standardized electronic health record templates, the structure of discharge summaries may hinder communication from inpatient settings to primary care providers (PCPs). We developed an enhanced electronic discharge summary template to improve PCP satisfaction with written discharge summaries targeting diagnoses, medication reconciliation, laboratory test results, specialist follow-up, and recommendations.
Methods:
Resident template usage was measured using statistical process control charts. PCP reviewers' discharge summary satisfaction was surveyed using 5-point Likert scales analyzed using the Mann-Whitney U test. Residents were surveyed for satisfaction.
Results:
Resident template usage increased from 61% initially to 72% of discharge summaries at 6 months. The PCP reviewers reported increased satisfaction for summaries using the template compared with those without (4.3 vs 3.9, P = .003). Surveyed residents desired template inclusion in the default electronic discharge summary (93%).
Conclusions:
This system-level resident-initiated quality improvement initiative created a novel discharge summary template that achieved widespread usage among residents and significantly increased outpatient PCP satisfaction.
Background
Clinician notes are structured in a variety of ways. This research pilot tested an innovative study design and explored the impact of note formats on diagnostic accuracy and documentation review time.
Objective
To compare two formats for clinical documentation (narrative format vs. list of findings) on clinician diagnostic accuracy and documentation review time.
Method
Participants diagnosed written clinical cases, half in narrative format, and half in list format. Diagnostic accuracy (defined as including correct case diagnosis among top three diagnoses) and time spent processing the case scenario were measured for each format. Generalised linear mixed regression models and bias-corrected bootstrap percentile confidence intervals for mean paired differences were used to analyse the primary research questions.
Results
Odds of correctly diagnosing list format notes were 26% greater than with narrative notes. However, there is insufficient evidence that this difference is significant (75% CI 0.8–1.99). On average the list format notes required 85.6 more seconds to process and arrive at a diagnosis compared to narrative notes (95% CI -162.3, −2.77). Of cases where participants included the correct diagnosis, on average the list format notes required 94.17 more seconds compared to narrative notes (75% CI -195.9, −8.83).
Conclusion
This study offers note format considerations for those interested in improving clinical documentation and suggests directions for future research. Balancing the priority of clinician preference with value of structured data may be necessary.
Implications
This study provides a method and suggestive results for further investigation in usability of electronic documentation formats.
Writing progress notes represent a critical activity of practicing clinicians in a variety of settings. They provide a way for medical practitioners, insurance companies, and others to communicate in a timely fashion regarding ongoing clinical care. Previous research showed that intervention components like didactic training, using note templates, and feedback improved the quality of progress notes. At least two questions remain despite several studies already addressing progress note writing. First, previous research most often used multiple intervention components to improve progress notes. Thus, the relative impact of two common components of interventions, such as didactic training and feedback, is unclear. Second, previous research has not evaluated the acceptability of improved progress notes for the practitioners that actually utilize them. Thus, the purpose of the current study evaluated the components of didactic training and feedback on improved progress note writing for four direct staff employed by a psychiatric inpatient unit. A second purpose of this study was to evaluate the acceptability of the training procedures by both (a) the direct‐care staff participating in this study and (b) four members of the psychiatric treatment team that used direct‐care staff progress notes to inform their clinical care. Results showed that feedback was necessary to improve the accuracy of progress notes for three of four participants. The direct‐care staff reported the training procedures as acceptable and the treatment team noted improvements in the quality of the progress notes after intervention. These data will be discussed in terms of ways to arrange effective training programs to improve direct‐care staff's progress notes.
Automatically summarizing patients' main problems from daily progress notes using natural language processing methods helps to battle against information and cognitive overload in hospital settings and potentially assists providers with computerized diagnostic decision support. Problem list summarization requires a model to understand, abstract, and generate clinical documentation. In this work, we propose a new NLP task that aims to generate a list of problems in a patient's daily care plan using input from the provider's progress notes during hospitalization. We investigate the performance of T5 and BART, two state-of-the-art seq2seq transformer architectures, in solving this problem. We provide a corpus built on top of progress notes from publicly available electronic health record progress notes in the Medical Information Mart for Intensive Care (MIMIC)-III. T5 and BART are trained on general domain text, and we experiment with a data augmentation method and a domain adaptation pre-training method to increase exposure to medical vocabulary and knowledge. Evaluation methods include ROUGE, BERTScore, cosine similarity on sentence embedding, and F-score on medical concepts. Results show that T5 with domain adaptive pre-training achieves significant performance gains compared to a rule-based system and general domain pre-trained language models, indicating a promising direction for tackling the problem summarization task.
Objectives This study set out to obtain a general profile of physician time expenditure and electronic health record (EHR) limitations in a large university medical center in Germany. We also aim to illustrate the merit of a tool allowing for easier capture and prioritization of specific clinical needs at the point of care for which the current study will inform development in subsequent work.
Methods Nineteen physicians across six different departments participated in this study. Direct clinical observations were conducted with 13 out of 19 physicians for a total of 2,205 minutes, and semistructured interviews were conducted with all participants. During observations, time was measured for larger activity categories (searching information, reading information, documenting information, patient interaction, calling, and others). Semistructured interviews focused on perceived limitations, frustrations, and desired improvements regarding the EHR environment.
Results Of the observed time, 37.1% was spent interacting with the health records (9.0% searching, 7.7% reading, and 20.5% writing), 28.0% was spent interacting with patients corrected for EHR use (26.9% of time in a patient's presence), 6.8% was spent calling, and 28.1% was spent on other activities. Major themes of discontent were a spread of patient information, high and often repeated documentation burden, poor integration of (new) information into workflow, limits in information exchange, and the impact of such problems on patient interaction. Physicians stated limited means to address such issues at the point of care.
Conclusion In the study hospital, over one-third of physicians' time was spent interacting with the EHR, environment, with many aspects of used systems far from optimal and no convenient way for physicians to address issues as they occur at the point of care. A tool facilitating easier identification and registration of issues, as they occur, may aid in generating a more complete overview of limitations in the EHR environment.
Effective documentation is considered a core competency for clinical ethics consultation. Ethics consultants within the Cleveland Clinic in Cleveland, Ohio, observed variation in the formatting of ethics chart notes across consultants and realized that this formatting was based on their own views of effectiveness. To minimize variation and optimize the readability and understandability of ethics chart notes for end users, a team undertook a quality improvement project to assess the formatting preferences of healthcare professionals who rely on ethics consultation notes. The team developed three sample templates and conducted interviews with stakeholders to understand their preferences. A single standardized template was developed based on the preferences that emerged, which all consultants on the ethics consultation service then utilized. In the first five months of implementation, the percentage of end user respondents marking the highest Likert scale option on a post-consultation survey regarding whether the ethics consultation service provided helpful documentation increased from 60 percent to 72 percent compared to the same five-month period in the year prior.
Background:
Electronic health records (EHRs) demand a significant amount of physician time for documentation, orders, and communication during care delivery. Resident physicians already work long hours as they gain experience and develop both clinical and socio-technical skills.
Objectives:
Measure how much time resident physicians spend in the EHR during clinic hours and after-hours, and how EHR usage changes as they gain experience over a 12-month period.
Methods:
Longitudinal descriptive study where participants were 622 resident physicians across postgraduate year cohorts (of 948 resident physicians at the institution, 65.6%) working in an ambulatory setting from July 2017 to June 2018. Time spent in the EHR per patient, patients records documented per day, and proportion of EHR time spent after-hours were the outcome, while the number of months of ambulatory care experience was the predictor.
Results:
Resident physicians spent an average of 45.6 minutes in the EHR per patient, with 13.5% of that time spent after-hours. Over 12 months of ambulatory experience, resident physicians reduced their EHR time per patient and saw more patients per day, but the proportion of EHR time after-hours did not change.
Conclusion:
Resident physicians spend a significant amount of time working in the EHR, both during and after clinic hours. While residents improve efficiency in reducing EHR time per patient, they do not reduce the proportion of EHR time spent after-hours. Concerns over the impact of EHRs on physician well-being should include recognition of the burden of EHR usage on early-career physicians.
Modern EHR systems are complex, and end-user behavior and training are highly variable. The need for clinicians to access key clinical data is a critical patient safety issue. This study used a mixed methods approach employing a high-fidelity EHR simulation environment, eye and screen tracking, surveys, and semi-structured interviews to characterize typical EHR usage by hospital physicians (hospitalists) as they encounter a new patient. The main findings were: 1) There were strong similarities across the groups in the information types the physicians looked at most frequently, 2) While there was no overall difference in case duration between the groups, we observed two distinct workflow types between the groups with respect to gathering information in the EHR and creating a note, and 3) A majority of the case time was devoted to note composition in both groups. This has implications for EHR interface design and raises further questions about what individual user workflows exist in the EHR.
The American Medical Association asked RAND Health to characterize the factors that affect physician professional satisfaction. RAND researchers sought to identify high-priority determinants of professional satisfaction by gathering data from 30 physician practices in six states, using a combination of surveys and semistructured interviews. This article presents the results of the subsequent analysis.
In the collaborative hospital environment, pharmacists are important members of the healthcare team, yet compared to physicians and nurses, little is known about pharmacists’ information needs or how they interact with the electronic health record (EHR). We directly observed seven clinical inpatient pharmacists as they interacted with the EHR preparing for clinical rounds using an eye-tracking camera and contextual inquiry. Pharmacists spent 50% of their time reading information from the EHR, such as notes and medication lists, and 27% of their time copying EHR data onto paper, such as patient history and laboratory results. In an environment where minutes count, the results of this study can help inform the development of CDS tools and/or EHR designs to facilitate the information needs of the pharmacists in providing care for their patients.
Purpose:
Survey studies of health information systems use tend to focus on availability of functionalities, adoption and intensity of use. Usability surveys have not been systematically conducted by any healthcare professional groups on a national scale on a repeated basis. This paper presents results from two cross-sectional surveys of physicians' experiences with the usability of currently used EHR systems in Finland. The research questions were: To what extent has the overall situation improved between 2010 and 2014? What differences are there between healthcare sectors?
Methods:
In the spring of 2014, a survey was conducted in Finland using a questionnaire that measures usability and respondents' user experiences with electronic health record (EHR) systems. The survey was targeted to physicians who were actively doing clinical work. Twenty-four usability-related statements, that were identical in 2010 and 2014, were analysed from the survey. The respondents were also asked to give an overall rating of the EHR system they used. The study data comprised responses from 3081 physicians from the year 2014 and from 3223 physicians in the year 2010, who were using the nine most commonly used EHR system brands in Finland.
Results:
Physicians' assessments of the usability of their EHR system remain as critical as they were in 2010. On a scale from 1 ('fail') to 7 ('excellent') the average of overall ratings of their principally used EHR systems varied from 3.2 to 4.4 in 2014 (and in 2010 from 2.5 to 4.3). The results show some improvements in the following EHR functionalities and characteristics: summary view of patient's health status, prevention of errors associated with medication ordering, patient's medication list as well as support for collaboration and information exchange between the physician and the nurses. Even so, support for cross-organizational collaboration between physicians and for physician-patient collaboration were still considered inadequate. Satisfaction with technical features had not improved in four years. The results show marked differences between the EHR system brands as well as between healthcare sectors (private sector, public hospitals, primary healthcare). Compared to responses from the public sector, physicians working in the private sector were more satisfied with their EHR systems with regards to statements about user interface characteristics and support for routine tasks. Overall, the study findings are similar to our previous study conducted in 2010.
Conclusions:
Surveys about the usability of EHR systems are needed to monitor their development at regional and national levels. To our knowledge, this study is the first national eHealth observatory questionnaire that focuses on usability and is used to monitor the long-term development of EHRs. The results do not show notable improvements in physician's ratings for their EHRs between the years 2010 and 2014 in Finland. Instead, the results indicate the existence of serious problems and deficiencies which considerably hinder the efficiency of EHR use and physician's routine work. The survey results call for considerable amount of development work in order to achieve the expected benefits of EHR systems and to avoid technology-induced errors which may endanger patient safety. The findings of repeated surveys can be used to inform healthcare providers, decision makers and politicians about the current state of EHR usability and differences between brands as well as for improvements of EHR usability. This survey will be repeated in 2017 and there is a plan to include other healthcare professional groups in future surveys.
Primary care physicians face cognitive overload daily, perhaps exacerbated by the form of electronic health record documentation. We examined physician information needs to prepare for clinic visits, focusing on past clinic progress notes.
This study used cognitive task analysis with 16 primary care physicians in the scenario of preparing for office visits. Physicians reviewed simulated acute and chronic care visit notes. We collected field notes and document highlighting and review, and we audio-recorded cognitive interview while on task, with subsequent thematic qualitative analysis. Member checks included the presentation of findings to the interviewed physicians and their faculty peers.
The Assessment and Plan section was most important and usually reviewed first. The History of the Present Illness section could provide supporting information, especially if in narrative form. Physicians expressed frustration with the Review of Systems section, lamenting that the forces driving note construction did not match their information needs. Repetition of information contained in other parts of the chart (eg, medication lists) was identified as a source of note clutter. A workflow that included a patient summary dashboard made some elements of past notes redundant and therefore a source of clutter.
Current ambulatory progress notes present more information to the physician than necessary and in an antiquated format. It is time to reengineer the clinic progress note to match the workflow and information needs of its primary consumer.
© Copyright 2015 by the American Board of Family Medicine.
Issues with workflow integration have contributed to slow rates of EHR adoption in ambulatory outpatient care settings. In response to workflow integration challenges with EHRs, clinicians often develop workarounds to complete clinical tasks in ways other than were intended by system designers. Based on the insights generated during collegial discussions with physician Subject Matter Experts (SMEs) and three interdisciplinary team meetings with clinical and human factors experts, we created process map visualizations. A wide range of opportunities to improve workflow through enhanced functionality with the EHR were identified. Targeted recommendations for EHR developers and ambulatory (outpatient) care centers are proposed to increase efficiency, allow for better eye contact between the physician and patient, improve physician’s information workflow, and reduce alert fatigue. These recommendations provide a first step in moving from a billing-centered perspective to a clinician-centered perspective.
Objective:
Several studies have documented the preference for physicians to attend to the impression and plan section of a clinical document. However, it is not clear how much attention other sections of a document receive. The goal of this study was to identify how physicians distribute their visual attention while reading electronic notes.
Methods:
We used an eye-tracking device to assess the visual attention patterns of ten hospitalists as they read three electronic notes. The assessment included time spent reading specific sections of a note as well as rates of reading. This visual analysis was compared with the content of simulated verbal handoffs for each note and debriefing interviews.
Results:
Study participants spent the most time in the "Impression and Plan" section of electronic notes and read this section very slowly. Sections such as the "Medication Profile", "Vital Signs" and "Laboratory Results" received less attention and were read very quickly even if they contained more content than the impression and plan. Only 9% of the content of physicians' verbal handoff was found outside of the "Impression and Plan."
Conclusion:
Physicians in this study directed very little attention to medication lists, vital signs or laboratory results compared with the impression and plan section of electronic notes. Optimizing the design of electronic notes may include rethinking the amount and format of imported patient data as this data appears to largely be ignored.
EHR clinical document synthesis by clinicians may be time-consuming and error-prone due to the complex organization of narratives, excessive redundancy within documents, and, at times, inadvertent proliferation of data inconsistencies. Development of EHR systems that are easily adaptable to the user's work processes requires research into visualization techniques that can optimize information synthesis at the point of care.
To evaluate the effect of a prototype visualization tool for clinically relevant new information on clinicians' synthesis of EHR clinical documents and to understand how the tool may support future designs of clinical document user interfaces.
A mixed methods approach to analyze the impact of the visualization tool was used with a sample of eight medical interns as they synthesized EHR clinical documents to accomplish a set of four pre-formed clinical scenarios using a think-aloud protocol.
Differences in the missing (unretrieved) patient information (2.3±1.2 [with the visualization tool] vs. 6.8±1.2 [without the visualization tool], p = 0.08) and accurate inferences (1.3±0.3 vs 2.3±0.3, p = 0.09) were not statistically significant but suggest some improvement with the new information visualization tool. Despite the non-significant difference in total times to task completion (43±4 mins vs 36±4 mins, p = 0.35) we observed shorter times for two scenarios with the visualization tool, suggesting that the time-saving benefits may be more evident with certain clinical processes. Other observed effects of the tool include more intuitive navigation between patient details and increased efforts towards methodical synthesis of clinical documents.
Our study provides some evidence that new information visualization in clinical notes may positively influence synthesis of patient information from EHR clinical documents. Our findings provide groundwork towards a more effective display of EHR clinical documents using advanced visualization applications.
Background:
Electronic health records (EHRs) are structured, distributed documentation systems that differ from paper charts. These systems require skills not traditionally used to navigate a paper chart and to produce a written clinic note. Despite these differences, little attention has been given to physicians' electronic health record (EHR)-writing and -reading competence.
Purposes:
This study aims to investigate physicians' self-assessed competence to document and to read EHR notes; writing and reading preferences in an EHR; and demographic characteristics associated with their perceived EHR ability and preference.
Methods:
Fourteen 5-point Likert scale items, based on EHR system characteristics and a literature review, were developed to measure EHR-writing and -reading competence and preference. Physicians in the midwest region of the United States were invited via e-mail to complete the survey online from February to April 2011. Factor analysis and reliability testing were conducted to provide validity and reliability of the instrument. Correlation and regression analysis were conducted to pursue answers to the research questions.
Results:
Ninety-one physicians (12.5%), from general and specialty fields, working in inpatient and outpatient settings, participated in the survey. Despite over 3 years of EHR experience, respondents perceived themselves to be incompetent in EHR writing and reading (Mean = 2.74, SD = 0.76). They preferred to read succinct, narrative notes in EHR systems. However, physicians with higher perceived EHR-writing and -reading competence had less preference toward reading succinct (r= - 0.33, p<0.001) and narrative (r= - 0.36, p<0.001) EHR notes than physicians with lower perceived EHR competence. Physicians' perceived EHR-writing and -reading competence was strongly related to their EHR navigation skills (r=0.55, p<0.0001).
Conclusions:
Writing and reading EHR documentation is different for physicians. Maximizing navigation skills can optimize non-linear EHR writing and reading. Pedagogical questions remain related to how physicians and medical students are able to retrieve correct information effectively and to understand thought patterns in collectively lengthier and sometimes fragmented EHR chart notes.
Many health care organizations are deploying electronic health records (EHRs). A health care provider's EHR progress notes are essential for effective communication. However, these notes may increase errors when they are difficult to read. Billing requirements, regulatory statements, and extensive inclusion of test results detract from progress note brevity and clarity. In our experience, EHR progress notes that include such elements can span 17 electronic pages, rendering actual clinical reasoning extraordinarily difficult to locate. Missing data can lead to lost productivity and increased cost. Health care providers' frustration with EHR progress notes may interfere with EHR adoption and deployment. Although the traditional SOAP (Subjective, Objective, Assessment, Plan) format mirrors the sequence of a clinical encounter, it translates poorly from paper medical charts to the EHR. Finding the Assessment and Plan requires considerable on-screen “scrolling.” We examined the adoption of an alternate APSO (Assessment, Plan, Subjective, Objective) format, which places the Assessment and Plan at the top of the note, where it is readily located when the EHR note is opened. We hypothesized it would improve readability and satisfaction and shorten the time to answer clinical questions.
Clinical documentation is central to patient care. The success of electronic health record system adoption may depend on how well such systems support clinical documentation. A major goal of integrating clinical documentation into electronic heath record systems is to generate reusable data. As a result, there has been an emphasis on deploying computer-based documentation systems that prioritize direct structured documentation. Research has demonstrated that healthcare providers value different factors when writing clinical notes, such as narrative expressivity, amenability to the existing workflow, and usability. The authors explore the tension between expressivity and structured clinical documentation, review methods for obtaining reusable data from clinical notes, and recommend that healthcare providers be able to choose how to document patient care based on workflow and note content needs. When reusable data are needed from notes, providers can use structured documentation or rely on post-hoc text processing to produce structured data, as appropriate.
To explore different user-interface designs for structured progress note entry, with a long-term goal of developing design guidelines for user interfaces where users select items from large medical vocabularies.
The authors created eight different prototypes of a pen-based progress-note-writing system called PEN-Ivory. Each prototype allows physicians to write patient progress notes using simple pen-based gestures such as circle, line-out, and scratch-out. The result of an interaction with PEN-Ivory is a progress note in English prose. The eight prototypes were designed in a principled way, so that they differ from one another in just one of three different user-interface characteristics.
Five of the eight prototypes were tested by measuring the time it took 15 users, each using a distinct prototype, to document three patient cases consisting of a total of 63 medical findings.
The prototype that allowed the fastest data entry had the following three user-interface characteristics: it used a paging rather than a scrolling form, it used a fixed palette of modifiers rather than a dynamic "pop-up" palette, and it made available all findings from the controlled vocabulary at once rather than displaying only a subset of findings generated by analyzing the patient's problem list.
Even simple design changes to a user interface can make dramatic differences in user performance. The authors discuss possible influences on performance, such as positional constancy, user uncertainty and system anticipation, that may contribute significantly to the effectiveness of systems that display menus of items from large controlled vocabularies of medicine.
To explore the use of an observational, cognitive-based approach for differentiating between successful, suboptimal, and failed entry of coded data by clinicians in actual practice, and to detect whether causes for unsuccessful attempts to capture true intended meaning were due to terminology content, terminology representation, or user interface problems.
Observational study with videotaping and subsequent coding of data entry events in an outpatient clinic at New York Presbyterian Hospital.
Eight attending physicians, 18 resident physicians, and 1 nurse practitioner, using the Medical Entities Dictionary (MED) to record patient problems, medications, and adverse reactions in an outpatient medical record system.
Classification of data entry events as successful, suboptimal, or failed, and estimation of cause; recording of system response time and total event time.
Two hundred thirty-eight data entry events were analyzed; 71.0 percent were successful, 6.3 percent suboptimal, and 22.7 percent failed; unsuccessful entries were due to problems with content in 13.0 percent of events, representation problems in 10.1 percent of events, and usability problems in 5.9 percent of events. Response time averaged 0.74 sec, and total event time averaged 40.4 sec. Of an additional 209 tasks related to drug dose and frequency terms, 94 percent were successful, 0.5 percent were suboptimal, and 6 percent failed, for an overall success rate of 82 percent.
Data entry by clinicians using the outpatient system and the MED was generally successful and efficient. The cognitive-based observational approach permitted detection of false-positive (suboptimal) and false-negative (failed due to user interface) data entry.
Introduction:
Cluttered documentation may contribute adversely to physician readers' cognitive load, inadvertently obscuring high-value information with less valuable information. We test the hypothesis that a novel, collapsible assessment, plan, subjective, objective (APSO) note design would be faster, more accurate, and more satisfying to use than a conventional electronic health record (EHR) subjective, objective, assessment, plan (SOAP) note for finding information needed for ambulatory chronic disease care.
Methods:
We iteratively developed physician clinic note prototypes with features designed to emphasize more important information and de-emphasize less clinically relevant information. Sixteen primary care physicians reviewed comparable clinic notes with the 4 note styles presented in random order to find key information in the notes during timed tasks. The 4 note styles were denoted A (traditional SOAP note), B (2-column APSO note), C (collapsible APSO note), and D (2-column collapsible APSO note). The 4 unique note styles were designed to have equal amounts of information in each section. We simulated their utility for clinical practice by imposing time limits and by interrupting 1 of the tasks with a typical clinical interruption. For each session, we recorded audio, computer-screen activity, eye tracking, and made field notes. We obtained usability ratings (System Usability Scale), new feature preference ratings, and performed semistructured post-task interviews with subsequent content analysis. We compared the effectiveness of the 4 note styles by measuring time on task, task success (accuracy), and effort as measured by NASA Task Load Index.
Results:
Note styles C and D were significantly faster than A and B for the Review of Systems and Physical Examination tasks, as we expected. Notes B and C had the best success (finding requested data) scores. Users strongly endorsed all the new note features incorporated into the new note prototypes. Previously expressed concerns about temporarily hiding parts of the note (using the accordion display design pattern) were allayed. Usability ratings for note A were worst but comparably better for note styles B, C, and D.
Discussion:
The new APSO note prototypes performed better than the traditional SOAP note format for speed, task success (accuracy), and usability for physician users acquiring information needed for a typical chronic disease visit in primary care. Moving Assessment and Plan to the top is 1 easily accomplished feature change. Innovative documentation displays of EHR data can safely improve information display without eliminating data from the record of the visit.
Objectives:
To evaluate the use, usability, and physician satisfaction of a locally developed problem-oriented clinical notes application that replaced paper-based records in a large Dutch university medical center.
Methods:
Using a clinical notes database and an application event log file and a cross-sectional survey of usability, authors retrospectively analyzed system usage for medical specialties, users, and patients over 4 years. A standardized questionnaire measured usability. Authors analyzed the effects of sex, age, professional experience, training hours, and medical specialty on user satisfaction via univariate analysis of variance. Authors also examined the correlation between user satisfaction in relation to users' intensity of use of the application.
Results:
In total 1,793 physicians used the application to record progress notes for 219,755 patients. The overall satisfaction score was 3.2 on a scale from 1 (highly dissatisfied) to 5 (highly satisfied). A statistically significant difference occurred in satisfaction by medical specialty, but no statistically significant differences in satisfaction took place by sex, age, professional experience, or training hours. Intensity of system use did not correlate with physician satisfaction.
Conclusions:
By two years after the start of the implementation, all medical specialties utilized the clinical notes application. User satisfaction was neutral (3.2 on a 1-5 scale). Authors believe that the significant factors facilitating this transition mirrored success factors reported by other groups: a generic, consistent, and transparent design of the application; intensive collaboration; continuous monitoring; and an incremental rollout.
Background:
Communication errors are identified as a root cause contributing to a majority of sentinel events. The clinical note is a cornerstone of physician communication, yet there are few published interventions on teaching note writing in the electronic health record (EHR). This is a prospective, two-site, quality improvement project to assess and improve the quality of clinical documentation in the EHR using a validated assessment tool.
Methods:
Internal Medicine (IM) residents at the University of Kentucky College of Medicine (UK) and Montefiore Medical Center/Albert Einstein College of Medicine (MMC) received one of two interventions during an inpatient ward month: either a lecture, or a lecture and individual feedback on progress notes. A third group of residents in each program served as control. Notes were evaluated with the Physician Documentation Quality Instrument 9 (PDQI-9).
Results:
Due to a significant difference in baseline PDQI-9 scores at MMC, the sites were not combined. Of 75 residents at the UK site, 22 were eligible, 20 (91%) enrolled, 76 notes in total were scored. Of 156 residents at MMC, 22 were eligible, 18 (82%) enrolled, 40 notes in total were scored. Note quality did not improve as measured by the PDQI-9.
Conclusion:
This educational quality improvement project did not improve the quality of clinical documentation as measured by the PDQI-9. This project underscores the difficulty in improving note quality. Further efforts should explore more effective educational tools to improve the quality of clinical documentation in the EHR.
Background:
Despite widespread electronic health record (EHR) adoption, poor EHR system usability continues to be a significant barrier to effective system use for end users. One key to addressing usability problems is to employ user testing and user-centered design.
Objectives:
To understand if redesigning an EHR-based navigation tool with clinician input improved user performance and satisfaction.
Methods:
A usability evaluation was conducted to compare two versions of a redesigned ambulatory navigator. Participants completed tasks for five patient cases using the navigators, while employing a think-aloud protocol. The tasks were based on Meaningful Use (MU) requirements.
Results:
The version of navigator did not affect perceived workload, and time to complete tasks was longer in the redesigned navigator. A relatively small portion of navigator content was used to complete the MU-related tasks, though navigation patterns were highly variable across participants for both navigators. Preferences for EHR navigation structures appeared to be individualized.
Conclusions:
This study demonstrates the importance of EHR usability assessments to evaluate group and individual performance of different interfaces and preferences for each design.
The modern medical record is not only used by providers to record nuances of patient care, but also is a document that must withstand the scrutiny of insurance payers and legal review. Medical documentation has evolved with the rapid growth in the use of electronic health records (EHRs). The medical software industry has created new tools and more efficient ways to document patient care encounters and record results of diagnostic testing. While these techniques have resulted in efficiencies and improvements in patient care and provider documentation, they have also created a host of new problems, including authorship attribution, data integrity, and regulatory concerns over the accuracy and medical necessity of billed services. Policies to guide provider documentation in EHRs have been developed by institutions and payers with the goal of reducing patient care risks as well as preventing fraud and abuse. In this article, we describe the major content-importing technologies that are commonly used in EHR documentation as well as the benefits and risks associated with their use. We have also reviewed a number of institutional policies and offer some best practice recommendations.
Time is a measurable and critical resource that affects the quality of services provided in clinical practice. There is limited insight into the effects of time restrictions on clinicians' cognitive processes with the electronic health record (EHR) in providing ambulatory care. Objective To understand the impact of time constraints on clinicians' synthesis of text-based EHR clinical notes.
We used an established clinician cognitive framework based on a think-aloud protocol. We studied interns' thought processes as they accomplished a set of four preformed ambulatory care clinical scenarios with and without time restrictions in a controlled setting.
Interns most often synthesized details relevant to patients' problems and treatment, regardless of whether or not the time available for task performance was restricted. In contrast to previous findings, subsequent information commonly synthesized by clinicians related most commonly to the chronology of clinical events for the unrestricted time observations and to investigative procedures for the time-restricted sessions. There was no significant difference in the mean number of omission errors and incorrect deductions when interns synthesized the EHR clinical notes with and without time restrictions (3.5 ± 0.5 vs 2.3 ± 0.5, p = 0.14).
Our results suggest that the incidence of errors during clinicians' synthesis of EHR clinical notes is not increased with modest time restrictions, possibly due to effective adjustments of information processing strategies learned from the usual time-constrained nature of patient visits. Further research is required to investigate the effects of similar or more extreme time variations on cognitive processes employed with different levels of expertise, specialty, and with different care settings.
In 2013, electronic documentation of clinical care stands at a crossroads. The benefits of creating digital notes are at risk of being overwhelmed by the inclusion of easily importable detail. Providers are the primary authors of encounters with patients. We must document clearly our understanding of patients and our communication with them and our colleagues. We want to document efficiently to meet without exceeding documentation guidelines. We copy and paste documentation, because it not only simplifies the documentation process generally, but also supports meeting coding and regulatory requirements specifically. Since the primary goal of our profession is to spend as much time as possible listening to, understanding and helping patients, clinicians need information technology to make electronic documentation easier, not harder. At the same time, there should be reasonable restrictions on the use of copy and paste to limit the growing challenge of ‘note bloat’. We must find the right balance between ease of use and thoughtless documentation. The guiding principles in this document may be used to launch an interdisciplinary dialogue that promotes useful and necessary documentation that best facilitates efficient information capture and effective display.
Citation: Shoolin J, Ozeran L, Hamann C, Bria W. II. Association of Medical Directors of Information Systems Consensus on Inpatient Electronic Health Record Documentation. Appl Clin Inf 2013; 4: 293–303
http://dx.doi.org/10.4338/ACI-2013-02-R-0012
Amazingly, people sometimes prefer software systems that hinder their ability to use them.
NASA-TLX is a multi-dimensional scale designed to obtain workload estimates from one or more operators while they are performing a task or immediately afterwards. The years of research that preceded subscale selection and the weighted averaging approach resulted in a tool that has proven to be reasonably easy to use and reliably sensitive to experimentally important manipulations over the past 20 years. Its use has spread far beyond its original application (aviation), focus (crew complement), and language (English). This survey of 550 studies in which NASA-TLX was used or reviewed was undertaken to provide a resource for a new generation of users. The goal was to summarize the environments in which it has been applied, the types of activities the raters performed, other variables that were measured that did (or did not) covary, methodological issues, and lessons learned
Clinicians utilize electronic health record (EHR) systems during time-constrained patient encounters where large amounts of clinical text must be synthesized at the point of care. Qualitative methods may be an effective approach for uncovering cognitive processes associated with the synthesis of clinical documents within EHR systems. We utilized a think-aloud protocol and content analysis with the goal of understanding cognitive processes and barriers involved as medical interns synthesized patient clinical documents in an EHR system to accomplish routine clinical tasks. Overall, interns established correlations of significance and meaning between problem, symptom and treatment concepts to inform hypotheses generation and clinical decision-making. Barriers identified with synthesizing EHR documents include difficulty searching for patient data, poor readability, redundancy, and unfamiliar specialized terms. Our study can inform recommendations for future designs of EHR clinical document user interfaces to aid clinicians in providing improved patient care.
Objectives:
To determine the prevalence and mechanism of copying among ICU physicians using an electronic medical record.
Design:
Retrospective cohort study.
Setting:
Medical ICU of an urban, academic medical center.
Patients:
Two thousand sixty-eight progress notes of 135 patients generated by 62 residents and 11 attending physicians between August 1, 2009, and December 31, 2009.
Interventions:
None.
Measurements and main results:
EIghty-two percent of all residents and 74% of all attending notes contained greater than or equal to 20% copied information (p = 0.001). Although residents authored more copied notes than attendings, residents copied less information between notes than attendings (55% vs. 61%, p < 0.001). Following greater than or equal to 1 day off, residents copied less often from their own prior notes compared to attendings (66% vs. 94%, p < 0.001). Of the copied information following a day off, there was no difference in the amount of information copied into notes of residents (59%) or attendings (61%, p = 0.17). In a regression model of attending notes, no patient factors were associated with copying. However, the levels of copying among attendings varied from 41% to 82% (p < 0.001).
Conclusions:
Copying among attendings and residents was common in this ICU-based cohort, with residents copying more frequently and attendings copying more information per note. The only factor that was independently associated with attending copying was the attending. Further studies should focus on further elucidating the factors influencing copying in the ICU and the effects of copying on patient outcomes.
We examine recent published research on the extraction of information from textual documents in the Electronic Health Record (EHR).
Literature review of the research published after 1995, based on PubMed, conference proceedings, and the ACM Digital Library, as well as on relevant publications referenced in papers already included.
174 publications were selected and are discussed in this review in terms of methods used, pre-processing of textual documents, contextual features detection and analysis, extraction of information in general, extraction of codes and of information for decision-support and enrichment of the EHR, information extraction for surveillance, research, automated terminology management, and data mining, and de-identification of clinical text.
Performance of information extraction systems with clinical text has improved since the last systematic review in 1995, but they are still rarely applied outside of the laboratory they have been developed in. Competitive challenges for information extraction from clinical text, along with the availability of annotated clinical text corpora, and further improvements in system performance are important factors to stimulate advances in this field and to increase the acceptance and usage of these systems in concrete clinical and biomedical research contexts.
Reading and writing: qualitative analysis of pharmacists' use of the EHR when preparing for team rounds
- S D Nelson
- J Lafleur
- Rgd Fiol
- C R Weir
Nelson SD, LaFleur J, Fiol RGD, Weir CR. Reading and writing:
qualitative analysis of pharmacists' use of the EHR when preparing for team rounds. In: AMIA Annual Symposium Proceedings. Vol. 2015. American Medical Informatics Association 2015:
943