ArticlePDF Available

Validating free-text order entry for a note-centric EHR

Authors:

Abstract

Electronic Health Records (EHRs) have increased the utility and portability of health information by storing it in structured formats. However, EHRs separate this structured data from the rich, free-text descriptions of clinical notes. The ultimate objective of our research is to develop an interactive progress note that unifies entry, access, and retrieval of structured and unstructured health information. In this study we present the design and subsequent testing with eight clinicians of a core element of this envisioned note: free-text order entry. Clinicians saw this new order-entry paradigm as a way to save time and preserve data quality by reducing double-documentation. However, they wanted the prototype to recognize more diverse types of shorthand and apply default values to fields that remain fairly constant across orders, such as number of refills and pickup location. Future work will test more complex orders, such as cascading orders, with a broader range of clinicians.
A preview of the PDF is not available
... 51 Inefficient navigation in EHRs has been shown to increase users' cognitive load, which may increase the likelihood of errors, reduce efficiency and increase fatigue. [52][53][54] Poor user experience, captured in the staff surveys and use-cases, was consistent with the peer-reviewed literature and the barriers to using EHRs. Better screen designs improve the user experience of an EHR. ...
... Roman et al. 49 and Rule et al. 52 highlighted the importance of site navigation in the design of electronic healthcare systems and described the difficulties that clinicians face by having to navigate through multiple screens and sections. By having to access the EMR, a state-or territorybased portal (if available) and the My Health Record system, ED clinicians may experience the phenomenon of 'display fragmentation', as termed by the Institute of Medicine. ...
... Better navigation can help address 'display fragmentation', to release the clinicians' cognitive resources for complex clinical reasoning and patient safety, and to increase efficiency. 52 It is well documented that poor usability design can significantly affect clinicians' satisfaction with a (digital health) system, and impede their ability to deliver safe and high-quality care. It is also a major contributor to clinician burnout. ...
... For instance, Gloyd et al [87] argued that incomplete, missing, and implausible data "was by far the most common challenge encountered." Scholars regularly identified that data fragmentation contributed to incompleteness, with a patient's medical record deemed incomplete owing to data being required from multiple systems and EHRs [18,37,[88][89][90][91][92][93]. "Data were also considered hidden within portals, outside systems, or multiple EHRs, frustrating efforts to assemble a complete clinical picture of the patient" [89]. ...
Article
Full-text available
Background: The promise of digital health is principally dependent on the ability to electronically capture data that can be analyzed to improve decision-making. However, the ability to effectively harness data has proven elusive, largely because of the quality of the data captured. Despite the importance of data quality (DQ), an agreed-upon DQ taxonomy evades literature. When consolidated frameworks are developed, the dimensions are often fragmented, without consideration of the interrelationships among the dimensions or their resultant impact. Objective: The aim of this study was to develop a consolidated digital health DQ dimension and outcome (DQ-DO) framework to provide insights into 3 research questions: What are the dimensions of digital health DQ? How are the dimensions of digital health DQ related? and What are the impacts of digital health DQ? Methods: Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, a developmental systematic literature review was conducted of peer-reviewed literature focusing on digital health DQ in predominately hospital settings. A total of 227 relevant articles were retrieved and inductively analyzed to identify digital health DQ dimensions and outcomes. The inductive analysis was performed through open coding, constant comparison, and card sorting with subject matter experts to identify digital health DQ dimensions and digital health DQ outcomes. Subsequently, a computer-assisted analysis was performed and verified by DQ experts to identify the interrelationships among the DQ dimensions and relationships between DQ dimensions and outcomes. The analysis resulted in the development of the DQ-DO framework. Results: The digital health DQ-DO framework consists of 6 dimensions of DQ, namely accessibility, accuracy, completeness, consistency, contextual validity, and currency; interrelationships among the dimensions of digital health DQ, with consistency being the most influential dimension impacting all other digital health DQ dimensions; 5 digital health DQ outcomes, namely clinical, clinician, research-related, business process, and organizational outcomes; and relationships between the digital health DQ dimensions and DQ outcomes, with the consistency and accessibility dimensions impacting all DQ outcomes. Conclusions: The DQ-DO framework developed in this study demonstrates the complexity of digital health DQ and the necessity for reducing digital health DQ issues. The framework further provides health care executives with holistic insights into DQ issues and resultant outcomes, which can help them prioritize which DQ-related problems to tackle first.
... Vendors of electronic health record systems have advocated the use of automation to facilitate record-keeping by use of structured data formats. [36][37][38] Natural language processing can be used to dissect patient-doctor conversations and create notes. 39 How to ensure the validity and reliability of such systems has been the subject of much research and remains a controversial topic. ...
Article
Full-text available
Objective To identify the extent to which administrative tasks carried out by primary care staff in general practice could be automated. Design A mixed-method design including ethnographic case studies, focus groups, interviews and an online survey of automation experts. Setting Three urban and three rural general practice health centres in England selected for differences in list size and organisational characteristics. Participants Observation and interviews with 65 primary care staff in the following job roles: administrator, manager, general practitioner, healthcare assistant, nurse practitioner, pharmacy technician, phlebotomist, practice nurse, pharmacist, prescription clerk, receptionist, scanning clerk, secretary and medical summariser; together with a survey of 156 experts in automation technologies. Methods 330 hours of ethnographic observation and documentation of administrative tasks carried out by staff in each of the above job roles, followed by coding and classification; semistructured interviews with 10 general practitioners and 6 staff focus groups. The online survey of machine learning, artificial intelligence and robotics experts was analysed using an ordinal Gaussian process prediction model to estimate the automatability of the observed tasks. Results The model predicted that roughly 44% of administrative tasks carried out by staff in general practice are ‘mostly’ or ‘completely’ automatable using currently available technology. Discussions with practice staff underlined the need for a cautious approach to implementation. Conclusions There is considerable potential to extend the use of automation in primary care, but this will require careful implementation and ongoing evaluation.
... Studies suggest workload caused by clinical documentation is an important barrier to effective patient care [12]. Using existing technologies, vendors of electronic health record systems and researchers have long looked for ways to facilitate the process by pulling in structured data/format into doctor's note to reduce duplicate work [13]. Recently some researchers have advocated for the application of AI systems to streamline and automate documentation tasks in healthcare [3,4], for example, by using natural language processing to dissect patient-doctor conversation and create notes [14]. ...
Conference Paper
Full-text available
The current conversation around automation and artificial intelligence technologies creates a future vision where humans may not possibly compete against intelligent machines, and that everything that can be automated through deep learning, machine learning, and other AI technologies will be automated. In this article, we focus on general practitioner documentation of the patients' clinical encounter, and explore how these work practices lend themselves to automation by AI. While these work practices may appear perfect to automate, we reveal potential negative consequences to automating these tasks, and illustrate how AI may render important aspect of this work invisible and remove critical thinking. We conclude by highlighting the specific features of clinical documentation work that could leverage the benefits of human-AI symbiosis.
Conference Paper
As healthcare providers have transitioned from paper to electronic health records they have gained access to increasingly sophisticated documentation aids such as custom note templates. However, little is known about how providers use these aids. To address this gap, we examine how 48 ophthalmologists and their staff create and use content-importing phrases - a customizable and composable form of note template - to document office visits across two years. In this case study, we find 1) content-importing phrases were used to document the vast majority of visits (95%), 2) most content imported by these phrases was structured data imported by data-links rather than boilerplate text, and 3) providers primarily used phrases they had created while staff largely used phrases created by other people. We conclude by discussing how framing clinical documentation as end-user programming can inform the design of electronic health records and other documentation systems mixing data and narrative text.
Article
Full-text available
The process of documentation is one of the most important parts of electronic health records (EHR). It is time-consuming, and up until now, available documentation procedures have not been able to overcome this type of EHR limitations. Thus, entering information into EHR still has remained a challenge. In this study, by applying the trigram language model, we presented a method to predict the next words while typing free texts. It is hypothesized that using this system may save typing time of free text. The words prediction model introduced in this research was trained and tested on the free texts regarding to colonoscopy, transesophageal echocardiogram, and anterior-cervical-decompression. Required time of typing for each of the above-mentioned reports calculated and compared with manual typing of the same words. It is revealed that 33.36% reduction in typing time and 73.53% reduction in keystroke. The designed system reduced the time of typing free text which might be an approach for EHRs improvement in terms of documentation.
Article
Background: Aspirin impacts risk for important outcomes such as cancer, cardiovascular disease, and gastrointestinal bleeding. However, ascertaining exposure to medications available both by prescription and over-the-counter such as aspirin for research and quality improvement purposes is a challenge. Objectives: Develop and validate a strategy for ascertaining aspirin exposure, utilizing a combination of structured and unstructured data. Research design: This is a retrospective cohort study. Subjects: In total, 1,869,439 Veterans who underwent usual care colonoscopy 1999-2014 within the Department of Veterans Affairs. Measures: Aspirin exposure and dose were obtained from an ascertainment strategy combining query of structured medication records available in electronic health record databases and unstructured data extracted from free-text progress notes. Prevalence of any aspirin exposure and dose-specific exposure were estimated. Positive predictive value and negative predictive value were used to assess strategy performance, using manual chart review as the reference standard. Results: Our combined strategy for ascertaining aspirin exposure using structured and unstructured data reached a positive predictive value and negative predictive value of 99.2% and 97.5% for any exposure, and 92.6% and 98.3% for dose-specific exposure. Estimated prevalence of any aspirin exposure was 36.3% (95% confidence interval: 36.2%-36.4%) and dose-specific exposure was 35.4% (95% confidence interval: 35.3%-35.5%). Conclusions: A readily accessible approach utilizing a combination of structured medication records and query of unstructured data can be used to ascertain aspirin exposure when manual chart review is impractical.
Article
Communication breakdowns in the referral process negatively impact clinical workflow and patient safety. There is a lack of evidence demonstrating the impact of published design recommendations addressing contributing issues with consultation order templates. This study translated the recommendations into a computer-based prototype and conducted a comparative usability evaluation. With a scenario-based simulation, 30 clinicians (referrers) participated in a within-group, counterbalanced experiment comparing the prototype with their present electronic order entry system. The prototype significantly increased satisfaction (Cohen's d = 1.80, 95% CI [1.19, 2.41], p <.001), and required significantly less mental effort (d = 0.67 [0.14, 1.20], p <.001). Regarding efficiency, the prototype required significantly fewer mouse clicks (mean difference = 29 clicks, p <.001). Although overall task time did not differ significantly (d = −0.05 [−0.56, 0.47]), the prototype significantly quickened identification of the appropriate specialty clinic (mean difference = 12 s, d = 0.98 [0.43, 1.52], p <.001). The experimental evidence demonstrated that clinician-centered interfaces significantly improved system usability during ordering of consultations.
Article
Full-text available
Clinical documentation was developed to track a patient's condition and communicate the author's actions and thoughts to other members of the care team. Over time, other stakeholders have placed additional requirements on the clinical documentation process for purposes other than direct care of the patient. More recently, new information technologies, such as electronic health record (EHR) systems, have led to further changes in the clinical documentation process. Although computers and EHRs can facilitate and even improve clinical documentation, their use can also add complexities; new challenges; and, in the eyes of some, an increase in inappropriate or even fraudulent documentation. At the same time, many physicians and other health care professionals have argued that the quality of the systems being used for clinical documentation is inadequate. The Medical Informatics Committee of the American College of Physicians has undertaken this review of clinical documentation in an effort to clarify the broad range of complex and interrelated issues surrounding clinical documentation and to suggest a path forward such that care and clinical documentation in the 21st century best serve the needs of patients and families.
Conference Paper
Full-text available
Despite the widespread introduction of information technology into primary health care within the United Kingdom, medical practitioners continue to use the more traditional paper medical record often alongside the computerized system. The resilience of the paper document is not simply a consequence of an impoverished design, but rather a product of the socially organized practices and reasoning which surround the use of the record within day to day consultative work. The practices that underpin the use of the medical records may have a range of important implications, not only for the general design of systems to support collaborative work, but also for our conceptions of `writers', `readers', `objects' and `records' utilized in those designs.
Conference Paper
Full-text available
We describe fieldwork in which we studied hospital ICU physicians and their strategies and documentation aids for composing patient progress notes. We then present a clinical documentation prototype, activeNotes, that supports the creation of these notes, using techniques designed based on our fieldwork. ActiveNotes integrates automated, context-sensitive patient data retrieval, and user control of automated data updates and alerts via tagging, into the documentation process. We performed a qualitative study of activeNotes with 15 physicians at the hospital to explore the utility of our information retrieval and tagging techniques. The physicians indicated their desire to use tags for a number of purposes, some of them extensions to what we intended, and others new to us and unexplored in other systems of which we are aware. We discuss the physicians' responses to our prototype and distill several of their proposed uses of tags: to assist in note content management, communication with other clinicians, and care delivery.
Article
Full-text available
Hospital computerized physician order entry (CPOE) systems are widely regarded as the technical solution to medication ordering errors, the largest identified source of preventable hospital medical error. Published studies report that CPOE reduces medication errors up to 81%. Few researchers, however, have focused on the existence or types of medication errors facilitated by CPOE. To identify and quantify the role of CPOE in facilitating prescription error risks. We performed a qualitative and quantitative study of house staff interaction with a CPOE system at a tertiary-care teaching hospital (2002-2004). We surveyed house staff (N = 261; 88% of CPOE users); conducted 5 focus groups and 32 intensive one-on-one interviews with house staff, information technology leaders, pharmacy leaders, attending physicians, and nurses; shadowed house staff and nurses; and observed them using CPOE. Participants included house staff, nurses, and hospital leaders. Examples of medication errors caused or exacerbated by the CPOE system. We found that a widely used CPOE system facilitated 22 types of medication error risks. Examples include fragmented CPOE displays that prevent a coherent view of patients' medications, pharmacy inventory displays mistaken for dosage guidelines, ignored antibiotic renewal notices placed on paper charts rather than in the CPOE system, separation of functions that facilitate double dosing and incompatible orders, and inflexible ordering formats generating wrong orders. Three quarters of the house staff reported observing each of these error risks, indicating that they occur weekly or more often. Use of multiple qualitative and survey methods identified and quantified error risks not previously considered, offering many opportunities for error reduction. In this study, we found that a leading CPOE system often facilitated medication error risks, with many reported to occur frequently. As CPOE systems are implemented, clinicians and hospitals must attend to errors that these systems cause in addition to errors that they prevent.
Article
Full-text available
The purpose of this evaluation was to assess perceptions of usability of a new semi-structured electronic clinical note. Two focus groups were held, one with attending physicians and one with residents. Physicians described their experiences with eNote and their perceptions about the system. Transcripts of the focus groups underwent content analysis, and four major themes emerged. These were "time", "hardware-system issues", "eNote application issues", and "patients' perceptions."
Conference Paper
Alan Calvitti Neal Farber Yunan Chen Danielle Zuest Lin Liu Kristin Bell Barbara Gray Zia Agha We develop temporal data mining and visualization methods to quantitatively profile physician Electronic Health Records (EHR) workflow and compare time-at-task versus click count distributions for top-level EHR functionality. The temporal data is based on time-resolved activity during outpatient visits, captured by usability software and audio-video recording and manual coding to physicians' activities.
Article
The US Federal Government is investing up to $29 billion in incentives for meaningful use of electronic health records (EHRs). However, the effect of EHRs on ambulatory quality is unclear, with several large studies finding no effect. To determine the effect of EHRs on ambulatory quality in a community-based setting. Cross-sectional study, using data from 2008. Ambulatory practices in the Hudson Valley of New York, with a median practice size of four physicians. We included all general internists, pediatricians and family medicine physicians who: were members of the Taconic Independent Practice Association, had patients in a data set of claims aggregated across five health plans, and had at least 30 patients per measure for at least one of nine quality measures selected by the health plans. Adoption of an EHR. We compared physicians using EHRs to physicians using paper on performance for each of the nine quality measures, using t-tests. We also created a composite quality score by standardizing performance against a national benchmark and averaging standardized performance across measures. We used generalized estimation equations, adjusting for nine physician characteristics. We included 466 physicians and 74,618 unique patients. Of the physicians, 204 (44 %) had adopted EHRs and 262 (56 %) were using paper. Electronic health record use was associated with significantly higher quality of care for four of the measures: hemoglobin A1c testing in diabetes, breast cancer screening, chlamydia screening, and colorectal cancer screening. Effect sizes ranged from 3 to 13 percentage points per measure. When all nine measures were combined into a composite, EHR use was associated with higher quality of care (sd 0.4, p = 0.008). This is one of the first studies to find a positive association between EHRs and ambulatory quality in a community-based setting.
Clinicians' perceptions of usability of eNote
  • J P Haas
  • S Bakken
  • T J Bright
  • G B Melton
  • P Stetson
  • S B Johnson
Haas JP, Bakken S, Bright TJ, Melton GB, Stetson P, Johnson SB. Clinicians' perceptions of usability of eNote. AMIA Annual Symposium Proceedings. American Medical Informatics Association; 2005. p. 973.