Natural language processing in the electronic medical record: assessing clinician adherence to tobacco treatment guidelines.
ABSTRACT Comprehensively assessing care quality with electronic medical records (EMRs) is not currently possible because much data reside in clinicians' free-text notes.
We evaluated the accuracy of MediClass, an automated, rule-based classifier of the EMR that incorporates natural language processing, in assessing whether clinicians: (1) asked if the patient smoked; (2) advised them to stop; (3) assessed their readiness to quit; (4) assisted them in quitting by providing information or medications; and (5) arranged for appropriate follow-up care (i.e., the 5A's of smoking-cessation care).
We analyzed 125 medical records of known smokers at each of four HMOs in 2003 and 2004. One trained abstractor at each HMO manually coded all 500 records according to whether or not each of the 5A's of smoking cessation care was addressed during routine outpatient visits.
For each patient's record, we compared the presence or absence of each of the 5A's as assessed by each human coder and by MediClass. We measured the chance-corrected agreement between the human raters and MediClass using the kappa statistic.
For "ask" and "assist," agreement among human coders was indistinguishable from agreement between humans and MediClass (p>0.05). For "assess" and "advise," the human coders agreed more with each other than they did with MediClass (p<0.01); however, MediClass performance was sufficient to assess quality in these areas. The frequency of "arrange" was too low to be analyzed.
MediClass performance appears adequate to replace human coders of the 5A's of smoking-cessation care, allowing for automated assessment of clinician adherence to one of the most important, evidence-based guidelines in preventive health care.
- [Show abstract] [Hide abstract]
ABSTRACT: AimsTo assess the association between doctors’ smoking status and the use of the 5As of smoking cessation.MethodsA systematic search of 11 databases covering English and Spanish language publications since 1996 was undertaken. Studies were included if they reported doctors’ smoking status (current, former or never smoker) and rates of practicing any of the "5 As" of smoking cessation (Ask; Advise; Assess; Assist; and Arrange). Frequencies and proportions were extracted from individual papers and risk ratios (RR) were calculated. Random-effects meta-analysis model was used to assess the effect of the doctor's personal smoking history. Covariate effects were explored using meta-regression for three pre-specified study characteristics: doctors’ role, smoking prevalence of the sample and study quality.ResultsTwenty studies were included in this systematic review. The risk ratio of always asking patients about their smoking was not significantly associated with doctors’ smoking status (RR=0.98; 95%CI= 0.94-1.02; p= 0.378; I2=0.0%; 10 studies). Meta-analysis suggested that doctors who were current smokers had a 17% increased risk of not advising their patients to quit compared with never-smokers (RR= 0.83; 95%CI= 0.77-0.90; p<0.000; I2=82.1%; 14 studies). However, high levels of heterogeneity were found that were not explained by the meta-regression. Regarding assisting patients to quit, never smokers were more likely to counsel than current smokers (RR=0.92; 95%CI=0.85-0.99; p=0.036; I2=0.0%; 3 studies) but less likely to make a referral (RR=1.40; 95%CI=1.09-1.79; p=0.009; I2=0.0%; 4 studies). No statistically significant differences were found in arranging future contact by smoking status (RR=0.80; 95%CI=0.52-1.23; p=0.315; I2=47.0%; 4 studies).Conclusions Smoking status of doctors may affect their delivery of smoking cessation treatments to patients, with smokers being less likely than non-smokers or ex-smokers to advise and counsel their patients to quite but more likely to refer them to smoking cessation programmes.Addiction 07/2014; · 4.60 Impact Factor
- [Show abstract] [Hide abstract]
ABSTRACT: Electronic health records (EHRs) and social media have the potential to enrich public health surveillance of diabetes. Clinical and patient-facing data sources for diabetes surveillance are needed given its profound public health impact, opportunity for primary and secondary prevention, persistent disparities, and requirement for self-management. Initiatives to employ data from EHRs and social media for diabetes surveillance are in their infancy. With their transformative potential come practical limitations and ethical considerations. We explore applications of EHR and social media for diabetes surveillance, limitations to approaches, and steps for moving forward in this partnership between patients, health systems, and public health.Current Diabetes Reports 03/2014; 14(3):468. · 3.38 Impact Factor
- [Show abstract] [Hide abstract]
ABSTRACT: Numerous population-based surveys indicate that overweight and obese patients can benefit from lifestyle counseling during routine clinical care. To determine if natural language processing (NLP) could be applied to information in the electronic health record (EHR) to automatically assess delivery of weight management-related counseling in clinical healthcare encounters. The MediClass system with NLP capabilities was used to identify weight-management counseling in EHRs. Knowledge for the NLP application was derived from the 5As framework for behavior counseling: Ask (evaluate weight and related disease), Advise at-risk patients to lose weight, Assess patients' readiness to change behavior, Assist through discussion of weight-loss methods and programs, and Arrange follow-up efforts including referral. Using samples of EHR data between January 1, 2007, and March 31, 2011, from two health systems, the accuracy of the MediClass processor for identifying these counseling elements was evaluated in postpartum visits of 600 women with gestational diabetes mellitus (GDM) compared to manual chart review as the gold standard. Data were analyzed in 2013. Mean sensitivity and specificity for each of the 5As compared to the gold standard was at or above 85%, with the exception of sensitivity for Assist, which was 40% and 60% for each of the two health systems. The automated method identified many valid Assist cases not identified in the gold standard. The MediClass processor has performance capability sufficiently similar to human abstractors to permit automated assessment of counseling for weight loss in postpartum encounter records.American journal of preventive medicine 05/2014; 46(5):457-64. · 4.24 Impact Factor