Meera Viswanathan’s research while affiliated with University of North Carolina at Chapel Hill and other places

Background. Data extraction is a critical but error-prone and labor-intensive task in evidence synthesis. Unlike other artificial intelligence (AI) technologies, large language models (LLMs) do not require labeled training data for data extraction. Objective. To compare an AI-assisted to a traditional y data extraction process. Design. Study within reviews (SWAR) utilizing a prospective, parallel group comparison with blinded data adjudicators. Setting. Workflow validation within six ongoing systematic reviews of interventions under real-world conditions. Intervention. Initial data extraction using an LLM (Claude versions 2.1, 3.0 Opus, and 3.5 Sonnet) verified by a human reviewer. Measurements: Concordance, time on task, accuracy, recall, precision, and error analysis. Results. The six systematic reviews of the SWAR contributed 9,341 data elements, extracted from 63 studies. Concordance between the two methods was 77.2%. The accuracy of the AI-assisted approach compared with enhanced human data extraction was 91.0%, with a recall of 89.4% and a precision of 98.9%. The AI-assisted approach had fewer incorrect extractions (9.0% vs. 11.0%) and similar risks of major errors (2.5% vs. 2.7%) compared to the traditional human-only method, with a median time saving of 41 minutes per study. Missed data items were the most frequent errors in both approaches. Limitations. Assessing the concordance of data extractions and classifying errors required subjective judgment. Tracking time on task consistently was challenging. Conclusion. The use of an LLM can improve accuracy of data extraction and save time in evidence synthesis. Results reinforce previous findings that human-only data extraction is prone to errors. Primary Funding Source: US Agency for Healthcare Research and Quality, RTI International Registration: SWAR28 Gerald Gartlehner (2023 FEB 11 2102).pdf

Importance Fragility fractures result in significant morbidity. Objective To review evidence on osteoporosis screening to inform the US Preventive Services Task Force. Data Sources PubMed, Embase, Cochrane Library, and trial registries through January 9, 2024; references, experts, and literature surveillance through July 31, 2024. Study Selection Randomized clinical trials (RCTs) and systematic reviews of screening; pharmacotherapy studies for primary osteoporosis; predictive and diagnostic accuracy studies. Data Extraction and Synthesis Two reviewers assessed titles/abstracts, full-text articles, study quality, and extracted data; when at least 2 similar studies were available, meta-analyses were conducted. Main Outcomes and Measures Hip, clinical vertebral, major osteoporotic, and total fractures; mortality; harms; accuracy. Results Three RCTs and 3 systematic reviews reported benefits of screening in older, higher-risk women. Two RCTs used 2-stage screening: Fracture Risk Assessment Tool estimate with bone mineral density (BMD) testing if risk threshold exceeded. One RCT used BMD plus additional tests. Screening was associated with reduced hip (pooled relative risk [RR], 0.83 [95% CI, 0.73-0.93]; 3 RCTs; 42 009 participants) and major osteoporotic fracture (pooled RR, 0.94 [95% CI, 0.88-0.99]; 3 RCTs; 42 009 participants) compared with usual care. Corresponding absolute risk differences were 5 to 6 fewer fractures per 1000 participants screened. The discriminative accuracy of risk assessment instruments to predict fracture or identify osteoporosis varied by instrument and fracture type; most had an area under the curve between 0.60 and 0.80 to predict major osteoporotic fracture, hip fracture, or both. Calibration outcomes were limited. Compared with placebo, bisphosphonates (pooled RR, 0.67 [95% CI, 0.45-1.00]; 6 RCTs; 12 055 participants) and denosumab (RR, 0.60 [95% CI, 0.37-0.97] from the largest RCT [7808 participants]) were associated with reduced hip fractures. Compared with placebo, no statistically significant associations were observed for adverse events. Conclusions and Relevance Screening in higher-risk women 65 years or older was associated with a small absolute risk reduction in hip and major fractures compared with usual care. No evidence evaluated screening with BMD alone or screening in men or younger women. Risk assessment instruments, BMD alone, or both have poor to modest discrimination for predicting fracture. Osteoporosis treatment with bisphosphonates or denosumab over several years was associated with fracture reductions and no meaningful increase in adverse events.

Background Telehealth services can increase access to care by reducing barriers. Telephone-administered care, in particular, requires few resources and may be preferred by communities in areas that are systemically underserved. Understanding the effectiveness of audio-based care is important to combat the current mental health crisis and inform discussions related to reimbursement privileges. Objectives We compared the effectiveness of audio-based care to usual care for managing mental health and substance use disorders (MHSUD). Design We used systematic review methods to synthesize available evidence. Studies We searched for English-language articles reporting randomized controlled trials (RCTs) of adults diagnosed with MHSUD published since 2012. Outcomes We abstracted data on clinical outcomes, patient-reported health and quality of life, health care access and utilization, care quality and experience, and patient safety. Results We included 31 RCTs of participants diagnosed with depression, post-traumatic stress disorder (PTSD), other serious mental illness (SMI), anxiety, insomnia, or substance use disorder (SUD). Most of the evidence was for interventions targeting depression, PTSD, and SUD. The evidence demonstrates promise for: (1) replacing in-person care with audio care for depression, other SMI, and SUD (very low to moderate certainty of comparable effectiveness); and (2) adding audio care to monitor or treat depression, PTSD, anxiety, insomnia, and SUD (low to moderate certainty of evidence favoring audio care for clinical outcomes). Conclusions MHSUD can be managed with audio care in certain situations. However, more evidence is needed across conditions, and specifically for anxiety and other conditions for which no research was identified.

Replies to comments made by Mattos et al. (see record 2025-49982-003) on the original article (see record 2024-19816-001). Mattos et al. critiqued our assessments of the certainty of evidence as being overly permissive and not adhering to the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) Working Group's guidelines. GRADE has become an international standard to describe the level of confidence that investigators have in estimates of effects. Like the risk of bias evaluations, determining the certainty of evidence involves subjective judgment. The true value of GRADE is not in yielding a definitive evidence certainty rating but in its emphasis on transparency. While we acknowledge and respect the differing viewpoints of Mattos et al. regarding our ratings, we caution against the rigid and formulaic use of the GRADE methodology. (PsycInfo Database Record (c) 2024 APA, all rights reserved).