Judy A Shea’s research while affiliated with Philadelphia University and other places

Background Institutions rely on student evaluations of teaching (SET) to ascertain teaching quality. Manual review of narrative comments can identify faculty with teaching concerns but can be resource and time-intensive. Aim To determine if natural language processing (NLP) of SET comments completed by learners on clinical rotations can identify teaching quality concerns. Setting and Participants Single institution retrospective cohort analysis of SET ( n = 11,850) from clinical rotations between July 1, 2017, and June 30, 2018. Program Description The performance of three NLP dictionaries created by the research team was compared to an off-the-shelf Sentiment Dictionary. Program Evaluation The Expert Dictionary had an accuracy of 0.90, a precision of 0.62, and a recall of 0.50. The Qualifier Dictionary had lower accuracy (0.65) and precision (0.16) but similar recall (0.67). The Text Mining Dictionary had an accuracy of 0.78 and a recall of 0.24. The Sentiment plus Qualifier Dictionary had good accuracy (0.86) and recall (0.77) with a precision of 0.37. Discussion NLP methods can identify teaching quality concerns with good accuracy and reasonable recall, but relatively low precision. An existing, free, NLP sentiment analysis dictionary can perform nearly as well as dictionaries requiring expert coding or manual creation.

Importance Adverse patient events are inevitable in surgical practice. Objectives To characterize the impact of adverse patient events on surgeons and trainees, identify coping mechanisms, and assess whether current forms of support are sufficient. Design, Setting, and Participants In this mixed-methods study, a validated survey instrument was adapted and distributed to surgical trainees from 7 programs, and qualitative interviews were conducted with faculty from 4 surgical departments in an urban academic health system. Main Outcomes and Measures The personal impact of adverse patient events, current coping mechanisms, and desired forms of support. Results Of 216 invited trainees, 93 (43.1%) completed the survey (49 [52.7%] male; 60 [64.5%] in third postgraduate year or higher; 23 [24.7%] Asian or Pacific Islander, 6 [6.5%] Black, 51 [54.8%] White, and 8 [8.6%] other race; 13 [14.0%] Hispanic or Latinx ethnicity). Twenty-three of 29 (79.3%) invited faculty completed interviews (13 [56.5%] male; median [IQR] years in practice, 11.0 [7.5-20.0]). Of the trainees, 77 (82.8%) endorsed involvement in at least 1 recent adverse event. Most reported embarrassment (67 of 79 trainees [84.8%]), rumination (64 of 78 trainees [82.1%]), and fear of attempting future procedures (51 of 78 trainees [65.4%]); 28 of 78 trainees (35.9%) had considered quitting. Female trainees and trainees who identified as having a race and/or ethnicity other than non-Hispanic White consistently reported more negative consequences compared with male and White trainees. The most desired form of support was the opportunity to discuss the incident with an attending physician (76 of 78 respondents [97.4%]). Similarly, faculty described feelings of guilt and shame, loss of confidence, and distraction after adverse events. Most described the utility of confiding in peers and senior colleagues, although some expressed unwillingness to reach out. Several suggested designating a departmental point person for event debriefing. Conclusions and Relevance In this mixed-methods study of the personal impact of adverse events on surgeons and trainees, these events were nearly universally experienced and caused significant distress. Providing formal support mechanisms for both surgical trainees and faculty may decrease stigma and restore confidence, particularly for underrepresented groups.

Importance The health care workforce continues to experience high rates of depression and anxiety. Finding ways to effectively support the mental health and well-being of health care workers is challenging. Objective To test the effectiveness of remote, pushed digital assessments and engagement to improve depression and anxiety among health care workers compared with usual care. Design, Setting, and Participants This was a 9-month randomized clinical trial with a 6-month intervention period. Participants were health care workers with self-reported daily access to a smartphone and at least 4 clinical hours per week. Participants were randomized to usual care or the intervention between January 2022 and March 2023. Data analyses were conducted between May and July 2023. Interventions All participants completed baseline, 6-month, and 9-month mental health, well-being, and burnout assessments. The control group had open access to a web-based mental health platform. Participants in the intervention group received monthly text messaging about mental health, mental health assessments, and linkages to care. Main Outcomes and Measures The primary outcomes were mean change in depression and anxiety scores at 6 months from baseline. Secondary outcomes include mean change in well-being, burnout, and self-reported workplace productivity. Results In this study, 1275 participants were randomized (642 [50.4%] to the intervention group and 633 [49.6%] to control group). Participants had a mean (SD) age of 38.6 (10.9) years, 1063 participants (83.4%) were female, 320 (25.1%) self-identified as Black, and 793 (62.2%) self-identified as White. Across the groups, the mean difference in depression score was significantly different at 6 months (−0.96 [95% CI, −1.52 to −0.40]) and at 9 months (−1.14 [95% CI, −1.69 to −0.58]). The mean difference in anxiety score from baseline to 6 months was statistically significantly larger for those in the intervention group vs usual care (−0.71 [95% CI, −1.25 to −0.17]) and held true at 9 months (−1.06 [95% CI, −1.59 to −0.52]). Conclusions and Relevance In a trial of health care workers, a proactive digital engagement strategy, including pushed text messaging, mobile mental health assessments, and connection to care, improved depression and anxiety over a 6-month period compared with simply making the same resources available for individuals to find and use. Trial Registration ClinicalTrials.gov Identifier: NCT05028075

OBJECTIVE Parents of pediatric patients are key stakeholders in the design and implementation of health-related social needs (HRSN) screening programs. Yet, there is little research exploring their perspectives on the documentation and sharing of HRSN data. We aimed to examine parents’ preferences regarding how HRSN data are documented and shared. METHODS We conducted semi-structured interviews with parents of hospitalized children participating in an HRSN screening program at a quaternary care children’s hospital. Interviews were coded using an inductive and deductive approach to identify emergent themes. RESULTS The 20 interviewed parents were uniformly female with 55% identifying as Black or African American and 20% identifying as Hispanic or Latino. Parents expressed comfort with electronic health record documentation of HRSN data and the use of International Classification of Diseases, 10th Revision Z codes as long as this information was used to provide families with meaningful support. Most parents viewed social workers and medical teams as the most appropriate recipients of HRSN data. Few parents felt comfortable with HRSN data being shared with payors. Parents desired transparency around HRSN data sharing. Many expressed concerns that documentation and sharing of HRSN data could lead to unwanted or unsafe disclosures or result in child welfare referrals. CONCLUSIONS Parents expressed comfort with HRSN documentation and sharing with health care providers, but requested that providers be transparent and respect parental preferences regarding data sharing to mitigate potential harms. When implementing HRSN support programs, health systems and payors should prioritize transparency around documentation and data sharing with families.

Objective The aim of this study was to develop and validate an instrument to measure Belonging in Surgery among surgical residents. Summary Background Data Belonging is the essential human need to maintain meaningful relationships and connections to one’s community. Increased belongingness is associated with better well-being, job performance and motivation to learn. However, no tools exist to measure belonging among surgical trainees. Methods A panel of experts adapted a belonging instrument for use among United States surgery residents. After administration of the 28-item instrument to residents at a single institution, a Cronbach’s alpha was calculated to measure internal consistency, and exploratory principal component analyses (PCA) were performed. Multiple iterations of analyses with successively smaller item samples suggested the instrument could be shortened. The expert panel was reconvened to shorten the instrument. Descriptive statistics measured demographic factors associated with Belonging in Surgery. Results The overall response rate was 52% (114 responses). The Cronbach’s alpha among the 28 items was 0.94 (95% CI: 0.93−0.96). The exploratory PCA and subsequent Promax rotation yielded one dominant component with an eigenvalue of 12.84 (70% of the variance). The expert panel narrowed the final instrument to 11 items with an overall Cronbach’s alpha of 0.90 (95% CI: 0.86, 0.92). Belonging in Surgery was significantly associated with race (Black and Asian residents scoring lower than White residents), graduating with one’s original intern cohort (residents who graduated with their original class scoring higher than those that did not), and inversely correlated with resident stress level. Conclusions An instrument to measure Belonging in Surgery was validated among surgical residents. With this instrument, Belonging in Surgery becomes a construct that may be used to investigate surgeon performance and well-being.

Background Long-acting reversible contraception (LARC) has long been regarded as highly effective and safe. However, access is limited and lengthy when specialty referrals are required. Objectives To integrate LARC services into an urban internal medicine primary care practice to decrease wait time for LARC procedures. Design/methods This pre–post with control group study took place at two large urban academic primary care practices (Practices A and B) and included patients ages 18 to 45 years assigned female sex at birth. Pre-implementation baseline data were collected retrospectively from 2019 to 2020 by identifying subjects who requested LARC insertion or removal via their primary care practice and were referred to Obstetrics and Gynecology (Ob/Gyn) for the procedure. Wait time was noted from time of initial request in the medical record to time of procedure. Practice A developed an integrated primary care LARC program in which one of their LARC-trained providers began offering these procedures within their own practice. All other providers within the practice were educated on how to counsel patients about the devices and procedures. Practice B did not have an in-house LARC provider and continued referring patients to Ob/Gyn. Post-implementation data were collected prospectively 2021–2022. Results Ninety-one patients in Practice A experienced a significant decrease in wait time (87 vs 21 days, p < 0.001) over the observation period, with a majority undergoing procedures on their first visit with the in-house LARC provider. Wait time for the 54 patients in Practice B remained unchanged (57 vs 47 days, p = .59), often requiring multiple specialty visits. Conclusion Integrating LARC services into a primary care internal medicine practice can significantly reduce wait times for these procedures with the potential to contribute to increased reproductive and menstrual autonomy.

Background Limited health literacy is associated with increased hospitalizations, emergency visits, health care costs, and mortality. The health literacy levels of caregivers of critically ill children are unknown. This mixed-methods study aims to quantitatively assess the health literacy of caregivers of children admitted to the pediatric intensive care unit (PICU) and qualitatively describe facilitators and barriers to implementing health literacy screening from the provider perspective. Methods Caregivers of patients admitted to our large, academic PICU (between August 12, 2022 and March 31, 2023) were approached to complete a survey with the Newest Vital Sign (NVS), which is a validated health literacy screener offered in English and Spanish. We additionally conducted focus groups of interdisciplinary PICU providers to identify factors which may influence implementation of health literacy screening using the Consolidated Framework for Implementation Research (CFIR) framework. Results Among 48 surveyed caregivers, 79% demonstrated adequate health literacy using the Newest Vital Sign screener. The majority of caregivers spoke English (96%), were mothers (85%), and identified as White (75%). 83% of caregivers were able to attend rounds at least once and 98% believed attending rounds was helpful. Within the PICU provider focus groups, there were 11 participants (3 attendings, 3 fellows, 2 nurse practitioners, 1 hospitalist, 2 research assistants). Focus group participants described facilitators and barriers to implementation, which were mapped to CFIR domains. Timing of screening and person administering screening were identified as modifiable factors to improve future implementation. Conclusion We found the health literacy levels of PICU caregivers in our setting is similar to prior assessments of parental health literacy. Participation in morning rounds was helpful for developing understanding of their child's illness, regardless of health literacy status. Qualitative feedback from providers identified barriers across all CFIR domains, with timing of screening and person administering screening as modifiable factors to improve future implementation.

Background Medical school acceptance rates in the United States (US) have been lower for applicants who identify as Underrepresented-in-Medicine (UiM) compared to non-UiM applicants. The gap between UiM and no-UiM groups is narrowing in recent years. Less well-studied are associations of acceptance decisions with family income and parental education. This study’s purpose is to evaluate the relationships between medical school acceptance and family income, parental education status, racial/ethnic background, Grade Point Average (GPA), Medical College Admission Test (MCAT) score, and participation in extracurricular activities. Methods This is a cross-sectional study of first-time US medical school applicants between 2017 and 2020. Acceptance rates for first-time applicants were calculated for first-generation (FG), low-income (LI), and UiM applicants. Associations of these attributes with MCAT scores, science GPAs, and seven categories of extracurricular activities were evaluated. Regression analyses estimated associations between acceptance to medical school with all variables with and without interaction terms (FG*URM, LI*URM, FG*LI). Results The overall acceptance rate for first-time applicants from 2017–2020 was 45.3%. The acceptance rates among FG, LI and UiM applicants were 37.9%, 39.6% and 44.2%, respectively. In univariable logistic regression analyses, acceptance was negatively associated with being FG (OR: 0.68, CI: 0.67–0.70), LI (OR: 0.70, CI: 0.69–0.72), and UiM (OR: 0.95, CI: 0.93–0.97). In multivariable regression, acceptance was most strongly associated with science GPA (OR: 7.15, CI: 6.78–7.54 for the highest quintile) and UiM (OR: 5.56, CI: 5.48–5.93) status and MCAT score (OR: 1.19, CI: 1.18–1.19), FG (OR: 1.14, CI: 1.10–1.18), and most extracurricular activities. Including interaction terms revealed a negative association between acceptance and LI (OR:0.90, CI: 0.87–0.94) and FG was no longer significant (OR:1.10, CI:0.96–1.08). Conclusions Collectively these results suggest medical school admissions committees may be relying on holistic admission practices. While MCAT and GPA scores continue to predict acceptance, individuals from racially and ethnically UiM backgrounds have favorable odds of acceptance when controlling for MCAT and GPA. However, these positive associations were not seen for low-income and first-generation applicants. Additional preparation for college and the MCAT for these latter groups may help further diversify the medical profession.