Figure - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Source publication
Attrition, or dropout, is a problem faced by many online health interventions, potentially threatening the inferential value of online randomized controlled trials.
In the context of a randomized controlled trial of an online weight management intervention, where 85% of the baseline participants were lost to follow-up at the 12-month measurement, t...
Contexts in source publication
Context 1
... left us with a total of 3260 baseline participants-1681 treatment and 1579 control participants. The pattern of response to the three online follow-up surveys is shown in Table 1. As shown in Table 1, the majority of participants (64.4% overall) did not complete any of the follow-up surveys, while 499 (15.3%) completed the 12-month follow-up (whether or not they completed earlier follow-up surveys). ...Context 2
... pattern of response to the three online follow-up surveys is shown in Table 1. As shown in Table 1, the majority of participants (64.4% overall) did not complete any of the follow-up surveys, while 499 (15.3%) completed the 12-month follow-up (whether or not they completed earlier follow-up surveys). The pattern of response across the three waves of follow-up is quite similar between the treatment and control groups. ...Citations
... Health surveys are an essential component of longitudinal data collection, yet having participants return to complete follow-up surveys can be difficult [2]. There are multiple reasons why individuals don't fill out follow-up surveys including issues with accessing or submitting the survey, technical issues, not receiving messages, lack of interest, survey being boring or too long, or no time or bad timing [3]. Large cohort studies that used surveys for data collection have had varying response rates, with only a few having online follow-up surveys [4][5][6]. ...
... This extensive literature on survey methods shows that multimodal survey methods are more effective than single-mode methods. Survey response rates for electronic, Web-based surveys improve when followed up by letter mailings, phone calls, or both [3], illustrating the importance of multimodal approaches. ...
... Although the study's interventions helped increase follow-up survey completion rates, All of Us still has significant follow-up survey completion challenges. Our pilot interventions align with the literature on survey methods indicating that single-mode survey methods are far less effective than multimode methods [3,[7][8][9][10][11][12] and increased completion rates by up to 25%. In a systematic review by Booker et al. [22] multiple retention methods, including face-to-face interviews, increased the survey completion rate by more than 70%. ...
Objective
Retention to complete follow-up surveys in extensive longitudinal epidemiological cohort studies is vital yet challenging. All of Us developed pilot interventions to improve response rates for follow-up surveys.
Study design and setting
The pilot interventions occurred from April 27, 2020, to August 3, 2020. The three arms were: (1) telephone appointment [staff members calling participants offering appointments to complete surveys over phone] (2) postal [mail reminder to complete surveys through U.S. Postal Service], and (3) combination of telephone appointment and postal. Controls received digital-only reminders [program-level digital recontact via email or through the participant portal]. Study sites chose their study arm and participants were not randomized.
Results
A total of 50 sites piloted interventions with 17,593 participants, while 47,832 participants comprised controls during the same period. Of all participants, 6,828 (10.4%) completed any follow-up surveys (1448: telephone; 522: postal; 486: combination; 4372: controls). Follow-up survey completions were 24% higher in the telephone appointment arm than in controls in bivariate analyses. When controlling for confounders, telephone appointment and combination arms increased rates of completion similarly compared to controls, while the postal arm had no significant effect (odds ratio [95% Confidence Interval], telephone appointment:2.01[1.81–2.23]; combination:1.91[1.66–2.20]; postal:0.92[0.79–1.07]). Although the effects of the telephone appointment and combination arms were similar, differential effects were observed across sub-populations.
Conclusion
Telephone appointments appeared to be the most successful intervention in our study. Lessons learned about retention interventions, and improvement in follow-up survey completion rates provide generalizable knowledge for similar cohort studies and demonstrate the potential value of precision reminders and engagement with sub-populations of a cohort.
... The causes behind the falling response rates are unclear, although the sheer number of requests for participation and increase in surveyors from the commercial space has been proposed as a factor (Dillman, 2015). In the past, surveys following up with non-responders have suggested issues of salience/relevance, burden, and lack of interest (Tait et al., 1995;Couper et al., 2007;Singer & Couper, 2017). Unsurprisingly, these same aspects are also well-represented in surveys of what respondents find bothersome in surveys (Johnston, 2014;Husebø et al., 2018;Mayer, 2019). ...
Smart surveys combine traditional survey questions with sensor-based data from smartphones, wearables, and other devices. By leveraging modern technology, these surveys can improve data quality, reduce participant burden, and yield more timely and granular insights. This deliverable reports on the first stage of the Smart Survey Implementation (SSI) project, whose overarching aim is to develop an end-to-end research methodology for smart surveys. Focusing on European Time Use Surveys (TUS) and Household Budget Surveys (HBS), it identifies knowledge gaps in the existing literature and outlines how the project will address them in subsequent stages.
The deliverable centers on four key challenges that hinder large-scale adoption of smart surveys: (1) recruiting and retaining diverse participants (including difficult-to-reach groups), (2) using machine learning to enhance human-computer interaction, (3) ensuring strong usability and user experience, and (4) integrating smart survey data with conventional surveys while controlling for mode effects. Drawing on previous ESSNET projects and broader international research, each chapter examines these challenges in detail and describes planned field tests and randomized controlled trials. The ultimate goal is to evaluate best practices and design trade-offs across multiple countries, thereby providing a robust methodology and concrete recommendations for implementing smart surveys in official European statistics.
... Mitigating this limitation is the finding that high participant attrition rates are typical of Frontiers in Psychology 09 frontiersin.org online studies and that non-response rates (<85%) do not tend to bias reported outcomes (Couper et al., 2007;Church and Brooks, 2010;Tabachnick and Fidell, 2014). A further limitation is the large gender disparity between participants. ...
Background
A plethora of literature has delineated the therapeutic benefits of meditation practice on psychological functioning. A novel meditative practice, EcoMeditation, includes elements of four evidence-based techniques: The Quick Coherence Technique for regulating heart rate variability (HRV), Emotional Freedom Techniques (EFT), mindfulness, and neurofeedback.
Objectives
Changes in psychological symptoms, including anxiety, depression, posttraumatic stress, pain, and happiness were measured following a one-day virtual EcoMeditation training workshop. The current study extended on previous literature by adding measures of transcendent experiences and flow states.
Methods
Participants were drawn from a convenience sample of 151 participants (130 female, 21 male) aged between 26 to 71 years (M = 45.1, SD = 9.19) attending a one-day virtual EcoMeditation workshop. They were assessed pre-workshop, post-workshop, and at 3-months follow-up.
Results
Post-workshop results (N = 111) indicated a significant reduction in anxiety (−42.3%, p < 0.001), depression (−37.5%, p < 0.001), posttraumatic stress (−13.0%, p < 0.001), and pain (−63.2%, p < 0.001) Likert mean scores when compared to pre-workshop. There was also a significant increase in happiness (+111.1%, p < 0.001), flow states (+17.4%, p < 0.001), and transcendent experiences (+18.5%, p < 0.001). At 3-months follow-up, a one-way repeated measures ANOVA (N = 72) found significant decreases in anxiety, depression, and pain symptoms between pre-test and post-test, as well between pre-test and follow-up. Flow, happiness, and transcendent experiences increased significantly between pre-test and post-test, as well as between pre-test and follow-up, with over 71% of participants experiencing clinically significant improvements. Significant reductions in posttraumatic stress and depression symptoms between pre-test and follow-up were also noted.
Conclusion
EcoMeditation is associated with significant improvements in psychological conditions such as anxiety, depression, pain, and posttraumatic stress. EcoMeditation was also shown to enhance flow states and transcendent experiences. The benefits identified were similar to those found in the existing literature and provide support for the use of EcoMeditation as an effective stress reduction method that improves psychological symptoms and enhances transcendent states.
... Of the 10 studies that compared telephone-only to mailonly mode, 4 reported telephone-only response rates higher than mail-only rates 21,29,34,43 and 4 found comparable response rates for the 2 modes (or did not have sufficient power to detect differences; Table 1). [22][23][24]45 A study of bereaved family caregivers of hospice patients and another study of hospital patients found lower response rates to telephone-only surveys than to mail-only (38% phone-only vs. 43% mail-only 26 ; 27% phone-only vs. 38% mail-only, respectively 26,28 ). ...
Background:
Data from surveys of patient care experiences are a cornerstone of public reporting and pay-for-performance initiatives. Recently, increasing concerns have been raised about survey response rates and how to promote equity by ensuring that responses represent the perspectives of all patients.
Objective:
Review evidence on survey administration strategies to improve response rates and representativeness of patient surveys.
Research design:
Systematic review adhering to the Preferred Reporting Items for Systematic reviews and Meta-Analyses guidelines.
Study selection:
Forty peer-reviewed randomized experiments of administration protocols for patient experience surveys.
Results:
Mail administration with telephone follow-up provides a median response rate benefit of 13% compared with mail-only or telephone-only. While surveys administered only by web typically result in lower response rates than those administered by mail or telephone (median difference in response rate: -21%, range: -44%, 0%), the limited evidence for a sequential web-mail-telephone mode suggests a potential response rate benefit over sequential mail-telephone (median: 4%, range: 2%, 5%). Telephone-only and sequential mixed modes including telephone may yield better representation across patient subgroups by age, insurance type, and race/ethnicity. Monetary incentives are associated with large increases in response rates (median increase: 12%, range: 7%, 20%).
Conclusions:
Sequential mixed-mode administration yields higher patient survey response rates than a single mode. Including telephone in sequential mixed-mode administration improves response among those with historically lower response rates; including web in mixed-mode administration may increase response at lower cost. Other promising strategies to improve response rates include in-person survey administration during hospital discharge, incentives, minimizing survey language complexity, and prenotification before survey administration.
... We revealed a large increase in the number of participants who met a certain guideline from the baseline to the intermediate measurement (week 25). This could be due to the fact that this measurement was executed by telephone, which may be more subject to social desirability bias than a written questionnaire [51]. A similar increase from baseline to the intermediate telephone survey was also found in the control group. ...
Worldwide, adherence to national guidelines for physical activity (PA), and fruit and vegetable consumption is recommended to promote health and reduce the risk for (chronic) disease. This study reports on the effectiveness of various social-cognitive interventions to improve adherence to guidelines and the revealed adherence predictors. Participants (n = 1,629), aged 45–70 years, randomly selected and recruited in 2005–2006 from 23 Dutch general practices, were randomized (centralized stratified allocation) to four groups to receive a 12-month lifestyle intervention targeting guideline adherence for PA and fruit and vegetable consumption. Study groups received either four computer-tailored print communication (TPC) letters (n = 405), four telephone motivational interviewing (TMI) sessions (n = 407), a combined intervention (two TPC letters and two TMI sessions, n = 408), or no intervention (control group, n = 409). After the baseline assessment, all parties were aware of the treatment groups. Outcomes were measured with self-report postal questionnaires at baseline, 25, 47 and 73 weeks. For PA, all three interventions were associated with better guideline adherence than no intervention. Odds ratios for TPC, TMI and the combined intervention were 1.82 (95% CI 1.31; 2.54), 1.57 (95% CI 1.13; 2.18), and 2.08 (95% CI 1.50; 2.88), respectively. No pedometer effects were found. For fruit and vegetable consumption, TPC seemed superior to those in the other groups. Odd ratio for fruit and vegetable consumption were 1.78 (95% CI 1.32; 2.41) and 1.73 (95% CI 1.28; 2.33), respectively. For each behaviour, adherence was predicted by self-efficacy expectations, habit strength and stages of change, whereas sex, awareness and the number of action plans predicted guideline adherence for fruit and vegetable intake. The season predicted the guideline adherence for PA and fruit consumption. The odds ratios revealed were equivalent to modest effects sizes, although they were larger than those reported in systematic reviews. This study indicated that less resource intensive interventions might have the potential for a large public health impact when widely implemented. The strengths of this study were the participation of lower educated adults and evaluation of maintenance effects. (Trial NL1035, 2007-09-06).
... It is also important to note that high participant attrition rates are found in online studies 40 and that nonresponse rates (up to 85%) do not tend to bias reported outcomes. 41,42 Future research could consider comparing EcoMeditation administered as individual therapy with group-based meditation training. The cost-effectiveness of both could be compared. ...
Background
A growing body of clinical research attests to the psychological and physiological benefits of meditation. EcoMeditation is a non-pharmacological therapeutic approach used to promote health and well-being, comprising four evidence-based techniques: The Quick Coherence Technique for regulating heart rate variability (HRV), Emotional Freedom Techniques (EFT), mindfulness, and neurofeedback.
Objectives
This study investigated changes in psychological symptoms of anxiety, depression, posttraumatic stress disorder (PTSD), pain, and happiness following a one-day EcoMeditation training workshop delivered in a large-group format and at 3-months post-intervention.
Methods
A convenience sample of 208 participants (137 women, 71 men) aged between 21 and 87 years ( M = 55.4 years; SD = 12.8 years) attended a one-day EcoMeditation training workshop. Participants completed a pen-and-paper survey pre-workshop and post-workshop, and an online survey three months following the EcoMeditation intervention.
Results
Post-workshop results revealed significant reductions in anxiety (−23.4%, p < .001), depression (−15.8%, p = .011), PTSD (−11.8%, p < .001), and pain (−18.5%, p < .001), while happiness scores increased significantly (+8.9%, p < .001). At 3-month follow-up, one-way repeated-measures ANOVA ( N = 65) found significant decreases in anxiety between pre-test and post-test, and pain between pre-intervention and 3-month follow-up. Differences in depression and PTSD scores were not significant over time. Happiness scores significantly increased from pre-test to 3-month follow-up. However, post-hoc analyses suggested that the final sample size was inadequate to detect significant differences between time points.
Conclusion
Findings provide preliminary support for EcoMeditation as a brief group-based stress reduction intervention with benefits for improved psychological functioning.
... Similarly, utilising communication strategies and maintaining between-assessment contacts (by for example, sending out newsletters, emails or telephone calls) has been found to significantly improve retention [11]. These strategies have been effectively employed in a range of interventions including mental health [17,18], weight loss [19,20], rare diseases [21], substance use [22], research involving minority ethnic groups [23,24] and vulnerable groups such as the elderly [25], or people living with HIV [26]. ...
Background:
Engaging and retaining young men in community-based interventions is highly challenging. The purpose of this study was to investigate the individual factors that predict intervention engagement and adherence in a sample of at-risk South African men.
Methods:
Baseline data were collected as a part of a cluster randomised control trial (RCT) situated in Khayelitsha and Mfuleni, two peri-urban settlements situated on the outskirts of Cape Town, South Africa. Neighbourhoods were randomised to one of three intervention conditions. We performed univariate descriptive statistics to report neighbourhood and individual socio-demographic factors, and ran multivariate models, adjusting for entry of study, to determine if high adherence and consistency of engagement with the intervention were associated with socio-behavioural demographics and risk behaviours, such as hazardous substance use, gangsterism, and criminal activity.
Results:
Total of 729 men were on average 22.5 years old (SD 2.8), with a mean of 10 years of education. More than half of the sample were single (94%), lived with their parents (66%) and had an income below ~$30 (52%). The overall mean of adherence is 0.41 (SD 0.24) and mean of consistency of engagement is 0.61 (SD 0.30). Our data indicated that completing more years of education, living with parents, and having higher socioeconomic status were significantly associated with higher rates of engagement and adherence. Men with a history of gang membership demonstrated higher levels of adherence and consistent engagement with the intervention, compared with other men who were recruited to the intervention. Crucially, our data show that young men with a history of substance use, and young men who report symptoms of depression and high levels of perceived stress are equally likely as other young men to adhere to the intervention and attend intervention sessions consistently.
Conclusion:
Our results may contribute to a better understanding of young men's patterns of engagement and adherence to public health interventions. The results may have important implications for policy and practice, as they may be useful in planning more effective interventions and could potentially be used to predict which young men can be reached through community-based interventions.
Trial registration:
ClinicalTrials.gov registration, NCT02358226. Prospectively registered 24 November 2014.
... For example, the literature shows that single mode surveys are far less effective than multimode methods [18][19][20] and that web-based survey response rates are improved when followed by letter mailings and/or phone calls. 21 Providing incentives for survey completion may also increase the response rate. Utilizing multiple modes of survey administration is even more important in diverse populations. ...
Health care systems are increasingly utilizing electronic medical record—associated patient portals to facilitate communication with patients and between providers and their patients. These patient portals are growing in recognition as potentially valuable research tools. While there is much information about the response rates and demographics of internet-based surveys as well as the demographics of patients who are portal members, not much is known about the response rate of internet-based surveys directed to a group of patient portal members or the demographics of which portal members respond to internet-based surveys issued within that specific population. The objective of these analyses was to determine the demographics of patient portal users who respond to an internet-based survey request. We hypothesized that respondents would more likely be: (1) older (65+), (2) European American, (3) married, (4) female, (5) college educated, (6) have higher medical care utilization, (7) have more comorbidities, and (8) have a private practice primary care physician (as opposed to a salaried group practice primary care physician). We found that our respondents tended to be older, of European geographic ancestry, and more frequent users of healthcare. While patient portal members are an easily identifiable and contactable group that are potentially valuable participants for research, it is important to understand that respondents to surveys solicited from this sampling frame may not be entirely representative. It will be important to develop strategies to more fully engage populations that represent the target population in order to increase overall and subgroup response rates.
... In accordance to Eysenbach et al (2005), less-frequent log-in rates over time are a serious problem of Web-based interventions [42]. According to Couper et al (2007), technical challenges (eg, no access to email, problems accessing or submitting the survey) and problems with the survey (eg, lack of interest in or lack of effectiveness of intervention, no time or bad timing, survey was boring or too long) are the main reasons for noncompletion [49]. Nevertheless, the only common feature of Web-based interventions is the delivery channel [50]. ...
... In accordance to Eysenbach et al (2005), less-frequent log-in rates over time are a serious problem of Web-based interventions [42]. According to Couper et al (2007), technical challenges (eg, no access to email, problems accessing or submitting the survey) and problems with the survey (eg, lack of interest in or lack of effectiveness of intervention, no time or bad timing, survey was boring or too long) are the main reasons for noncompletion [49]. Nevertheless, the only common feature of Web-based interventions is the delivery channel [50]. ...
Background:
In the primary and secondary prevention of civilization diseases, regular physical activity is recommended in international guidelines to improve disease-related symptoms, delay the progression of the disease, or to enhance postoperative outcomes. In the preoperative context, there has been a paradigm shift in favor of using preconditioning concepts before surgery. Web-based interventions seem an innovative and effective tool for delivering general information, individualized exercise recommendations, and peer support.
Objective:
Our first objective was to assess feasibility of our Web-based interventional concept and analyze similarities and differences in a sustained exercise implementation in different diseases. The second objective was to investigate the overall participants' satisfaction with our Web-based concept.
Methods:
A total of 4 clinical trials are still being carried out, including patients with esophageal carcinoma scheduled for oncologic esophagectomy (internet-based perioperative exercise program, iPEP, study), nonalcoholic fatty liver disease (hepatic inflammation and physical performance in patients with nonalcoholic steatohepatitis, HELP, study), depression (exercise for depression, EXDEP, study), and cystic fibrosis (cystic fibrosis online mentoring for microbiome, exercise, and diet, COMMED, study). During the intervention period, the study population had access to the website with disease-specific content and a disease-specific discussion forum. All participants received weekly, individual tailored exercise recommendations from the sports therapist. The main outcome was the using behavior, which was obtained by investigating the log-in rate and duration.
Results:
A total of 20 participants (5 from each trial) were analyzed. During the intervention period, a regular contact and a consequent implementation of exercise prescription were easily achieved in all substudies. Across the 4 substudies, there was a significant decrease in log-in rates (P<.001) and log-in durations (P<.001) over time. A detailed view of the different studies shows a significant decrease in log-in rates and log-in durations in the HELP study (P=.004; P=.002) and iPEP study (P=.02; P=.001), whereas the EXDEP study (P=.58; P=.38) and COMMED study (P=.87; P=.56) showed no significant change over the 8-week intervention period. There was no significant change in physical activity within all studies (P=.31). Only in the HELP study, the physical activity level increased steadily over the period analyzed (P=.045). Overall, 17 participants (85%, 17/20) felt secure and were not scared of injury, with no major differences in the subtrials.
Conclusions:
The universal use of the Web-based intervention appears to be applicable across the heterogonous collectives of our study patients with regard to age and disease. Although the development of physical activity shows only moderate improvements, flexible communication and tailored support could be easily integrated into patients' daily routine.
Trial registration:
iPEP study: ClinicalTrials.gov NCT02478996; https://clinicaltrials.gov/ct2/show/NCT02478996 (Archived by WebCite at http://www.webcitation.org/6zL1UmHaW); HELP study: ClinicalTrials.gov NCT02526732; http://www.webcitation.org/6zJjX7d6K (Archived by WebCite at http://www.webcitation.org/6Nch4ldcL); EXDEP study: ClinicalTrials.gov NCT02874833; https://clinicaltrials.gov/ct2/show/NCT02874833 (Archived by WebCite at http://www.webcitation.org/6zJjj7FuA).
... Due to declining survey response rates and persistent concerns about nonresponse bias compromising the quality of survey estimates, many social science studies conduct follow-up surveys with nonrespondents (Couper, Peytchev, Strecher, Rothert, and Anderson 2007;Schouten 2007;Olson, Lepkowski, and Garabrant 2011). Nonresponse follow-up (NRFU) surveys aim to collect additional information on nonrespondents and improve the representativity of the survey results. ...
To evaluate and adjust for nonresponse bias in household surveys, many social science studies conduct follow-up surveys with nonrespondents. By recruiting additional respondents, the goal of nonresponse follow-up (NRFU) surveys is to reduce nonresponse bias and make the respondent pool more representative of the characteristics of the sample as a whole. Often a change of data collection mode or a shorter questionnaire is implemented to increase response rates. However, whether these design features actually reduce nonresponse bias is usually unknown. What is also unknown is the effect of NRFU studies on measurement error, particularly when interviewer- and self-administered modes are used to administer sensitive questions susceptible to misreporting. Few studies have explicitly examined the joint impact of nonresponse and measurement error bias in NRFU surveys due to the lack of auxiliary validation data about the respondents and nonrespondents. We overcome this deficiency in an economic survey initially administered by telephone with mail nonresponse follow-up and administrative validation records available for the entire sample. This situation permits the estimation of both nonresponse and measurement error bias before and after the NRFU. We find that the NRFU survey succeeds in bringing in respondents who differ from the telephone respondents, but that these additional respondents are not always representative of the final nonrespondents. This results in reduced nonresponse bias for some items, but increased nonresponse bias for others. We also find that combining the mail NRFU respondents with the telephone respondents reduces measurement error bias for economic estimates. Lastly, we report a paradoxical finding in which adding NRFU respondents to the respondent pool produces greater total bias in some survey estimates despite reducing both nonresponse and measurement error bias separately. We conclude with a discussion of the practical implications of these findings and speculate on their possible causes. © The Author 2017. Published by Oxford University Press on behalf of the American Association for Public Opinion Research. All rights reserved.
