Content uploaded by Courtney Kennedy
Author content
All content in this area was uploaded by Courtney Kennedy on Jul 16, 2014
Content may be subject to copyright.
A preview of the PDF is not available
Gauging the Impact of Growing Nonresponse on Estimates From a National RDD Telephone Survey
Abstract and Figures
Declining contact and cooperation rates in random digit dial (RDD) national telephone surveys raise serious concems about the validity of estimates drawn from such research. While research in the 1990s indicated that nonresponse bias was relatively small, response rates have continued to fall since then. The current study replicates a 1997 methodological experiment that compared results from a "Standard" 5-day survey employing the Pew Research Center's usual methodology with results from a "Rigorous" survey conducted over a much longer field period and achieving a significantly higher response rate. As with the 1997 study, there is little to suggest that unit nonre-sponse within the range of response rates obtained seriously threatens the quality of survey estimates. In 77 out of 84 comparable items, the two surveys yielded results that were statistically indistinguishable. While the "Rigorous" study respondents tended to be somewhat less politically engaged, they did not report consistently different behaviors or attitudes on other kinds of questions. With respect to sample composition, the Standard survey was closely aligned with estimates from the U.S. Census and other large govemment surveys on most vari-ables. We extend our analysis of nonresponse to include comparisons with the hardest-to-reach respondents and with respondents who termi-nated the interview prior to completion.
Figures - uploaded by Courtney Kennedy
Author content
All figure content in this area was uploaded by Courtney Kennedy
Content may be subject to copyright.
... Health care seeking behavior is poor and only a small proportion of children receive appropriate treatment [20]. The top five leading causes of morbidity in Ethiopia for children under five years are infectious diseases: diarrhea (20%), pneumonia (19%), acute respiratory infections (ARI) (15%), and acute febrile illnesses (AFI) (7%) [21]. Furthermore, they are major causes of under-five mortality as well as inpatient admissions [22]. ...
Appropriate health care-seeking behaviour of mothers/caregivers for common childhood illnesses could prevent a significant number of child deaths due to childhood illnesses, currently, there are no studies in Ambo town, Oromia Regional State, Ethiopia. Assessment of Health Care Seeking Behaviour for Common Childhood Illnesses and Associated Factors among Mothers' Having Under 5 Children in Ambo Town. A total of 476 mothers with under-five children who experienced diseases within eight weeks before the survey were selected for this study. A community-based cross-sectional study design was employed from June to August 2020. All children who are under 5 years old in the town were included in the study. A sampling technique was used to interview the mother. Data were entered in Epi-data version 3.1 and exported to SPSS version 16.1. The completeness and consistency of the data were checked and cleaned. Logistic regression was applied to see the association between dependent and independent variables. A total of 476 mothers were enrolled in the study giving an overall response rate of 100%. A total surveyed mothers who had 234 sick children observed these children had a cough at any time in the last 8 weeks 119 (58.9%) when had an illness with a cough, did he/she breathe faster than usual with short, rapid breaths or had difficulty breathing 63 (33.7%) for less than two weeks 167 (89.3%). Awareness of common childhood illnesses, perceived illness severity, perception of early treatment, and child age between 12-24 months were positively associated with mothers' or caregivers' healthcare-seeking behaviour. Efforts should be made by government and non-governmental organizations to improve the mothers' healthcare-seeking behaviour by providing information.
... A second limitation of the study is its response rate, which may have implications for bias and statistical precision. In general, there is an assumption that greater response rates are tied to more representative empirical claims about populations, but methodological research has demonstrated that nonresponse bias cannot be assumed from response rates (Keeter et al., 2006;Rindfuss et al., 2015). Because the Work and Well-Being in Science Study was not primarily about the primary dependent variable in our study, and given that our response patterns showed no systematic bias along the core independent variables of interest (such as gender and discipline), we do not believe the Bias Attribution Index is systematically biased. ...
Colleges and universities across the globe have implemented a variety of bureaucratic reforms aimed at diversification of the professoriate. Initiatives related to bias, such as implicit bias training, are especially common, reflecting an assumption that underrepresentation of women and racialized minorities in academe originates from biased evaluative processes. To date, however, existing research lacks representative data assessing the extent to which members of the academic profession attribute insufficient diversity to bias in hiring and promotion. Drawing on a survey of 1,923 physicists and biologists in the United States, the United Kingdom, Italy, and India, we examine the extent to which scientists believe that insufficient diversity in science reflects bias in hiring and promotion. The results demonstrate that perceptions of bias are related to gender, race, political ideology, and national context. We consider the practical implications of these findings alongside evidence from organizational theory that demonstrates how the effects of diversity training are often short-lived and counterproductive.
... However, evidence suggests that response rates may not be as strongly correlated with survey quality or representativeness as once thought. 80,81 Moreover, this survey included representation from central, east, and west Canadian provinces, and a French-speaking province, thus covering key geographical areas of Canada. Finally, our institution is comparable in size with others across the country. ...
Purpose: Globally, there is a lack of access to health care providers who offer gynecological procedures. Understanding the practice patterns of academic family physicians (AFPs) and whether additional training impacts the provision of care is critical. This study surveys the practice patterns of AFPs regarding gynecological procedures offered, identifies barriers, and explores the impact of additional training.
Methods: We circulated an anonymous, cross-sectional survey to all 17 family medicine programs across Canada, receiving responses from 71 AFPs. We computed descriptive statistics and bivariate associations.
Results: A total of 71 respondents from five universities participated. Most participants (97.2%) performed Papanicolaou (Pap) smears; 67.6% provided intrauterine device (IUD) insertion, and only 54.9% offered endometrial biopsy. Numbers decreased significantly for routine pessary care (29.5%), punch biopsy of the vulva (15.5%), and pessary fitting (5.6%). Eighteen participants (26.9%) had received enhanced skills training with a certificate of added competence (CAC), of which 55.6% were in women’s health. CAC holders in women’s health provided IUD insertions (100% vs. 67.3%; p = 0.049, V = 0.28) and endometrial biopsies (90.0% vs. 53.1%; p = 0.036, V = 0.28) at higher rates than general AFPs. Frequently cited barriers to offering gynecological procedures included lack of knowledge, procedural skills, and insufficient patient volumes to maintain competence. During the COVID-19 pandemic, 44% of respondents reported reducing or ceasing to provide Pap smears.
Conclusions: Many AFPs in Canada do not provide essential gynecological procedures. This impacts patient access and the training of the next generation of family physicians and thus requires innovative strategies to address the persistent procedural skills educational gap for trainees.
... These rates are comparable to those found in other large telephone surveys in recent years (see Kohut et al., 2012;Lavrakas et al., 2017;Marken, 2018). It should be noted, however, that response rates are poor measure of data quality and nonresponse bias, and de facto do not have a substantial impact on survey estimates, including crime-related attitudes such as fear of neighborhood crime (Curtin et al., 2000;Groves, 2006;Keeter et al., 2000Keeter et al., , 2006Lee et al., 2009). ...
Core normative value theory asserts that perceptions of the legitimacy of law enforcement authorities are not just a product of these agencies’ conduct and performance, but are also based on deeply embedded ideological worldviews. This study further tests the theory, exploring the diverse relationships between values and legitimacy in multicultural societies. We use Israel’s highly divided society as a case study. Our data were derived from a survey among a representative sample of more than 1,600 Israelis from seven social groups—mainstream Jews, ultra-Orthodox Jews, Jewish residents of the occupied territories, immigrants from the former Soviet Union, Druze, Muslims, and Arab Christians. The results point to significant differences in levels of the core normative values and legitimacy across the seven groups, and differences between the groups in associations between the values and legitimacy, controlling for socio-demographic variables. The findings support the main premises of the theory.
... A limitation of the data is lack of information about the response rate in each case organization. However, a low response rate is not necessarily an indicator of poor survey quality (Keeter et al., 2006). Important though, is whether there is reason to believe that the themes addressed systematically influenced the willingness of certain groups of workers to respond. ...
Individualized services are provided under complex conditions, as a variety of factors can affect the ability of a street‐level organization to adapt its services to individual needs and circumstances. Especially challenging are tensions between the means of control and standardization following new public management (NPM) and post‐NPM ideas of holistic and coordinated services. Through a fuzzy‐set qualitative comparative analysis of Norwegian sector‐spanning street‐level organizations, we show three different configurations that can promote individualized services. These consist of variations of structural circumstances (size, service variety); organizational responses (goal coherence, cross working); and manager capacity (professional background, managerial orientation). Service individualization is not an outcome of the interaction between street‐level workers and clients alone, but an outcome of street‐level organizations and their managers' use of measures and competencies across service sectors, and of their capacity to develop a shared perception of goals and an organization that handles institutional complexity.
... The HHGP-HK cohort study is composed of 1,025 healthy Chinese volunteers recruited in Hong Kong stratified according to sex (512 female subjects and 513 male subjects); and age (6 decades of age: [20][21][22][23][24][25][26][27][28][29] (Table S1). The response and cooperation rates (Table S1) were comparable to those attained for random or population-based studies locally and internationally including pre-and peripandemic studies 19 . ...
Immune responses are highly variable from one individual to another, with this variability determined by factors such as age, sex, genetics and environment, which also affect physical and mental well-being. Systems immunology studies have been applied to human cohort studies to better understand these effects, but most are performed in Western populations of mostly European ancestry, neglecting the vast majority of human global diversity. To overcome this limitation, we established the Healthy Human Global Project Hong Kong (HHGP-HK) cohort to allow the study of immune variability in a healthy Asian population. Taking inspiration from the established French Milieu Interieur study we adapted inclusion and exclusion criteria to identify healthy donors, and to facilitate cross comparison between the two cohorts we collected similar demographic, medical and lifestyle information. To study the effects of age and sex, donors were stratified for both ranging from 20-79 years. We report here significant age and sex effects on multiple clinical laboratory measures, many of which were consistently observed in the Milieu Interieur cohort. C-reactive protein and liver enzymes were two exceptions perhaps reflecting different environmental effects between the two cohorts. We applied biological aging, physiological, and mental health scores to our cohort which highlights the need to develop and adapt such scores for application to populations of non-European descent. We also included wearable fitness trackers from which we observed significant age and sex differences for sleep, physical activity, and heart rate variability. This unique cohort will enable the better understanding of factors that determine immune variability within Asian populations.
... The analysis of (online) textual data has the advantage of considering the expressions of a high number of stakeholders at a low cost while also reducing some of the biases that can arise when administering brand equity surveys (Bhoi et al., 2022;Grandcolas et al., 2003;Olson, 2006;Wang et al., 2023;Warner, 1965;Zhu et al., 2023). Traditional survey methods often suffer from nonresponse and social desirability biases, which can skew results (Fronzetti Colladon, 2018;Keeter et al., 2006). In contrast, analyzing spontaneous online discussions provides more authentic insights into stakeholders' true opinions and attitudes (Fronzetti Colladon, 2018;Rust et al., 2021;Zhang & Moe, 2021). ...
This paper presents a new decision support system offered for an in-depth analysis of semantic networks, which can provide insights for a better exploration of a brand's image and the improvement of its connectivity. In terms of network analysis, we show that this goal is achieved by solving an extended version of the Maximum Betweenness Improvement problem, which includes the possibility of considering adversarial nodes, constrained budgets, and weighted networks-where connectivity improvement can be obtained by adding links or increasing the weight of existing connections. Our contribution includes a new algorithmic framework and the integration of this framework into a software system called Brand Network Booster (BNB), which supports brand connectivity evaluation and improvement. We present this new system together with three case studies, and we also discuss its performance. Our tool and approach are valuable to both network scholars and in facilitating strategic decision-making processes for marketing and communication managers across various sectors, be it public or private.
During armed conflict civilians often inhabit areas of contested governance or areas where rebel groups, NGOs, and/or criminal syndicates vie for authority and challenge the control of the state. As non-state actors confront the authority and legitimacy of the state, civilians become central players in that competition asked to uphold or undercut these alternative governance claims. In this paper we examine the long-term impact of rebel governance for citizens living in spaces where state governance is challenged. Leveraging survey data from areas historically under PKK control in Southeastern Turkey, we focus on the ways in which contestation over governance during the conflict influenced future trust and engagement with the Turkish state. Specifically, we find that individual engagement with rebel governance institutions and personal conflict experience are important factors in understanding the effects of contested governance. Our findings increase our understanding of the long-term impact of armed conflict on civilians and the potential lasting impacts of rebel governance on the post-conflict state.
The high cost of US childcare has become burdensome to most families with young children, yet little is known about how childcare costs shapes parental childcare decisions. This study addresses a critical gap in knowledge by examining how economic hardships shape parental childcare decisions, along with the individual and community characteristics associated with the influence of cost on childcare decisions. Guided by the accommodation model and family stress model, we used data from a statewide consumer survey in Iowa to conduct the first known study to examine associations between household constraints (i.e., economic hardships) and respondent reports that cost influenced their childcare decisions at least once in the previous year (i.e., systemic barriers to childcare). As hypothesized, each form of economic hardship was associated with increased odds of cost influencing childcare decisions. Our logistic regression demonstrated that every standard deviation increase in our financial stress scale resulted in a 110% increase in the odds of cost influence, while each additional experience of material hardship (out of 16) increased the odds of cost influence by 10%. Additionally, we identified demographic variations, with cost playing a more prominent role for people who were Hispanic/any race, unemployed, and were between the ages of 18- to 34-years old. Overall, findings demonstrate that economic hardships are constraints that, coupled with systemic barriers like cost, shape childcare decisions across demographic groups, communities, and income brackets. Findings also suggest the need for more public support to alleviate the costs of childcare among all families, such as expanding childcare subsidies or funding the provision of care, which may mitigate the risk of negative outcomes associated with economic hardship.
We correct, update, and elaborate Curtin, Presser, and Singer’s (2000) report that the University of Michigan’s Survey of Consumer Attitudes (SCA) experienced only a small response rate decline
between 1979 and 1996, contrary to the widespread perception of plunging response rates. Our aims are to (1) correct errors
in the SCA response rate data that affected Curtin, Presser, and Singer’s (2000) result, (2) examine the trend in SCA response rates after 1996, when caller identification technology became widespread,
and (3) describe the roles played by the various sources of SCA nonresponse over time. The results show that the response
rate decline from 1979 to 1996 was larger than described by Curtin, Presser, and Singer (2000); the response rate drop was significantly steeper from 1996 to 2003 than from 1979 to 1996; and the 1979 to 2003 trends differed
substantially for refusals and noncontacts.
While a low survey response rate may indicate that the risk of nonresponse error is high, we know little about when nonresponse causes such error and when nonresponse is ignorable. Leverage-salience theory of survey participation suggests that when the survey topic is a factor in the decision to participate, noncooperation will cause nonresponse error. We test three hypotheses derived from the theory: (1) those faced with a survey request on a topic of interest to them cooperate at higher rates than do those less interested in the topic; (2) this tendency for the "interested" to cooperate more readily is diminished when monetary incentives are offered; and (3) the impact of interest on cooperation has nonignorability implications for key statistics. The data come from a three-factor experiment examining the impact on cooperation with surveys on (a) five different topics, using (b) samples from five different populations that have known attributes related to the topics, with
We correct, update, and elaborate Curtin, Presser, and Singer's (2000) report that the University of Michigan's Survey of Con- sumer Attitudes (SCA) experienced only a small response rate decline between 1979 and 1996, contrary to the widespread perception of plunging response rates. Our aims are to (1) correct errors in the SCA response rate data that affected Curtin, Presser, and Singer's (2000) result, (2) examine the trend in SCA response rates after 1996, when caller identification technology b ecame widespread, and (3) describe the roles played by the various sources of SCA nonresponse over time. The results show that the response rate decline from 1979 to 1996 was larger than described by Curtin, Presser, and Singer (2000); the response rate drop was significantly steeper from 1996 to 2003 than from 1979 to 1996; and the 1979 to 2003 trends differed substantially for refusals and noncontacts.
The key to understanding why people overreport is that those who are under the most pressure to vote are the ones most likely to misrepresent their behavior when they fail to do so. Among all nonvoters, the most likely to overreport are the more educated, partisan, and religious, and those who have been contacted and asked to vote for a candidate. The greater the concentration of African-American and Latino nonvoters in a district, the greater the probability of overreporting in those districts, both among those in the relevant minority group and among white Anglos. White nonvoters are more likely to overreport in the Deep South than elsewhere. Overreporting matters: using reported votes in place of validated votes substantially distorts standard multivariate explanations of voting, increasing the apparent importance of independent variables that are related in the same direction to both overreporting and voting and sharply decreasing the apparent importance of independent variables related in opposing directions to those two variables.
Methods for estimating nonresponse bias are reviewed and several methods are tried on the 1980 GSS. The results indicate that
various estimating procedures are inappropriate and that even the more promising techniques can provide faulty estimates of
nonresponse bias. By its nature, nonresponse bias is very difficult to assess accurately and no simple, certain method exists.
The Urban Institute conducts the National Survey of America's Families (NSAF) to assess the New Federalism by surveying children and adults in 12 states and in the balance of the nation. The 12 states account for about 50 percent of the country's population. The initial round of the NSAF took place in 1997, with followup rounds in 1999 and 2002. The major objective of the study is assessing the effects of the devolution of responsibility for major social programs. A consortium of foundations, led by the Annie E. Casey Foundation, funds NSAF. Westat is responsible for the sample design, data collection, and weighting. The survey is a dual frame design with a random digit dialing (RDD) sample and an area probability sample. In this paper we consider only the RDD sample. See Cunningham, Martin, and Brick (2003) and the Urban Institute's URL for more details on the survey and its methodology (www.urban.org). Recent literature on random digit dial (RDD) telephone surveys suggests that increased levels of effort are required to contact households due to changes in technology. For example, see Curtin et al. 2000. Answering machines, caller-ID devices, TeleZappers and privacy managers are examples of the types of technologies that are being used at an increasing rate to shield the households from unwanted calls. In addition, it is speculated that more telephone numbers are devoted to purposes such as fax machines and computers and may never be answered no matter how many times the numbers are dialed. This paper uses Round 1 (1997) and Round 3 (2002) NSAF data to examine some of these issues. The next section gives some basic background information needed to help understand the data collection efforts in the two rounds. 2. Background
Many surveys of the U.S. household population are experiencing higher refusal rates. Nonresponse can, but need not, induce
nonresponse bias in survey estimates. Recent empirical findings illustrate cases when the linkage between nonresponse rates
and nonresponse biases is absent. Despite this, professional standards continue to urge high response rates. Statistical expressions
of nonresponse bias can be translated into causal models to guide hypotheses about when nonresponse causes bias. Alternative
designs to measure nonresponse bias exist, providing different but incomplete information about the nature of the bias. A
synthesis of research studies estimating nonresponse bias shows the bias often present. A logical question at this moment
in history is what advantage probability sample surveys have if they suffer from high nonresponse rates. Since postsurvey
adjustment for nonresponse requires auxiliary variables, the answer depends on the nature of the design and the quality of
the auxiliary variables.
Critics of public opinion polls often claim that methodological shortcuts taken to collect timely data produce biased results. This study compares two random digit dial national telephone surveys that used identical questionnaires but very different levels of effort: a "Standard" survey conducted over a 5-day period that used a sample of adults who were home when the interviewer called, and a "Rigorous" survey conducted over an 8-week period that used random selection from among all adult household members. Response rates, computed according to AAPOR guidelines, were 60.6 percent for the Rigorous and 36.0 percent for the Standard study. Nonetheless, the two surveys produced similar results. Across 91 comparisons, no difference exceeded 9 percentage points, and the average difference was about 2 percentage points. Most of the statistically significant differences were among demographic items. Very few significant differences were found on attention to media and engagement in politics, social trust and connectedness, and most social and political attitudes, including even those toward surveys.