No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Explaining Interviewer Effects: A Research Synthesis
Abstract
A rich and diverse literature exists on the effects that human interviewers can have on different aspects of the survey data collection process. This research synthesis uses the Total Survey Error (TSE) framework to highlight important historical developments and advances in the study of interviewer effects on a variety of important survey process outcomes, including sample frame coverage, contact and recruitment of potential respondents, survey measurement, and data processing. Included in the scope of the synthesis is research literature that has focused on explaining variability among interviewers in these effects and the different types of variable errors that they can introduce, which can ultimately affect the efficiency of survey estimates. We first consider common tasks with which human interviewers are often charged and then use the TSE framework to organize and synthesize the literature discussing the variable errors that interviewers can introduce when attempting to execute each task. Based on our synthesis, we identify key gaps in knowledge and then use these gaps to motivate an organizing model for future research investigating explanations for interviewer effects on different aspects of the survey data collection process.
Supplementary resource (1)
... Due to the semi-structured nature of administration of POS, and the large number of interviewers involved in data collection, we hypothesised that a large proportion of variance in the data set was influenced by between-interviewer variability. There are a number of ways interviewers could have introduced error into the data-this could be at participant selection, for example, in the way that they made contact and gained cooperation from participants, or it could be measurement error introduced in the way that interviewers deliver questions (West & Blom, 2017). ...
... Bias in the sample could have been introduced at selection as participants were selected according to organisation and interviewer convenience (West & Blom, 2017). (e.g., interviewers may have been more likely to approach participants at services where they already held a relationship with either staff or participants). ...
... A review of the literature on interviewer effect (Schaeffer et al., 2010) of questions may provide more opportunity for probing or using a more conversational format to deliver questions (West & Blom, 2017). ...
Background
The Personal Outcomes Scale (POS) is a scale developed to measure quality of life of adults (18+) with intellectual disability. Previous studies have reported good fit for Spanish and Portuguese language versions of POS.
Aims
This study aimed to evaluate the factor structure of the English language version of POS when used to measure the quality of life of adults (18+) with intellectual disability in the UK.
Materials and Methods
Analysis was conducted on POS data from 310 adults with an intellectual disability. First and second order factor models and multi‐level models were used to assess fit.
Results
There was poor fit to the data for all tested models. We estimated that 23% of variance in POS scores was accounted for by interviewer cluster.
Discussion
This was the first UK‐based evaluation of POS and our data did not confirm the factor structure of the POS measure. The identification of systematic variability within the dataset indicates that inter‐rater reliability is a potential limitation of the POS tool.
Conclusion
Further research is needed to investigate inter‐rater reliability of POS interviewers and to explore factor structure.
... While some recent studies have assessed impacts of respondent and survey characteristics on response quality in web surveys (Hofelich Mohr et al., 2016;Meitinger et al., 2019;, there is little systematic research concerning the mechanisms of interviewer effects in OEQs. This is even more surprising given the fact that studies reveal a high intra-interviewer correlation coefficient in OEQs (expressing the amount of variance explained by the interviewer) (Schaeffer et al., 2010;Schnell & Kreuter, 2005;West & Blom, 2017). Despite these findings, interviewer effects are seldom controlled in research using OEQs, and little is known about the ways in which interviewers and respondents may (jointly and interactively) impact on response quality. ...
... There is a second dimension of mechanisms which can generate or distort quality at the level of the interviewer. Interviewers can have a number of influences in the survey process, from differences in contact practices and realized responses rate to measurement variability, not to mention the errors introduced by the falsification of parts of or the entire interview (Blasius & Thiessen, 2018;Haunberger, 2006;West & Blom, 2017). Interviewer behavior impacts on response quality include neglecting interview instructions, directive probing, prompting the respondent to answer more quickly, giving subtle hints of displeasure or contentment, processing errors such as misclassification or selective reporting of respondents' answers, or skipping or falsifying items (Blasius & Thiessen, 2018;Brunton-Smith et al., 2017;Hanson & Marks, 1958;Holbrook et al., 2003;Houtkoop-Steenstra, 1996;Mangione et al., 1992;Mitchell et al., 2008;Smyth & Olson, 2019). ...
... Many studies, most of them examining standardized questions, have assessed whether interviewer characteristics can explain such behavior. Numerous researchers have found effects of interviewers' age, gender, and ethnicity, albeit with results pointing into different directions, suggesting interaction effects with both question and respondent characteristics (West & Blom, 2017). There seems to be a slight tendency, however, for female interviewers to generate higher quality data (Freeman & Butler, 1976;Groves & Fultz, 1985;Hill, 1991;Liu & Wang 2016) in both face-to face and telephone surveys. ...
... The reliance on trained human interviewers may impose substantial costs on researchers (Groves, 2005) and induce effects such as socially desirable responding. Studies suggest that field interviewers, when asked to 'live code' information from respondents, are prone to errors like mistyping and mishearing (Olson & Smyth, 2015;West & Blom, 2017). Moreover, integrating conversational or interactive elements into self-administered web surveys has historically been challenging. ...
... Active coding refers to a textbot's ability to detect concepts within open-ended answers in real time, as a field interviewer might (West & Blom, 2017) without the risks of human error (Olson & Smyth, 2015). This functionality draws on machine learning techniques such as text classification, sentiment analysis, and topic modelling (Puri & Catanzaro, 2019). ...
Standardized surveys scale efficiently but sacrifice depth, while conversational interviews improve response quality at the cost of scalability and consistency. This study bridges the gap between these methods by introducing a framework for AI-assisted conversational interviewing. To evaluate this framework, we conducted a web survey experiment where 1,800 participants were randomly assigned to text-based conversational AI agents, or "textbots", to dynamically probe respondents for elaboration and interactively code open-ended responses. We assessed textbot performance in terms of coding accuracy, response quality, and respondent experience. Our findings reveal that textbots perform moderately well in live coding even without survey-specific fine-tuning, despite slightly inflated false positive errors due to respondent acquiescence bias. Open-ended responses were more detailed and informative, but this came at a slight cost to respondent experience. Our findings highlight the feasibility of using AI methods to enhance open-ended data collection in web surveys.
... Their ability to follow standardized procedures and manage interactions with respon-24 dents can greatly influence the reliability and validity of the data collected. Non-zero correlations among 25 responses collected by a particular interviewer, often referred to as interviewer effects, are a common 26 concern in survey research (West & Blom, 2017). Research suggests that interviewer effects can vary 27 across three primary domains: respondent characteristics, interviewer characteristics, and survey design 28 features. ...
... Two-way combinations that are significantly associated with interviewer variance are shown in Table 369 2. The use of showcards in combination with other characteristics indicated an increase in ICCs. Scalar Interviewers in face-to-face surveys are both a key to securing high data quality and a source of instability 376 in measurement (West & Blom, 2017). Understanding the circumstances under which interviewer vari-377 ance is more likely to occur is an important area of research. ...
Interviewer effects are a common challenge in face-to-face surveys. Understanding the conditions that make interviewer variance more likely to occur is essential in tackling sources of bias. Earlier evidence suggests that certain features of the survey instrument provide more ground for interviewer influence. For instance, attitudinal, sensitive, complex or open-ended questions invite more interviewer variance. In this paper, we aim to validate earlier results, previously derived from single-country studies, by using the large cross-national sample of the European Social Survey. We compare 31,270 intraclass-correlations (ICCs) derived from 1004 survey questions from 28 countries using data from 10 waves of the ESS. The questions were manually coded based on several characteristics. These features of survey questions were then used as predictors of ICCs in multilevel models. The results show that question characteristics account for a significant portion of the variation in ICCs, with certain types, such as attitude and non-factual questions, items appearing later in the survey, and those using showcards, being especially susceptible to interviewer effects. Our findings have important implications for both interviewer training and questionnaire design.
... On the other hand, questions on sexual health and other chronic disorders (i.e., HIV symptoms) were more likely to be described if interviewers were men. This is likely attributed to the fact that "participants recognize men as more 'doctor-like'" (Kianersi et al. 2019, quoted in Edwards & Berk 1993West & Blom 2017). Finally, interviewers who were men were more likely to elicit a non-response: "Don't Know" or "Refuse", a finding consistent with similar studies undertaken on gendersensitive sexual behaviour surveys conducted in the US and Taiwan (West & Blom 2017, quoted in Catania et al. 1996Tu and Liao 2007). ...
... This is likely attributed to the fact that "participants recognize men as more 'doctor-like'" (Kianersi et al. 2019, quoted in Edwards & Berk 1993West & Blom 2017). Finally, interviewers who were men were more likely to elicit a non-response: "Don't Know" or "Refuse", a finding consistent with similar studies undertaken on gendersensitive sexual behaviour surveys conducted in the US and Taiwan (West & Blom 2017, quoted in Catania et al. 1996Tu and Liao 2007). ...
... Samplegenerating characteristics such as social status, contacts, or willingness to participate in a survey can be assumed to be highly correlated with the dependent variable (Cornesse et al., 2020). For example, research has shown that respondents who are well integrated in a country are more likely to be involved politically and to share their opinions with others and are therefore also more likely to participate in surveys (Tina et al., 1991;West & Blom, 2017). As close social contacts more often share the same views, similar effects may occur when using origin-based snowball convenience sampling whereby respondents "nominate" friends or relatives as additional potential respondents (Beauchemin & González-Ferrer, 2011). ...
... Compared with respondents in self-administered surveys, respondents in interviewer-assisted surveys may be more likely to give more socially desirable responses, depending on the race and ethnicity of the interviewer (Deding et al., 2008;Holbrook et al., 2019). In this case, the mechanisms for giving socially desirable responses are activated by interviewer characteristics such as having the same cultural background as the respondent (Davis et al., 2019;Kühne, 2018;West & Blom, 2017). Furthermore, it can be expected that immigrants who have been living for some time in and are literate in the language of the residence country will be able to participate in self-administered surveys in that language Salentin, 2014). ...
... Depending on the mode and sampling design of data collection, they may need to list addresses to generate sampling frames, recruit respondents, ask survey questions, and record participants' responses. Therefore, from a total survey error framework, interviewers can affect survey data quality by generating or reducing coverage error, nonresponse error, measurement error, and processing error (West and Blom 2017). Most research examining interviewers' effects focuses on measurement error (Schuman and Converse 1971;Hanson and Marks 1958;Ehrlich and Riesman 1961), which can be further decomposed into a systematic part, the bias due to interviewers (when respondents alter answers either because of the presence of interviewers or their observable traits), and a random component, interviewer variance. ...
... Second, identifying the questions associated with large interviewer variance mode effects can inform how interviewer variance is generated and thus might be reduced. For example, researchers show that attitudinal, sensitive, ambiguous, complex, and open-ended questions are generally more vulnerable to interviewer effects (Schaeffer, Dykema, and Maynard 2010), as those questions introduce more opportunities for the interviewer to help the respondents (West and Blom 2017). If sensitive questions only present a large interviewer effect in FTF but not in TEL, that may suggest the questions bring a burden to field interviewers. ...
This study examines whether interviewer variances remain consistent across different modes in mixed-mode studies, using data from two distinct designs. In the first design, when interviewers are responsible for either face-to-face or telephone mode, we examine whether there are mode differences in interviewer variances for 1) sensitive political questions, 2) international items, 3) and item missing indicators on international items, using the Arab Barometer wave 6 Jordan data. In the second design, we draw on Health and Retirement Study (HRS) 2016 core survey data to examine the question on three topics when interviewers are responsible for both modes. The topics cover 1) the CESD depression scale, 2) interviewer observations, and 3) the physical activity scale. To account for the lack of interpenetrated designs in both data sources, we include respondent-level covariates in our models. We find significant differences in interviewer variances on one item (twelve items in total) in the Arab Barometer study; whereas for HRS, the results are three out of eighteen. Overall, we find the magnitude of the interviewer variances larger in FTF than TEL on sensitive items. We conduct simulations to understand the power to detect mode effects in the typically modest interviewer sample sizes.
... Subsequently, we turn to the cross-country analysis of interviewer effects on module duration. Interviewers play a large part in survey implementation, with poten-tial effects on how respondents answer questions, nonresponse, measurement errors, and interview length [19,20,21,22]. Regarding interviewer effects on interview length in particular, previous research has focused exclusively on high-income settings [12]. ...
... Interviewers can contribute to the variability of respondents' answers, non-response/survey participation, measurement error or bias, and interview length. West and Blom [19] summarize findings concerning interviewer effects, focusing on high-income countries. In low-and middleincome countries, interviewer characteristics such as gender [20,21] and ethnicity [24] have been found to affect responses. ...
Over the past decade, national statistical offices in low- and middle-income countries have increasingly transitioned to computer-assisted personal interviewing and computer-assisted telephone interviewing for the implementation of household surveys. The byproducts of these types of data collection are survey paradata, which can unlock objective, module- and question-specific, actionable insights on survey respondent burden, survey costs, and interviewer effects – all of which have been understudied in low- and middle-income contexts. This study uses paradata generated by Survey Solutions, a computer-assisted personal interviewing platform used in recent national household surveys implemented by the national statistical offices of Cambodia, Ethiopia, and Tanzania. Across countries, the average household interview, based on a socioeconomic household questionnaire, ranges from 82 to 120 minutes, while the average interview with an adult household member, based on a multi-topic individual questionnaire, takes between 13 to 25 minutes. The paper further provides guidelines on the use of paradata for module-level analysis to aid in operational survey decisions, such as using interview length to estimate unit cost for budgeting purposes as well as understanding interviewer effects using a multilevel model. Our findings, particularly by module, point to where additional interviewer training, fieldwork supervision, and data quality monitoring may be needed in future surveys.
... Subjective questions such as those related to perceptions and attitudes have larger interviewer effects compared to factual questions such as respondent demographics. Finally, lengthy questions are also associated with larger interviewer effects (Mangione, Fowler, and Louis 1992;Pickery and Loosveldt 2001;Schnell and Kreuter 2005;West and Blom 2017). ...
... 2) Type of question (factual versus non-factual, or attitudinal/subjective): Questions that are subjective, such as attitudinal or opinion-related questions, may be more prone to interviewer effects compared to objective questions such as factual and demographic questions (West and Blom 2017). In responding to subjective questions, respondents might be influenced by interviewer characteristics and/or the environment to respond in a certain way compared to factual questions. ...
This DHS methodological report is an extension of previous work done in examining the effects of interviewer characteristics on data quality in DHS surveys (MR24). In this report we use multilevel models to estimate interviewer effects in
DHS surveys, while accounting for the structure of the interviewer assignments and the characteristics of both respondents and interviewers.
Based on data from 24 recent DHS surveys and more than 100 questions
from the Woman’s Questionnaire in each survey, this report examines interviewer effects across countries and across different characteristics of questions, such as length (longer versus shorter questions), sensitivity (questions on sensitive topics versus questions on non-sensitive topics), social desirability (questions prone
to social desirability bias versus questions not prone to social desirability bias), complexity and/or difficulty (complex or difficult questions versus questions that are not complex or difficult), and question type (whether the information collected by the question was factual or non-factual).
Long questions, non-factual questions, and questions on complex or difficult topics were associated with larger interviewer effects compared to the shorter questions, factual questions, and questions on less complex or difficult topics. These differences were consistent across most surveys.
The analysis in this report can be extended to additional questions and surveys in the future. Results from these analyses can improve the quality of interviews and data collected by improving training for interviewers before fieldwork and monitoring interviewer performance during fieldwork.
... It is a known phenomenon that the sociodemographic characteristics of interviewers may have an impact on response rates (Haunberger, 2010;Durrant & D'Arrigo, 2014;Vercruyssen et al., 2017;West & Blom, 2017;Holbrook, Johnson, & Krysan, 2019). Studies into these role-independent interviewer effects are conceptually linked to the theory of like (Groves, Cialdini, & Couper, 1992) and the notion of social distance (Williams, 1964). ...
... conditions (Durrant & D'Arrigo, 2014;West & Blom, 2017;Holbrook, Johnson, & Krysan, 2019) and although we had previous experience with respect to interviewers' effects in our own work (Giordano, Alarcon-Henriquez, & Jacobs, 2018), we could only partly anticipate the importance of such effects and the consequences for both our results and our team. ...
... Although interviewer effects in surveys are well-established (West and Blom 2017), little is known about the respondent side of the data collection. Interviewers, who have principal roles in surveys (e.g., contacting sample units, asking questions, and recording answers), are also responsible for maintaining the motivation of respondents during the interviews (Schaeffer et al. 2010;West and Blom 2017). ...
... Although interviewer effects in surveys are well-established (West and Blom 2017), little is known about the respondent side of the data collection. Interviewers, who have principal roles in surveys (e.g., contacting sample units, asking questions, and recording answers), are also responsible for maintaining the motivation of respondents during the interviews (Schaeffer et al. 2010;West and Blom 2017). Researchers have focused on the relationship between motivation and the resulting quality of estimates in recent years (Groves et al. 2004;Blom and Korbmacher 2013;Schaeffer et al. 2010), although a few studies touched on the determinants of respondent motivation in earlier times (Dijkstra 1987). ...
Survey quality would be enhanced if respondents willingly and accurately provided their responses. This seems feasible in ideal interview settings where respondents fully understand questions and provide truthful responses. Therefore, survey outcomes may be impacted by respondents’ feelings during the interview. The purpose of this study is to determine whether there is any relationship between respondent motivation, as operationalized by willingness and ability, and the item-nonresponse level for split-ballot designed questions. The 9th round of the European Social Survey (ESS9), a cross-national survey conducted across Europe, is the source of the data. This study assesses item-nonresponse using “don’t know” and “no answer” response choices for a set of questions designed with the split-ballot technique, which has not been dealt with much. In addition to the percentage distribution of item-nonresponse levels, the correlation analysis is used to understand the relationship between respondent motivation and item-nonresponse descriptively. Multivariable analyses use multiple linear regression modeling to explore the impact of respondent motivation on item-nonresponse, controlling for respondent and interviewer characteristics. Furthermore, bivariate relationships between outcome variable and covariates in the models were assessed using design-adjusted Wald-F tests. The findings pointed out that respondent motivation and item-nonresponse have a significant, negative, and moderate association. According to the complex sample design adjusted estimates of the statistical models, it may be able to reduce item-nonresponse with the increased motivation of respondents. This study offers several recommendations for questionnaire design, interviewing techniques, and interviewer evaluations for each respondent at the end.
... Взаимодействие интервьюера и респондента, а также влияние этого взаимо действия на результаты опросов широко обсуждаются в литературе, посвященной методологии социологических исследований [Журавлева, 2009;Мягков, Журав лева, 2004;West, Blom, 2017]. Искажения результатов, вызванные эффектами интервьюера, особенно проблематично оценить в силу того, что они не предпо лагают какихлибо конкретных техник контроля. ...
... Изучение роли эффекта интервьюера занимает важное место в исследованиях, посвященных модели общей ошибки исследования. Эффекты интервьюера везде сущи: они могут проявляться в ошибке покрытия, в вероятности отказа от ответа, ошибке измерения и ошибке обработки [West, Blom, 2017]. В частности, даваемые респондентами ответы связаны с установками, поведением, опытом и ожидания ми интервьюеров [Blom, Korbmacher, 2013]. ...
В статье рассматривается связь характеристик интервьюера и ответов на гендерно окрашенные вопросы в странах с разным уровнем гендерного равенства. В научной литературе можно выделить два основных подхода к объяснению эффекта интервьюера: модель социальной атрибуции и модель социальной дистанции. Согласно первому подходу, респонденты меняют ответы в зависимости от характеристик интервьюера; второй же подход предполагает также учет разницы в социальных характеристиках респондента и интервьюера. В представленном исследовании рассмотрены эффекты пола интервьюера (модель социальной атрибуции) и разницы в возрасте респондента и интервьюера (модель социальной дистанции) на оценку идеального возраста для различных событий в жизни женщин и мужчин. На основе данных девятого раунда Европейского социального исследования показано, что разница в оценке идеального возраста наступления демографических событий в жизни мужчин и женщин оказывается больше, если опрос проводится женщиной. В то же время данный эффект менее выражен в странах с высоким уровнем гендерного равенства. Кроме того, возраст, указываемый в качестве идеального для жизненных событий, оказывается меньше при большей разнице в возрасте респондента и интервьюера, и данный эффект остается схожим во всех исследуемых странах. Результаты исследования свидетельствуют о необходимости учета характеристик интервьюера при анализе опросных данных. Благодарность. Статья подготовлена в рамках Программы фундаментальных исследований Национального исследовательского университета «Высшая школа экономики» (НИУ ВШЭ).
... Subsequently, we turn to the cross-country analysis of interviewer effects on module duration. Interviewers play a large part in survey implementation, with potential effects on how respondents answer questions, non-response, measurement errors, and interview length (West and Blom 2017;Flores-Macias et al. 2008;Vollmer et al. 2021;Maio and Fiala 2020). Regarding interviewer effects on interview length, past research has focused exclusively on high-income settings (Couper and Kreuter, 2013). ...
... Interviewers can contribute to the variability of respondents' answers, non-response/survey participation, measurement error or bias, and interview length. West and Blom (2017) summarizes findings concerning interviewer effects, focusing on high-income countries. In low-and middle-income countries, interviewer characteristics such as gender (Flores-Macias et al. 2008;Vollmer et al. 2021) and ethnicity (Adida et al. 2016) have been found to affect responses. ...
... [62,63]). Other studies have investigated interviewer effects using a different method, namely hierarchical regression models [64][65][66][67][68]. One of these studies showed that interviewers have a significant impact on responses, with interviewer effects even outweighing respondent differences [67]. ...
Background
Self-efficacy refers to an individual's belief in their ability to accomplish specific tasks and achieve goals, and plays an essential role in achieving positive outcomes in a wide range of domains. Central to the measurement of any form of self-efficacy is the assessment without bias, also in case of an interview situation.
Methods
Outpatients with macular edema, an eye disease, participated in this questionnaire-based cross-sectional study. The study assessed self-efficacy using the General Self-Efficacy Scale (GSE) in German. Interviewers read questionnaires aloud to patients. Differential item functioning (DIF) was investigated using likelihood-ratio χ2 tests for interviewer, sex, age, education, working status, income, diagnosis, and health-status.
Results
The analysis included N = 556 patients. Median age was 68.4 (IQR: 62.0 – 76.0) years and mean overall GSE score 32.8 (SD: 4.81). No DIF was detected for interviewer. However, DIF was found in item 1 for education (uniform DIF, NCDIFno degree vs. degree = 0.042; easier with degree vs. none), in item 1 and 3 for income (item 1: non-uniform DIF, NCDIF<€ 1,125 vs.≥€ 1,125≤€ 1,950 = 0.050 / NCDIF< € 1,125 vs.≥€ 1,950 = 0.099; item 3: uniform DIF, NCDIF<€ 1,125 vs.≥€ 1,125—≤€ 1,950 = 0.024 / NCDIF< € 1,125 vs.≥€ 1,950: 0.095; both easier with higher income), in item 2 for working status (uniform DIF, NCDIFretired vs. other = 0.017; easier if working) and in item 3 for sex (non-uniform DIF, NCDIFmale vs. female = 0.043; easier for women in low ability, harder for them from medium ability on).
Conclusions
Given that no DIF was detected concerning interviewers, our findings indicate that an objective assessment of self-efficacy in a face-to-face interview may be feasible, provided that interviewers receive appropriate training. Since DIF effects concerning other patients characteristics found were small, the GSE may provide a relatively bias free way to assess self-efficacy in an interview setting.
... Following this concept of socially desirable reporting, several studies analyze the role of gender-of-interviewer effects (see West and Blom 2017, for an overview). Most of the previous research has focused on normative views, for example about marriage (Liu and Stainback 2013) or women's issues and gender equality (Huddy et al. 1997). ...
Gender differences in the amount of housework performed and the role of social norms in explaining these persistent gaps have received increasing attention from both policymakers and researchers in recent years. However, norms may not only affect the actual division of housework but also potentially influence the reporting behavior in surveys. We study how retrospective responses about time-use in face-to-face interviews are influenced by the gender of the interviewer. Our findings show that women tend to report significantly more hours of housework when interviewed by a woman rather than by a man. This effect is not observable for male respondents, resulting in an interviewer gender gap in the housework gender gap. Exploring the effect in relation to several norm-related characteristics indicates that social norms play an important role in the reporting of housework hours. Therefore, gender gap estimates based on face-to-face interviews should be interpreted with great caution.
... Majority of the respondents agreed that mental illness is violent in nature and that those who suffer from it should be isolated because they pose a threat to society, that the main cause of mental illness is lack of self-discipline and are mostly of the opinion that it takes longer to recover from mental illness than other diseases. Nevertheless, the participant's answers are based on interviewer-provided variables, which may introduce bias (West & Blom, 2017) and lack the depth of a genuine viewpoint. ...
Background: Public perception of mental illness in Nigeria has a greater impact on the attitude portrayed towards mental disorders. Adequate management of mental illness within the population is hindered by poor awareness and inadequate mental health services. Aim: This study explores existing studies on mental illness, recognized the characteristics that are pertinent to this research to discuss factors that shapes public perception and management of mental illness in Nigeria. Methods: This is a literature review which was performed on studies that covers the years 2014 to 2024 using a sample of 118 journal articles chosen from a pool of 258 journal articles obtained from search of electronic databases which include Scopus, Medline, CINAHL and PubMed which were further screened for full text using inclusion and exclusion criteria to obtained 9 journal articles used for the review. Conclusion: The study concluded that without government creation of public awareness of mental illness and provision of accessible mental health services, the management of mental disorders will continue to be challenging to individual experiencing mental illness.
... All participants remained anonymous which helped to ensure responses that were given were true to the social style and personality of that participant, thus reducing bias through the 'Interviewer Effect' (Marsden and Wright, 2010). The 'Interviewer Effect' is where the interviewee responds with answers that they believe to be the desired response from the question, rather than their own opinion (West and Blom, 2016). The results of the questionnaire were statistically analysed to showcase the relevant findings through a visual representation thus highlighting the key findings of the study. ...
The UK construction industry is vital to the UK's economy. The industry is a significant consumer of resources and a producer of carbon emissions, waste, and pollution. More so now than ever, it is vital that the construction industry adopts sustainability practices. Literature has outlined the economy, environment, and society as being key elements needed to achieve sustainable development (SD). The construction industry impacts largely on all three of these elements and therefore has a critical role in this endeavour. The UK Government has continued to target the industry by promoting the delivery of sustainable construction (SC) practices through a range of policies and guidance documents. However, considering its history in past failures paired with the ambitious targets which the UK Government is asking of the construction industry, there is a growing concern about whether the UK construction industry will ever actually operate sustainably. The aim of this study was to establish how effective the UK construction industry is in achieving SD whilst reviewing the role of the UK Government. This study utilised a ‘mixed method’ approach to outline both the drivers and barriers impacting the implementation of SC practices within construction-related organisations. As a result, the study has determined the effective approaches used, whilst suggesting recommendations to the Government, the construction industry, and academics. The findings of the research emphasise a need for the Government and construction industry to deliver a more effective collaborative approach towards the sustainability agenda within the UK. Documents need to be well understood and transparency in guidance is needed for the construction industry to adhere to Government regulations. The Government should focus on the cost benefits that can come to organisations that adopt sustainable practices. After all, no business will function without its generation of profits. Cost has been a theme throughout this study and has proven to be the biggest factor that determines the adoption of SC practices. It has been made clear that if SC practices are seen as a risky investment, then construction organisations will choose to avoid adopting them.
... The researchers conducted interviews at this time to allow students to reflect on their field course experiences while also preventing the influence of retrospective recall bias (Berney and Blane, 1997). Even still, it is possible that other forms of biases, such as social desirability bias (Nederhof, 1985) or interviewer effects (West and Blom, 2017), could have impacted how students talked about their field biology course experiences and the identities they disclosed. ...
Field biology courses can be formative learning experiences that develop students’ science identities. Yet, they can also pose challenges to students that may disaffirm their science identities—especially to those who identify with underrepresented, excluded, and minoritized groups. It is largely unknown how students’ social (e.g., gender) and personal (e.g., where they grew up) identities intersect with their science identities in field biology courses. Therefore, we used the Expanded Model of Science Identity to determine: 1) the factors that influence students’ science, social, and personal identities; and 2) whether and how these identities intersect in field biology courses. Using a card sorting task during semistructured interviews, we found variation in science identities with which students identified, mediated by social factors (e.g., social comparison). These social factors influenced how students’ social and personal identities intersected with their science identities. Intersections between students’ social and science identities were also facilitated by structural factors (e.g., privilege, lack of representation) that perpetuate inequities in field biology. Based on our findings, we offer suggestions to support welcoming, equitable, and inclusive field biology education that nurtures the science identities of all students.
... This is consistent with findings in the international literature, whereby women are generally more likely to be enrolled in surveys than men, especially phone-based surveys. Studies suggest that women may be more accessible during survey times and potentially more willing to participate in survey research [60,61]. Gender imbalances were also identified in previous studies about cats, where 78% to 85% of respondents identified as female [9,62,63]. ...
Simple Summary
Management of stray cats in urban areas is an ongoing challenge in Australia, and many are euthanized, particularly in disadvantaged regions. The Australian Pet Welfare Foundation aimed to assess the impact of a free cat sterilization program in Ipswich, Queensland. Before that intervention, a situational analysis was conducted to evaluate cat and dog ownership behaviours. In a sample of 343 residents from the area, 35% owned cats, and 3% fed strays. Cats were mainly obtained from family or friends and shelters, while 53% of respondents owned dogs, mostly sourced from breeders and family acquaintances. A higher percentage of owned cats (91%; but only 74% for those aged 4 to <12 months) were sterilized compared to owned dogs (78%). Among cat owners, 51% contained their cats all the time and 18% at night. Our findings suggest that community-based sterilization programs targeting both owned and semi-owned cats, and assisting semi-owners in becoming owners, would assist in reducing unplanned litters and the stray cat population. It is also recommended that assistance with cat containment be provided where vulnerable native species are present in urban and peri-urban areas. These insights are crucial for developing effective policies aligned with One Welfare principles.
Abstract
Managing stray cats in urban areas is an ongoing challenge, and in Australia, many are euthanized. Most stray cats are from disadvantaged areas and are under 1 year of age. The Australian Pet Welfare Foundation intended to assess the impact of a free cat sterilization program in an area with high shelter intake of cats in the city of Ipswich, Queensland. The aims of this pre-intervention study were to undertake a situational analysis of cat ownership, semi-ownership and cat caring behaviours, and compare those in the same demographic with dog ownership and caring behaviours relating to sterilization rates, to provide a basis against which to assess the program’s effectiveness. In a sample of 343 participants from that area, 35% owned cats and 3% fed stray cats. Cats were predominantly obtained from family or friends (31%) and shelters (20%). More respondents owned dogs (53%), which were most often sourced from breeders (36%) and family acquaintances (24%). More owned cats than owned dogs were sterilized (91% versus 78%). However, only 74% of cats aged 4 to <12 months were sterilized. Cat containment practices varied, with 51% of owners containing their cat(s) at all times, and a further 18% doing so at night. These results suggest the need for community-based programs that focus on sterilizing owned and semi-owned cats, and assisting semi-owners in becoming full owners to reduce stray cat populations and associated negative impacts. This includes assistance with cat containment where vulnerable native species are present. Public understanding of the causes and effective solutions for free-roaming cats, alongside legislative changes, are required to facilitate these efforts. Assistive programs aligned with One Welfare principles are expected to benefit the wellbeing of animals, humans and their environments.
... Survey methodology literature thoroughly discusses interviewer effects, which may affect coverage error, response rate, measurement error as well as paradata. Interviewer personality traits, such as positive attitude, optimism and self-confidence were found to increase respondent cooperation (see West & Blom, 2017). Some issues experienced at the Hungarian Central Statistical Office (HCSO) are, presumably, related to these factors: high administrative burden of field interviewers, high risk of burnout and high attrition rates of field interviewer staff. ...
Survey methodology literature thoroughly discusses interviewer effects, which may affect coverage error, response rate, measurement error as well as paradata. Interviewer traits, such as positive attitude, optimism and self-confidence were found to increase respondent cooperation. To improve these, issues of high administrative burden of field interviewers, high risk of burnout and high fluctuation rates of interviewers has to be addressed at the Hungarian Central Statistical Office. Funded by the LFS 2024 module grant, our early stage project focuses on developing a dedicated smartphone application for fieldwork-related tasks in the Labour Force Survey (excluding questionnaire administration). It is expected to improve fieldwork quality and enable real-time monitoring and timely interventions. Three main modules will be available for interviewers. The first will include fieldwork assistance components, such as smart case management, visit/call planner, route planning, intuitive nonresponse administration, case-specific notebook, visit history, etc. in order to reduce interviewer burden and optimise performance. The second module will be a contact and assistance centre: live chat with supervisors, interviewer forum, technical assistance surface, panic button, etc. for prompt feedback and support. The third will be a complex education and motivation module: it will create a gamified environment for the entire fieldwork experience (except questionnaire administration), including an education refresher component to keep interviewers’ knowledge up to date. The gamified environment will give interviewers feedback, rewards and positive reinforcements to provide a better overall experience and increase motivation and attitudes. Gamified tasks and challenges may also facilitate better fieldwork in neighbourhoods where interviewer performance is usually less efficient, such as the fringes of settlements, very high/very low status areas, etc.
Apart from these, additional benefits over similar systems on larger devices are the ease of use and inconspicuousness. In contrast with having to handle a tablet or laptop in open space, using a phone on the street can go virtually unnoticeable. Interviewers can thus “keep a low profile” and are less likely to encounter adverse reactions from locals, especially in less safe areas.
For fieldwork supervision, monitoring and management, the application provides more granular data on what happens in the field, opportunity for more efficient supervision and targeted interventions or development of tailored visit protocols to assist interviewers.
Major phases of the project:
1. Needs assessment: individual and group consultations with interviewers and supervisors (currently ongoing).
2. Detailed specification of functions and development of a beta application.
3. 1st round live fieldwork test, followed by feedback consultations with interviewers and supervisors.
4. Application updates, bug fixes, etc.
5. 2nd round live fieldwork test.
6. Analysis of the results, reporting.
A brief overview of the project and planned application components will be presented at the workshop, together with the results of the needs assessment phase. Given the early stage, we would appreciate attendants’ feedback and possible experiences with such endeavours, e.g. gamification in interviewer training and fieldwork, especially in the Labour Force Survey. Best practices of and unexpected results in interviewer motivation schemes for staff retention and to prevent burnout, with a particular focus on novel solutions would also be useful.
... A challenge with estimating interviewer effects on the basis of survey data is the lack of a proper design wherein respondents are randomly assigned to interviewers. Such a design is untenable in large-scale surveys due to cost constraints or other logistic reasons, such as dialect requirements (West and Blom 2016). In reality, interviewers are often assigned based on geographic location, making it hard to distinguish between interviewer effects and area effects. ...
Interviewer ratings of respondents’ physical appearance have been collected in several major social surveys. While researchers have made good use of such ratings data in substantive studies, empirical evidence on their measurement properties is rather limited. This study evaluates two potential threats to the quality of interviewer ratings of physical appearance: interviewer effects and halo effects. Using data from the China Family Panel Studies, we show large interviewer effects on interviewer ratings of respondents’ physical appearance based on cross-classified models. We also provide possible evidence for halo effects based on high correlations between physical appearance ratings and other theoretically distinct constructs, after controlling for interviewer effects. However, we find support for convergent and discriminant validity of physical appearance ratings when both interviewer effects and halo effects are controlled for. Empirical studies using interviewer observation data should take into account interviewer effects and halo effects when possible or at least discuss their potential impact on the substantive findings.
... The use of the Delphi technique offers advantages over workshop or focus group methods as it helps prevent single participant response dominance. Moreover, compared to the interview method, it eliminates the potential impact of interviewer effects (Nyumba et al., 2018;West & Blom, 2017). Secondly, in comparison with a single survey study, the Delphi technique allows for an iterative approach with interspersed controlled feedback to each participant in between survey rounds (Drumm et al., 2022). ...
The many challenges faced by instructors in distance education have been extensively documented in current literature. Nevertheless, the relative importance of such challenges has not been researched to the same extent. As instructors and institutions face limitations in terms of time and budget, the ability to effectively prioritise challenges becomes increasingly vital. This paper aims to investigate the relative importance of various distance education challenges to instructors. For this purpose, a ranking-type Delphi study of 90 instructors was conducted at a Swedish higher education institution of technology. In total 26 unique challenges were identified, out of which the top 13 challenges were further ranked by instructors. The ranking revealed that pedagogical challenges were regarded as being of highest relative importance, whereas technological and organisational challenges were of lesser significance. Instructors rated the following challenges as the most significant: lack of student responsiveness, diminished collaboration due to social distancing among students, substantial upfront course design investments by instructors, and limitations in instructor accessibility. These rankings were found to be largely consistent across different instructor characteristics. The findings of the study indicate the importance of investing in faculty training and support functions within higher education institutions to ensure the quality of distance education.
... Its use offers advantages over workshop or focus-group methods as it helps prevent single-participant response dominance because respondents are anonymous. Compared with the interview method, it eliminates the potential impact of interviewer effects (West and Blom 2017). Compared with a single-survey study, it represents an iterative approach with interspersed controlled feedback for each participant between survey rounds. ...
This paper presents the results of a ranking-type Delphi study on the critical risk factors for the adoption of an electric road system (e-road) for trucks on the main motorways in Sweden. The investment cost of such a system is high, necessitating an upfront evaluation of the adoption risk factors to reduce the likelihood of budget overruns and project delays. Participating Swedish e-road experts (N = 52) from the public sector, private sector, and academia identified 32 unique risk factors, which were divided into five categories. The three most critical risk factors, as ranked by the experts, were 'low expansion rate,' 'low utilization rate,' and 'lengthy public-sector evaluation.' Overall, market and financial risks were ranked as more important than institutional, technological, and sustainability risks. This study has important implications for policymakers in countries considering e-road adoption. ARTICLE HISTORY
... Cet effet a été modélisé par certains auteurs dans le cadre de la Total Survey Error (TSE) framework (West et al., 2013(West et al., , 2018West & Blom, 2017). Les auteurs parlent alors de deux biais qui expliquent cet effet. ...
... Interviewers can also affect respondent behaviour and data quality regardless of survey mode. Evidence of interviewer effects in surveys has been widely documented in high-income country settings (West and Blom, 2016). A recent study by Di Maio and Fiala (2020) found limited interviewer effects for an in-person survey in Uganda, but the literature on interviewer effects in lowincome settings is thin and virtually non-existent for interviewer effects in phone surveys. ...
The Coronavirus disease pandemic has disrupted survey systems globally and especially in low- and middle-income countries. Phone surveys started being implemented at a national scale in many places that previously had limited experience with them. As in-person data collection resumes, the experience gained provides the grounds to reflect on how phone surveys may be incorporated into survey and data systems in low- and middle-income countries. Reviewing evidence and experiences from before and during the pandemic, the paper provides guidance on the scope of and considerations for using phone surveys for agricultural data collection and the integration of phone interviews with in-person data.
... All three of the interviewers identified as White, as did the members of the analytic team. Although none of the interview questions explicitly addressed race or ethnicity, some research suggests that sociodemographic features of interviewers may affect participants' responses (West & Blom, 2017). Third, it should be noted that this study was conducted shortly after the murder of George Floyd (Eichstaedt et al., 2021) and other high-profile attacks on people of color in the U.S. (e.g., Tessler et al., 2020). ...
Objective: This study examined how family factors impacted parents’ attitudes toward integrated behavioral health (IBH) in pediatric primary
care during the COVID-19 pandemic. We hypothesized that COVID-19 impact would predict family functioning challenges, and that pre-existing
familial contextual factors would predict parents’ interest in IBH modalities.
Methods: Parents of children ages 1.5–5 years (N ¼ 301) from five primary care clinics completed a survey with measures assessing familial con�textual factors (income, race and ethnicity, and parents’ childhood adversity), COVID-19 impact on family relationships and wellbeing, family func�tioning (child behavior, parenting self-efficacy, and parent psychological functioning), and parents’ preferences for behavioral support in
primary care. A subsample of parents (n ¼ 23) completed qualitative interviews to provide deeper insights into quantitative relationships.
Results: Higher COVID-19 impact was significantly associated with worse parent mental health and child behavior problems, as well as lower
interest in IBH virtual support options. Overall, lower SES and racial and/or ethnic minority parents both indicated greater interest in IBH modal�ities compared to higher SES and White parents, respectively. Qualitative interviews identified how pandemic stressors led to increases in
parents’ desire for behavioral support from pediatricians, with parents sharing perspectives on the nature of support they desired, including
proactive communication from providers and variety and flexibility in the behavioral supports offered.
Conclusions: Findings have important implications for the provision of behavioral supports for families in primary care, underlying the need to
increase parents’ access to IBH services by proactively providing evidence-based resources and continuing to offer telehealth support.
Keywords: behavior problems; COVID-19; parenting; preschool children; primary care.
Age-group differences play an essential role in social interaction across sub-Saharan Africa. However, the social effects of these differences remain understudied. We hypothesize that age-group differences will affect response patterns and use Afrobarometer data to test this hypothesis. We also explore three mechanisms through which age-group differences may induce response-pattern variation: in-group loyalty, social acquiescence, and social distance. As hypothesized, we find relatively large and statistically significant effects for age-group differences across a variety of questions. Our findings support in-group loyalty and social acquiescence rather than social distance when questions do not address age-related issues directly. However, social distance may play a more important role when questions address age-specific issues. Additionally, we show preliminary evidence that age-group differences induce larger response-pattern variation than coethnicity. Our findings speak to the importance of age in social interaction in Africa and provide important lessons for the survey research community.
This study investigates biases introduced when soliciting customer satisfaction survey respondents to post online reviews on different platforms. We examine review nonresponse bias and measurement bias arising when customers post reviews on TripAdvisor or Google Travel. Our findings indicate that platform heterogeneity significantly impacts both the likelihood of posting a review and the ratings provided, leading to biases. Review nonresponse bias occurs when not all survey respondents post reviews, creating a selection bias. Measurement bias arises when posted ratings differ from survey responses. Our analysis reveals that Google Travel users with negative experiences are less likely to post reviews, but those who do post tend to give lower ratings compared to their survey responses. Conversely, TripAdvisor users show less review nonresponse bias, but their posted ratings are more positive when survey responses are negative. Understanding platform-specific biases can help managers adjust opinion weights to ensure reliability. Academically, this research advances the literature by distinguishing review nonresponse bias from measurement bias and encourages further comparative studies across platforms.
Interviewers have long been identified as a source of error in face-to-face surveys. However, previous studies have typically focused on a single source of interviewer error and single-country cross-sectional surveys. We extend this literature by investigating interviewer errors from multiple dimensions in the Oesterreichische Nationalbank Euro Survey, a cross-national survey conducted annually in 10 Central, Eastern, and Southeastern European countries. Using data from 10 rounds (i.e. 100 country-years), we apply several data quality indicators on various dimensions of interviewer error and investigate country-years with particularly exceptional patterns. To combine the indicators, we use a multivariate tree-based outlier detection method (isolation forest) that flags country-years and interviewers with outlying values and combine it with methods from the interpretable machine learning literature to identify the respective exceptional feature values. Lastly, we document the effects of interviewer errors on the bias and variance of survey estimates. In several instances, our results identify fieldwork institutes and supervisors rather than interviewers as the main source of error.
This indispensable collection provides extensive, yet accessible, coverage of conceptual and practical issues in research design in personality and social psychology. Using numerous examples and clear guidelines, especially for conducting complex statistical analysis, leading experts address specific methods and areas of research to capture a definitive overview of contemporary practice. Updated and expanded, this third edition engages with the most important methodological innovations over the past decade, offering a timely perspective on research practice in the field. To reflect such rapid advances, this volume includes commentary on particularly timely areas of development such as social neuroscience, mobile sensing methods, and innovative statistical applications. Seasoned and early-career researchers alike will find a range of tools, methods, and practices that will help improve their research and develop new conceptual and methodological possibilities. Supplementary online materials are available on Cambridge Core.
This study investigated whether the age of examiner effect, that is, the positive relation between cognitive performance of older people and age of their examiner, relies on stereotype threat. Data from the Survey of Health, Ageing and Retirement in Europe (SHARE), European Social Survey, European Statistical Office (Eurostat) and the United Nations Economic Commission for Europe were crossed. Older people from different countries were tested on verbal short‐ and long‐term recall components of memory, by examiners of different ages. Country‐specific indicators of negative age stereotypes, related to the stereotype threat mechanism, were used. We expected that the age of examiner effect would be stronger in countries with more negative age stereotypes. The results replicated the age of examiner effect, but contrary to our hypothesis, this effect was not moderated by countries’ indicators of negative age stereotypes. Negative age stereotypes across countries, such as experienced discrimination among older people, lack of active ageing index, unemployment ratio and perceived negative competence of older people decrease memory. The results are discussed in relation to stereotype threat theory.
In a time where new research methods are constantly being developed and science is evolving, researchers must continually educate themselves on cutting-edge methods and best practices related to their field. The second of three volumes, this Handbook provides comprehensive and up-to-date coverage of a variety of issues important in developing, designing, and collecting data to produce high-quality research efforts. First, leading scholars from around the world provide an in depth explanation of various advanced methodological techniques. In section two, chapters cover general important methodological considerations across all types of data collection. In the third section, the chapters cover self-report and behavioral measures and their considerations for use. In the fourth section, various psychological measures are covered. The final section of the handbook covers issues that directly concern qualitative data collection approaches. Throughout the book, examples and real-world research efforts from dozens of different disciplines are discussed.
The Indigenous Connectedness Framework (ICF) describes mechanisms that help or hinder Indigenous children’s development of family, community, environmental, intergenerational, and cultural/spiritual connectedness, or belonging, and healthy identity. It has been applied in considerations of child welfare system-involved youth and alumni and studies of Indigenous mental health, but not yet to Indigenous adoptees who may not have experienced child welfare involvement prior to adoption and are reconnecting to their origins as adults. Although various factors influence the extent of (re)connectedness, Indigenous adoptees’ reunification with biological family, community, land/place, ancestors, and culture is generally linked with increases in their senses of connectedness, or belonging, and Indigenous/Tribal identity. To gain initial insights into the suitability of the ICF to describe components of Indigenous adoptee (re)connectedness, this study used a Two-Eyed Seeing approach with Indigenous storytelling and narrative inquiry methodologies to explore experiences and perceptions of reunification with four Indigenous adults who experienced adoption by non-Indigenous families as children and were raised in non-Indigenous communities. Stories about reconnection with family, community, and land/place after adoption, as well as stories of perceived supports for, and challenges of, reunification were obtained during conversational interviews conducted via Zoom or phone and then mapped onto the ICF. Disconnectedness mechanisms, (re)connectedness mechanisms, and themes related to adoptees’ internal worlds and resilience, relational identity development, and contributions to intergenerational connectedness were revealed. Suggested pathways for social work to support (re)connectedness or belonging for Indigenous adoptees and implications for future research are provided.
What determines one's ethnic identity? I argue that contact with a co‐ethnic encourages an enhanced ethnic identity compared to contact with a non‐co‐ethnic. This is most likely to hold where identities are more ethnic and less national, as in many African countries, thus creating a sense of trust among co‐ethnics. To test this claim, I conducted a field experiment with participants from the West African countries of Côte d'Ivoire and Senegal using two designs to test my claims including if enumerators and participants share an ethnic identity established directly with face‐to‐face contact, and indirectly with enumerators employing ethnic sounding names. Social contact in which ethnicity is shared increases identification with one's ethnic group. Not sharing an ethnic identity does not increase the chances of a national identity. Variation by ethnic groups is strongest for members of groups who have recently lost political power or are marginalized. Sharing an ethnic identity affects political views. These findings contribute to our understanding about the potential for contact to affect prejudice and bias, and they raise issues about data collection done with personal contact.
Paradata are widely used in conjunction with surveys, from predicting behavior for targeted interventions, monitoring data quality and interviewer performance, to understanding and correcting biases in the data. We define survey paradata broadly: as nonsubstantive data that relate to the survey and its processes in at least one of three ways—they are produced by survey processes, describe them, or are used to manage and evaluate them. They typically would not exist without the survey. They may be automatically produced (e.g., keystrokes), actively collected (e.g., interviewer observations), or constructed later on (e.g., when a human labeler rates respondent–interviewer rapport by listening to recordings).
First, we review other data types (auxiliary, contextual, and metadata) because their overlaps with paradata can make it difficult to grasp paradata precisely. We discuss paradata definitions, including their weaknesses, arriving at our definition.
Second, we offer an overview of our field’s practice and literature: paradata examples, heterogeneity across paradata types and design options, applications, and challenges.
With paradata a somewhat mature concept in our field, survey methodology, we hope to provide a stimulating, broad introduction to practice and literature in our field, accessible to anyone irrespective of professional background. We hope that this chapter provides a valuable backdrop for the conceptualizations of paradata in other disciplines, as presented in this volume.
Past research indicates interviewer effects lead to an underestimation of network size and higher nonresponse to the “important matters” name generator. Self-administered surveys offer a potential solution, but evidence is mixed and context-specific. We employ a logistic multilevel regression, estimated using a Bayesian Markov Chain Monte Carlo approach, to analyze nonresponse to this name generator from 33 post-electoral surveys across 21 countries in the Comparative National Election Project. We find higher nonresponse in interviewer-administered surveys compared to self-administered surveys, particularly among specific demographic groups. Finally, we discuss the trade-offs in selecting survey modes for collecting ego-network data using this instrument.
We designed and randomly evaluated the impact of a textbooks for self-study scheme in eastern DRC targeting student achievement in primary schools. Students in treatment schools were seven percentage points more likely to pass the national exam, and those who passed obtained higher scores. We also evidence higher scores on a French language test. The effects are primarily driven by student interest in textbooks, frequency of doing homework and motivation to go to school and continue education. Student achievement can thus be improved by intensified and diversified use of existing learning materials in poor and fragile settings.
Although vacancies in the real estate market have always been of key interest to both private and public stakeholders, measuring the impact of vacancy rates on property prices is challenging. This study attempts to quantify the extent of the influence of vacancy rates on the price decline of commercial properties. The dataset used in this study was collected and collated by valuation experts. We attempt to alleviate the bias inherent in the price estimates provided by the surveyors. A Bayesian multilevel estimation model was employed, and the results revealed that while some experts do not show deviations from the standard tendency of a peer group, others do diverge from this norm. Based on these findings, a bias-controlling price decline rate was derived. As many real estate statistics are produced based on survey data compiled by experts, the approach adopted in this study is expected to be applied to various real estate practices to mitigate the inherent and inevitable surveyor bias.
Introduction
Causes of deaths often go unrecorded in lower income countries, yet this information is critical. Verbal autopsy is a questionnaire interview with a family member or caregiver to elicit the symptoms and circumstances preceding a death and assign a probable cause. The social and cultural aspects of verbal autopsy have gotten less attention than the technical aspects and have not been widely explored in South and Southeast Asia settings.
Methods
Between October 2021 and March 2023, prior to implementing a verbal autopsy study at rural sites in Bangladesh, Cambodia, Laos, Myanmar and Thailand, focus group discussions were conducted with village heads, religious leaders and community members from varied demographic backgrounds. Thematic analysis elucidated customs and traditional views surrounding death to understand local ethnocultural sensitivities.
Results
We found that death rituals varied greatly among religions, ethnicities and by socioeconomic status. Mourning periods were reported to last 3–100 days and related to the cause of death, age and how close the deceased person was to the family. Participants advised that interviews should happen after mourning periods to avoid emotional distress, but not long after so as to avoid recall bias. Interviewers should be introduced to respondents by a trusted local person. To provide reassurance and confidentiality, a family’s residence is the preferred interview location. Interview questions require careful local language translation, and community sensitisation is important before data collection.
Conclusion
Verbal autopsy is acceptable across a wide range of cultural settings in Southeast Asia, provided that local norms are preidentified and followed.
Interviewers’ postinterview evaluations of respondents’ performance (IEPs) are paradata, used to describe the quality of the data obtained from respondents. IEPs are driven by a combination of factors, including respondents’ and interviewers’ sociodemographic characteristics and what actually transpires during the interview. However, relatively few studies examine how IEPs are associated with features of the response process, including facets of the interviewer-respondent interaction and patterns of responding that index data quality. We examine whether features of the response process—various respondents’ behaviors and response quality indicators—are associated with IEPs in a survey with a diverse set of respondents focused on barriers and facilitators to participating in medical research. We also examine whether there are differences in IEPs across respondents’ and interviewers’ sociodemographic characteristics. Our results show that both respondents’ behaviors and response quality indicators predict IEPs, indicating that IEPs reflect what transpires in the interview. In addition, interviewers appear to approach the task of evaluating respondents with differing frameworks, as evidenced by the variation in IEPs attributable to interviewers and associations between IEPs and interviewers’ gender. Further, IEPs were associated with respondents’ education and ethnoracial identity, net of respondents’ behaviors, response quality indicators, and sociodemographic characteristics of respondents and interviewers. Future research should continue to build on studies that examine the correlates of IEPs to better inform whether, when, and how to use IEPs as paradata about the quality of the data obtained.
The ‘My Choice of Care’ project is the result of a collaboration with the Australian GLBTIQA+ Multicultural Council (AGMC). The aim of the project was to improve awareness among LGBTIQA+ individuals from multicultural multifaith (MCMF) backgrounds of their rights and options and create strategies of equity and inclusion among palliative care service providers. This report provides an overview of the project design and outcomes, research findings and recommendations.
Dyadic surveys aim to interview pairs of respondents, such as partners in a relationship. In dyadic surveys, it is often necessary to obtain the anchors’ consent to contact their partners and invite them to a survey. If the survey is operated in self-administered modes, no interviewer is present to improve the consent rate, for example, by providing convincing arguments and additional information. To overcome the challenges posed by self-administered modes for dyadic surveys and to improve consent rates, it is important to identify aspects that positively influence the likelihood of anchors giving consent to contact their partners. Ideally, these aspects are in the hands of the researchers, such as the survey design and aspects of the questionnaire. Thus, in this study, we analyzed the relationship between anchors’ survey experience and their willingness to consent to surveying their partners in self-administered modes. Based on data from the German Family Demography Panel Study (FReDA), we found that the anchors’ perceptions of the questionnaire as “interesting” or “too personal” were related to consent rates. These relationships were consistent across different survey modes and devices. Effects of other aspects of the questionnaire, such as “important for science” and “diverse” varied between modes and devices. We concluded with practical recommendations for survey research and an outlook for future research.
* This article belongs to a special issue on “Family Research and Demographic Analysis – New Insights from the German Family Demography Panel Study (FReDA)”.
Interviewers are the first in line when it comes to data collection. Therefore, it is important
that they perform their tasks diligently, so that the data they collect are comparable and that errors are minimized. This paper analyzes how interviewers conducted interviews for the Programme for the International Assessment of Adult Competencies (PIAAC) and which kinds of mistakes they made. We approached these questions with audio interview recordings collected during the fieldwork of PIAAC in Germany (carried out in 2011/2012), as well as with an interviewer survey conducted with the German PIAAC interviewers. First,
we introduce the data and the coding scheme used to evaluate interviewers’ behavior with
audio recordings. Subsequently, we describe the interviewers’ actual behavior with regard
to standardized interviewing techniques and investigate whether interviewer characteristics are associated with data quality. Our results demonstrate that interviewers do deviate from the expected behavior in all the aspects we examined. However, we identified only few associations with interviewers’ background characteristics.
Because survey response rates are consistently declining worldwide, survey researchers strive to obtain as much auxiliary information on sampled units as possible. Surveys using in-person interviewing often request that interviewers collect observations on key features of all sampled units, given that interviewers are the eyes and ears of the survey organization. Unfortunately, these observations are prone to error, which decreases the effectiveness of nonresponse adjustments based on the observations. No studies have investigated the strategies being used by interviewers tasked with making these observations, or examined whether certain strategies improve observation accuracy. This study is the first to examine the associations of observational strategies used by survey interviewers with the accuracy of observations collected by those interviewers. A qualitative analysis followed by multilevel models of observation accuracy show that focusing on relevant correlates of the feature being observed and considering a diversity of cues are associated with increased observation accuracy.
Social desirability is generally thought to underlie the propensity for survey respondents to tailor their answers to what they think would satisfy or please the interviewer. While this may in fact be the underlying motivation, especially on attitudinal and opinion questions, social desirability does not seem to be an adequate explanation for interviewer effects on factual questions. Borrowing from the social psychology literature on stereotype threat, we test an alternative account of the race-of-interviewer effects. Stereotype threat maintains that the pressure to disconfirm and to avoid being judged by negative and potentially degrading stereotypes interferes with the processing of information. We argue that the survey context contains many parallels to a testing environment in which stereotype threat might alter responses to factual questions. Through a series of framing experiments in a public opinion survey and the reliance on the sensitivity to the race of the interviewer, our results are consistent with expectations based on a theory of "stereotype threat." African American respondents to a battery of questions about political knowledge get fewer answers right when interviewed by a white interviewer than when interviewed by an African American interviewer. The observed differences in performance on the political knowledge questions cannot be accounted for by differences in the educational background or gender of the respondents.
Paradata related to reasons for refusals in face-to-face surveys can be useful for research on refusers and on refusal rates. This article addresses the measurement quality of reasons for refusals. Multilevel multinomial models were used to examine the effects of interviewers on the collection of reasons for refusals in a survey on the German population, the General German Social Survey (ALLBUS 2008). In this survey, reasons for refusals were collected by interviewers, predominantly in an open-ended answer format. We categorized reasons for refusals - our dependent variable - through content analysis. The results of the subsequent multilevel analysis show that interviewers' gender, age, and workload are related to the collection of some reasons for refusals. At the same time, we obtained high unexplained interviewer variance, which indicates that interviewers tend to report certain reasons for refusals. Further research should investigate how to reduce the effects of interviewers on collection of paradata.
There is evidence that survey interviewers may be tempted to manipulate answers to filter questions in a way that minimizes the number of follow-up questions. This becomes relevant when ego-centered network data are collected. The reported network size has a huge impact on interview duration if multiple questions on each alter are triggered. We analyze interviewer effects on a network-size question in the mixed-mode survey “Panel Study ‘Labour Market and Social Security’” (PASS), where interviewers could skip up to 15 follow-up questions by generating small networks. Applying multilevel models, we find almost no interviewer effects in CATI mode, where interviewers are paid by the hour and frequently supervised. In CAPI, however, where interviewers are paid by case and no close supervision is possible, we find strong interviewer effects on network size. As the area-specific network size is known from telephone mode, where allocation to interviewers is random, interviewer and area effects can be separated. Furthermore, a difference-in-difference analysis reveals the negative effect of introducing the follow-up questions in Wave 3 on CAPI network size. Attempting to explain interviewer effects we neither find significant main effects of experience within a wave, nor significantly different slopes between interviewers.
Record linkage is becoming more important as survey budgets are tightening while at the same time demands for more statistical information are rising. Not all respondents consent to linking their survey answers to administrative records, threatening inferences made from linked data sets. So far, several studies have identified respondent-level attributes that are correlated with the likelihood of providing consent (e.g., age, education), but these factors are outside the control of the survey designer. In the present study three factors that are under the control of the survey designer are evaluated to assess whether they impact respondents' likelihood of linkage consent: 1) the wording of the consent question; 2) the placement of the consent question and; 3) interviewer attributes (e.g., attitudes toward data sharing and consent, experience, expectations). Data from an experiment were used to assess the impact of the first two and data from an interviewer survey that was administered prior to the start of data collection are used to examine the third. The results show that in a telephone setting: 1) indicating time savings in the wording of the consent question had no effect on the consent rate; 2) placement of the consent question at the beginning of the questionnaire achieved a higher consent rate than at the end and; 3) interviewers' who themselves would be willing to consent to data linkage requests were more likely to obtain linkage consent from respondents.
The half-open interval procedure seems to offer an inexpensive method of reducing undercoverage in housing unit frames during data collection. Interviewers check the areas near their selected cases, and, if they find any units missing from the frame, give them a chance of selection. However, the effectiveness of the method in the field has not been tested. This paper reviews how the procedure should work and presents evidence from two surveys about its performance in practice. We show experimentally that the procedure often fails to reduce undercoverage and can introduce overcoverage. We conclude with thoughts about the appropriate role for the half-open interval procedure in household surveys in the future.
When developing household surveys, much emphasis is understandably placed on developing survey instruments that can elicit accurate and comparable responses. In order to ensure that carefully crafted questions are not undermined by ‘interviewer effects’, standardised interviewing tends to be utilised in preference to conversational techniques. However, by drawing on a behaviour coding analysis of survey paradata arising from the 2012 UK Poverty and Social Exclusion Survey we show that in practice standardised survey interviewing often involves extensive unscripted conversation between the interviewer and the respondent. Whilst these interactions can enhance response accuracy, cooperation and ethicality, unscripted conversations can also be problematic in terms of survey reliability and the ethical conduct of survey interviews, as well as raising more basic epistemological questions concerning the degree of standardisation typically assumed within survey research. We conclude that better training in conversational techniques is necessary, even when applying standardised interviewing methodologies. We also draw out some theoretical implications regarding the usefulness of the qualitative–quantitative dichotomy.
“Rapport” has been used to refer to a range of positive psychological features of an interaction -- including a situated sense of connection or affiliation between interactional partners, comfort, willingness to disclose or share sensitive information, motivation to please, or empathy. Rapport could potentially benefit survey participation and response quality by increasing respondents’ motivation to participate, disclose, or provide accurate information. Rapport could also harm data quality if motivation to ingratiate or affiliate caused respondents to suppress undesirable information. Some previous research suggests that motives elicited when rapport is high conflict with the goals of standardized interviewing.
We examine rapport as an interactional phenomenon, attending to both the content and structure of talk. Using questions about end-of-life planning in the 2003-2005 wave of the Wisconsin Longitudinal Study, we observe that rapport consists of behaviors that can be characterized as dimensions of responsiveness by interviewers and engagement by respondents. We identify and describe types of responsiveness and engagement in selected question-answer sequences and then devise a coding scheme to examine their analytic potential with respect to the criterion of future study participation. Our analysis suggests that responsive and engaged behaviors vary with respect to the goals of standardization—some conflict with these goals, while others complement them.
In quantitative data collection, there is an inherent tension between traditional norms and survey protocols, particularly
in the case of subjective or sensitive questions, which can lead to interviewer effects. This article makes use of a household
survey in Timor-Leste to study these effects. In addition to their age and gender, data were collected from the interviewers
regarding their opinions on the subjective questions before fieldwork. Fixed effects and mixed effects logit models examine
the main effects and interactions between interviewer and respondent characteristics. The article finds interviewer effects
in both subjective and objective data, but the magnitude is stronger for subjective questions. It also finds female respondents
appear more susceptible to influence based on the interviewer’s beliefs.
This study investigates how an onscreen virtual agent's dialog capability and facial animation affect survey respondents' comprehension and engagement in “face-to-face” interviews, using questions from US government surveys whose results have far-reaching impact on national policies. In the study, 73 laboratory participants were randomly assigned to respond in one of four interviewing conditions, in which the virtual agent had either high or low dialog capability (implemented through Wizard of Oz) and high or low facial animation, based on motion capture from a human interviewer. Respondents, whose faces were visible to the Wizard (and videorecorded) during the interviews, answered 12 questions about housing, employment, and purchases on the basis of fictional scenarios designed to allow measurement of comprehension accuracy, defined as the fit between responses and US government definitions. Respondents answered more accurately with the high-dialog-capability agents, requesting clarification more often particularly for ambiguous scenarios; and they generally treated the high-dialog-capability interviewers more socially, looking at the interviewer more and judging high-dialog-capability agents as more personal and less distant. Greater interviewer facial animation did not affect response accuracy, but it led to more displays of engagement—acknowledgments (verbal and visual) and smiles—and to the virtual interviewer's being rated as less natural. The pattern of results suggests that a virtual agent's dialog capability and facial animation differently affect survey respondents' experience of interviews, behavioral displays, and comprehension, and thus the accuracy of their responses. The pattern of results also suggests design considerations for building survey interviewing agents, which may differ depending on the kinds of survey questions (sensitive or not) that are asked.
Examines a fundamental problem for opinion polls and those who use them.
The author applies a hierarchical regression model to analyze the simultaneous effects of Hispanic and Anglo interviewer and respondent ethnicity and gender on response quality in survey research. Although few significant main effects were found, response quality was affected significantly by interaction effects of respondent and interviewer ethnicity and gender. For the most part, ethnically homophilous and gender-heterophilous interviews generate the highest response quality. Both Hispanic and Anglo respondents deferred to an interviewer of a different ethnic background when queried about the interviewer's culture, but not when asked noncultural, albeit sensitive, questions.
Two studies of interviewer bias and variance are reported. One is a controlled experiment which yielded estimates of interviewer variability. The other, in which a controlled experiment was not feasible, yielded indications of a relationship between “success” in collecting income data and interviewer characteristics and attitudes.
Will the investigator's racial characteristics affect responses to a Negro attitude scale when a group administered, anonymous, self-enumerative procedure is employed? Since the questionnaires were group administered, two investigators were used for each group. In a portion of the groups both investigators were white. In the remainder of the groups there was a Negro and a white investigator. The results indicate that socially acceptable answers to the Negro attitude scale are reported with greater frequency when one of the investigators is a Negro. However, this phenomena is more pronounced among certain strata of respondents than among others suggesting that we might profitably view such effects as the result of interaction between investigator and respondent characteristics.
Privacy (or the lack of it) is an important feature of the interview. Researchers rely on interviewers to ensure a private setting. In reality, interviewers are guests in respondents’ homes and might find it difficult to achieve privacy. Thus, a substantial proportion of interviews are conducted in the presence of a third party. We investigate whether there are cultural and interviewer variations in interview privacy and whether respondent characteristics associated with interview privacy vary by culture. We analyzed data from 14 countries. The results highlight the direct and indirect role the country-level characteristics (level of individualism and wealth) play in explaining cultural differences in rates of interview privacy and the significant between-interviewer variation in interview privacy.
Alternative Views on the Role of the ObserverThe Role of the Survey InterviewerDesigns for Measuring Interviewer VarianceInterviewer Effects in Personal Interview SurveysInterviewer Effects in Centralized Telephone SurveysExplaining the Magnitude of Interviewer EffectsSummary of Research on Interviewer VarianceMeasurement of Interviewer Compliance with Training GuidelinesExperiments in Manipulating Interviewer BehaviorSocial Psychological and Sociological Explanations for Response Error Associated with InterviewersSummary
In July 1995, Social and Community Planning Research (SCPR) began a programme of research to take an in-depth look at the role of interviewers in the survey nonresponse „process. The programme is divided into three sub-projects. This article describes the analysis undertaken for Sub-Project 2 which deals mainly with the noncontact component of non response. The analysis is based on call record data from the random half of the 1995/96 Family Resources Survey (FRS) undertaken by SCPR. The particular aim of the analysis is to examine how day of week and time of day of calling affect (a) the probability of contact and (b) the probability of subsequently achieving a productive interview. Combining these estimates with information on costs and survey designs, definitions of efficient calling strategies can, in principle, be developed.
Of all data collection procedures in the social sciences, the survey-interview is most frequently used (cf. Brown & Glimartin, 1969; Wahlke, 1979). Considerable doubt has been expressed, however, concerning the validity and reliability of the information it yields (e.g., Phillips, 1971). The term “response effects” refers to the effects of variables that influence or distort the responses, such as the race of the Interviewer, the social desirability of the response alternatives provided, or the way questions are formulated.
Introduction Data Collection Measures of Reliability and Validity Results Discussion Acknowledgments
Although there is a large body of literature on interviewer gender effect, there are still two areas that merit further research. First, very limited attention has been paid to non-Western settings. Second, previous research has focused on responses to a limited number of survey questions and ignored other broader response behaviors. In this study, we use the Asian Barometer Study data to examine the gender effect on acquiescent response style for 53 agree/disagree questions asked in 11 Asian countries and societies. In general, we find that in six countries/societies, male interviewers elicited more acquiescent responses than female interviewers. We also discover that the interviewer gender effect is similar for male and female interviewees.
The abstract for this document is available on CSA Illumina.To view the Abstract, click the Abstract button above the document title.
The paper analysis focuses on the extent to which there is scope for reducing rates of refusal on large-scale household surveys below current standard levels. Our data consisted of over 300 tape-recorded doorstep interactions, drawn from substantive surveys at two different organisations. Tape-recorded interactions were classified in terms of the degree of reluctance expressed by the respondent and the consequent scope for the interviewer to deploy interpersonal skills and persuasion. Interactions where there was both reluctance and scope for persuasion were classified according to whether they resulted in co-operation or refusal and the interviewer tactics associated with the two types of outcome were compared. Conclusions are drawn about the prevalence of different types of interaction and the scope for reducing refusal rates through training interviewers to use techniques likely to minimise refusals on the doorstep.
Although a small number of studies are available that evaluate the effects of interviewer characteristics in substance use surveys conducted in person, none have done so using information collected via telephone interviews. We address this issue by examining the utility of social attribution and social desirability models for detecting the presence of interviewer effects in a large statewide telephone survey concerned with substance use. The specific outcome variables of interest were reports of lifetime and 18-month composite drug use. Analyses focus on the direct effects of individual interviewer characteristics (to assess social attribution) and a summary measure of interviewer-respondent similarity (to assess social distance) and employ random effects regression models to control for respondent clustering by interviewer. Results are most consistent with a social distance model and suggest that social distance between respondent and interviewer may decrease the probability of respondents reporting substance use behavior.
We evaluated the importance of interviewer and subject effects on cocaine and marijuana use disclosure in a sample of over 3,000 male juvenile arrestees. Analyses evaluated the viability of Social Attribution and Conditional Social Attribution models of interviewer effects. The viability of alternative models was investigated in the context of comparative analyses excluding and including statistical adjustments for the clustering of responses by interviewers. Interviewer effects were more salient in models predicting marijuana disclosure than in models predicting cocaine disclosure. Logistic regression analyses provided support for Social Attribution and Conditional Social Attribution models of interviewer effects. Models suggested large interviewer cluster effects. Cluster adjustment altered interpretation of effects for both cocaine and marijuana. Subject race/ethnicity effects were salient in models predicting disclosure for both drugs, but were especially large in models predicting cocaine disclosure.
Examines the psychological processes involved in answering different types of survey questions. The book proposes a theory about how respondents answer questions in surveys, reviews the relevant psychological and survey literatures, and traces out the implications of the theories and findings for survey practice. Individual chapters cover the comprehension of questions, recall of autobiographical memories, event dating, questions about behavioral frequency, retrieval and judgment for attitude questions, the translation of judgments into responses, special processes relevant to the questions about sensitive topics, and models of data collection. The text is intended for: (1) social psychologists, political scientists, and others who study public opinion or who use data from public opinion surveys; (2) cognitive psychologists and other researchers who are interested in everyday memory and judgment processes; and (3) survey researchers, methodologists, and statisticians who are involved in designing and carrying out surveys. (PsycINFO Database Record (c) 2012 APA, all rights reserved)