No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Sensitive Survey Questions with Auxiliary Information
Abstract
Scholars increasingly rely on indirect questioning techniques to reduce social desirability bias and item nonresponse for sensitive survey questions. The major drawback of these approaches, however, is their inefficiency relative to direct questioning. We show how to improve the statistical analysis of the list experiment, randomized response technique, and endorsement experiment by exploiting auxiliary information on the sensitive trait. We apply the proposed methodology to survey experiments conducted among voters in a controversial antiabortion referendum held during the 2011 Mississippi General Election. By incorporating the official county-level election results, we obtain precinct- and individual-level estimates that are more accurate than standard indirect questioning estimates and occasionally even more efficient than direct questioning. Our simulation studies shed light on the conditions under which our approach can improve the efficiency and robustness of estimates based on indirect questioning techniques. Open-source software is available for implementing the proposed methodology.
... While applying RRT, many authors utilized auxiliary information to suggest further improved estimation of a sensitive mean or proportion. In this regards [37,38,39], are some of the authors to be mentioned amongst many. Similarly, some sort of prior information about the parameter of interest may also be available in some studies. ...
... The expression derived in inequality (37), reduces to the lower and upper limits values for given combinations of parameters, where the [46] method of aiding data based estimator performs better than [13] estimator. Table 5, comprehends the numerical results evaluated using inequality (37) for various combinations of parameters involved in study. ...
... In this section, we explore the performance of [13] estimator when it is aided through shrinkage estimation procedures. We derive the efficiency conditions while considering the optimal values and mid-point values of weights (k 3 and k 4 ) over optimal range (see equations (37) and (42)), for both Searls and Thompson techniques. The performance analysis is then conducted for both shrinkage techniques with respect to Ref. [13] estimator. ...
Asking direct questions in face to face surveys about sensitive traits is an intricate issue. One of the solutions to this issue is the randomized response technique (RRT). Being the most widely used indirect questioning technique to obtain truthful data on sensitive traits in survey sampling RRT has been applied in a variety of fields including behavioral science, socio-economic, psychological, epidemiology, biomedical, criminology, data masking, public health engineering, conservation studies, ecological studies and many others. This paper aims at exploring the methods to subsidize the randomized response technique through additional information relevant to the parameter of interest. Specifically, we plan to contribute by proposing more efficient hybrid estimators compared to existing estimator based on (Kuk, 1990) [31] family of randomized response models. The proposed estimators are based on the methodology of incorporating the pertinent information, available on the basis of either historical records or expert opinion. Specifically, in case of availability of auxiliary information, the regression-cum-ratio estimator is found to be the best to further enhance the estimation through (Kuk, 1990) [31] model while the (Thompson, 1968) [49] shrinkage estimation is observed to be yielding more precise and accurate estimator of sensitive proportion. The findings in this study signify the importance of the proposed methodology. Additionally, to support the mathematical findings, a detailed numerical investigation to evaluate the comparative performances is also conducted. Based on performance analysis, overwhelming evidences are witnessed in the favor of proposed strategies.
... In their applications, the combined estimator decreased variance by 12% to 50%. Chou, Imai, and Rosenfeld (2018) provide a generalization of Aronow et al. (2015) to any subgroup among whom the true prevalence rate is known. In their application to support for an antiabortion ballot measure, auxiliary information in the form of known vote totals reduced the variance of the list experiment by 88%. ...
... Using auxiliary information (Chou, Imai, and Rosenfeld 2018) 88% 733% ...
Eliciting honest answers to sensitive questions is frustrated if subjects withhold the truth for fear that others will judge or punish them. The resulting bias is commonly referred to as social desirability bias, a subset of what we label sensitivity bias. We make three contributions. First, we propose a social reference theory of sensitivity bias to structure expectations about survey responses on sensitive topics. Second, we explore the bias-variance trade-off inherent in the choice between direct and indirect measurement technologies. Third, to estimate the extent of sensitivity bias, we meta-analyze the set of published and unpublished list experiments (a.k.a., the item count technique) conducted to date and compare the results with direct questions. We find that sensitivity biases are typically smaller than 10 percentage points and in some domains are approximately zero.
... This limitation is intrinsic to the list experiment method which has substantially higher variance than direct question methods.47 Future studies could aim to improve the precision around estimates through surveys with larger sample sizes or by using combined data techniques.61 Finally, our study is inherently observational in nature. ...
Objective
To estimate the association of Medicaid coverage of abortion care with cumulative lifetime abortion incidence among women insured by Medicaid.
Data Sources and Study Setting
We use 2016–2019 (Pre‐Dobbs) data from the Survey of Women studies that represent women aged 18–44 living in six U.S. states. One state, Maryland, has a Medicaid program that has long covered the cost of abortion care. The other five states, Alabama, Delaware, Iowa, Ohio, and South Carolina, have Medicaid programs that do not cover the cost of abortion care. Our sample includes 8972 women residing in the study states.
Study Design
Our outcome, cumulative lifetime abortion incidence, is identified using an indirect survey method, the double list experiment. We use a multivariate regression of cumulative lifetime abortion on variables including whether women were Medicaid‐insured and whether they were residing in Maryland versus in one of the other five states.
Data Collection/Extraction Methods
This study used secondary survey data.
Principal Findings
We estimate that Medicaid coverage of abortion care in Maryland is associated with a 37.0 percentage‐point (95% CI: 12.3–61.4) higher cumulative lifetime abortion incidence among Medicaid‐insured women relative to women not insured by Medicaid compared with those differences by insurance status in states whose Medicaid programs do not cover the cost of abortion care.
Conclusions
We found that Medicaid coverage of abortion care is associated with a much higher lifetime incidence of abortion among individuals insured by Medicaid. We infer that Medicaid coverage of abortion care costs may have a very large impact on the accessibility of abortion care for low‐income women.
... This helps in facilitating the widespread implementation of a variant of the list experiment that improves along the bias-variance frontier. This is compatible with previous efforts to increase precision, such as using responses to direct questions (Aronow et al. 2015) or auxiliary information (Chou, Imai, and Rosenfeld 2017) to adjust estimates. ...
Social scientists use list experiments in surveys to estimate the prevalence of sensitive attitudes and behaviors in a population of interest. However, the cumulative evidence suggests that the list experiment estimator is underpowered to capture the extent of sensitivity bias in common applications. The literature suggests double list experiments (DLEs) as an alternative to improve along the bias-variance frontier. This variant of the research design brings the additional burden of justifying the list experiment identification assumptions in both lists, which raises concerns over the validity of DLE estimates. To overcome this difficulty, this paper outlines two statistical tests to detect strategic misreporting that follows from violations to the identification assumptions. I illustrate their implementation with data from a study on support toward anti-immigration organizations in California and explore their properties via simulation.
... The use of auxiliary information to enhance the performance of estimation procedures is well documented in the statistical and allied sciences literature, see for example, Zhang and Chambers (2004), Chou et al. (2017) and Bai et al. (2021). In practice auxiliary information is obtainable from various sources, such as census, survey reports and expert opinion, see also Rao et al. (1990) and Biemer and Peytchev (2013). ...
The use of auxiliary information has a long history in statistical theory and estimation procedures. The utility of supplementary knowledge becomes vital when information about the study variable is limited. In this paper, we present a more competent mechanism to utilise auxiliary information in the estimation of the finite population mean. We propose a new exponential type of estimator for the estimation of finite population mean in the scenario where a simple random sampling scheme is adopted. Our proposed procedure is based on the dual use of the supportive information to maximise additional gain and involves the use of the mean of the auxiliary variable along with its rank to increase the extent of relevant information. The comparative performance of the proposed scheme is demonstrated with respect to 10 most used, classic, and some recent procedures in estimation theory literature. These are the classic mean estimator ̅ , the so-called traditional ratio, product, and regression estimators ̅ , ̅ and ̅ , respectively, along with the difference type estimator. . In addition, the more recent estimators investigated are the ratio-product exponential type ̅ , , difference exponential type ̅ , ratio exponential ̅ , , product exponential ̅ , and the ratio-product-exponential ̅ , all used for comparison. Moreover, we consider three data sets of a multidisciplinary nature, encompassing health surveillance, industrial production and poultry. The choice of data sets is mainly motivated by two reasons; (i) these data sets have been topics of contemporary techniques and, (ii) the considered data sets do offer a wide range of parametric settings, including lower extent of correlation between the study variable with the auxiliary variable and they also vary in sample sizes. In addition, we consider cases of a higher positive and higher negative degree of linear relationship extant between the study variable and auxiliary variable in these data sets. Along with the opportunity of conducting a fair comparison of our suggested strategy with contemporary techniques, the above approach allows for us to observe various patterns prevalent in the resultant gains of our newly devised scheme. Improvements are quantified by the mean square errors of the competing estimators, which are further transformed into relative percentage efficiencies to attain a comprehensive view of the research effort. Overall, we observe a noticeable amount of decrease in mean square error for our proposed estimator as compared to existing estimators, evident for all the considered data sets. However, there are a few observant patterns in the efficiency gains coinciding with assigned pre-defined parametric settings, in that the extent of the correlation between the auxiliary variable and output variable plays a pivotal role in the performance of estimation procedures. The improvement in the efficiency becomes more obvious as the degree of linear relationship between the output variable with the auxiliary variable strengthens. For example, minimum gain in percentage relative efficiencies (PREs) is observed for the 1 st data set, wherein the correlation coefficient, , , remains minimal. For the two other data sets the gain remains clearer as the correlation coefficient takes higher values, say, | , | > 0.85. We also note the varying performance hierarchy among contemporary estimators with respect to varying features of each population. Our proposed estimator outperforms the existing methods studied here in all cases. The mathematical expressions for the bias and mean squared error of the proposed estimator is derived under the first order of approximation. The theoretical and empirical studies show that the proposed estimator performs uniformly better than the existing estimators in terms of the percentage relative efficiency. We advocate that in future exponential smoothing will be used to quantify changes given updates by auxiliary information and recent observations.
... Even if unobtrusive indicators were available as readily, they are rather ill-suited for delivering dependents of explanatory models: the anonymity guarantee awarded by ICT and similar procedures comes at the price of severing any tie between individual respondents, on one hand, and scores of the sensitive item, on the other. This drawback was recently eased by the development of imputation techniques (Blair and Imai 2012;Chou, Imai and Rosenfeld 2017;Corstange 2009;Holbrook and Krosnick 2010;Imai 2011), but these entail high standard errors. Thus, from a model-optimization perspective, the aim of discerning ATII determinants is best served when all variables-including the dependent-originate in DQs. ...
Extant scholarship on attitudes toward immigration and immigrants relies mostly on direct survey items. Thus, little is known about the scope of social desirability bias, and even less about its covariates. In this paper, we use probability-based mixed-modes panel data collected in the Southern Spanish region of Andalusia to estimate anti-immigrant sentiment with both the item-count technique, also known as list experiment, and a direct question. Based on these measures, we gauge the size of social desirability bias, compute predictor models for both estimators of anti-immigrant sentiment, and pinpoint covariates of bias. For most respondent profiles, the item-count technique produces higher estimates of anti-immigrant sentiment than the direct question, suggesting that self-presentational concerns are far more ubiquitous than previously assumed. However, we also find evidence that among people keen to position themselves as all-out xenophiles, social desirability pressures persist in the list-experiment: the full scope of anti-immigrant sentiment remains elusive even in non-obtrusive measurement.
... In addition, since the literature suggests to use auxiliary questions in developing the research variables (e.g. Chou et al., 2017), two more items which describe the level of and the respondents satisfaction with the level of AM adoption were added in the questionnaire. Although AM is available in the production for many years, it has not reached the level of market saturation yet. ...
3D printing technologies are well-established as offering unique capabilities that may benefit the automotive industries, but empirical evidence has highlighted that only a partial adoption has been achieved by Original Equipment Manufacturers and their suppliers. However, the review of the relevant literature reflects that no work has been done to explore the level of 3D printing adoption and the specific barriers enabling appropriate adoption within the automotive supply chains. This chapter considers the technology adoption model and factors that inhibit 3D printing adoption; in doing so, both technical and non-technical barriers are discussed.
... A similar approach was taken in Agostini and Nosella (2019), who measured the level of adoption of industry 4.0 technologies (including Additive Manufacturing). Also, two auxiliary question (Chou et al., 2017) items (AM_9 and AM_10), describing the level of and respondent's satisfaction with the level of Additive Manufacturing adoption, were added in the research questionnaire. The multi-item reflective measures for the SCF and SCP constructs, as shown in Table A1, were adapted from scales established in extant research (refer to Section 2.2. and 2.3.): ...
The purpose of this paper is to provide a conceptual framework for analyzing the relationships among Additive Manufacturing adoption, flexibility, and performance in the supply chain context. No empirical study was found in the supply chain literature that specifically examines the relationships among Additive Manufacturing adoption, flexibility and performance; the paper therefore fills an important gap in the supply chain literature. The research is based on a quantitative approach using a questionnaire survey from a total of 124 medium-and large-sized European Union automotive manufacturing companies. The hypothesized relationships are tested using partial least square structural equation modeling (PLS-SEM). The research provides insights into how supply chain flexibility mediates the effect of Additive Manufacturing adoption on supply chain performance in the context of European automotive industry. Research findings indicate that Additive Manufacturing adoption positively impacts supply chain flexibility and that, in turn, supply chain flexibility positively impacts supply chain performance. This suggests that companies should focus on flexibilities in the supply chain to improve its performance. Overall, these findings provide important insights into the value of Additive Manufacturing adoption for supply chain flexibility and performance.
... These were derived from Wohers (2016) research according to which Additive Manufacturing is used in direct parts production (28.1 per cent), fit and assembly (17.5 per cent), prototype tooling (11.3 per cent), metal castings (10.8 per cent), visual aids (10.40 per cent) and prototypes (9.50 per cent). As the literature suggests to use auxiliary questions in developing the research variables to prove the authenticity of the answers (Chou et al., 2017), two more items describing the respondents satisfaction with the level of Additive Manufacturing adoption (AM_9 and AM_10) were added in the research questionnaire (Table A1). ...
Purpose
Additive Manufacturing offers much potential for industry, but at the same time is likely to have a significant impact on supply chain theory and practice. To-date there has been limited empirical work exploring the effect of Additive Manufacturing, and this study aims to provide a detailed appraisal of how supply chain integration, supply chain performance and firm performance may be affected by the adoption of Additive Manufacturing. These are critical factors for supply chain management, but have received little quantified attention to date.
Design/methodology/approach
A theoretical model is developed from a detailed review of the literature, from which a quantitative analysis is performed. Using data collected from 124 automotive manufacturers in European Union and the partial least square- structural equation modeling method, the research examines the relationships among different dimensions of supply chain integration, supply chain performance and firm performance from the perspective of Additive Manufacturing adoption.
Findings
The findings indicate that Additive Manufacturing adoption positively influences supply chain performance and as a consequence, firm performance. In addition, supporting the resource-based view perspective, the results show a positive indirect effect of supply chain integration on the supply chain and firm performance improvements, enabled by the Additive Manufacturing adoption. This helps to explain some inconsistent findings in previous research regarding the impacts of supply chain integration on performance.
Research limitations/implications
The results of this study support the view that Additive Manufacturing can make a positive contribution to the supply chain, but this is not achieved solely by the technologies alone. Many of the traditional activities of supply chain management (i.e. integration) are still needed when using Additive Manufacturing, and research needs to understand whether Additive Manufacturing adoption will necessitate changes to the way these traditional activities are undertaken. Building on the findings of the current study, much more work is therefore needed to understand how operations within the supply chain may be changed, and how this may affect the integration and performance of the supply chain.
Practical implications
This study provides quantitative evidence to show that the adoption of Additive Manufacturing has the potential to affect both firm and supply chain performance. This is significant for those companies considering the adoption of Additive Manufacturing, and may serve as a valuable insight in the strategic decision-making process. For those already using Additive Manufacturing, this study serves to underline the potential for firm performance to be influenced, by focusing on improvements to their production strategies and policies.
Originality/value
This study provides an initial insight into some fundamental supply chain concepts within an Additive Manufacturing context, which have received very little research attention. It develops a novel conceptual model, and through a large-scale industry survey provides quantified evidence of the impact of Additive Manufacturing on the supply chain. To date, much of the supply chain research is exploratory and qualitative; the quantitative evidence presented in this work, therefore, makes an important and original contribution to both research and practice.
Professional athletes in the United States have protested racism in various ways for decades. Kneeling during the national anthem became a prominent form of such activism, ever since American football player Colin Kaepernick popularized the practice in 2016. “Anthem protests” gained renewed attention after the police killing of George Floyd and nationwide unrest in the summer of 2020. This article explores whether public approval of those protests was weaker than scholars and journalists suggested, because survey respondents were reluctant to admit that they considered the protesters disrespectful. A list experiment confirms hidden opposition to anthem protests, especially among people of color, who may feel heightened pressure to support racialized protesters. A second experiment reveals that social desirability bias persists even after respondents hear reassurance that nobody will judge their views. These findings indicate that mainstream surveys misrepresent attitudes toward contemporary racial issues, and that anthem protests have yet to gain wide acceptance in the general U.S. population.
Istilah ekstremisme yang berbau kekerasan (violent extremism)
sebenarnya sangat problematik. Jika kita percaya ada violent extremism,
berarti kita dengan sendirinya mengakui ada kelompok ekstremis yang
tidak memakai kekerasan (non-violent extremism). Lantas, di luar masalah
pemakaian instrumen kekerasan dalam mencapai tujuan, seberapa jauh
perbedaan di antara kedua kelompok ekstremis tersebut? Apakah mereka
memiliki pandangan dan ideologi yang sama?
In this article, I review recent developments of the item-count technique (also known as the unmatched-count or list-experiment technique) and introduce a new package, kict, for statistical analysis of the item-count data. This package contains four commands: kict deff performs a diagnostic test to detect the violation of an assumption underlying the item-count technique. kict ls and kict ml perform least-squares estimation and maximum likelihood estimation, respectively. Each encompasses a number of estimators, offering great flexibility for data analysis. kict pfci is a postestimation command for producing confidence intervals with better coverage based on profile likelihood. The development of the item-count technique is still ongoing. I will continue to update the kict package accordingly.
Measurement error threatens the validity of survey research, especially when studying sensitive questions. Although list experiments can help discourage deliberate misreporting, they may also suffer from nonstrategic measurement error due to flawed implementation and respondents’ inattention. Such error runs against the assumptions of the standard maximum likelihood regression ( MLreg ) estimator for list experiments and can result in misleading inferences, especially when the underlying sensitive trait is rare. We address this problem by providing new tools for diagnosing and mitigating measurement error in list experiments. First, we demonstrate that the nonlinear least squares regression ( NLSreg ) estimator proposed in Imai (2011) is robust to nonstrategic measurement error. Second, we offer a general model misspecification test to gauge the divergence of the MLreg and NLSreg estimates. Third, we show how to model measurement error directly, proposing new estimators that preserve the statistical efficiency of MLreg while improving robustness. Last, we revisit empirical studies shown to exhibit nonstrategic measurement error, and demonstrate that our tools readily diagnose and mitigate the bias. We conclude this article with a number of practical recommendations for applied researchers. The proposed methods are implemented through an open-source software package.
The rise of professionalism within the nonprofit sector has transformed the sector’s reliance on well-meaning volunteers to paid professionals. While the professionalization of the nonprofit workforce is likely to continue, nonprofits are increasingly challenged for their inability to pay competitive wages. Our study argues that a social expectation for nonprofit employees to forgo some of their wages influences the donative labor narrative, which in turn impacts low nonprofit wages. We present data from an online survey experiment of executive directors at 467 nonprofits, along with their organizations’ Form 990 filings, to contrast socially biased attitudes and genuine views toward the donative labor hypothesis. The findings illustrate that the donative labor narrative should be understood as a result of social expectations for sacrifice of nonprofit employees, rather than a simple outcome of supply and demand in the labor market. We discuss the need to reframe the widespread donative labor narrative.
This article assesses the validity of responses to sensitive questions using four different methods. In an experimental setting, the authors compared a computer-assisted self-interview (CASI), face-to-face direct questioning, and two different varieties of randomized response. All respondents interviewed had been identified as having committed welfare and unemployment benefit fraud. The interviewers did not know that respondents had been caught for fraud, and the respondents did not know that the researchers had this information. The results are evaluated by comparing the percentage of false negatives. The authors also looked for variables that might explain why some respondents admit fraud and others do not. The proportions of respondents admitting fraud are relatively low, between 19 percent and 49 percent. The two randomized response conditions were superior in eliciting admissions of fraud. A number of background variables, notably gender, age, still receiving benefit, and duration and perception of fraud, are related to admitting fraud. Although the randomized response conditions performed much better than face-to-face direct questioning and CASI, the percentage of respondents admitting fraud is only around 50 percent. Some possible reasons for this are discussed.
Eliciting information from experts for use in constructing prior distributions for logistic regression coeffcients can be challenging. The task is especially diffcult when the model contains many predictor variables, because the expert is asked to provide summary information about the probability of "success" for many subgroups of the population. Often, however, experts are confident only in their assessment of the population as a whole. This paper is about incorporating such overall information easily into a logistic regression data analysis using g-priors. We present a version of the g-prior such that the prior distribution on the overall population logistic regression probabilities of success can be set to match a beta distribution. A simple data augmentation formulation allows implementation in standard statistical software packages.
This article is an empirical contribution to the evaluation of the randomized response technique (RRT), a prominent procedure to elicit more valid responses to sensitive questions in surveys. Based on individual validation data, we focus on two questions: First, does the RRT lead to higher prevalence estimates of sensitive behavior than direct questioning (DQ)? Second, are there differences in the effects of determinants of misreporting according to question mode? The data come from 552 face-to-face interviews with subjects who had been convicted by a court for minor criminal offences in a metropolitan area in Germany. For the first question, the answer is negative. For the second, it is positive, that is, effects of individual and situational determinants of misreporting differ between the two question modes. The effect of need for social approval, for example, tends to be stronger in RRT than in DQ mode. Interviewer experience turns out to be positively related to answer validity in DQ and negatively in RRT mode. Our findings support a skeptical position toward RRT, shed new light on long-standing debates within survey methodology, and stimulate theoretical reasoning about response behavior in surveys.
Many validation studies deal with item nonresponse and measurement error in
earnings data. In this article, the author explores respondents’ motives for failing to
reveal earnings using the German Socio-Economic Panel (SOEP). The SOEP collects
socioeconomic information from private households in the Federal Republic of Germany.
The author explains the evolution of income nonresponse in the SOEP and demonstrates
the importance of discriminating between refusing to state income and responses of
“don’t know.”
Survey respondents may give untruthful answers to sensitive questions when asked directly. In recent years, researchers have
turned to the list experiment (also known as the item count technique) to overcome this difficulty. While list experiments
are arguably less prone to bias than direct questioning, list experiments are also more susceptible to sampling variability.
We show that researchers need not abandon direct questioning altogether in order to gain the advantages of list experimentation.
We develop a nonparametric estimator of the prevalence of sensitive behaviors that combines list experimentation and direct
questioning. We prove that this estimator is asymptotically more efficient than the standard difference-in-means estimator,
and we provide a basis for inference using Wald-type confidence intervals. Additionally, leveraging information from the direct
questioning, we derive two nonparametric placebo tests for assessing identifying assumptions underlying list experiments.
We demonstrate the effectiveness of our combined estimator and placebo tests with an original survey experiment.
This article examines the problem of response error in survey earnings data.
Comparing workers’ earnings reports in the U.S. Census Bureau’s Survey of
Income and Program Participation (SIPP) to their detailed W-2 earnings
records from the Social Security Administration, we employ ordinary least
squares (OLS) and quantile regression models to assess the effects of earnings
determinants and demographic variables on measurement errors in
2004 SIPP earnings in terms of bias and variance. Results show that measurement
errors in earnings are not classical, but mean-reverting. The directions
of bias for subpopulations are not constant, but varying across levels of earnings.
Highly educated workers more correctly report their earnings than less
educated workers at higher earnings levels, but they tend to overreport at
lower earnings levels. Black workers with high earnings underreport to a greater degree than comparable whites, while black workers with low earnings
overreport to a greater degree. Some subpopulations exhibit higher variances
of measurement errors than others. Blacks, Hispanics, high school
dropouts, part-year employed workers, and occupation ‘‘switchers’’ tend
to misreport—both over- and underreport—their earnings rather than unilaterally
in one direction. The implications of our findings are discussed.
Despite its shortcomings, cross-level or ecological inference remains a
necessary part of some areas of quantitative inference, including in United
States voting rights litigation. Ecological inference suffers from a lack of
identification that, most agree, is best addressed by incorporating
individual-level data into the model. In this paper we test the limits of such
an incorporation by attempting it in the context of drawing inferences about
racial voting patterns using a combination of an exit poll and precinct-level
ecological data; accurate information about racial voting patterns is needed to
assess triggers in voting rights laws that can determine the composition of
United States legislative bodies. Specifically, we extend and study a hybrid
model that addresses two-way tables of arbitrary dimension. We apply the hybrid
model to an exit poll we administered in the City of Boston in 2008. Using the
resulting data as well as simulation, we compare the performance of a pure
ecological estimator, pure survey estimators using various sampling schemes and
our hybrid. We conclude that the hybrid estimator offers substantial benefits
by enabling substantive inferences about voting patterns not practicably
available without its use.
This paper studies the problem of identification of the long regression E(y j x# z) when the short conditional distributions P (y j x) and P (z j x) are known but the long conditional distribution P (y j x# z) is not known. The nature of the problem is revealed by the Law of Total Probability, P (y j x)= X j2Z Pr(z = j j x)P (y j x# z = j): (1) Knowledge of P (y j x) and P (z j x) restricts [P (y j x# z = j)#j 2 Z]toJ-vectors of distributions that satisfy (1). Our objective is to determine the implied restrictions on E(y j x# z). Aspects of the problem of inference on E(y j x# z)have been studied in several literatures with varying concerns and terminology. The classical literature on linear regression compares the parameter estimates obtained in a least squares fit of y to x with those obtained in a least squares fit of y to (x# z). The expected difference between the estimated coefficients on x in the former and the latter fits is sometimes called "omitted variable bias". The findings are specific to least squares estimation of linear regressions and so do not directly inform the present nonparametric analysis. We do, however, borrow the terms short regression and long regression from Goldberger (1991), Sec. 17.2. Stimulated by Simpson (1951), statisticians havebeenintrigued by the fact that E(y j x) may be increasing in a scalar x and yet all J components of [E(y j x# z = j)#j 2 Z] may be decreasing in x. Studies of Simpson's Paradox have soughttocharacterize the circumstances in which this phenomenon occurs. See, for example, Lindley and Novick (1981) and Zidek (1984). Stimulated by Robinson (1950), social scientists have studied the ecological inference problem, often described as inference on individual behavior from aggregate data (e.g., King, 1997). A leadin...
In both political behavior research and voting rights litigation, the turnout and vote choice for different racial groups
are often inferred using aggregate election results and racial composition. Over the Past several decades, many statistical
methods have been proposed to address this ecological inference problem. We propose an alternative method to reduce aggregation
bias by predicting individual-level ethnicity from voter registration records. Building on the existing methodological literature,
we use Bayes’s rule to combine the Census Bureau’s Surname List with various information from geocoded voter registration
records. We evaluate the performance of the proposed methodology using approximately nine million voter registration records
in Florida, where self-reported ethnicity is available. We find that it is possible to reduce the false positive rate among
Black and Latino voters to 6% and 3%, respectively, while maintaining the true positive rate above 80%. Moreover, we use our
predictions to estimate the turnout by race and find that our estimates yields substantially less amounts of bias and root
mean squared error than standard ecological inference estimates. The open-source software is available for implementing the
proposed methodology.
About a half century ago, in 1965, Warner proposed the randomized response method as a survey technique to reduce potential bias due to nonresponse and social desirability when asking questions about sensitive behaviors and beliefs. This method asks respondents to use a randomization device, such as a coin flip, whose outcome is unobserved by the interviewer. By introducing random noise, the method conceals individual responses and protects respondent privacy. While numerous methodological advances have been made, we find surprisingly few applications of this promising survey technique. In this article, we address this gap by (1) reviewing standard designs available to applied researchers, (2) developing various multivariate regression techniques for substantive analyses, (3) proposing power analyses to help improve research designs, (4) presenting new robust designs that are based on less stringent assumptions than those of the standard designs, and (5) making all described methods available through open-source software. We illustrate some of these methods with an original survey about militant groups in Nigeria.
When studying sensitive issues, including corruption, prejudice, and sexual behavior, researchers have increasingly relied upon indirect questioning techniques to mitigate such known problems of direct survey questions as underreporting and nonresponse. However, there have been surprisingly few empirical validation studies of these indirect techniques because the information required to verify the resulting estimates is often difficult to access. This article reports findings from the first comprehensive validation study of indirect methods. We estimate whether people voted for an anti-abortion referendum held during the 2011 Mississippi General Election using direct questioning and three popular indirect methods: list experiment, endorsement experiment, and randomized response. We then validate these estimates against the official election outcome. While direct questioning leads to significant underestimation of sensitive votes against the referendum, indirect survey techniques yield estimates much closer to the actual vote count, with endorsement experiment and randomized response yielding the least bias.
Using Carmines and Stimson's issue evolution model of partisan change, I argue that the abortion issue has transformed the two major United States political parties and that this process follows a predictable pattern, as outlined by Carmines and Stimson. By applying the theory of issue evolution to abortion, I develop three hypotheses: 1) The reputation for each party's stance on abortion among party elites has grown clearer and more distinct over the last 20 years; 2) At the mass level, people have changed their party identification in a manner consistent with their attitudes on abortion; 3) The changes among party elites and masses are causally connected, with elite-level changes producing mass-level responses. Simple time series analyses are used on two datasets, roll call votes on abortion in the United States Congress and public opinion polls from General Social Surveys, 1972-94. Democrats and Republicans shift dramatically on the abortion issue at both the elite and mass levels. Moreover, this change closely follows with the issue evolution model. The process unfolds gradually, and causality appears to run from elites to masses, rather than from masses to elites.
The abstract for this document is available on CSA Illumina.To view the Abstract, click the Abstract button above the document title.
Qualitative studies of vote buying find the practice to be common in many Latin American countries, but quantitative studies using surveys find little evidence of vote buying. Social desirability bias can account for this discrepancy. We employ a survey-based list experiment to minimize the problem. After the 2008 Nicaraguan municipal elections, we asked about vote-buying behavior by campaigns using a list experiment and the questions traditionally used by studies of vote buying on a nationally representative survey. Our list experiment estimated that 24% of registered voters in Nicaragua were offered a gift or service in exchange for votes, whereas only 2% reported the behavior when asked directly. This detected social desirability bias is nonrandom and analysis based on traditional obtrusive measures of vote buying is unreliable. We also provide systematic evidence that shows the importance of monitoring strategies by parties in determining who is targeted for vote buying. C lientelistic electoral linkages are characterized by a transaction of political favors in which politi-cians offer immediate material incentives to cit-izens or groups in exchange for electoral support. 1 Vote buying, which is a more particularized form of clien-telism involving the exchange of goods for votes at the individual level (Stokes 2007), has generated numerous ethnographies and surveys to measure its incidence and test-related hypotheses. While qualitative research rou-tinely finds vote buying to be pervasive in the developing world (e.g., Auyero 2001), individual-level surveys often uncover low levels of such exchanges (e.g., Transparency Ezequiel Gonzalez-Ocantos is a Ph.
How are civilian attitudes toward combatants affected by wartime victimization? Are these effects conditional on which combatant inflicted the harm? We investigate the determinants of wartime civilian attitudes towards combatants using a survey experiment across 204 villages in five Pashtun-dominated provinces of Afghanistan—the heart of the Taliban insurgency. We use endorsement experiments to indirectly elicit truthful answers to sensitive questions about support for different combatants. We demonstrate that civilian attitudes are asymmetric in nature. Harm inflicted by the International Security Assistance Force (ISAF) is met with reduced support for ISAF and increased support for the Taliban, but Taliban-inflicted harm does not translate into greater ISAF support. We combine a multistage sampling design with hierarchical modeling to estimate ISAF and Taliban support at the individual, village, and district levels, permitting a more fine-grained analysis of wartime attitudes than previously possible.
List and endorsement experiments are becoming increasingly popular among social scientists as indirect survey techniques for sensitive questions. When studying issues such as racial prejudice and support for militant groups, these survey methodologies may improve the validity of measurements by reducing nonresponse and social desirability biases. We develop a statistical test and multivariate regression models for comparing and combining the results from list and endorsement experiments. We demonstrate that when carefully designed and analyzed, the two survey experiments can produce substantively similar empirical findings. Such agreement is shown to be possible even when these experiments are applied to one of the most challenging research environments: contemporary Afghanistan. We find that both experiments uncover similar patterns of support for the International Security Assistance Force (ISAF) among Pashtun respondents. Our findings suggest that multiple measurement strategies can enhance the credibility of empirical conclusions. Open-source software is available for implementing the proposed methods.
Background: Self-reported data are commonly used to estimate the prevalence of health conditions and the use of preventive health services in the population, but the validity of such data is often questioned.Methods: The Behavioral Risk Factor Survey (BRFS) was administered by telephone to a stratified, random sample of health maintenance organization (HMO) subscribers in Colorado in 1993, and self-reports were compared with HMO medical records for 599 adults aged >21. Sensitivity and specificity were calculated for three chronic conditions and use of six preventive services.Results: Sensitivity was highest for hypertension (83%), moderate for diabetes (73%), and lowest for hypercholesterolemia (59%); specificity was >80% for all three conditions. Sensitivity ranged from 86% to 99% for influenza immunization, clinical breast examination, blood cholesterol screening, mammography, Pap test, and blood pressure screening; specificity was Conclusions: Self-reports are reasonably accurate for certain chronic conditions and for routine screening exams and can provide a useful estimate for broad measures of population prevalence.
This study presents a survey-based method for conducting inference into the determinants of sensitive political behavior. The approach combines two well-established literatures in statistical methods in the social sciences: the randomized response (RR) methodology utilized to reduce evasive answer bias and the generalized propensity score methodology utilized to draw inferences about causal effects in observational studies. The approach permits one to estimate the causal impact of a multivalued predictor variable of interest on a given sensitive behavior in the face of unknown interaction effects between the predictor and the confounders as well as nonlinearities in the relationship between the confounders and the sensitive behavior. Simulation results point to the superior performance of the RR relative to direct survey questioning using this method for samples of moderate to large size. The utility of the approach is illustrated through an application to corruption in the public bureaucracy in three countries in South America.
This study examines the accuracy of a survey question about the number of times a subject has been arrested. Specifically, the study answers questions dealing with extent of response error, random versus systematic response errors, the reasons for response errors (lapses in memory and motivational factors that lead to under- reporting or overreporting), and the extent to which the survey response is an adequate substitute for actual data. The results demonstrate that there is a considerable amount of response error in the measure and that both memory lapses and motivational factors contribute to it. Some strategies for dealing with the problem are explored.
An experimental CATI-survey (N=2041), asking sensitive questions about xenophobia and anti-Semitism in Germany, was conducted to compare the randomized response technique (RRT) and the direct questioning technique. Unlike the vast majority of RRT surveys measuring the prevalence of socially undesirable behaviors, only few studies have explored the effectiveness of the RRT with respect to the disclosure of socially undesirable opinions. Results suggest that the RRT is an effective method eliciting more socially undesirable opinions and yielding more valid prevalence estimates of xenophobia and anti-Semitism than direct questioning ('more-is-better' assumption). Furthermore, the results indicate that with increasing topic sensitivity, the benefits of using the RRT also increase. Finally, adapted logistic regression analyses show that several covariates such as education and generalized trust are related to the likelihood of being prejudiced towards foreigners and Jews.
Regression coefficients specify the partial effect of a regressor on the dependent variable. Sometimes the bivariate or limited multivariate relationship of that regressor variable with the dependent variable is known from population-level data. We show here that such population-level data can be used to reduce variance and bias about estimates of those regression coefficients from sample survey data. The method of constrained MLE is used to achieve these improvements. Its statistical properties are first described. The method constrains the weighted sum of all the covariate-specific associations (partial effects) of the regressors on the dependent variable to equal the overall association of one or more regressors, where the latter is known exactly from the population data. We refer to those regressors whose bivariate or limited multivariate relationships with the dependent variable are constrained by population data as being “directly constrained.” Our study investigates the improvements in the estimation of directly constrained variables as well as the improvements in the estimation of other regressor variables that may be correlated with the directly constrained variables, and thus “indirectly constrained” by the population data. The example application is to the marital fertility of black versus white women. The difference between white and black women's rates of marital fertility, available from population-level data, gives the overall association of race with fertility. We show that the constrained MLE technique both provides a far more powerful statistical test of the partial effect of being black and purges the test of a bias that would otherwise distort the estimated magnitude of this effect. We find only trivial reductions, however, in the standard errors of the parameters for indirectly constrained regressors.
Theory and evidence suggests that respondents are likely to overreport voter turnout in election surveys because they have a strong incentive to offer a socially desirable response. We suggest that contextual influences may affect the socially desirable bias, leading to variance in the rate of overreporting across countries. This leads us to hypothesize that nonvoters will be more likely to overreport voting in elections that have high turnout. We rely on validated turnout data to measure overreporting in five countries which vary a great deal in turnout: Britain, New Zealand, Norway, Sweden, and the United States. We find that in national settings with higher levels of participation, the tendency to overreport turnout may be greater than in settings where low participation is the norm.
Objective. Immigration scholars have found that the highly educated and political liberals are considerably less likely to support restrictionist immigration policies than other groups. I ask whether the influence of social desirability pressures in the survey interview is responsible for this finding.
Methods. An unobtrusive questioning technique known as the list experiment is used to measure Americans' support for immigration restrictionism. The list experiment can easily be embedded in a standard telephone survey and has been used by previous investigators to study racial attitudes.
Results. Restrictionist sentiments are found to be more widespread among the U.S. populace than previous studies have estimated, especially among college graduates and political liberals.
Conclusion. My findings have implications for immigration scholars and social scientists who study other sensitive attitudes and behaviors. The most commonly employed strategies to reduce socially desirable responding may not be enough.
The item count technique is a survey methodology that is designed to elicit respondents’ truthful answers to sensitive questions such as racial prejudice and drug use. The method is also known as the list experiment or the unmatched count technique and is an alternative to the commonly used randomized response method. In this article, I propose new nonlinear least squares and maximum likelihood estimators for efficient multivariate regression analysis with the item count technique. The two-step estimation procedure and the Expectation Maximization algorithm are developed to facilitate the computation. Enabling multivariate regression analysis is essential because researchers are typically interested in knowing how the probability of answering the sensitive question affirmatively varies as a function of respondents’ characteristics. As an empirical illustration, the proposed methodology is applied to the 1991 National Race and Politics survey where the investigators used the item count technique to measure the degree of racial hatred in the United States. Small-scale simulation studies suggest that the maximum likelihood estimator can be substantially more efficient than alternative estimators. Statistical efficiency is an important concern for the item count technique because indirect questioning means loss of information. The open-source software is made available to implement the proposed methodology.
In this paper we analyze the estimation of coefficients in regression models under moment restrictions in which the moment restrictions are derived from auxiliary data. The moment restrictions yield weights for each observation that can subsequently be used in weighted regression analysis. We discuss the interpretation of these weights under two assumptions: that the target population (from which the moments are constructed) and the sampled population (from which the sample is drawn) are the same, and that these populations differ. We present an application based on omitted ability bias in estimation of wage regressions. The National Longitudinal Survey Young Men's Cohort (NLS) - in addition to containing information for each observation on wages, education, and experience - records data on two test scores that may be considered proxies for ability. The NLS is a small dataset, however, with a high attrition rate. We investigate how to mitigate these problems in the NLS by forming moments from the joint distribution of education, experience, and log wages in the 1% sample of the 1980 U.S. Census and using these moments to construct weights for weighted regression analysis of the NLS. We analyze the impacts of our weighted regression techniques on the estimated coefficients and standard errors of returns to education and experience in the NLS controlling for ability, with and without the assumption that the NLS and the Census samples are random samples from the same population. © 1999 by the President and Fellows of Harvard College and the Massachusetts Institute of Technolog
For various reasons individuals in a sample survey may prefer not to confide to the interviewer the correct answers to certain questions. In such cases the individuals may elect not to reply at all or to reply with incorrect answers. The resulting evasive answer bias is ordinarily difficult to assess. In this paper it is argued that such bias is potentially removable through allowing the interviewee to maintain privacy through the device of randomizing his response. A randomized response method for estimating a population proportion is presented as an example. Unbiased maximum likelihood estimates are obtained and their mean square errors are compared with the mean square errors of conventional estimates under various assumptions about the underlying population.
Gaining valid answers to so-called sensitive questions is an age-old problem in survey research. Various techniques have been developed to guarantee anonymity and minimize the respondent's feelings of jeopardy. Two such techniques are the randomized response technique (RRT) and the unmatched count technique (UCT). In this study we evaluate the effectiveness of different implementations of the RRT (using a forced-response design) in a computer-assisted setting and also compare the use of the RRT to that of the UCT. The techniques are evaluated according to various quality criteria, such as the prevalence estimates they provide, the ease of their use, and respondent trust in the techniques. Our results indicate that the RRTs are problematic with respect to several domains, such as the limited trust they inspire and non-response, and that the RRT estimates are unreliable due to a strong false "no" bias, especially for the more sensitive questions. The UCT, however, performed well compared to the RRTs on all the evaluated measures. The UCT estimates also had more face validity than the RRT estimates. We conclude that the UCT is a promising alternative to RRT in self-administered surveys and that future research should be directed towards evaluating and improving the technique.
Census reports can be interpreted as providing nearly exact knowledge of moments of the marginal distribution of economic
variables. This information can be combined with cross-sectional or panel samples to improve accuracy of estimation. In this
paper we show how to do this efficiently. We show that the gains from use of marginal information can be substantial. We also
discuss how to test the compatibility of sample and marginal information.
To obtain information about the contribution of individual and area level factors to population health, it is desirable to use both data collected on areas, such as censuses, and on individuals, e.g. survey and cohort data. Recently developed models allow us to carry out simultaneous regressions on related data at the individual and aggregate levels. These can reduce 'ecological bias' that is caused by confounding, model misspecification or lack of information and increase power compared with analysing the data sets singly. We use these methods in an application investigating individual and area level sociodemographic predictors of the risk of hospital admissions for heart and circulatory disease in London. We discuss the practical issues that are encountered in this kind of data synthesis and demonstrate that this modelling framework is sufficiently flexible to incorporate a wide range of sources of data and to answer substantive questions. Our analysis shows that the variations that are observed are mainly attributable to individual level factors rather than the contextual effect of deprivation. Copyright 2008 Royal Statistical Society.
This paper studies estimators that make sample analogues of population orthogonality conditions close to zero. Strong consistency and asymptotic normality of such estimators is established under the assumption that the observable variables are stationary and ergodic. Since many linear and nonlinear econometric estimators reside within the class of estimators studied in this paper, a convenient summary of the large sample properties of these estimators, including some whose large sample properties have not heretofore been discussed, is provided.
The authors investigate the small sample properties of three alternative generalized method of moments estimators of asset pricing models. The estimators that they consider include ones in which the weighting matrix is iterated to convergence and ones in which the weighting matrix is changed with each choice of the parameters. Particular attention is devoted to assessing the performance of the asymptotic theory for making inferences based directly on the deterioration of GMM criterion functions.
We consider the problem of inference about a quantity of interest given different sources of information linked by a deterministic population dynamics model. Our approach consists of translating all the available information into a joint pre-model distribution on all the model inputs and outputs, and then restricting this to the submanifold defined by the model to obtain the joint post-model distribution. Marginalizing this yields inference, conditional on the model, about quantities of interest which can be functions of model inputs, model outputs, or both. Samples from the postmodel distribution are obtained by importance sampling and Rubin's SIR algorithm. The framework includes as a special case the situation where the pre-model information about the outputs consists of measurements with error; this reduces to standard Bayesian inference. The results are in the form of a sample from the post-model distribution and so can be examined using the full range of exploratory data analysis...
Statistical Analysis of Endorsement Experiments: Measuring Support for Militant Groups in Pakistan
- W Bullock
- K Imai
- J N Shapiro
Statistical Analysis of List Experiments
- G Blair
- K Imai
“rr: Statistical Methods for the Randomized Response
- G Blair
- Y Y Zhou
- K Imai
Neighborhood Effects on Crime for Female and Male Youth: Evidence from a Randomized Housing Voucher Experiment
- J R Kling
- J Ludwig
- L F Katz
Endorse: R Package for Analyzing Endorsement Experiments
- Y Shiraito
- K Imai
List: Statistical Methods for the Item Count Technique and List Experiment.” Comprehensive R Archive Network (CRAN)
- G Blair
- K Imai
Racial Attitudes and the “New South.”
- J H Kuklinski
- M D Cobb
- M Gilens
Improved Regression Estimation of a Multivariate Relationship with Population Data on the Bivariate Relationship
- M S Handcock
- M S Rendall
- J E Cheadle