Barry Schouten

Centraal Bureau voor de Statistiek | CBS

Anyone with a smartphone can take pictures anytime anywhere. This opens the opportunity for researchers to collect photos to augment traditional Web survey data. We conducted an experimental survey asking 2700 members of the Dutch LISS panel about their dwelling conditions. Depending on the condition, they were asked to either take several photos o...

Passively-generated location data have the potential to augment mobility and transportation research, as demonstrated by a decade of research. A common trait of these data is a high proportion of missingness. Naïve handling, including list-wise deletion of subjects or days, or linear interpolation across time gaps, has the potential to bias summary...

Data collection staff involved in sampling designs, monitoring and analysis of surveys often have a good sense of the response rate that can be expected in a survey, even when this survey is new or done at a relatively low frequency. They make expectations of response rates, and, subsequently, costs on an almost continuous basis. Rarely, however, a...

This study investigates how an auto-forward design, where respondents navigate through a web survey automatically, affects response times and navigation behavior in a long mixed-device web survey. We embedded an experiment in a health survey administered to the general population in The Netherlands to test the auto-forward design against a manual-f...

Smartphone sensors allow measurement of phenomena that are difficult or impossible to capture via self-report (e.g., geographical movement, physical activity). Sensors can reduce respondent burden by eliminating survey questions and improve measurement accuracy by replacing/augmenting self-reports. However, if respondents who are not willing to col...

Advances in smartphone technology have allowed for individuals to have access to near-continuous location tracking at a very precise level. As the backbone of mobility research, the Travel Diary Study, has continued to offer decreasing response rates over the years, researchers are looking to these mobile devices to bridge the gap between self-repo...

The growing smartphone penetration and the integration of smartphones into people’s everyday practices offer researchers opportunities to augment survey measurement with smartphone-sensor measurement or to replace self-reports. Potential benefits include lower measurement error, a widening of research questions, collection of in situ data, and a lo...

Adaptive survey design has attracted great interest in recent years, but the number of case studies describing actual implementation is still thin. Reasons for this may be the gap between survey methodology and data collection, practical complications in differentiating effort across sample units and lack of flexibility of survey case management sy...

The relation between answer behaviour and measurement error has been studied extensively. Answer behaviour may be considered undesirable, like answering ‘don’t know’ or ‘won’t tell’. It is not clear to what degree undesirable answer behaviour from the same respondents is present across different surveys. In this study, we investigated to what exten...

In this paper we discuss the implications of using mobile devices for online survey completion. With more and more people accessing online surveys on mobile devices, online surveys need to be redesigned in order to be able to meet the characteristics of mobile device usage, such as small screens and short messaging. We discuss mobile friendly desig...

Mapping the movement of the general population is important for the development of mobility related policy. CBS collected the time-location data of approximately one week of almost 700 respondents for this purpose. Determining the start and end point of each track is a first step in determining specific points of interest and to find out how a trac...

We discuss an evidence-based approach to guiding real-time design decisions during the course of survey data collection. We call it responsive and adaptive design (RAD), a scientific framework driven by cost-quality tradeoff analysis and optimization that enables the most efficient production of high-quality data. The notion of RAD is not new; nor...

In the design of surveys a number of input parameters, such as contact propensities, participation propensities and costs per sample unit, play a decisive role. In on-going surveys, these survey design parameters are usually estimated from previous experience and updated gradually with new experience. In new surveys, these parameters are estimated...

In mixed-mode surveys, mode differences in measurement bias, also called measurement effects or mode effects, continue to pose a problem to survey practitioners. In this paper, we discuss statistical adjustment of measurement bias to the level of a measurement benchmark mode in the context of inference from mixed-mode data. Doing so requires auxili...

Item characteristics can have a significant effect on survey data quality and may be associated with measurement error. Literature on data quality and measurement error is often inconclusive. This could be because item characteristics used for detecting measurement error are not coded unambiguously. In our study, we use a systematic coding procedur...

Adaptive survey designs (ASDs) optimize design features, given 1) the interactions between the design features and characteristics of sampling units and 2) a set of constraints, such as a budget and a minimum number of respondents. Estimation of the interactions is subject to both random and systematic error. In this article, we propose and evaluat...

In most real life studies, auxiliary variables are available and are employed to explain and understand missing data patterns and to evaluate and control causal relationships with variables of interest. Usually their availability is assumed to be a fact, even if the variables are measured without the objectives of the study in mind. As a result, in...

Adaptive survey designs (ASDs) provide a framework for data-driven tailoring of data collection procedures to different sample members, often for cost and bias reduction. People vary in how likely they are to respond and in how they respond. This variation leads to opportunities to selectively deploy design features in order to control both nonresp...

Surveys can be conducted with a variety of survey modes. Over the past decade, and following the emergence of the Web, many surveys have moved to designs in which multiple modes are combined, the so-called mixed-mode surveys. The contemporary modes are telephone, face-to-face, mail/paper, and online through various devices (desktop/laptop, tablet,...

Recently, survey methodology literature has put forward responsive and adaptive survey designs as means to make efficient tradeoffs between survey quality and survey costs. The designs, however, restrict quality-cost assessments to nonresponse error, while there are various design features that impact also measurement error, e.g. the survey mode, t...

Recently, various indicators have been proposed as indirect measures of non-response error in surveys. They employ auxiliary variables, external to the survey, to detect non-representative or unbalanced response. A class of designs known as adaptive survey designs maximizes these indicators by applying different treatments to different subgroups. T...

Assessing the impact of mode effects on survey estimates has become a crucial research objective due to the increasing use of mixed-mode designs. Despite the advantages of a mixed-mode design, such as lower costs and increased coverage, there is sufficient evidence that mode effects may be large relative to the precision of a survey. They may lead...

This study compares the extent of selection error (non-response and coverage error) evoked by the four major contemporary modes of data collection (face to face, telephone, mail and Web) and three sequential mixed mode designs (telephone, mail and Web with face-to-face follow-up) for the case of the Dutch Crime Victimization Survey. Sociodemographi...

This study evaluated three types of bias—total, measurement, and selection bias (SB)—in three sequential mixed-mode designs of the Dutch Crime Victimization Survey: telephone, mail, and web, where nonrespondents were followed up face-to-face (F2F). In the absence of true scores, all biases were estimated as mode effects against two different types...

Recent survey literature shows an increasing interest in survey designs that adapt data collection to characteristics of the survey target population. Given a specified quality objective function, the designs attempt to find an optimal balance between quality and costs. Finding the optimal balance may not be straightforward as corresponding optimis...

Short-term statistics (STS) are important early indicators of economic activity. The statistics are obligatory for all EU countries and also serve as input to national accounts. In most countries, short-term Statistics are based on business surveys. However, in recent years a number of countries have gradually replaced their business surveys with b...

A large-scale mixed-mode experiment linked to the Dutch Crime Victimization Survey was conducted in 2011. The experiment consisted of two waves; one wave with random assignment to one of the modes web, paper, telephone and face-to-face, and one follow-up wave to the full sample with interviewer modes only. The objective of the experiment is to esti...

Paradata provides useful observations on sampled households and persons in the evaluation, monitoring, and reduction of nonresponse error and response error. This chapter demonstrates how to use paradata for these purposes. It also discusses how to include paradata observations on sample units as auxiliary variables in monitoring and design of data...

This study applies ordinal confirmatory factor analysis for multiple groups to assess equivalence of scale, random errors and systematic (nonrandom) errors of attitudinal questions surveyed on rating scales under different survey modes (Face-to-Face [F2F], Telephone, Paper, and Web). Empirical findings from a large-scale experiment are presented. C...

Resource allocation is a relatively new research area in survey designs and has not been fully addressed in the literature. Recently, the declining participation rates and increasing survey costs have steered research interests towards resource planning. Survey organizations across the world are considering the development of new mathematical model...

We used a tailored survey design to obtain a more representative response. Paradata from previous consumer sentiments surveys and register information were used to stratify the sample into groups that differed in contact and co-operation propensity. We approached an experimental sample of 3000 households with a Web—mail—computer-assisted telephone...

Non-response is a common source of error in many surveys. Because surveys often are costly instruments, quality-cost trade-offs play a continuing role in the design and analysis of surveys. The advances of telephone, computers, and Internet all had and still have considerable impact on the design of surveys. Recently, a strong focus on methods for...

In 2011, Statistics Netherlands conducted a large-scale mixedmode experiment linked to the Crime Victimization Survey. The experiment consisted of two waves; one wave with random assignment to one of the modes web, paper, telephone and face-to-face, and one follow-up wave to the full sample with interviewer modes only. The objective of the experime...

Nonresponse is a major source of estimation error in sample surveys. The response rate is widely used to measure survey quality associated with nonresponse, but is inadequate as an indicator because of its limited relation with nonresponse bias. Schouten et al. (2009) proposed an alternative indicator, which they refer to as an indicator of represe...

In most surveys all sample units receive the same treatment and the same design features apply to all selected people and households. In this paper, it is explained how survey designs may be tailored to optimize quality given constraints on costs. Such designs are called adaptive survey designs. The basic ingredients of such designs are introduced,...

The increasing efforts and costs required to achieve survey response have led to a stronger focus on survey data collection monitoring by means of paradata and to the rise of adaptive and responsive survey designs. Indicators that support data collection monitoring, targeting and prioritising in such designs are not yet available. Subgroup response...

This volume presents an all-inclusive guide to the problem of nonresponse in household surveys, providing an overview of the theory while also describing practical implications. The book begins with a general overview of the nonresponse problem, outlining existing sources of error and guidelines for calculating response rates according to various i...

Adaptive survey designs assume that different persons or households can receive different treatments. These designs find their origin in the literature on medical statistics where treatments are varied beforehand over patient groups but also depend on the responses of patients, namely measurements during data collection. This chapter talks about st...

This chapter discusses the selection of auxiliary variables for nonresponse adjustment. Auxiliary variables are variables that are external to the survey; they are (also) measured outside the survey. The auxiliary variables used for nonresponse adjustment are often termed weighting variables, so adhering to the fact that they form the input to adju...

The idea of using the propensity score method in survey methodology was introduced by Harris Interactive. Schonlau et al. elaborate on the technique of the propensity score method. The methods presented in this chapter use response probabilities, or response propensities, for the nonresponse bias adjustment without the use of a reference survey. Th...

There are two ways to obtain information on the nonrespondents: linking data sets and re-approaching nonrespondents. For the latter case, the literature suggests two methods: the callback approach (CBA) and the basic-question approach (BQA). This chapter presents the theory behind the CBA, the BQA, and also the Politz-Simmons approach. The applicat...

There are two ways to do something about the nonresponse problem. One way is to prevent nonresponse from happening during the fieldwork of the survey. The second way of doing something about nonresponse is to correct estimates for a possible bias. This chapter is about reducing nonresponse in the field and focuses on the psychological aspects of no...

The response rate of a survey and the composition of the nonresponses are determined by many different factors. One of these factors may be the country in which a survey is conducted. The meso level consists of factors affecting nonresponse due to the survey’s design and the organization of the fieldwork. Depending on the type and amount of data av...

Nonresponse is the phenomenon where persons in the sample from the population do not provide the requested information or provide information that is not usable. Two types of nonresponses can be distinguished: unit nonresponse and item nonresponse. Item nonresponse can occur, among other ways, as a consequence of a data-editing procedure. There is...

This chapter discusses a number of situations that require a different approach to nonresponse. It describes methods to deal with a combination of item and unit nonresponse. The chapter talks about panel surveys that have become a popular way of collecting data with the introduction of the Internet as a data collection mode. A key element of the an...

Half Title Wiley Series Page Title Copyright Contents Preface

This chapter presents an overview of the developments that have led to various forms of data collection: face-to-face surveys, telephone surveys, mail surveys, CAPI, CATI, CASI, Web surveys, and mixed mode surveys. They all have a specific impact on the size and structure of the nonresponse problem. The chapter describes the several modes of data c...

This chapter describes how bivariate and multivariate methods can be used to analyze response behavior with the use of auxiliary variables. The bivariate methods can be used to evaluate the relationship between one auxiliary variable and the response behavior. Both the statistical measure and the graphical representation of a bivariate relationship...

This chapter presents models to analyze different types of nonresponse. The nested logit model and the bivariate probit model with sample selection are discussed. The chapter outlines the difference between these models with respect to the sequential nature of the response process. The correlation between the different response types is also outlin...

This chapter introduces the concept of representative response. From representative response it is a small step to quality indicators, called representativeness indicators or R-indicators for short. First, the chapter defines the representative response. It also defines R-indicators that are further linked to nonresponse bias. The chapter talks abo...

For its social and demographic surveys, Statistics Netherlands favours CAPI over cheaper modes that employ web or telephone. Due to the persuasive power and assistance of interviewers visiting selected persons or households, nonresponse in CAPI surveys is relatively low and data quality is high. However, the costs of this mode of interviewing are r...

Nonresponse is a major source of estimation error in sample surveys. The response rate is widely used to measure the quality of a sample survey associated with nonresponse. It is, however, inadequate as an indicator because of its limited relation with nonresponse bias. Schouten et al. (2009) proposed an alternative indicator, which they refer to a...

Many survey organisations focus on the response rate as being the quality indicator for the impact of non-response bias. As a consequence, they implement a variety of measures to reduce non-response or to maintain response at some acceptable level. However, response rates alone are not good indicators of non-response bias. In general, higher respon...

This volume presents an all-inclusive guide to the problem of nonresponse in household surveys, providing an overview of the theory while also describing practical implications. The book begins with a general overview of the nonresponse problem, outlining existing sources of error and guidelines for calculating response rates according to various i...