ArticlePDF Available

Abstract

Smartphone sensors allow measurement of phenomena that are difficult or impossible to capture via self-report (e.g., geographical movement, physical activity). Sensors can reduce respondent burden by eliminating survey questions and improve measurement accuracy by replacing/augmenting self-reports. However, if respondents who are not willing to collect sensor data differ on critical attributes from those who are, the results can be biased. Research on the mechanisms of willingness to collect sensor data mostly comes from (nonprobability) online panels and is hypothetical (i.e., asks participants about the likelihood of participation in a sensor-based study). In a cross-sectional general population randomized experiment, we investigate how BELLA STRUMINSKAYA is an assistant professor in the features of the request and respondent characteristics influence willingness to share (WTS) and actually sharing smartphone-sensor data. We manipulate the request to either mention or not mention (1) how participation will benefit the participant, (2) participants' autonomy over data collection, and (3) that data will be kept confidential. We assess nonparticipation bias using the administrative records. WTS and actually sharing varies by sensor task, participants' autonomy over data sharing, their smartphone skills, level of privacy concerns, and attitudes toward surveys. Fewer people agree to share photos and a video than geolocation, but all who agreed to share photos or a video actually did. Some nonresponse and nonparticipation biases are substantial and make each other worse, but others jointly reduce the overall bias. Our findings suggest that sensor-data-sharing decisions depend on sample members' situation when asked to share and the nature of the sensor task rather than the sensor type.
SHARING DATA COLLECTED WITH SMARTPHONE
SENSORS
WILLINGNESS, PARTICIPATION, AND
NONPARTICIPATION BIAS
BELLA STRUMINSKAYA*
PETER LUGTIG
VERA TOEPOEL
BARRY SCHOUTEN
DEIRDRE GIESEN
RALPH DOLMANS
Abstract Smartphone sensors allow measurement of phenomena that
are difficult or impossible to capture via self-report (e.g., geographical
movement, physical activity). Sensors can reduce respondent burden
by eliminating survey questions and improve measurement accuracy
by replacing/augmenting self-reports. However, if respondents who are
not willing to collect sensor data differ on critical attributes from those
who are, the results can be biased. Research on the mechanisms of
willingness to collect sensor data mostly comes from (nonprobability)
online panels and is hypothetical (i.e., asks participants about the
likelihood of participation in a sensor-based study). In a cross-sectional
general population randomized experiment, we investigate how
BELLA STRUMINSKAYA is an assistant professor in the Department of Methodology and Statistics at
Utrecht University, Utrecht, The Netherlands. PETER LUGTIG is an associate professor in the
Department of Methodology and Statistics at Utrecht University, Utrecht, The Netherlands. VERA
TOEPOEL is an assistant professor in the Department of Methodology and Statistics at Utrecht
University, Utrecht, The Netherlands. BARRY SCHOUTEN is a senior methodologist at Statistics
Netherlands, The Hague, The Netherlands, and a professor in the Department of Methodology
and Statistics at Utrecht University, Utrecht, The Netherlands. DEIRDRE GIESEN is a senior method-
ologist at Statistics Netherlands, The Hague, The Netherlands. RALPH DOLMANS is an information
and communication technology developer at the Blaise team at Statistics Netherlands, Heerlen,
The Netherlands. The authors thank Ole Mussmann for assistance in preparing the sensor plug-
ins; Jelmer de Groot, Annemieke Luiten, and Vivian Meertens for assistance with data collection;
three anonymous reviewers for feedback on an earlier draft of the manuscript; and Frederick
Conrad for his invaluable feedback during the finalization of the paper. Data collection was
funded by an internal grant from Statistics Netherlands. *Address correspondence to
Bella Struminskaya, Utrecht University, Padualaan 14, 3584CH, Utrecht, The Netherlands;
email: b.struminskaya@uu.nl.
doi:10.1093/poq/nfab025
V
CThe Author(s) 2021. Published by Oxford University Press on behalf of American Association for Public Opinion Research.
This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial License (http://cre-
ativecommons.org/licenses/by-nc/4.0/), which permits noncommercial re-use, distribution, and reproduction in any medium, provided
the original work is properly cited. For commercial re-use, please contact journals.permissions@oup.com
Public Opinion Quarterly
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
features of the request and respondent characteristics influence willing-
ness to share (WTS) and actually sharing smartphone-sensor data. We
manipulate the request to either mention or not mention (1) how partic-
ipation will benefit the participant, (2) participants’ autonomy over
data collection, and (3) that data will be kept confidential. We assess
nonparticipation bias using the administrative records. WTS and actu-
ally sharing varies by sensor task, participants’ autonomy over data
sharing, their smartphone skills, level of privacy concerns, and atti-
tudes toward surveys. Fewer people agree to share photos and a video
than geolocation, but all who agreed to share photos or a video actually
did. Some nonresponse and nonparticipation biases are substantial and
make each other worse, but others jointly reduce the overall bias. Our
findings suggest that sensor-data-sharing decisions depend on sample
members’ situation when asked to share and the nature of the sensor
task rather than the sensor type.
Introduction
In recent years, social and behavioral scientists have increasingly used sens-
ing technology to gain insights into human behavior. Native smartphone sen-
sors allow much more frequent and detailed measurement of phenomena that
are difficult or impossible to capture via self-report. For example, GPS sen-
sors and accelerometers can measure geographic mobility (e.g., Geurs et al.
2015) and physical activity (e.g., Rosli et al. 2013;Kapteyn et al. 2018); log-
ging smartphone use can provide knowledge about social interactions
(Stopczynski et al. 2014); and the combined use of light and ambient noise
sensors allows measurement of sleep (e.g., Wang et al. 2014).
Smartphone sensing technology is also attractive for social research be-
cause it can reduce social desirability and recall biases inherent in self-
reports. Furthermore, sensor data can be linked to auxiliary data and in-situ
self-reports collected via Ecological Momentary Assessment (e.g., Larson
and Csikszentmihalyi 1983), enabling research on topics such as how green
spaces influence a person’s mood and well-being (e.g., MacKerron and
Mourato 2013), and how physical activity is related to happiness (e.g.,
Lathia et al. 2017).
However, it remains unclear whether smartphone sensing is well suited to
studying the general population. Most studies have relied on specific popula-
tions such as students (e.g., Wang et al. 2014), recently paroled inmates
(Sugie 2018), elderly people (Fritz et al. 2017;York Cornwell and Cagney
2017), and opt-in participants. Researchers have often provided devices to
study participants. Only a few recent feasibility studies using smartphone
Page 2 of 40 Struminskaya et al.
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
sensors have collected data from general populations (Scherpenzeel 2017;
Ja¨ckle et al. 2019;Kreuter et al. 2020;McCool, Schouten, Mussmann, and
Lugtig 2021). More typically, these studies are conducted with volunteer
panels rather than randomly sampled members of the population. The pre-
existing relationship that such participants have with the organization
requesting data-sharing likely increases their trust in the organization and the
“survivorship bias” (Lynn and Lugtig 2017) in which only loyal members re-
main in a panel; both might increase willingness and sharing rates.
The general population members have to be willing and able to share
smartphone-sensor data. If the willing and unwilling sample members differ
systematically on an attribute measured by the sensor, the results will be bi-
ased. Several studies have investigated willingness to collect sensor data
(Pinter 2015;Scherpenzeel 2017;Keusch et al. 2019;Revilla, Couper, and
Ochoa 2019;Wenz, Ja¨ckle, and Couper 2019;Kreuter et al. 2020).
However, they too used data from panel studies, possibly inflating partici-
pants’ motivation and trust in the research (Hillygus, Jackson, and Young
2014;Matthijsse, De Leeuw, and Hox 2015). Moreover, previous studies
have focused on hypothetical willingness to share sensor data (Keusch et al.
2019;Wenz et al. 2019). How hypothetical willingness relates to actual par-
ticipation/sharing is unclear. Whatever the relationship, nonparticipation bias
in studies using smartphone sensing has not received much attention.
In the current study, we quantify participants’ willingness to share and the
extent to which they actually share different sensor measurements as well as
nonparticipation bias (difference between those who do and do not share) in
a representative sample of the Netherlands’ general population. We explore
the mechanisms underlying participation in sensor-based studies by asking
relevant survey questions and nonparticipation bias by comparing register
data for the survey respondents who do and do not share.
Background and Research Questions
Previous research has shown large variability in willingness to participate
and actual participation in a range of sensor measurements and tasks (e.g.,
sharing geolocation, allowing smartphone use tracking, using a smartphone
camera to take a picture or a video
1
), study populations, and countries. The
reasons for this could be variation in sample sources (e.g., probability vs.
1. We refer to smartphone cameras as sensors, although technically cameras are equipped with
image sensors. Several smartphone measurement studies include camera-related tasks: scanning
bar codes; taking pictures of receipts (Ja¨ckle et al. 2019); taking pictures of food for food diaries
(e.g., Yan 2019); or combining sensor measurements (e.g., geolocation and pictures of neighbor-
hoods, Fritz et al. 2017), making smartphone cameras important measurement tools in surveys.
Note that our definition of smartphone sensor measurement refers to any nontextual information
collected using smartphones, whether it requires active involvement of the participant for each
Sharing Data Collected with Smartphone Sensors Page 3 of 40
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
nonprobability), respondents’ trust in the survey organization and its corre-
sponding influence on their confidentiality concerns, or the perception of
particular sensors as more or less intrusive. Revilla et al. (2016) found a will-
ingness of 25 52 percent for taking pictures using one’s smartphone cam-
eras, and a willingness of 19 37 percent for sharing geolocation across six
countries. In the Dutch LISS Panel, the willingness varied from 18 percent
for taking a photo of oneself to 30 percent for sharing geolocation and
60 percent for wearing a fitness bracelet (Struminskaya et al. 2020). In an-
other study, 37 percent of the LISS Panel respondents were willing to share
their geolocation and accelerometry data, 81 percent of whom actually shared
these data; and 57 percent were willing to wear a fitness bracelet, 90 percent
of whom actually did (Scherpenzeel 2017). In the UK Understanding Society
Innovation Panel, 17 percent downloaded a budget app to photograph pur-
chase receipts (Ja¨ckle et al. 2019). In the German PASS Panel, 16 percent
downloaded a research app that tracked location and social interactions of
the participants (Kreuter et al. 2020).
To account for this variability in willingness, several studies have focused
on the mechanisms underlying willingness to participate in smartphone-sen-
sor data collection (e.g., Keusch et al. 2019;Wenz et al. 2019;Struminskaya
et al. 2020). Generally, they point to study, task, and respondent characteris-
tics that might influence stated willingness to share sensor data.
Among study/task characteristics that have been shown to affect willing-
ness to share (WTS) are: (1) autonomy, with WTS higher for tasks where
participants have perceived or actual autonomy over data collection (Keusch
et al. 2019;Revilla et al. 2019;Struminskaya et al. 2020); (2) study sponsor,
with WTS higher for university vs. market research and government statisti-
cal office (Keusch et al. 2019;Struminskaya et al. 2020); (3) framing, with
emphasizing benefit for the respondents (e.g., time savings) or for research-
ers (e.g., scientific value) showing a slight but nonsignificant increase in
WTS compared to the neutral framing (Silber et al. 2018), and a lower odds
of WTS if the benefit framing emphasized time savings (Struminskaya et al.
2020); (4) incentives, with a positive effect on WTS (e.g., Pinter 2015;
Keusch et al. 2019); and (5) providing/promising feedback, associated with
greater likelihood of WTS (Struminskaya et al. 2020).
Among respondent characteristics that have been shown to influence WTS
are: (6) privacy concerns, with high concerns about data privacy and/or secu-
rity associated with lower WTS (Ja¨ckle et al. 2019;Keusch et al. 2019;
Revilla et al. 2019;Wenz et al. 2019;Struminskaya et al. 2020); (7) smart-
phone skills, with the more frequently activities such as using GPS, taking
pictures, and online banking are performed on smartphones the higher
individual task (e.g., photographs) or a one-time active involvement and follow-up passive data
collection (e.g., geolocation).
Page 4 of 40 Struminskaya et al.
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
the WTS (Keusch et al. 2019;Wenz et al. 2019;Struminskaya et al. 2020);
(8) experience with smartphone apps or sharing sensor data, with previously
downloading a research app being associated with a higher likelihood of
WTS (Keusch et al. 2019;Struminskaya et al. 2020); and (9) perceived value
of surveys with respondents rating the survey as important for scientific
research being associated with greater willingness to share sensor data
(Struminskaya et al. 2020).
2
Thus, our first goal is to understand the WTS mechanisms among the general
population. A second goal is to investigate how patterns of nonparticipation bias
could affect substantive results. To our knowledge, only Elevelt, Lugtig, and
Toepoel (2019) have investigated nonparticipation bias in a sensor study and
found large biases on the dependent variable time use betweenthosewhoshared
geolocation and those who did not. It is important to assess how nonparticipa-
tion bias can compromise the accuracy of sensor data. However, benchmark
data that contain the information on both respondents and nonrespondents as
well as participants and nonparticipants of sensor studies is not widely available.
We improve upon earlier studies by using survey data that were collected from
a cross-sectional sample so that consent rates would not be inflated by partici-
pants’ existing relationship with the research organization and administrative
data linked to the survey data, making available information on nonrespondents
and nonparticipants in the smartphone-sensor measurements.
To address our twin goals, we ask the following research questions:
(1) What are the rates for WTS sensor data and actually participating in
particular sensor measurement tasks? (2) What study and task characteristics
influence WTS and actual sharing of smartphone-sensor data? (3) What re-
spondent characteristics influence WTS and actually sharing sensor data? (4)
What is the extent of nonparticipation bias in relation to nonresponse bias?
We tested mechanisms for WTS proposed in the literature that are con-
cerned with consent to perform smartphone sensor measurements. Here we
manipulate the wording of the consent request, in particular whether the re-
quest emphasizes benefits of participation as time saved, whether participants
are promised autonomy over data collection, and how they are assured their
data will be confidential. Empirical tests of these mechanisms have been con-
ducted on hypothetical WTS (e.g., Keusch et al. 2019;Struminskaya et al.
2020), which was affected by (1) framing (i.e., positive or negative presenta-
tion), (2) the degree to which autonomy over data collection is emphasized,
and (3) the degree to which privacy is emphasized. For mechanisms of actual
data sharing, there are no studies on smartphone sensors; however, positive
(“allow”) framing increased the rates of disclosure of sensitive information
compared to negative (“prohibit”) framing (Samat and Acquisti 2017).
2. Note that Keusch et al. (2019) did not find a significant relationship between WTS attitudes to-
ward surveys.
Sharing Data Collected with Smartphone Sensors Page 5 of 40
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
Regarding autonomy, Brandimarte, Acquisti, and Loewenstein (2013) found
that individuals who believe they have control over who can access and use
the data shared more personal information. However, Peer and Acquisti (2016)
found that emphasizing either the reversibility (i.e., ability to change the
permission granted to the researcher) or merely pointing out the irreversibility
of data-sharing decreased disclosure of sensitive information. Emphasizing
privacy practices caused participants to reflect on privacy (Tan et al. 2014;
Shih, Liccardi, and Weitzner 2015), which led to changes in data-sharing be-
havior. We test how these mechanisms apply to willingness to share and actu-
ally sharing sensor data.
Hypotheses
Studies have shown that willingness to use smartphone cameras is higher than
allowing one’s location to be tracked (Revilla et al. 2019;Wenz et al. 2019;
Keusch, Struminskaya, et al. 2020). However, different tasks performed with
cameras might differ in their perceived privacy, potentially reversing this pat-
tern. For example, respondents in a nonprobability panel were more willing to
allow geolocation tracking than to take a video of their face (Revilla et al.
2019). Although no theoretical justification for this has been proposed, geolo-
cation coordinates might be more abstract than pictures and reveal information
that participants find less sensitive. In our study, sponsored by a national statis-
tical office, respondents at home might have recognized that their addresses
were already known to the researchers, but pictures and videos of their envi-
ronment or themselves would reveal information they preferred to keep pri-
vate. We expect:
H1. WTS and actual sharing will be lower for pictures and videos of personal
content than sharing of geolocation.
In addition, features of the request affect participants’ sharing decisions.
Studies on the consent to link survey data to administrative data find positive
effects of benefit framing in terms of time savings (e.g., Sakshaug, Stegmaier,
et al. 2019), but this has failed to replicate in sensing studies (Silber et al.
2018;Struminskaya et al. 2020). Panel members, the participants in these stud-
ies, might see less benefit in being asked fewer questions—their membership
in the panel suggests they are not particularly burdened by answering survey
questions—than members of a cross-sectional sample for whom the reduced
burden of fewer questions may appear more beneficial. We hypothesize:
H2. Framing the request in terms of benefits, in particular time savings, will
increase the WTS and actual sharing of sensor-collected data compared to
neutral framing.
Page 6 of 40 Struminskaya et al.
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
Several studies have found that WTS is higher for tasks in which participants
have autonomy over data collection. WTS is higher when participants can
turn off data collection (Keusch et al. 2019) or when they can anticipate and
control what content they ultimately share with researchers. Compare being
asked to take photos or scan bar codes (more autonomy) with agreeing to have
one’s geolocation tracked or their social media posts analyzed (less autonomy).
Revilla et al. (2019) report a significant relationship between autonomy
(“Respondent Control”) and WTS. Autonomy is closely related to how ac-
tively or passively participants engage in the measurement process. Generally,
WTS is higher for sensor tasks requiring active participation, like taking photos
or scanning bar codes, than if data are collected passively, for example, track-
ing travel or posts (Wenz et al. 2019). Thus, we expect that emphasizing auton-
omy will increase perceived control over data collection and increase WTS:
H3. Emphasizing autonomy over the data-sharing process will be associated with
higher willingness to share and actual sharing of sensor data compared to a
request with no such emphasis.
When asked to share data collected on smartphones that goes beyond com-
pleting questionnaires (e.g., taking pictures, scanning bar codes, sharing
accelerometry measures or geolocation, downloading an app to track smart-
phone use), some participants express concerns about privacy (Keusch,
Struminskaya, et al. 2020). Accordingly, willingness to share sensor data is
lower for respondents with high privacy concerns (Keusch et al. 2019;
Revilla et al. 2019;Struminskaya et al. 2020). We expect that addressing
these concerns in the sharing request will lead to higher WTS and increased
rates of actual sharing because it might increase trust:
H4. Emphasizing privacy protection will be associated with increased willingness
to share and actual sharing of sensor data.
3
The final block of hypotheses is about the influence of respondent character-
istics. Participants may find some sensor data to be private (e.g., pictures or
videos of the respondent or their surroundings) or their potential use to be
abstract (e.g., geolocation). Both perceptions might raise concerns about
sharing data. Survey respondents’ concerns that the collection of paradata
might violate their privacy have been shown to negatively affect their will-
ingness to participate in surveys (Couper et al. 2008,2010;Couper and
Singer 2013). Similarly, high levels of concern about data privacy and/or
data security are associated with lower stated WTS (Keusch et al. 2019;
3. The competing hypothesis would be that emphasizing privacy increases respondents’ aware-
ness about data sensitivity and could lower the willingness, as has been shown for survey re-
sponse (Singer, Hippler, and Schwartz 1992) and actually sharing sensitive data (Peer and
Acquisti 2016).
Sharing Data Collected with Smartphone Sensors Page 7 of 40
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
Revilla et al 2019;Wenz et al. 2019;Struminskaya et al. 2020). Thus, we
expect:
H5. Respondents with high privacy concerns and high concerns about data sharing
will be less willing to share and less likely to actually share sensor-collected data.
However, privacy concerns may be mitigated by increased smartphone skill
(Keusch, Struminskaya, et al. 2020), which is associated with higher stated
willingness (e.g., Pinter 2015;Keusch et al. 2019). Thus, we hypothesize:
H6.1. The more activities participants perform on their smartphones, the higher
the WTS and actual sharing rates.
H6.2. The more frequently participants use certain sensors for specific tasks on
their smartphone, the more likely they will be willing to share and will actually
share data collected with that sensor.
Consistent with the positive association between experience downloading re-
search apps and higher stated willingness (Keusch et al. 2019;Struminskaya
et al. 2020), we expect:
H7.1. Previously downloading a research app will be associated with higher
likelihood of WTS and actual sharing.
H7.2. Previously sharing sensor-collected data will be associated with higher
likelihood of WTS and actual sharing.
Methods
SURVEY DATA
The current experiment is embedded in a survey. The sample consists of
respondents from one of the cross-sectional general population surveys that
Statistics Netherlands periodically conducts. The sample members had pre-
viously participated in at least one Statistics Netherlands survey using ei-
ther a smartphone or tablet in the preceding six months and had indicated
they were willing to be contacted again to participate in another survey.
The preceding survey(s) were conducted in the web mode and followed by
the telephone and/or face-to-face interviews; the sampling was done from
the population register, which is centrally available in the Netherlands.
Overall, 3,618 persons were invited to participate in the online survey
4
by
postal mail and received up to two reminders. Of those, 1,965 persons
4. The questionnaire was browser based and implemented in an experimental version of Blaise
5.3 that enables sensor-data collection.
Page 8 of 40 Struminskaya et al.
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
participated (AAPOR COOP2
5
¼54.3 percent). The fieldwork took place
in July–August 2018. The median questionnaire completion time was 16.2
minutes. A total of 15 (0.76 percent) respondents indicated that they were
not users of smartphones or tablets, and for those whose device could not
automatically be detected, 82 (4.17 percent) reported that they completed
the questionnaire on a PC or laptop or “other device.” These respondents
were not asked about WTS and were excluded, yielding an analytical sam-
ple of 1,868.
ADMINISTRATIVE DATA
For all survey respondents and nonrespondents, we use data from several
administrative data sources that are maintained by various Dutch authori-
ties. This information can be legally accessed and linked for all sample
members by Statistics Netherlands for the purposes of research by Statistics
Netherlands without respondents’ consent. We use these data to investigate
nonresponse bias by comparing respondents to the gross sample (i.e.,
respondents and nonrespondents), and to study nonparticipation bias by
comparing those respondents who also participate in smartphone sensor
measurement to all respondents. Nonresponse and nonparticipation bias
was calculated for age, gender, marital status,andethnic background from
the general population register maintained by municipalities; level of edu-
cation from the education register maintained by the Dutch Ministry of
Education, Culture and Science and in which all persons entering the edu-
cation system are registered (however, immigrants and persons whose edu-
cation began prior to mid-1990s are missing); number of household
members (derived from information about marital status and income from
the households register
6
maintained by Statistics Netherlands); car owner-
ship and possession of a driver’s license from the register maintained by
the Ministry of Infrastructure and Water Management and in which all mo-
torized vehicles are registered; home ownership from the dwelling register
maintained by the Ministry of the Interior and Kingdom Relations and in
which addresses are classified as self-owned or rented; employment status
from the employment register maintained by the Ministry of Social Affairs;
income information (the percentile of the respondent’s income) from the
tax register maintained by the Ministry of Finance; and urbanization
(urbanicity, size of township) derived by aggregating the addresses from
the general population register over postal codes.
5. Since our study sampled not from the population register directly but from those persons who
participated in at least one of the general population surveys and for whom it was known that
they owned a device, a cooperation rate rather than a response rate is reported.
6. Because the register does not contain household composition data for about 10 percent of the
households, this information is inferred.
Sharing Data Collected with Smartphone Sensors Page 9 of 40
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
SMARTPHONE SENSOR-DATA-SHARING REQUESTS
Respondents were asked to share their current geolocation, and photos and
videos taken on their smartphone (or tablet) (H1). Willing respondents were
shown a map with their current location (measured through GPS), which
they then had to confirm (figure 1). Respondents were asked if they would
be willing to use their smartphone or tablet camera to take a video of their
surroundings; a photo of the exterior of their house; a photo of a receipt of
a recent purchase; a photo of themselves (“selfie”). After agreeing, respond-
ents were transferred to the camera app that is used on their device for taking
photos or videos. After taking a photo or video, respondents could review it
Figure 1. Screenshots of the in-browser survey and sensor measurements.
Panel a: Screenshot for a question to share a picture of the house (We would
like to know in what kind of house you live. To receive this information, we
would like to ask you to take a photo of your house.) with benefit framing (By
sharing this information, you can skip some questions so that the completion
time will be shorter.), autonomy over data collection (You can see what infor-
mation you are sending to Statistics Netherlands and undo the measurements
later if you like.), and emphasis on privacy (The data you provide will be
treated confidentially. It will only be available to researchers conducting this
study and your personal information will not be shared with third parties. The
results of the survey will only be made available in the anonymized form. Your
data is safe in all of our surveys. Personal information can never be inferred
from the statistical information collected by Statistics Netherlands.). Panel b:
Screen on which GPS location was shown to respondent, with a question (Is
this your current location?). Panel c: Author Peter Lugtig demonstrating the
camera mode that was used for taking pictures and video.
Page 10 of 40 Struminskaya et al.
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
before sharing it with the researchers or retake the photo. We collected latitude
and longitude coordinates for geolocation and time stamps, file sizes, and other
metadata for the photos and video, but no actual photos or videos were cap-
tured, to which participants were alerted at the end of the questionnaire.
THE EXPERIMENTAL DESIGN
Survey respondents were randomly assigned to one of six sensor-data-
sharing requests, corresponding to a fully crossed 2 (framing) x 2 (auton-
omy) x 2 (privacy) design.
Benefit framing (H2): When survey respondents were asked to share their
sensor data, they were either told this would save them time (experimental
condition) by reducing the number of questions they would be asked, or the
request made no mention of time (control). For each sensor measurement,
respondents who were not willing to share data from a particular sensor re-
ceived additional questions, for example, concerning their whereabouts if
they were not willing to share geolocation, whether there were other people
present if they were not willing to share a video of their surroundings,
whether the dwelling had a balcony or terrace if they were not willing to
share a photo of the house, a question about the type of purchase, store, and
total amount if they were not willing to share a photo of a receipt, and their
happiness if they were not willing to take a selfie.
7
Autonomy over data collection (H3): The experimental condition specified
that respondents would be able to view what information they were sending
to Statistics Netherlands and could undo the measurements later. The control
condition did not include such text.
Assurance of confidentiality (H4): The experimental condition included
the following statement:
The data you provide will be treated confidentially. It will only be avail-
able to researchers conducting this study and your personal information will
not be shared with third parties. The results of the survey will only be made
available in anonymized form. Your data is safe in all of our surveys.
Personal information can never be inferred from the statistical information
collected by Statistics Netherlands. The control condition did not include
such text.
Willingness to share sensor data was measured using a binary choice
(yes/no). The individual data requests were asked in one fixed order
7. The question about happiness was asked since the request to take a selfie was prefaced with
“Using new technologies, it is possible to estimate your age and recognize your emotions from a
photo. To try it out, we would like to ask you to make a photo of your face.” This phrasing was used
to provide the rationale for the request; the recognition of the emotions was not actually performed.
Respondents were debriefed at the end of the questionnaire that the images they took were not stored.
Sharing Data Collected with Smartphone Sensors Page 11 of 40
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
(geolocation, a video, and photos of the house, receipt, and self) to maxi-
mize statistical power within the individual cells.
8
Respondents were ran-
domized in the same conditions for all five data-sharing requests.
Respondents who were not willing to share smartphone sensor data were
asked why they were not willing. Furthermore, respondents received ques-
tions about their smartphone skills, device use (H6.1, H6.2), previous data
sharing and app download experience (H7.1, H7.2), attitudes toward privacy
and data sharing (H5), attitudes toward surveys, and socio-demographic char-
acteristics. See appendix for the question wording.
Analysis Plan
To answer our first research question about the rates of willingness to
share and actually sharing smartphone-sensor data, we calculated propor-
tions of respondents who (1) answered yes to the request to share and (2)
actually shared data by the sensor tasks. To answer our second research
question about the influence of study characteristics and task features, we
used five logistic regressions predicting (1) WTS and (2) sharing condi-
tional on WTS. We account for the dependency of observations by using
clustered standard errors.
9
For our third research question, which con-
cerned the influence of respondent characteristics, we add respondent
characteristics to the logistic regressions predicting WTS and sharing. For
the survey attitude scale, we conducted a factor analysis following De
Leeuw et al. (2019), assigning items to the factors of enjoyment, value,
andburden(seeQuestion8inAppendix B), and calculating mean scores
for each dimension.
To answer our fourth research question, which concerned the prevalence
of nonresponse and nonparticipation bias, we calculated the biases using the
following formulae for the variables from the registry:
10
8. Studies that randomized the order of the hypothetical data-sharing requests have found that the
first request evoked higher WTS (Silber et al. 2018;Struminskaya et al. 2020). We look for any
evidence that the order of sharing requests might have affected the outcome in tables A2 and A3.
9. In these models, we tested the interactions of the main experimental conditions as well as con-
cerns about privacy and data sharing because perceived risk has been shown to moderate the
effects of framing (Samat and Acquisti 2017), autonomy over data collection (Peer and Acquisti
2016), and consequences for privacy (Gates et al. 2014) on data sharing. We also included the
interactions of main effects with the number of smartphone activities since concerns about sen-
sor-data sharing are negatively associated with smartphone skills (Keusch, Struminskaya, et al.
2020). The results appear in the Supplementary Material,tables S1 and S2.
10. For ease of interpretation, continuous variables were recoded into categories (age: 16–24,
25–34, 35–44, 45–54, 55–64, 65þ; number of household members: 1, 2, 3, 4þ; income percen-
tiles: 1–24, 25–49, 50–74, 75–100) with the modal category chosen as reference. Other variables
are dichotomized.
Page 12 of 40 Struminskaya et al.
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
Non Response Bias yADMIN
ðÞ
¼yADMIN;respondents yADMIN ;gross sample
Non Participation Bias yADMIN
ðÞ
¼yADMIN;participants yADMIN ;respondents
To calculate the standard errors of the differences, we adapted Lee’s
(2006) calculation of noncoverage bias to our context.
Nonresponse bias:
se prespondents pgross sample
ðÞ
¼ngross sample nrespondents
ngross sample ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
var prespondents
ðÞ
þvarðpnonrespondents Þ
q
Nonparticipation bias:
se pparticipants prespondents
ðÞ
¼nrespondents nparticipants
nrespondents ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
var pparticipants
ðÞ
þvarðpnonparticipants Þ
q;
where pis the proportion for each variable in the reference category.
To compare nonresponse and nonparticipation biases, we used one-sample
t-tests treating the sample data as benchmark data.
11
We calculated the
Average Absolute Bias across the variables from the registry by summing
the absolute value of individual biases and dividing the result by the num-
ber of variables included in the calculation. The analyses are based on
unweighted data.
Results
WTS RATES AND SHARING OF SENSOR DATA
Table 1 shows the willingness to share and actual sharing rates, with the lat-
ter being a percent of all those who indicated they were willing to share. For
example, for the first question, 67 percent of respondents are willing to share
their geolocation, but 69 percent of those (that is, 46 percent of all respond-
ents) actually share it. Each treatment’s effect on the two measures is com-
pared to the no-text (control) condition and its significance is tested with
one-sided z-tests.
11. This method is common for studies on bias (e.g., Yeager et al. 2011 used t-tests; Antoun
2015;Sakshaug, Cernat, and Raghunathan 2019; and Keusch, Ba¨hr, et al. 2020 all used z-tests).
With our large sample, t-tests and z-tests produce comparable results.
Sharing Data Collected with Smartphone Sensors Page 13 of 40
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
Table 1. Willingness to share and actual sharing conditional on willing-
ness by sensor tasks (percentages)
GPS
location
Video of
surroundings
Photo of
house
Photo of
receipt
Photo of
self
Overall (N¼1,868)
Willingness to share 66.6 15.7 12.4 18.7 14.5
Actual sharing 68.6 100.0 100.0 100.0 100.0
No additional text (N¼235)
Willingness to share 63.4 17.0 12.8 17.0 13.2
Actual sharing 74.5 100.0 100.0 100.0 100.0
Benefit framing (N¼237)
Willingness to share 59.9 12.7 9.3 14.4 12.2
0.782 0.909 0.886 0.788 0.622
Actual sharing 69.7 100.0 100.0 100.0 100.0
0.818
Autonomy over data
collection (N¼237)
Willingness to share 71.7 17.3 13.1 23.6 16.9
0.027 0.468 0.460 0.037 0.131
Actual sharing 66.5 100.0 100.0 100.0 100.0
0.942
Assurance of privacy (N¼265)
Willingness to share 62.3 15.1 12.1 17.4 16.6
0.604 0.720 0.592 0.460 0.142
Actual sharing 71.5 100.0 100.0 100.0 100.0
0.724
Benefit & autonomy (N¼224)
Willingness to share 75.9 17.9 15.2 20.1 16.5
0.002 0.407 0.229 0.200 0.159
Actual sharing 71.2 100.0 100.0 100.0 100.0
0.747
Benefit & privacy (N¼227)
Willingness to share 59.5 13.2 12.3 17.6 13.7
0.807 0.873 0.556 0.433 0.442
Actual sharing 70.4 100.0 100.0 100.0 100.0
0.781
Autonomy & privacy (N¼233)
Willingness to share 67.8 18.0 12.9 15.0 11.1
0.158 0.388 0.486 0.722 0.749
Actual sharing 60.8 100.0 100.0 100.0 100.0
0.995
(continued)
Page 14 of 40 Struminskaya et al.
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
Across the main experimental conditions, WTS and actual sharing rates do
not differ much from the rates in the control condition. Emphasizing auton-
omy over data collection for geolocation increased WTS by 8.3 p.p.
(p¼.027) and 6.6 for sharing a photo of a receipt (p¼.037). Emphasizing au-
tonomy over data collection together with benefit framing increased the
WTS for sharing geolocation by 12.5 p.p. (p¼.002). Emphasizing autonomy,
assuring privacy, and mentioning the benefits of sharing increases WTS by
9.5 p.p. (p¼.017) for geolocation and by 8.2 p.p. (p¼.016) for a photo of a
receipt. Given that autonomy over data collection is the only significant
main effect, it is possible that it might drive significant interaction effects.
The overall WTS rates for camera-related tasks (sharing a video of one’s
surroundings and three photos) vary between about 12 percent (photo of the
house) and 19 percent (photo of a receipt), and are much lower than the geo-
location rates. This provides partial support for H1. The proportion of
respondents who report willingness to share their data from camera-related
tasks who actually share their data is 100 percent.
One explanation for the lower willingness rates for camera-related tasks
might be that respondents learn from the (earlier) geolocation question that
the survey not only asks about willingness to share, but actually requests
sharing data.
A related explanation is that the results are an artifact of question order. In
studies that randomized the order of multiple data-sharing requests, the first
request achieved higher willingness than later requests (Silber et al. 2018;
Walzenbach et al. 2019;Struminskaya et al. 2020). Our experimental design
precludes directly testing whether more sharing of geolocation data is be-
cause the request is the first of many or whether respondents are simply less
Table 1. (continued)
GPS
location
Video of
surroundings
Photo of
house
Photo of
receipt
Photo of
self
Benefit & autonomy &
privacy (N¼210)
Willingness to share 72.9 14.8 11.9 25.2 15.2
0.016 0.742 0.609 0.017 0.269
Actual sharing 64.7 100.0 100.0 100.0 100.0
0.968
NOTE.—Noverall ¼1,868; actually sharing conditional on willingness to share. Treatment
effects are compared to no additional text (control) condition (Row 1). Significance based on
one sided z-tests (significant effects in bold; p-values in italics). Less than 1% of respondents
changed their initial willingness to will not share after providing the sensor-collected data;
they are treated as nonwilling.
Sharing Data Collected with Smartphone Sensors Page 15 of 40
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
open to sharing the data from camera-related tasks. To check whether sharing
one piece of data influences the subsequent data sharing for other tasks, we
analyzed the sharing patterns (tables A2 and A3). The largest group, 51.3
percent, declined all requests and did not share any data; 25 percent shared
only geolocation; 18 percent shared 2.6 pieces of data on average across all
subsequent requests, and only 5.7 percent declined all subsequent requests
after sharing two or more pieces of data. These findings argue against the or-
der of the requests determining the consent patterns, suggesting that it is the
nature of the task, that is, sharing geolocation versus camera-related tasks,
that is responsible. But we can’t conclude this definitively.
MECHANISMS OF WTS AND SHARING OF SENSOR DATA
In addition to the effects of how the data-sharing request was worded and
the nature of the task, we are interested in the effects of respondent character-
istics on WTS and actual sharing. For this, we turn to multivariate analyses
shown in table 2. We differentiate between willingness to share and actual
sharing for geolocation, but for the camera tasks, we report only WTS since
all who were willing to share did in fact share (see table 1).
Model 1 (baseline) tests the effects of the experimental conditions for the
four sensor tasks. The only significant effect is for autonomy over data col-
lection, which leads to 1.62 times higher odds of willingness to share geolo-
cation, and 1.33 higher odds of willingness to share a photo of a receipt. The
highest average marginal effect (AME) is estimated at 0.11 for geolocation
data (table 1). Thus, H3 (autonomy) is partially supported while H2 (benefit
framing) and H4 (privacy) are not.
Model 2 adds respondent characteristics as predictors. In addition to
effects of autonomy observed in model 1, some respondent characteristics af-
fect WTS in model 2. First, respondents who perform more activities on their
smartphones are more likely to be willing to share data from camera-related
tasks (supporting H6.1), although for each additional smartphone activity,
respondents are only about 1 p.p. (¼AME 100) more willing to share
data. Frequency of using smartphone camera functions does not affect WTS;
however, frequency of using GPS is significantly related to some tasks (par-
tially supporting H6.2). Also, self-assessed smartphone skills do not predict
WTS or sharing sensor data, consistent with Keusch et al. (2019) and
Struminskaya et al. (2020).
Second, previously having downloaded an app increases the odds of shar-
ing a photo of a receipt (þ12.6 p.p.); paradoxically, having been asked to
download but not downloading an app increases the odds of sharing a video
of surroundings (þ13.5 p.p.) and taking a photo of the house (þ9.7 p.p.);
taken together, these results provide partial support for H7.1. Sample sizes
are small, though, with only a handful of respondents being asked to
Page 16 of 40 Struminskaya et al.
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
Table 2. Logistic regressions predicting sensor data sharing: odds ratios, robust standard errors in parentheses, and
average marginal effects; for the camera tasks, all who were willing to share did in fact share; therefore, actual shar-
ing is not modeled, since it is identical with willingness to share
Willingness
GPS AME
Shared
GPS AME
Video of
surroundings AME
Photo of
house AME
Photo of
receipt AME
Photo of
self AME
Model 1: Experimental conditions only
Benefit framing 1.027
(0.101)
1.041
(0.128)
0.844
(0.108)
0.951
(0.134)
1.062
(0.127)
0.985
(0.130)
0.787 0.746 0.185 0.721 0.614 0.911
Autonomy 1.621
(0.161)
0.106 0.762
(0.094)
0.058 1.207
(0.154)
1.164
(0.163)
1.328
(0.158)
0.043 1.077
(0.142)
0.000 0.028 0.140 0.280 0.017 0.572
Privacy assurance 0.911
(0.090)
0.840
(0.103)
0.934
(0.119)
0.980
(0.138)
0.998
(0.119)
0.965
(0.127)
0.345 0.155 0.594 0.885 0.984 0.784
Intercept 1.641
(0.158)
2.705
(0.340)
0.190
(0.024)
0.136
(0.019)
0.194
(0.024)
0.167
(0.022)
0.000 0.000 0.000 0.000 0.000 0.000
Model fit statistics:
Log Likelihood 1178.76 769.46 811.05 700.22 896.64 771.50
Pseudo R-squared 0.0105 0.0044 0.0026 0.0009 0.0033 0.0003
N1,868 1,242 1,868 1,868 1,868 1,868
(continued)
Sharing Data Collected with Smartphone Sensors Page 17 of 40
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
Table 2. (continued)
Willingness
GPS AME
Shared
GPS AME
Video of
surroundings AME
Photo of
house AME
Photo of
receipt AME
Photo of
self AME
Model 2: Experimental conditions and respondent characteristics
Benefit framing 1.027
(0.110)
1.045
(0.133)
0.816
(0.109)
0.945
(0.140)
1.070
(0.133)
0.978
(0.137)
0.807 0.729 0.128 0.700 0.585 0.875
Autonomy 1.646
(0.177)
0.096 0.788
(0.100)
1.207
(0.160)
1.174
(0.173)
1.328
(0.165)
0.040 1.075
(0.150)
0.000 0.061 0.155 0.278 0.022 0.606
Privacy assurance 0.830
(0.089)
0.853
(0.108)
0.873
(0.116)
0.949
(0.140)
0.952
(0.118)
0.945
(0.130)
0.084 0.211 0.306 0.723 0.693 0.684
# Smartphone activities 1.035
(0.026)
1.018
(0.030)
1.079
(0.037)
0.009 1.107
(0.044)
0.010 1.069
(0.032)
0.010 1.116
(0.042)
0.012
0.170 0.547 0.025 0.010 0.026 0.004
Smartphone skill 1.107
(0.072)
1.144
(0.089)
1.022
(0.082)
0.922
(0.082)
0.974
(0.072)
1.060
(0.090)
0.120 0.084 0.785 0.363 0.718 0.490
Frequency GPS 1.190
(0.036)
0.033 1.100
(0.036)
0.019 1.029
(0.034)
1.059
(0.040)
1.063
(0.033)
1.089
(0.039)
0.010
0.000 0.003 0.391 0.132 0.051 0.016
Frequency photos 0.930
(0.046)
1.115
(0.069)
1.048
(0.066)
1.133
(0.077)
1.053
(0.059)
1.031
(0.063)
0.145 0.081 0.455 0.065 0.358 0.616
(continued)
Page 18 of 40 Struminskaya et al.
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
Table 2. (continued)
Willingness
GPS AME
Shared
GPS AME
Video of
surroundings AME
Photo of
house AME
Photo of
receipt AME
Photo of
self AME
Frequency videos 0.928
(0.057)
0.919
(0.077)
0.975
(0.079)
0.951
(0.084)
0.930
(0.068)
0.924
(0.077)
0.229 0.315 0.760 0.573 0.318 0.347
Privacy concern 0.850
(0.047)
0.031 1.051
(0.069)
0.942
(0.065)
0.933
(0.070)
0.978
(0.063)
0.897
(0.061)
0.004 0.449 0.384 0.358 0.730 0.113
Concern data sharing 0.898
(0.054)
0.910
(0.062)
0.911
(0.066)
0.907
(0.076)
0.989
(0.068)
0.897
(0.153)
0.072 0.170 0.200 0.242 0.869 0.153
Survey enjoyment 1.225
(0.065)
0.039 0.898
(0.056)
1.417
(0.090)
0.043 1.376
(0.097)
0.032 1.244
(0.074)
0.031 1.380
(0.090)
0.036
0.000 0.085 0.000 0.000 0.000 0.000
Survey value 0.120
(0.074)
0.035 0.998
(0.073)
1.074
(0.082)
1.134
(0.094)
1.220
(0.088)
0.028 1.183
(0.096)
0.019
0.003 0.979 0.351 0.129 0.006 0.039
Survey burden 0.925
(0.050)
0.949
(0.064)
0.992
(0.064)
0.923
(0.067)
0.944
(0.058)
1.047
(0.068)
0.150 0.440 0.900 0.270 0.344 0.483
Part. online surveys 0.990
(0.207)
1.324
(0.317)
0.864
(0.242)
1.056
(0.310)
1.188
(0.294)
0.630
(0.200)
0.961 0.241 0.602 0.853 0.486 0.147
(continued)
Sharing Data Collected with Smartphone Sensors Page 19 of 40
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
Table 2. (continued)
Willingness
GPS AME
Shared
GPS AME
Video of
surroundings AME
Photo of
house AME
Photo of
receipt AME
Photo of
self AME
Part. smartphone survey 1.149
(0.255)
1.102
(0.282)
1.474
(0.428)
0.988
(0.301)
0.999
(0.254)
1.355
(0.442)
0.532 0.703 0.181 0.967 0.998 0.352
Shared data (ref¼never invited)
not shared 0.386
(0.123)
0.183 1.176
(0.635)
0.314
(0.199)
0.672
(0.356)
0.272
(0.156)
0.185 0.615
(0.335)
0.003 0.764 0.068 0.453 0.024 0.373
shared 1.452
(0.957)
0.949
(0.518)
1.191
(0.611)
0.988
(0.616)
0.585
(0.328)
1.231
(0.702)
0.572 0.924 0.733 0.985 0.339 0.715
Download app (ref¼never invited)
not downloaded 1.409
(0.630)
1.454
(0.806)
2.973
(1.281)
0.135 2.630
(1.212)
0.097 2.267
(1.001)
1.580
(0.767)
0.443 0.500 0.011 0.036 0.064 0.346
downloaded 1.987
(0.938)
1.035
(0.442)
1.728
(0.750)
1.203
(0.572)
2.428
(0.938)
0.126 1.250
(0.589)
0.146 0.936 0.208 0.698 0.022 0.636
Age 1.011
(0.004)
0.002 1.004
(0.004)
1.010
(0.005)
0.001 1.022
(0.005)
0.002 1.008
(0.004)
1.016
(0.005)
0.002
0.004 0.329 0.033 0.000 0.056 0.001
(continued)
Page 20 of 40 Struminskaya et al.
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
Table 2. (continued)
Willingness
GPS AME
Shared
GPS AME
Video of
surroundings AME
Photo of
house AME
Photo of
receipt AME
Photo of
self AME
Gender (ref¼female) 1.097
(0.122)
0.914
(0.119)
0.968
(0.133)
1.191
(0.182)
0.757
(0.096)
0.040 1.053
(0.152)
0.408 0.493 0.814 0.251 0.028 0.719
Education (ref <HS)
HS & vocational 0.943
(0.150)
1.124
(0.211)
0.977
(0.194)
0.839
(0.179)
1.107
(0.207)
1.123
(0.230)
0.713 0.533 0.907 0.411 0.585 0.571
Applied uni. & Uni. 0.928
(0.146)
0.841
(0.154)
0.951
(0.191)
0.741
(0.161)
0.986
(0.186)
0.782
(0.165)
0.635 0.342 0.801 0.168 0.939 0.243
Intercept 0.275
(0.151)
1.447
(0.993)
0.012
(0.009)
0.005
(0.004)
0.011
(0.007)
0.003
(0.003)
0.018 0.590 0.000 0.000 0.000 0.000
Model fit statistics:
Log Likelihood 1051.10 737.74 747.81 631.67 836.54 693.45
Pseudo R-squared 0.1095 0.0407 0.0736 0.0885 0.0623 0.0902
N1,853 1,234 1,853 1,853 1,853 1,853
NOTE.—Marginal effects are shown only for significant predictors. Significant effects in bold; p-values in italics.
Sharing Data Collected with Smartphone Sensors Page 21 of 40
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
participate in app studies. Previously refusing to share sensor data leads to
lower willingness to share geolocation (18.3 p.p.) and lower odds of photo-
graphing a receipt (18.5 p.p.), partially supporting H7.2.
Third, being concerned about privacy reduced the odds of being willing to
share geolocation (3 p.p.) but did not affect sharing decisions for the other
sensor tasks. Concerns about data sharing are not statistically significant.
12
This provides partial support for H5.
Fourth, respondents who enjoy surveys are more likely to share all five
types of sensor data and those who value surveys are more likely to share
photos, but frequency of survey participation does not significantly affect
sharing decisions.
Finally, we find a small effect of age. A 10-year increase in age leads to
an AME of 0.02, implying a 2 p.p. increase in willingness to share geoloca-
tion or a video/photo.
NONRESPONSE AND NONPARTICIPATION BIAS
It is possible that survey respondents who are willing to share sensor data, in
addition to survey responses, differ from respondents who are not willing to
share on characteristics that may be relevant to the sensed behavior. For
example, respondents willing to share geolocation may differ in their patterns
of mobility from respondents who are not willing to share. Thus, we assessed
bias due to respondents’ nonparticipation in sensor tasks as well as the
cumulative effects of nonparticipation and survey nonresponse; that is, if
the survey respondents differ from nonrespondents on behavior, such as
mobility, this could amplify bias due to nonparticipation. Nonresponse and
nonparticipation bias were calculated using register data. Table 3 shows the
population percentages from the register variables for the sampled individu-
als, the bias in these statistics when based only on survey respondents’
answers, and when based only on the answers from participants in each
sensor task.
The nonparticipation bias is about the same magnitude as the nonresponse
bias. Because the characteristics that would ideally be used to measure non-
participation bias, such as a current location that can affect sharing of geolo-
cation, are probably not in the register, we need to use proxies. For example,
to estimate nonparticipation bias for geolocation, register variables such as
car ownership, possession of a driver’s license, and urbanicity may be rea-
sonable proxies, as they are related to mobility. In fact, nonparticipation
12. But it is possible that under certain circumstances they are significant. Since some respondent
characteristics might interact with the experimental conditions, we included these interaction
terms in a set of models presented in Supplementary Material tables S1 and S2. The benefit fram-
ing decreased the odds of sharing geolocation with higher data-sharing concerns. However, the
main effect of benefit framing on sharing geolocation was not significant.
Page 22 of 40 Struminskaya et al.
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
Table 3. Nonresponse and nonparticipation bias by sensor-tasks: percentages, standard errors in parentheses
Administrative data variables
Sample
value
(%)
Nonresponse bias
(%)
Nonparticipation bias (%)
GPS
shared
Video of
surround.
Photo of
house
Photo of
receipt
Photo of
self
Age (25–34) 21.90 -1.65
(0.063)
-1.34
(0.087)
2.81
(0.211)
-2.37
(0.229)
-0.47
(0.185)
-2.03
(0.212)
0.014 0.131 0.003 0.006 0.599 0.020
Gender (man) 42.52 0.93
(0.075)
1.10
(0.109)
1.36
(0.250)
6.34
(0.297)
-3.91
(0.227)
3.76
(0.273)
0.260 0.327 0.228 0.000 0.000 0.001
Education (high) 37.13 3.40
(0.089)
-2.22
(0.124)
-0.88
(0.280)
-6.79
(0.338)
-2.38
(0.265)
-7.54
(0.302)
0.000 0.089 0.505 0.000 0.069 0.000
Ethnic background (non-Dutch) 16.26 -1.79
(0.057)
-1.61
(0.076)
0.78
(0.181)
2.96
(0.225)
2.42
(0.173)
-2.59
(0.179)
0.002 0.034 0.339 0.001 0.004 0.000
Marital status (married) 45.79 2.66
(0.076)
0.47
(0.109)
-2.67
(0.250)
3.47
(0.297)
1.41
(0.231)
0.63
(0.273)
0.001 0.967 0.018 0.002 0.212 0.580
No. of household members (2 people) 35.78 2.87
(0.073)
-0.83
(0.106)
-0.67
(0.243)
3.47
(0.293)
-0.26
(0.225)
1.49
(0.268)
0.000 0.446 0.544 0.002 0.814 .0178
Owns a car 46.51 2.50
(0.076)
-1.44
(0.109)
0.99
(0.251)
2.48
(0.297)
-0.86
(0.231)
-0.68
(0.273)
0.003 0.202 0.379 0.028 0.442 0.549
(continued)
Sharing Data Collected with Smartphone Sensors Page 23 of 40
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
Table 3. (continued)
Administrative data variables
Sample
value
(%)
Nonresponse bias
(%)
Nonparticipation bias (%)
GPS
shared
Video of
surround.
Photo of
house
Photo of
receipt
Photo of
self
Has a driver’s license 82.93 2.79
(0.058)
-0.34
(0.079)
0.64
(0.174)
1.51
(0.200)
-1.19
(0.168)
1.64
(0.184)
0.000 0.671 0.408 0.045 0.144 0.029
Homeowner 74.38 2.15
(0.068)
0.10
(0.096)
-2.03
(0.223)
-4.60
(0.270)
-2.27
(0.207)
-1.24
(0.239)
0.003 0.918 0.043 0.000 0.024 0.211
Urban (>¼1500 addresses/km
2
) 51.50 -0.86
(0.076)
0.06
(0.109)
5.86
(0.249)
7.66
(0.293)
6.96
(0.229)
0.66
(0.273)
0.302 0.959 0.000 0.000 0.000 0.557
Size of township (>50,000) 54.24 -1.00
(0.076)
-0.57
(0.109)
5.20
(0.248)
6.34
(0.292)
3.78
(0.229)
0.67
(0.273)
0.228 0.614 0.000 0.000 0.001 0.555
In paid work 60.54 -0.38
(0.075)
0.53
(0.107)
0.75
(0.246)
-2.47
(0.294)
-1.26
(0.228)
-1.95
(0.270)
0.645 0.629 0.498 0.027 0.260 0.080
Income percentile (75
th
–100
th
) 39.99 4.18
(0.074)
-0.16
(0.108)
1.76
(0.250)
1.33
(0.297)
-1.92
(0.229)
-0.56
(0.273)
0.000 0.886 0.119 0.240 0.086 0.620
(continued)
Page 24 of 40 Struminskaya et al.
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
Table 3. (continued)
Administrative data variables
Sample
value
(%)
Nonresponse bias
(%)
Nonparticipation bias (%)
GPS
shared
Video of
surround.
Photo of
house
Photo of
receipt
Photo of
self
Average abs. bias 2.09 0.83 2.03 3.98 2.24 1.96
N3,608 1,961 862 308 235 349 269
N(education) 2,631 1,389 616 232 163 249 194
N(homeowner) 3,509 1,900 830 298 228 334 263
N(in paid work) 3,596 1,958 860 307 234 348 268
N(inc. percentile) 3,586 1,954 859 307 233 348 266
NOTE.—Reference category in parentheses for variables that initially had more than two categories. Significant effects in bold; p-values in italics.
Sharing Data Collected with Smartphone Sensors Page 25 of 40
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
biases for sharing geolocation in car ownership, possession of a driver’s li-
cense, and urbanicity are small (under 2 p.p., n.s.). Participation bias for pho-
tographing one’s home and videorecording surroundings could be affected
by the type of community one lives in. A reasonable proxy could be register
variables urbanicity and township size. The difference on these variables for
those willing to share a video of their surroundings and those who are not is
over 5 p.p. each, both statistically significant. In studies that measure home
ownership, willingness to share pictures of one’s house may be vulnerable to
nonparticipation bias, as homeowners might feel their home is more personal
than might renters and thus be less willing to share. In fact, there is signifi-
cant nonparticipation bias for this task in home ownership (4.6 p.p.) and
urbanicity (6–7 p.p.). To the extent that taking photos of receipts suffers
from nonparticipation bias, this could be due to financial attributes, although
the direction of any effects is not clear on intuitive grounds. In fact, nonparti-
cipation bias in photographing receipts for the register variables paid work
and income are small (n.s.). For taking pictures of oneself, age and gender
could be important, since younger people and women can be more willing to
take photos.
13
However, biases in age, gender, and ethnic background are
about 2-3 p.p. For this task, we find a large significant bias for education (-8
p.p.). However, we do not have a theoretical explanation for this effect.
The nonparticipation bias is conditional on nonresponse bias; that is, in or-
der to participate or not in the sensor task, one must first be a survey respon-
dent. In some cases, both biases are in the same direction, but in others they
are in opposite directions. For example, for education, a positive nonresponse
bias and a negative nonparticipation bias result in a small total bias for most
sensors. Some other biases aggravate each other; for example, non-Dutch are
less likely to participate in the survey compared to the gross sample, but
those who do respond to the survey are less likely to share a photo of them-
selves. Across sensor tasks, nonresponse and nonparticipation biases move
estimates in different directions, while there are more significant differences
for nonresponse than nonparticipation bias. More research is needed to deter-
mine whether biases generally aggravate each other or nonparticipation bias
is masked by the opposing nonresponse bias.
Discussion
Data collection using smartphone sensors has been investigated by statistical
agencies and large-scale panels in numerous studies (e.g., Ja¨ckle et al. 2019,
Kreuter et al. 2020,McCool et al. 2021). If participants who share smart-
phone sensor data differ on key outcomes from those who refuse, research
13. A study by the Pew Research Center shows that women and young adults are among the
most likely groups to use Instagram, a photo- and video-sharing platform (Auxier 2020).
Page 26 of 40 Struminskaya et al.
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
conclusions can be biased. To avoid potential biases, it is necessary to under-
stand the mechanisms of participation.
This paper validates and extends the research that focuses on hypothetical
willingness to share sensor data. We study the relationship between willing-
ness to share and actually sharing geolocation, photos, and a video. To exam-
ine this, we manipulated the wording in the invitation to share in three ways:
(1) benefit vs. neutral framing of the request, (2) emphasis on autonomy over
data collection, and (3) assurance of privacy, hypothesizing that such empha-
ses would increase both hypothetical willingness to share and actual sharing.
More private camera tasks (e.g., photographing or videotaping one’s sur-
rounding or home) evoked lower WTS (under 20 percent), whereas a request
to share geolocation, which may not reveal as much personal information as
the camera tasks, elicited the highest WTS (about 67 percent).
Emphasizing autonomy increased willingness to share and actually sharing
geolocation and increased willingness to share a photo of a receipt.
Autonomy also interacted with benefit framing to increase willingness to
share geolocation. Overall, effects of request wording are small and the ex-
planatory power of our models that only contain experimental effects is low.
Framing effects are also small or nonexistent for disclosure of sensitive infor-
mation (Gluck et al. 2016;Samat and Acquisti 2017) and hypothetical WTS
(Struminskaya et al. 2020).
Consistent with previous studies (e.g., Keusch et al. 2019), the number of
tasks performed on one’s device, having prior experience with sharing GPS,
and being asked to download an app are associated with an increased likeli-
hood of WTS and sharing, even if the prior request to share sensor data had
been denied.
Contrary to our expectations, privacy concerns decreased WTS and actual
sharing only of geolocation. Privacy preferences and subsequent sharing be-
havior do not always align: people stating they care about their privacy share
more data than they indicated they were willing to share (cf. privacy para-
dox,Norberg, Horne, and Horne 2007). In other contexts, however, individu-
als make data-sharing decisions consistent with their privacy attitudes
(Kokolakis 2017).
Being willing to share data in four camera tasks perfectly predicted actual
sharing (i.e., of those participants who indicated willingness to share, 100
percent actually shared). Presumably, the personal nature of the task re-
stricted WTS to the minority of respondents who were unphased and eager
to share photos and a video of their circumstances and themselves and so
consistently followed through on their “promise” to share. Possibly, the par-
ticular uses to which respondents were asked to put the camera rather than
using the camera per se are responsible for this pattern. A more mundane use
of the camera (e.g., photographing the nearest traffic light) may have pro-
duced higher WTS and possibly lower follow-through. In any event, for the
Sharing Data Collected with Smartphone Sensors Page 27 of 40
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
camera-related tasks examined here, the respondents were reluctant to share
the data irrespective of their privacy concerns or level of autonomy over data
collection. The camera tasks might produce information that is not germane
to the task (e.g., a messy home) and too private for most participants com-
pared with GPS coordinates. Although geolocation can potentially lead to a
greater violation of privacy than a photo, it might not be perceived that way
by respondents. If the information to be shared is perceived as very private,
varying the request will not affect the underlying—and presumably nega-
tive—preferences about sharing.
Why did benefit framing not increase data sharing? We propose this may
be due to the request coming from a trustworthy sponsor, creating a ceiling
effect above which benefit framing has no impact. This is consistent with
the finding of large framing effects only for a less trustworthy sponsor
(Samat and Acquisti 2017), and sponsorship effects on hypothetical WTS
(Keusch et al. (2019) found that participants from a market research panel
were more willing to share if the request came from a market research firm
or a university rather than a statistical agency; Struminskaya et al. (2020)
found in a university-housed panel that a university sponsor evoked the
highest hypothetical WTS, followed by a statistical agency and a market re-
search firm. Our respondents have participated in at least one survey con-
ducted by Statistics Netherlands and agreed to be invited to participate
again. They might not perceive question answering as burdensome. Thus,
eliminating some questions is not make sharing more attractive. The effect
of the study sponsor that is familiar to the respondents might also upwardly
bias WTS. Thus, our sharing rates likely fall between those of panel mem-
bers and a true population cross-section. We hope to see future research
with participants who have no experience with the study sponsor.
One implication of the current findings is that the rationale behind the
sharing request is key. Explaining the value of the information and the pur-
pose of its collection (cf. Nissenbaum 2009), thus, becomes critical in seek-
ing respondents’ consent. In our study, the purpose of the sensor tasks was
not clearly communicated mostly because the tasks were administered as
part of a methodological experiment, not to gain substantive knowledge.
The nature of the sensor task and the kind of data it captures are also key.
Our study concentrated on app-free geolocation and camera measurements to
avoid biases associated with selecting sample members who would have
downloaded an app. In practice, researchers might wish to use data from
other smartphone sensors (e.g., a microphone, a light sensor, or an acceler-
ometer). We believe our results generalize to other sensors. This is because
sensors can be classified based on the extent to which participants can di-
rectly and intentionally influence what is measured by the sensor (e.g., they
can decide what will be in a picture but not their current longitude and lati-
tude). Where a sensor falls on this continuum may affect participants’ WTS
Page 28 of 40 Struminskaya et al.
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
and help explain why relatively few who “promise” to share actually share
for some tasks (inadequate influence over what data are captured), but all of
those who indicate they are willing to share follow through for other tasks
(sufficient influence over what is captured).
For some sensors (e.g., an accelerometer), it is unlikely that participants
will understand what is captured well enough to consider if they can affect
what is captured when deciding whether or not to share the data. Thus,
researchers may wish to show the participants what kind of data they are be-
ing asked to share. We visualized the respondents’ location prior to actually
sharing, and participants indicated willingness to share these data more often
than they actually did. To avoid such “backfire effects,” researchers will
need to strike a balance between the need to increase sharing and the need to
provide detailed information about the data participants are asked to share.
Researchers must strike a similar balance when deciding whether to use sen-
sor measurements over which participants have autonomy, such as camera-
related tasks. While this kind of control increases participants’ WTS, it may
increase the very social desirability and reactivity of measurement, for exam-
ple, only photographing surroundings they believe reflect well on them, that
sensor measurement promises to mitigate.
Because the current study concerns the sharing decisions of smartphone
and tablet users, one should not generalize the results to other types of data
sharing involving the general population, such as financial records that both
users and nonusers could be asked to share. Mobile device users are younger
than nonusers, more urban, more affluent, and more educated (Antoun et al.
2018;Couper et al. 2018;Keusch, Ba¨hr, et al. 2020), and may think about
digital traces differently than nonusers. Instead, our findings indicate that
sharing decisions might be quite specific, not just to sensor data but to the
nature of the particular sensor task for which data sharing is requested, how
the purpose of collecting the sensor data is communicated, and who is asking
participants to share these data. It should no longer surprise us that partici-
pants’ decisions about their involvement in research studies requesting they
share sensor data entail considerable nuance.
Sharing Data Collected with Smartphone Sensors Page 29 of 40
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
Appendix A. Descriptive Statistics and Additional Analyses
Table A1. Descriptive statistics of variables used in the analysis
(N¼1,868)
M SD Min Max
Missing
values
Number of smartphone activities 10.588 3.522 0 16 0
Frequency of sharing GPS location 2.746 2.414 0 6 0
Frequency of taking photos 2.400 1.750 0 5 0
Frequency of taking videos 1.247 1.366 0 5 0
Self-assessed smartphone skills 3.894 1.016 1 5 0
Concern about privacy 3.819 1.504 1 7 0
Concern about sharing data with firms 4.056 1.599 1 7 0
Concern about sharing data with
governmental agencies 3.378 1.678 1 7 0
Concern about sharing data on social media 4.902 1.823 1 7 0
Mean score concern data sharing 4.112 1.419 1 7 0
Survey attitude: enjoyment 4.043 1.283 1 7 0
Survey attitude: value 5.359 1.087 1 7 0
Survey attitude: burden 3.046 1.199 1 7 0
Participated in online surveys in last 30 days .3672 .4822 0 1 0
Participated in a smartphone survey
in last 30 days* .2928 .4022 0 1 0
Sharing of GPS, photos, videos 0
Invited not shared .0246 .1550 0 1
Never invited .9668 .1792 0 1
Shared .0176 .1318 0 1
Download of research app 0
Invited not downloaded .0155 .1237 0 1
Never invited .9668 .1792 0 1
Downloaded .0177 .1318 0 1
Age 43.515 18.274 16 98 0
Gender (male) .4288 .4950 0 1 0
Education:
**
15
Less than HS (ref.) .1743 .3795 0 1
HS & vocational .3815 .4859 0 1
Applied univ. & univ. .4441 .4970 0 1
*
The question about participating in a smartphone survey was only asked to those who had
indicated to participate in an online survey; people who did not participate in an online survey
are recoded as 0.
**
For 29 respondents who chose the category “other,” information on education was
replaced by the information from the registry data from 2016, reducing the number of missing
values to 15 (0.9%).
Page 30 of 40 Struminskaya et al.
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
Appendix B. Question Wordings Used in the Analyses
Prior to starting a questionnaire, a respondent had to provide their name, sex,
date of birth, and marital status, which were checked against the frame data of
Statistics Netherlands. If respondents’ sex or date of birth deviated from the
record, they were not allowed to proceed with the questionnaire. The question
wordings of those checks are not included.
Note that the wording of questions was adapted for the type of device
(smartphone or tablet). For better readability, we only present smartphone
wordings here.
Questions marked with * are modeled after Keusch et al. (2019). Questions
marked with ** are modeled after Couper et al. (2008;2010).
Table A3. Distribution of sharing by other pattern
Shared data for: Percent N
one of the tasks 12.50 42
two of the tasks 37.50 126
three of the tasks 27.08 91
four of the tasks 22.92 77
Total 100.00 336
Mean, (SD) other pattern 2.60 .97
Table A2. Percentage of participants by sharing pattern
Pattern Percent N
Shared data for all five tasks: geolocation, video, photo house,
photo receipt, photo self 1.93 36
Shared geolocation, video, photo house, photo receipt; No photo
self 1.07 20
Shared geolocation, video, photo house; No photo receipt, no
photo self 0.80 15
Shared geolocation, video; No photo house, no photo receipt, no
photo self 1.93 36
Shared geolocation only; No video, no photo house, no photo re-
ceipt, no photo self 25.00 467
Did not share anything 51.28 958
Other pattern 17.99 336
Total 100.00 1,898
Sharing Data Collected with Smartphone Sensors Page 31 of 40
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
Q1.* Do you use your smartphone for the following activities? (yes/no)
Q1a. Sending messages (for example, through SMS, WhatsApp,
or Telegram)
Q1b. Browsing websites
Q1c. Reading and/or writing email
Q1d. Taking photos
Q1e. Taking videos
Q1f. Looking at content on social media websites/apps (for example
looking at text, images, videos on Facebook, Twitter, Instagram)
Q1g. Posting content to social media websites/apps (for example post-
ing text, images, videos on Facebook, Twitter, Instagram)
Q1h. Making purchases (for example buying books or clothes,
booking train tickets, ordering food)
Q1i. Online banking (for example checking account balance,
transferring money)
Q1j. Installing new apps (for example from the App Store,
Google Play Store)
Q1k. Using GPS/location-aware apps (for example Google Maps,
Foursquare, Yelp)
Q1l. Connecting to other electronic devices via Bluetooth (for ex-
ample smartwatches, fitness bracelets, step counter)
Q1m. Calling (also through Skype or Facetime)
Q1n. Playing games
Q1o. Streaming videos or music
Q1p. Other, please specify _________________________
If Q1d¼yes
Q2. How often do you take photos using your smartphone?
1. Several times a day or more often
2. Every day
3. Several times a week
4. Several times a month
5. Once a month or less often
If Q1e ¼yes
Q3. How often do you take videos using your smartphone?
1. Several times a day or more often
2. Every day
3. Several times a week
4. Several times a month
5. Once a month or less often
If Q1k ¼yes
Page 32 of 40 Struminskaya et al.
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
Q4. How often do you use GPS/location-aware apps using your smartphone?
1. GPS is always on
2. Several times a day or more often
3. Every day
4. Several times a week
5. Several times a month
6. Once a month or less often
Q5.* Generally, how would you rate your skills of using your smartphone on
a scale from 1 ¼Beginner to 5 ¼Advanced?
1. Beginner
2. Somewhat more than a beginner
3. Average skilled
4. Somewhat more than average
5. Advanced
Q6.* How concerned are you about whether or not each of the following
organizations will share personal data with other parties? (1 Not at all con-
cerned, 7 Very concerned)
a. Private companies
b. Government agencies (such as municipality, national
government)
c. Social media platforms such as Facebook, Twitter, or
Instagram
Q7.** In general, how worried are you about your personal privacy? (1 Not
worried at all, 7 Very worried)
Q8. *** We would like to ask you some questions about research. Could you
please indicate to what extent you agree or disagree with the following state-
ments? (1 ¼totally disagree, 7 ¼totally agree)
a. I generally enjoy responding to questionnaires through
the mail or Internet. (E)
b. I really enjoy being interviewed for a survey. (E)
c. Surveys are interesting in themselves. (E)
d. Surveys are important for society. (V)
e. A lot can be learned from information collected
through surveys. (V)
f. Completing surveys is a waste of time. (V, neg.)
g. I receive far too many requests to participate in sur-
veys. (B)
h. Opinion polls are an invasion of privacy. (B)
i. It is exhausting to answer so many questions in a sur-
vey. (B)
Sharing Data Collected with Smartphone Sensors Page 33 of 40
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
(Letters in parentheses indicate constructs of the survey attitude scale by De
Leeuw et al. (2019): Enjoyment, Value, Burden). For the survey attitude
scale, we conducted a factor analysis that showed a three-factor solution but
one survey value item loaded on both the value and the burden factors. We
followed De Leeuw et al. (2019) in assigning items to the factors enjoyment,
value, and burden.
Q9. In addition to the questions in this survey we would like to collect data on
the location where you are completing this survey. This can be done using
sensors of a smartphone or a tablet. We are interested whether we can make
use of your location data. We will do it once, only for this questionnaire.
[Experimental conditions repeated for all sensor data requests]:
Benefit framing: By sharing this information, you can skip some questions so
that the completion time will be shorter
Autonomy: You can see what information you are sending to Statistics
Netherlands and undo the measurements later if you like
Assurance of confidentiality: The data you provide will be treated confiden-
tially. It will only be available to researchers conducting this study and your
personal information will not be shared with third parties. The results of the
survey will only be made available in the anonymized form. Your data is safe
in all of our surveys. Personal information can never be inferred from the sta-
tistical information collected by Statistics Netherlands.
Do you give permission to share your location?
1. Yes, I give permission to share my location
2. No, I do not give permission to share my location
Q10. We are interested in which situations respondents fill out surveys. For
example, whether they are surrounded by other people or alone, and in what
kind of space they are. Would you make a short video of your surroundings?
Maximal 5 seconds.
1. Yes, I will make a video
2. No, I will not make a video
Q11. We would like to know in what kind of house you live. To receive this
information, we would like to ask you to take a photo of your house.
1. Yes, I will take a photo of my house
2. No, I will take a photo of my house
Page 34 of 40 Struminskaya et al.
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
Q12. We would like to know for our research whether it is possible to infer in-
formation about purchases from a photo of a receipt. Would you take a photo
of a recent receipt? If possible, take a photo of a full receipt.
1. Yes, I will take a photo of a receipt.
2. No, I will not take a photo of a receipt.
Q13. Using new technologies, it is possible to estimate your age and recog-
nize your emotions from a photo. To try it out, we would like to ask you to
make a photo of your face.
1. Yes, I will take a photo of myself.
2. No, I will not take a photo of myself.
Q14. How many online questionnaires did you complete in the past 30 days?
This can be short questionnaires of one or more questions. Do not include this
questionnaire in your count.
1. None
2. 1–5 surveys
3. 6–10 surveys
4. More than 10 surveys
If Q14 >1
Q15. How many of these online questionnaires did you complete on your
smartphone? Do not include this questionnaire in your count.
1. None
2. 1 survey
3. 2 surveys
4. 3 surveys
5. 4 surveys
6. 5 surveys
7. 6-10 surveys (shown if Q14 >2)
8. More than 10 surveys (shown if Q14 >3)
Q16.* Have you ever participated in a study, where you were asked to share
your geographic location, take photos or videos using your smartphone or
tablet?
1. Yes
2. No
IF Q16 ¼yes
Sharing Data Collected with Smartphone Sensors Page 35 of 40
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
Q17.* Did you then share your geographic location, take photos or videos us-
ing your smartphone or tablet?
1. Yes
2. No
Q18.*Have you ever participated in a study, where you were asked to down-
load a research app that automatically collects data such as about your GPS-
location, the apps you are using, or the websites you are visiting?
1. Yes
2. No
IF Q18 ¼yes
Q19.* Did you actually download the research app to your smartphone or
tablet?
1. Yes
2. No
Q20. What is the highest level of education that you have completed?
1. Basisonderwijs, lager onderwijs [Basic] (less than HS)
2. LBO, VMBO, VBO, lwoo, vso, vglo, mavo, ulo,
mulo [Low] (less than HS)
3. MBO, havo, atheneum, VWO, gymnasium, mms, hbs
[Middle] (Vocational)
4. HBO, WO [High] (Uni & Applied Uni.)
5. Een andere opleiding [Other]
6. Geen opleiding voltooid [None] (Less than HS)
Age and gender verified at the beginning of the survey based on the registry data.
Note
For all analyses, we used Stata version 16.0.
Data Availability Statement
REPLICATION DATA AND DOCUMENTATION are not available because
of the permission policy of the original data collector. All data used in this study
are stored at Statistics Netherlands and require on-site secure access of authorized
persons due to the sensitive nature of the sensor data such as the geolocation and
administrative data. The editors have waived the journal’s replication policy for
this manuscript. Please contact the corresponding author for more information.
The statistical code for the analysis is available here: https://github.com/peterlug
tig/data_archive_POQ21_Shari_data_collected_with_smartphone_sensors.
Page 36 of 40 Struminskaya et al.
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
Supplementary Material
SUPPLEMENTARY MATERIAL may be found in the online version of
this article: https://doi.org/10.1093/poq/nfab025.
References
Antoun, Christopher. 2015. “Mobile Web Surveys: A First Look at Measurement,
Nonresponse, and Coverage Errors.” Doctoral dissertation, University of Michigan.
Antoun, Christopher, Frederick G. Conrad, Mick P. Couper, and Brady T. West. 2018.
“Simultaneous Estimation of Multiple Sources of Error in a Smartphone-Based Survey.”
Journal of Survey Statistics and Methodology 7:93–117.
Auxier, Brooke. 2020. “8 Facts about Americans and Instagram. Pew Research Center,
October 21, 2020.” Available at https://www.pewresearch.org/fact-tank/2020/10/21/8-facts-
about-americans-and-instagram/.
Brandimarte, Laura, Alessandro Acquisti, and George Loewenstein. 2013. “Misplaced
Confidences: Privacy and the Control Paradox.” Social Psychological and Personality
Science 4:340–47.
Couper, Mick P., Garret Gremel, William Axinn, Heidi Guyer, James Wagner, and Brady T.
West. 2018. “New Options for National Population Surveys: The Implications of Internet
and Smartphone Coverage.” Social Science Research 73:221–35.
Couper, Mick P., and Eleanor Singer. 2013. “Informed Consent for Web Paradata Use.”
Survey Research Methods 7:57–67.
Couper, Mick P., Eleanore Singer, Frederick G. Conrad, and Robert M. Groves. 2008. “Risk
of Disclosure, Perceptions of Risk, and Concerns about Privacy and Confidentiality as
Factors in Survey Participation. “Journal of Official Statistics 24:255–75.
———. 2010. “Experimental Studies of Disclosure Risk, Disclosure Harm, Topic Sensitivity,
and Survey Participation.” Journal of Official Statistics 26:287–300.
De Leeuw, Edith D., Joop J. Hox, Henning Silber, Bella Struminskaya, and Corrie Vis. 2019.
“Development of an International Survey Attitude Scale: Measurement Equivalence,
Reliability, and Predictive Validity.” Measurement Instruments for the Social Sciences 1(9):
1–10.
Elevelt, Anne, Peter Lugtig, and Vera Toepoel. 2019. “Doing a Time Use Survey on
Smartphones Only: What Factors Predict Nonresponse at Different Stages of the Survey
Process?” Survey Research Methods 13:195–213.
Fritz, Heather, Tarraf Wassim, Dan J. Saleh, and Malcolm P. Cutchin. 2017. “Using a
Smartphone-Based Ecological Momentary Assessment Protocol with Community Dwelling
Older African Americans.” Journal of Gerontology: Psychological Sciences and Social
Sciences 72:876–87.
Gates, Christopher S., Jing Chen, Ninghui Li, and Robert W. Proctor. 2014. “Effective Risk
Communication for Android Apps. 2014.” IEEE Transactions on Dependable and Secure
Computing 11:252–65.
Geurs, Karst T., Tom Thomas, Marcel Bijlsma, and Salima Douhou. 2015. “Automatic Trip
and Mode Detection with MoveSmarter: First Results from the Dutch Mobile Mobility
Panel.” Transportation Research Procedia 11:247–62.
Gluck, Joshua, Florian Schaub, Amy Friedman, Hana Habib, Norman Sadeh, Lorrie Faith
Cranor, and Yuvraj Agarwal. 2016. “How Short Is Too Short? Implications of Length and
Framing on the Effectiveness of Privacy Notices.” In Twelfth Symposium on Usable Privacy
Sharing Data Collected with Smartphone Sensors Page 37 of 40
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
and Security (SOUPS 2016), pp. 321–40. Available at https://www.usenix.org/conference/
soups2016/technical-sessions/presentation/gluck.
Hillygus, D. Sunshine, Natalie Jackson, and McKenzie Young. 2014. “Professional
Respondents in Nonprobability Online Panels.” In Online Panel Research: A Data Quality
Perspective, edited by. M. Callegaro, P. J. Lavrakas, J. A. Krosnick, R. Baker, J. D.
Bethlehem, and A. S. Go¨ ritz, pp. 219–37. New York: Wiley.
Ja¨ ckle, Annette, Jonathan Burton, Mick P. Couper, and Carli Lessof. 2019. “Participation in a
Mobile App Survey to Collect Expenditure Data as Part of a Large-Scale Probability
Household Panel: Coverage and Participation Rates and Biases.” Survey Research Methods
13:23–44.
Kapteyn, Arie, James Banks, Mark Hamer, James P. Smith, Andrew Steptoe, Arthur Van
Soest, Annemarie Koster, and Saw Htay Wah. 2018. “What They Say and What They Do:
Comparing Physical Activity across the USA, England, and the Netherlands.” Journal of
Epidemiological Community Health 72:471–76.
Keusch, Florian, Sebastian Ba¨ hr, Georg-Christoph Haas, Frauke Kreuter, and Mark
Trappmann. 2020. “Coverage Error in Data Collection Combining Mobile Surveys With
Passive Measurement Using Apps: Data from a German National Survey.” Sociological
Methods and Research.https://doi-org.proxy.library.uu.nl/10.1177/0049124120914924.
Keusch, Florian, Bella Struminskaya, Christopher Antoun, Mick P. Couper, and Frauke
Kreuter. 2019. “Willingness to Participate in Passive Mobile Data Collection.” Public
Opinion Quarterly 83:210–35.
Keusch, Florian, Bella Struminskaya, Kreuter Frauke, and Weichbold. Martin 2020.
“Combining Active and Passive Mobile Data Collection: A Survey of Concerns.” In Big
Data Meets Survey Science: A Collection of Innovative Methods, edited by Craig A. Hill,
Paul P. Biemer, Buskirk Trent, Japec Lilli, Kirchner Antje, Kolenikov Stas, and Lars E.
Lyberg 657–82. Hoboken, NJ: Wiley.
Kokolakis, Spyros. 2017. “Privacy Attitudes and Privacy Behaviour: A Review of Current
Research on the Privacy Paradox Phenomenon.” Computers and Security 64:122–34.
Kreuter, Frauke, Georg-Christof Haas, Florian Keusch, Sebastian Ba¨hr, and Mark. Trappman
2020. “Collecting Survey and Smartphone Sensor Data with an App: Opportunities and
Challenges Around Privacy and Informed Consent.” Social Science Computer Review 38:
533–49.
Larson, Reed, and Mihaly Csikszentmihalyi. 1983. “The Experience Sampling Method.” In
Naturalistic Approaches to Studying Social Interaction. New Directions for Methodology of
Social and Behavioral Science, edited by H. T. Reis, 15:41–56. San Francisco: Jossey-Bass.
Lathia, Neal, Gillian M. Sandstrom, Cecilia Mascolo, and Peter J. Rentfrow. 2017. “Happier
people live more active lives: Using smartphones to link happiness and physical activity.”
PloS One, 12(1):e0160589.
Lee, Sunghee. 2006. “An Evaluation of Nonresponse and Coverage Errors in a Prerecruited
Probability Web Panel.” Social Science Computer Review 24:460–75.
Lynn, Peter, and Peter Lugtig. 2017. “Total Survey Error for Longitudinal Surveys.” In Total
Survey Error in Practice, edited by P. P. Biemer, E. de Leeuw, S. Eckman, B. Edwards, F.
Kreuter, L. E. Lyberg, N. C. Tucker, and B. West, pp. 279–98. New York: Wiley.
MacKerron, George, and Susana Mourato. 2013. “Happiness Is Greater in Natural
Environments.” Global Environmental Change 23:992–1000.
Matthijsse, Suzette M., Edith D. De Leeuw, and Joop J. Hox. 2015. “Internet Panels,
Professional Respondents, and Data Quality.” Methodology 11(3):81–88.
McCool, Danielle, Schouten Barry, Ole Mussmann, and Peter Lugtig. 2021. “An app-assisted
travel survey in official statistics. Possibilities and challenges.” Journal of Official Statistics
37:149–70.
Nissenbaum, Helen. 2009. Privacy in Context. Stanford, CA: Stanford University Press.
Page 38 of 40 Struminskaya et al.
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
Norberg, Patricia A., Daniel R. Horne, and David A. Horne. 2007. “The Privacy Paradox:
Personal Information Disclosure Intentions versus Behaviors.” Journal of Consumer Affairs
41:100–126.
Peer, Eyal, and Alessandro Acquisti. 2016. “The Impact of Reversibility on the Decision to
Disclose Personal Information.” Journal of Consumer Marketing 33:428–36.
Pinter, Robert. 2015. “Willingness of Online Access Panel Members to Participate in
Smartphone Application-Based Research.” In Mobile Research Methods, edited by D.
Toninelli, R. Pinter, and P. de Pedraza, 141–56. London: Ubiquity Press.
Revilla, Melanie, Mick P. Couper, and Ochoa. Carlos 2019. “Willingness of Online Panelists
to Perform Additional Tasks.” Methods, data, analyses 13:223–52.
Revilla, Melanie, Daniele Toninelli, Carlos Ochoa, and Loewe German 2016. “Do Online
Access Panels Need to Adapt Surveys for Mobile Devices?” Internet Research 26:1209–27.
Rosli, Mohd Hafiz, Mohamad Taufik Hidayat Baharuldin, Muhammad Nazrul Hakim
Abdullah, Mohd Sofian Omar Fauzee, Yunus Adam, and Azhar Yaacob. 2013. “Coupling
GPS with Accelerometer to Measure Physical Activity.” Procedia—Social and Behavioral
Sciences 91:171–78.
Sakshaug, Joseph, Alexandru Cernat, and Trivellore E. Raghunathan. 2019. “Do Sequential
Mixed-Mode Surveys Decrease Nonresponse Bias, Measurement Error Bias, and Total Bias?
An Experimental Study.” Journal of Survey Statistics and Methodology 7:545–71.
Sakshaug, Joseph, Jens Stegmaier, Mark Trappmann, and Frauke Kreuter. 2019. “Does Benefit
Framing Improve Record Linkage Consent Rates? A Survey Experiment.” Survey Research
Methods 13:289–304.
Samat, Sonam, and Alessandro Acquisti. 2017. “Format vs. Content: The Impact of Risk and
Presentation on Disclosure Decisions.” Paper presented at the Thirteenth Symposium on
Usable Privacy and Security (SOUPS), 377–84. Available at https://www.usenix.org/confer
ence/soups2017/technical-sessions/presentation/samat-disclosure.
Scherpenzeel, Annette. 2017. “Mixing Online Panel Data Collection with Innovative
Methods.” In Methodische Probleme von Mixed-Mode-Ansa¨ tzen in der Umfrageforschung
[Methodological Problems of Mixed-Mode Approaches in Survey Research], edited by S.
Eifler and F. Faulbaum,27–49. Wiesbaden: Springer.
Shih, Fuming, Ilaria Liccardi, and Daniel Weitzner. 2015. “Privacy Tipping Points in
Smartphones Privacy Preferences.” Proceedings of the 33rd Annual ACM Conference on
Human Factors in Computing Systems, 807–16.
Silber, Henning, Bernd Weiss, Florian Keusch, Christoph Beuthner, and Jette Schro¨der. 2018.
“Framing Consent Questions in Mobile Surveys: Experiments on Question Wording.” Paper
presented at the BigSurv18 Conference, Barcelona, Spain.
Singer, Eleanor, Hans-Ju¨ rgen Hippler, and Norbert Schwartz. 1992. “Confidentiality
Assurances in Surveys: Reassurance or Threat?” International Journal of Public Opinion
Research 4:256–68.
Stopczynski, Arkadiusz, Vedran Sekara, Piotr Sapiezynski, Andrea Cuttone, Mette My
Madsen, Jakob Eg Larsen, and Sune Lehmann. 2014. “Measuring Large Scale Social
Networks with High Resolution.” PLOS One 9(4):e95978.
Struminskaya, Bella, Vera Toepoel, Peter Lugtig, Barry Schouten, Marieke Haan, and
Annemieke Luiten. 2020. “Understanding Willingness to Share Smartphone-Sensor Data.”
Public Opinion Quarterly 84(3):725–59.
Sugie, Naomie F. 2018. “Utilizing Smartphones to Study Disadvantaged and Hard-to-Reach
Groups.” Sociological Methods and Research 47:458–91.
Tan, Joshua, Khanh Nguyen, Michael Theodorides, Heidi Negro´ n-Arroyo, Christopher
Thompson, Serge Egelman, and David Wagner. 2014. “The Effect Of Developer-Specified
Explanations for Permission Requests on Smartphone User Behavior.” In Proceedings of the
SIGCHI Conference on Human Factors in Computing Systems, 91–100.
Sharing Data Collected with Smartphone Sensors Page 39 of 40
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
Walzenbach, Sandra, Jon Burton, Mick P. Couper, Thomas F. Crossley, and Annette Ja¨ckle.
2019. “Foot-in-the-Door or Door-in-the-Face? A Survey Experiment on Multiple Requests
for Consent to Data Linkage.” Paper presented at the European Survey Research
Association (ESRA) Conference, Zagreb, Croatia.
Wang, Rui, Fanglin Chen, Zhenyu Chen, Tianxing Li, Gabriella Harari, Stefanie Tignor, Xia
Zhou, Dror Ben-Zeev, and Andrew T. Campbell. 2014. “StudentLife: Assessing Mental
Health, Academic Performance and Behavioral Trends of College Students Using
Smartphones.” In Proceedings of the 2014 ACM International Joint Conference on
Pervasive and Ubiquitous Computing, 3–14.
Wenz, Alexander, Annette Ja¨ckle, and Mick P. Couper. 2019. “Willingness to Use Mobile
Technologies for Data Collection in a Probability Household Panel.” Survey Research
Methods 13:1–22. doi: 10.18148/srm/2019.v1i1.7298
Yan, T. 2019. “The Feasibility of Using Smartphones to Record Food Acquisitions.” Paper
presented at the first Mobile Apps and Sensors in Surveys (MASS) Workshop, Mannheim,
Germany.
Yeager, David S., Jon A. Krosnick, LinChiat Chang, Harold S. Javitz, Matthew S.
Levendusky, Alberto Simpser, and Rui Wang. 2011. “Comparing the Accuracy of RDD
Telephone Surveys and Internet Surveys Conducted with Probability and Non-Probability
Samples.” Public Opinion Quarterly 75:709–47.
York Cornwell, Erin, and Kathleen A. Cagney. 2017. “Aging in Activity Space: Results from
Smartphone-Based GPS-Tracking of Urban Seniors.” Journals of Gerontology: Series B 72:
864–75.
Page 40 of 40 Struminskaya et al.
Downloaded from https://academic.oup.com/poq/advance-article/doi/10.1093/poq/nfab025/6363382 by guest on 06 September 2021
... Thirty-seven percent of them indicated their willingness to participate, and 68 percent of the willing panelists actually participated for an overall participation rate of 19 percent. Struminskaya et al. (2021) asked smartphone users in the general Dutch population who previously had participated in at least one of Statistics Netherlands' surveys to share their current GPS location once. Sixty-seven percent of the survey respondents said they were willing to do so, and 69 percent of those who reported willingness actually did share the data in a subsequent step, for an overall participation rate of 46 percent. ...
... For example, compared to the other studies, the relatively high participation rate reported in Sugie (2018) might be explained by the specific target population-parolees recently released from prison-who might be more acquiescent as well as the fact that participants received an Android smartphone, which they could keep after the study, and plans with unlimited call, text, and data for the duration of the data collection. Struminskaya et al. (2021) asked the survey participants to share their geoposition only once, which might be perceived as less intrusive than continuous collection of GPS data. For a study like the one described in this paper that does not provide a smartphone and data plan as incentives and that continuously collects different types of data over a longer period, we expect to find a lower participation rate. ...
... In the IAB-SMART study, we found that out of the 4,293 German Android smartphone owners invited to the study, 14.5 percent downloaded the app, went through the installation process, completed the welcome survey, and were verified as eligible participants. This participation rate is somewhat lower than what other studies reported when recruiting participants from the general population (Elevelt et al., 2019;Scherpenzeel, 2017;Struminskaya et al., 2021). While the relatively low participation rate seems disappointing on first sight, one has to consider that most earlier studies have collected one type of sensor data only (e.g. ...
Article
Full-text available
Research apps allow to administer survey questions and passively collect smartphone data, thus providing rich information on individual and social behaviours. Agreeing to this novel form of data collection requires multiple consent steps, and little is known about the effect of non-participation. We invited 4,293 Android smart-phone owners from the German Panel Study Labour Market and Social Security (PASS) to download the IAB-SMART app. The app collected data over six months through (a) short in-app surveys and (b) five passive mobile data collection functions. The rich information on PASS members from previous survey waves allows us to compare participants and non-participants in the IAB-SMART study at the individual stages of the participation process and across the different types of data collected. We find that 14.5 percent of the invited smartphone users installed the app, between 12.2 and 13.4 percent provided the different types of passively collected data, and 10.8 percent provided all types of data at least once. Likelihood to participate was smaller among women, decreased with age and increased with educational attainment, German citizen-This is an open access article under the terms of the Creative Commons Attribution-NonCommercial License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes.
Article
Full-text available
Images might provide richer and more objective information than text answers to open‐ended survey questions. Little is known, nonetheless, about the consequences for data quality of asking participants to answer open‐ended questions with images. Therefore, this paper addresses three research questions: (1) What is the effect of answering web survey questions with images instead of text on breakoff, noncompliance with the task, completion time and question evaluation? (2) What is the effect of including a motivational message on these four aspects? (3) Does the impact of asking to answer with images instead of text vary across device types? To answer these questions, we implemented a 2 × 3 between‐subject web survey experiment (N = 3043) in Germany. Half of the sample was required to answer using PCs and the other half with smartphones. Within each device group, respondents were randomly assigned to (1) a control group answering open‐ended questions with text; (2) a treatment group answering open‐ended questions with images; and (3) another treatment group answering open‐ended questions with images but prompted with a motivational message. Results show that asking participants to answer with images significantly increases participants' likelihood of noncompliance as well as their completion times, while worsening their overall survey experience. Including motivational messages, moreover, moderately reduces the likelihood of noncompliance. Finally, the likelihood of noncompliance is similar across devices.
Article
With the ubiquity of smartphones, it is possible to collect self-reports as well as to passively measure behaviors and states (e.g., locations, movement, activity, and sleep) with native sensors and the smartphone’s operating system, both on a single device that usually accompanies participants throughout the day. This research synthesis brings structure to a rapidly expanding body of literature on the combined collection of self-reports and passive measurement using smartphones, pointing out how and why researchers have combined these two types of data and where more work is needed. We distinguish between five reasons why researchers might want to integrate the two data sources and how this has been helpful: (1) verification, for example, confirming start and end of passively detected trips, (2) contextualization, for example, asking about the purpose of a passively detected trip, (3) quantifying relationships, for example, quantifying the association between self-reported stress and passively measured sleep duration, (4) building composite measures, for example, measuring components of stress that participants are aware of through self-reports and those they are not through passively measured speech attributes, and (5) triggering measurement, for example, asking survey questions contingent on certain passively measured events or participant locations. We discuss challenges of collecting self-reports and passively tracking participants’ behavior with smartphones from the perspective of representation (e.g., who owns a smartphone and who is willing to share their data), measurement (e.g., different levels of temporal granularity in self-reports and passively collected data), and privacy considerations (e.g., the greater intrusiveness of passive measurement than self-reports). While we see real potential in this approach it is not yet clear if its impact will be incremental or will revolutionize the field.
Article
Full-text available
The increasing volume of "Big Data" produced by sensors and smart devices can transform the social and behavioral sciences. Several successful studies used digital data to provide new insights into social reality. This special issue argues that the true power of these data for the social sciences lies in connecting new data sources with surveys. While new digital data are rich in volume, they seldomly cover the full population nor do they provide insights into individuals' feelings, motivations, and attitudes. Conversely, survey data, while well suited for measuring people's internal states, are relatively poor at measuring behaviors and facts. Developing a methodology for integrating the two data sources can mitigate their respective weaknesses. Sensors and apps on smartphones are useful for collecting both survey data and digital data. For example, smartphones can track people's travel behavior and ask questions about its motives. A general methodology on the augmentation of surveys with data from sensors and apps is currently missing. Issues of representativeness, processing, storage, data linkage, and how to combine survey data with sensor and app data to produce one statistic of interest pertain. This editorial to the special issue on "Using Mobile Apps and Sensors in Surveys" provides an introduction to this new field, presents an overview of challenges , opportunities, and sets a research agenda. We introduce the four papers in this special issue that focus on these opportunities and challenges and provide practical applications and solutions for integrating sensor-and app-based data collection into surveys.
Article
Full-text available
Advances in smartphone technology have allowed for individuals to have access to near-continuous location tracking at a very precise level. As the backbone of mobility research, the Travel Diary Study, has continued to offer decreasing response rates over the years, researchers are looking to these mobile devices to bridge the gap between self-report recall studies and a person’s underlying travel behavior. This article details an open-source application that collects real-time location data which respondents may then annotate to provide a detailed travel diary. Results of the field test involving 674 participants are discussed, including technical performance, data quality and response rate.
Article
Full-text available
The growing smartphone penetration and the integration of smartphones into people’s everyday practices offer researchers opportunities to augment survey measurement with smartphone-sensor measurement or to replace self-reports. Potential benefits include lower measurement error, a widening of research questions, collection of in situ data, and a lowered respondent burden. However, privacy considerations and other concerns may lead to nonparticipation. To date, little is known about the mechanisms of willingness to share sensor data by the general population, and no evidence is available concerning the stability of willingness. The present study focuses on survey respondents’ willingness to share data collected using smartphone sensors (GPS, camera, and wearables) in a probability-based online panel of the general population of the Netherlands. A randomized experiment varied study sponsor, framing of the request, the emphasis on control over the data collection process, and assurance of privacy and confidentiality. Respondents were asked repeatedly about their willingness to share the data collected using smartphone sensors, with varying periods before the second request. Willingness to participate in sensor-based data collection varied by the type of sensor, study sponsor, order of the request, respondent’s familiarity with the device, previous experience with participating in research involving smartphone sensors, and privacy concerns. Willingness increased when respondents were asked repeatedly and varied by sensor and task. The timing of the repeated request, one month or six months after the initial request, did not have a significant effect on willingness.
Article
Full-text available
Researchers are combining self-reports from mobile surveys with passive data collection using sensors and apps on smartphones increasingly more often. While smartphones are commonly used in some groups of individuals, smartphone penetration is significantly lower in other groups. In addition, different operating systems (OSs) limit how mobile data can be collected passively. These limitations cause concern about coverage error in studies targeting the general population. Based on data from the Panel Study Labour Market and Social Security (PASS), an annual probability-based mixed-mode survey on the labor market and poverty in Germany, we find that smart-phone ownership and ownership of smartphones with specific OSs are correlated with a number of sociodemographic and substantive variables.
Article
Full-text available
Survey researchers are increasingly seeking opportunities to link interview data with administrative records. However, obtaining consent from all survey respondents (or certain subgroups) remains a barrier to performing record linkage in many studies. We experimentally investigated whether emphasizing different benefits of record linkage to respondents in a telephone survey of employee working conditions improves respondents' willingness to consent to linkage of employment administrative records relative to a neutral consent request. We found that emphasizing linkage benefits related to "time savings" yielded a small, albeit statistically significant , improvement in the overall linkage consent rate (86.0) relative to the neutral consent request (83.8 percent). The time savings argument was particularly effective among "busy" respondents. A second benefit argument related to "improved study value" did not yield a statistically significant improvement in the linkage consent rate (84.4 percent) relative to the neutral request. This benefit argument was also ineffective among the subgroup of respondents considered to be most likely to have a self-interest in the study outcomes. The article concludes with a brief discussion of the practical implications of these findings and offers suggestions for possible research extensions.
Article
Full-text available
Declining response rates worldwide have stimulated interest in understanding what may be influencing this decline and how it varies across countries and survey populations. In this paper, we describe the development and validation of a short 9-item survey attitude scale that measures three important constructs, thought by many scholars to be related to decisions to participate in surveys, that is, survey enjoyment, survey value, and survey burden. The survey attitude scale is based on a literature review of earlier work by multiple authors. Our overarching goal with this study is to develop and validate a concise and effective measure of how individuals feel about responding to surveys that can be implemented in surveys and panels to understand the willingness to participate in surveys and improve survey effectiveness. The research questions relate to factor structure, measurement equivalence, reliability, and predictive validity of the survey attitude scale. The data came from three probability-based panels: the German GESIS and PPSM panels and the Dutch LISS panel. The survey attitude scale proved to have a replicable three-dimensional factor structure (survey enjoyment, survey value, and survey burden). Partial scalar measurement equivalence was established across three panels that employed two languages (German and Dutch) and three measurement modes (web, telephone, and paper mail). For all three dimensions of the survey attitude scale, the reliability of the corresponding subscales (enjoyment, value, and burden) was satisfactory. Furthermore, the scales correlated with survey response in the expected directions, indicating predictive validity.
Article
Full-text available
p>This paper examines non-response in a mobile app study designed to collect expenditure data. We invited 2,383 members of the nationally representative Understanding Society Innovation Panel in Great Britain to download an app to record their spending on goods and services: Participants were asked to scan receipts or report spending directly in the app every day for a month. We examine participation at different stages of the process. We further use data from the prior wave of the panel to examine the prevalence of potential barriers to participation, including access, ability and willingness to use different mobile technologies, and biases in the types of people who participate, considering socio-demographic characteristics, financial position and financial behaviours. While the participation rate was low, drop out was also low: Over 80% of participants remained in the study for the full month. The main barriers to participation were access to, and frequency of use of mobile devices, willingness to download an app for a survey, and general cooperativeness with the survey. While there were strong biases in who participated in terms of socio-demographic characteristics (with women, younger, and more educated sample members being more likely to participate), and in terms of financial behaviours (with respondents who already use mobile devices to monitor their finances being more likely to participate), we found no biases in correlates of spending.</p
Article
Full-text available
The rising penetration of smartphones now gives researchers the chance to collect data from smartphone users through passive mobile data collection via apps. Examples of passively collected data include geolocation, physical movements, online behavior and browser history, and app usage. However, to passively collect data from smartphones, participants need to agree to download a research app to their smartphone. This leads to concerns about nonconsent and nonparticipation. In the current study, we assess the circumstances under which smartphone users are willing to participate in passive mobile data collection. We surveyed 1,947 members of a German nonprobability online panel who own a smartphone using vignettes that described hypothetical studies where data are automatically collected by a research app on a participant’s smartphone. The vignettes varied the levels of several dimensions of the hypothetical study, and respondents were asked to rate their willingness to participate in such a study. Willingness to participate in passive mobile data collection is strongly influenced by the incentive promised for study participation but also by other study characteristics (sponsor, duration of data collection period, option to switch off the app) as well as respondent characteristics (privacy and security concerns, smartphone experience).
Article
Full-text available
We asked members of the Understanding Society Innovation Panel about their willingness to participate in various data collection tasks on their mobile devices. We find that stated willingness varies considerably depending on the type of activity involved: respondents are less willing to participate in tasks that involve downloading and installing an app, or where data are collected passively. Stated willingness also varies between smartphones and tablets, and between types of respondents: respondents who report higher concerns about the security of data collected with mobile technologies and those who use their devices less intensively are less willing to participate in mobile data collection tasks.
Chapter
This chapter reviews existing literature on concern with and willingness to engage in active and passive forms of mobile data collection. It describes four online surveys conducted in two countries that all administered a similar set of questions on concern with five different forms of mobile data collection. The chapter uses these data to analyze differences in concern across the five tasks and study correlates of concern. It discusses the author's findings, their practical implications, and suggestions for further research. For all five tasks and in all four samples, the more activities a respondent reported to do on their smartphone, the lower the likelihood for high concern for all five tasks. With each additional smartphone activity reported, the likelihood of having high concerns decreases by about two percentage points.