Available via license: CC BY 2.0
Content may be subject to copyright.
168
Section Editors: Jeannine M . Brant, Marilyn L. Haas-Haseman, Steven H. Wei, and Rita Wickham
TRANSLATING RESEARCH INTO PRACTICE
Understanding and Evaluating
Survey Research
JULIE PONTO, PhD, APRN, AGCNS-BC, AOCNS®
From Winona State University, Rochester,
Minnesota
Author's disclosures of potential conflicts of
interest are found at the end of this article.
Correspondence to: Julie Ponto, PhD, APRN,
AGCNS-BC, AOCNS®, Winona State University,
Graduate Programs in Nursing, 859 30th Avenue
South East, Rochester, MN 55904.
E-mail: jponto@winona.edu
© 2015 Harborside Press®
A
variety of methodo-
logic approaches ex-
ist for individuals in-
terested in conducting
research. Selection of a research
approach depends on a number of
factors, including the purpose of
the research, the type of research
questions to be answered, and the
availability of resources. The pur-
pose of this article is to describe
survey research as one approach
to the conduct of research so that
the reader can critically evaluate
the appropriateness of the con-
clusions from studies employing
survey research.
SURVEY RESEARCH
Survey research is defined as
“the collection of information from
a sample of individuals through their
responses to questions” (Check &
Schutt, 2012, p. 160). This type of re-
search allows for a variety of methods
to recruit participants, collect data,
and utilize various methods of instru-
mentation. Survey research can use
quantitative research strategies (e.g.,
using questionnaires with numerical-
ly rated items), qualitative research
strategies (e.g., using open-ended
questions), or both strategies (i.e.,
mixed methods). As it is often used to
describe and explore human behav-
ior, surveys are therefore frequently
used in social and psychological re-
search (Singleton & Straits, 2009).
Information has been obtained
from individuals and groups through
the use of survey research for de-
cades. It can range from asking a few
targeted questions of individuals on
a street corner to obtain information
related to behaviors and preferences,
to a more rigorous study using mul-
tiple valid and reliable instruments.
Common examples of less rigorous
surveys include marketing or politi-
cal surveys of consumer patterns and
public opinion polls.
Survey research has historically
included large population-based data
collection. The primary purpose of
this type of survey research was to ob-
tain information describing charac-
teristics of a large sample of individu-
als of interest relatively quickly. Large
census surveys obtaining information
reflecting demographic and personal
characteristics and consumer feed-
back surveys are prime examples.
These surveys were often provided
through the mail and were intended
to describe demographic characteris-
tics of individuals or obtain opinions
on which to base programs or prod-
ucts for a population or group.J Adv Pract Oncol 2015;6:168–171
169
TRANSLATING RESEARCH INTO PRACTICE
SURVEY RESEARCH
More recently, survey research has developed
into a rigorous approach to research, with scientifi-
cally tested strategies detailing who to include (rep-
resentative sample), what and how to distribute
(survey method), and when to initiate the survey
and follow up with nonresponders (reducing non-
response error), in order to ensure a high-quality
research process and outcome. Currently, the term
“survey” can reflect a range of research aims, sam-
pling and recruitment strategies, data collection in-
struments, and methods of survey administration.
Given this range of options in the conduct of
survey research, it is imperative for the consumer/
reader of survey research to understand the poten-
tial for bias in survey research as well as the tested
techniques for reducing bias, in order to draw ap-
propriate conclusions about the information re-
ported in this manner. Common types of error in
research, along with the sources of error and strat-
egies for reducing error as described throughout
this article, are summarized in the Table.
SAMPLING
The goal of sampling strategies in survey re-
search is to obtain a sucient sample that is rep-
resentative of the population of interest. It is often
not feasible to collect data from an entire popula-
tion of interest (e.g., all individuals with lung can-
cer); therefore, a subset of the population or sample
is used to estimate the population responses (e.g.,
individuals with lung cancer currently receiving
treatment). A large random sample increases the
likelihood that the responses from the sample will
accurately reflect the entire population. In order
to accurately draw conclusions about the popu-
lation, the sample must include individuals with
characteristics similar to the population.
It is therefore necessary to correctly identify
the population of interest (e.g., individuals with
lung cancer currently receiving treatment vs. all
individuals with lung cancer). The sample will
ideally include individuals who reflect the intend-
ed population in terms of all characteristics of the
population (e.g., sex, socioeconomic characteris-
tics, symptom experience) and contain a similar
distribution of individuals with those character-
istics. As discussed by Mady Stovall beginning on
page 162, Fujimori et al. (2014), for example, were
interested in the population of oncologists. The
authors obtained a sample of oncologists from two
hospitals in Japan. These participants may or may
not have similar characteristics to all oncologists
in Japan.
Participant recruitment strategies can aect
the adequacy and representativeness of the sam-
ple obtained. Using diverse recruitment strategies
can help improve the size of the sample and help
ensure adequate coverage of the intended popula-
tion. For example, if a survey researcher intends
to obtain a sample of individuals with breast can-
cer representative of all individuals with breast
cancer in the United States, the researcher would
want to use recruitment strategies that would re-
cruit both women and men, individuals from ru-
ral and urban settings, individuals receiving and
not receiving active treatment, and so on. Because
of the diculty in obtaining samples representa-
tive of a large population, researchers may focus
Table. Sources of Error in Survey Research and Strategies to Reduce Error
Type of error Source of error Strategies to reduce error
Coverage error Unknown or zero chance of individuals in the
population being included in the sample
Multimode design
Sampling error Individuals included in the sample do
not represent the characteristics of the
population
Clearly identified population of interest; diverse
participant recruitment strategies; large, random
sample
Measurement
error
Questions/instruments do not accurately
reflect the topic of interest; questionnaires/
interviews do not evoke truthful answers
Valid, reliable instruments; pretest questions; user-
friendly graphics, visual characteristics
Nonresponse
error
Lack of response from all individuals in
sample
User-friendly survey design; follow-up procedures
for nonresponders
Note. Information from Dillman et al. (2014), Singleton & Straits (2009), Check & Schutt (2012).
170
TRANSLATING RESEARCH INTO PRACTICE PONTO
the population of interest to a subset of individu-
als (e.g., women with stage III or IV breast can-
cer). Large census surveys require extremely large
samples to adequately represent the characteris-
tics of the population because they are intended to
represent the entire population.
DATA COLLECTION METHODS
Survey research may use a variety of data col-
lection methods with the most common being
questionnaires and interviews. Questionnaires
may be self-administered or administered by a
professional, may be administered individually or
in a group, and typically include a series of items
reflecting the research aims. Questionnaires may
include demographic questions in addition to val-
id and reliable research instruments (Costanzo,
Stawski, Ry, Coe, & Almeida, 2012; DuBenske et
al., 2014; Ponto, Ellington, Mellon, & Beck, 2010).
It is helpful to the reader when authors describe
the contents of the survey questionnaire so that
the reader can interpret and evaluate the poten-
tial for errors of validity (e.g., items or instruments
that do not measure what they are intended to
measure) and reliability (e.g., items or instruments
that do not measure a construct consistently).
Helpful examples of articles that describe the sur-
vey instruments exist in the literature (Buerhaus
et al., 2012).
Questionnaires may be in paper form and
mailed to participants, delivered in an electronic
format via email or an Internet-based program
such as SurveyMonkey, or a combination of both,
giving the participant the option to choose which
method is preferred (Ponto et al., 2010). Using a
combination of methods of survey administration
can help to ensure better sample coverage (i.e., all
individuals in the population having a chance of
inclusion in the sample) therefore reducing cover-
age error (Dillman, Smyth, & Christian, 2014; Sin-
gleton & Strait, 2009). For example, if a researcher
were to only use an Internet-delivered question-
naire, individuals without access to a computer
would be excluded from participation. Self-ad-
ministered mailed, group, or Internet-based ques-
tionnaires are relatively low cost and practical for
a large sample (Check & Schutt, 2012).
Dillman et al. (2014) have described and tested
a tailored design method for survey research. Im-
proving the visual appeal and graphics of surveys
by using a font size appropriate for the respon-
dents, ordering items logically without creating
unintended response bias, and arranging items
clearly on each page can increase the response
rate to electronic questionnaires. Attending to
these and other issues in electronic questionnaires
can help reduce measurement error (i.e., lack of
validity or reliability) and help ensure a better re-
sponse rate.
Conducting interviews is another approach to
data collection used in survey research. Interviews
may be conducted by phone, computer, or in per-
son and have the benefit of visually identifying the
nonverbal response(s) of the interviewee and sub-
sequently being able to clarify the intended ques-
tion. An interviewer can use probing comments
to obtain more information about a question or
topic and can request clarification of an unclear
response (Singleton & Strait, 2009). Interviews
can be costly and time intensive, and therefore are
relatively impractical for large samples.
Some authors advocate for using mixed meth-
ods for survey research when no one method is
adequate to address the planned research aims,
to reduce the potential for measurement and non-
response error, and to better tailor the study meth-
ods to the intended sample (Dillman et al., 2014;
Singleton & Strait, 2009). For example, a mixed
methods survey research approach may begin with
distributing a questionnaire and following up with
telephone interviews to clarify unclear survey re-
sponses (Singleton & Straits, 2009). Mixed meth-
ods might also be used when visual or auditory
deficits preclude an individual from completing a
questionnaire or participating in an interview.
FUJIMORI et al.: SURVEY RESEARCH
Fujimori et al. (2014) described the use of sur-
vey research in a study of the eect of communi-
cation skills training for oncologists on oncologist
and patient outcomes (e.g., oncologist’s perfor-
mance and confidence and patient’s distress, sat-
isfaction, and trust). A sample of 30 oncologists
from two hospitals was obtained and though the
authors provided a power analysis concluding an
adequate number of oncologist participants to
detect dierences between baseline and follow-
up scores, the conclusions of the study may not
171
TRANSLATING RESEARCH INTO PRACTICE
SURVEY RESEARCH
be generalizable to a broader population of on-
cologists. Oncologists were randomized to either
an intervention group (i.e., communication skills
training) or a control group (i.e., no training).
Fujimori et al. (2014) chose a quantitative
approach to collect data from oncologist and pa-
tient participants regarding the study outcome
variables. Self-report numeric ratings were used
to measure oncologist confidence and patient
distress, satisfaction, and trust. Oncologist confi-
dence was measured using two instruments each
using 10-point Likert rating scales. The Hospital
Anxiety and Depression Scale (HADS) was used
to measure patient distress and has demonstrated
validity and reliability in a number of populations
including individuals with cancer (Bjelland, Dahl,
Haug, & Neckelmann, 2002). Patient satisfaction
and trust were measured using 0 to 10 numeric
rating scales. Numeric observer ratings were used
to measure oncologist performance of commu-
nication skills based on a videotaped interaction
with a standardized patient. Participants com-
pleted the same questionnaires at baseline and
follow-up.
The authors clearly describe what data were
collected from all participants. Providing addi-
tional information about the manner in which
questionnaires were distributed (i.e., electronic,
mail), the setting in which data were collected
(e.g., home, clinic), and the design of the survey
instruments (e.g., visual appeal, format, content,
arrangement of items) would assist the reader in
drawing conclusions about the potential for mea-
surement and nonresponse error. The authors de-
scribe conducting a follow-up phone call or mail
inquiry for nonresponders, using the Dillman et al.
(2014) tailored design for survey research follow-
up may have reduced nonresponse error.
CONCLUSIONS
Survey research is a useful and legitimate
approach to research that has clear benefits in
helping to describe and explore variables and
constructs of interest. Survey research, like all re-
search, has the potential for a variety of sources
of error, but several strategies exist to reduce the
potential for error. Advanced practitioners aware
of the potential sources of error and strategies to
improve survey research can better determine
how and whether the conclusions from a survey
research study apply to practice. l
Disclosure
The author has no potential conflicts of inter-
est to disclose.
References
Bjelland, I., Dahl, A. A., Haug, T. T., & Neckelmann, D. (2002).
The validity of the Hospital Anxiety and Depression
Scale: An updated literature review. Journal of Psycho-
somatic Research, 52, 69–77. http://dx.doi.org/10.1016/
S0022-3999(01)00296-3
Buerhaus, P. I., DesRoches, C., Applebaum, S., Hess, R., Nor-
man, L. D., & Donelan, K. (2012). Are nurses ready for
health care reform? A decade of survey research. Nursing
Economics, 30, 318–330.
Check, J., & Schutt, R. K. (2012). Survey research. In J. Check
& R. K. Schutt (Eds.). Research methods in education. (pp.
159–185). Thousand Oaks, CA: Sage Publications.
Costanzo, E. S., Stawski, R. S., Ry, C. D., Coe, C. L., & Al-
meida, D. M. (2012). Cancer survivors’ responses to daily
stressors: Implications for quality of life. Health Psychol-
ogy, 31, 360–370. http://dx.doi.org/10.1037/a0027018
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet,
phone, mail, and mixed-mode surveys: The tailored design
method (4th ed). Hoboken, NJ: John Wiley & Sons, Inc.
DuBenske, L. L., Gustafson, D. H., Namkoong, K. H., Atwood,
R. P., Brown, R. L., Chih, M. Y.,…Cleary, J. (2014). CHESS
improves cancer caregivers’ burden and mood: Results of
an eHealth RCT. Health Psychology, 33, 1261–1272. http://
dx.doi.org/10.1037/a0034216
Fujimori, M., Shirai, Y., Asai, M., Kubota, K., Katsumata, N., &
Uchitomi, Y. (2014). Eect of communication skills train-
ing program for oncologists based on patient preferences
for communication when receiving bad news: A random-
ized controlled trial. Journal of Clinical Oncology, 32,
2166–2172. http://dx.doi.org/10.1200/JCO.2013.51.2756
Ponto, J. A., Ellington, L., Mellon, S., & Beck, S. L. (2010).
Predictors of adjustment and growth in women with
recurrent ovarian cancer. Oncology Nursing Forum, 37,
357–364. http://dx.doi.org/10.1188/10.ONF.357-364
Singleton, R. A., & Straits, B. C. (2009). Approaches to social
research (5th ed.). New York: Oxford University Press.
