Content uploaded by Theoni Stathopoulou
Author content
All content in this area was uploaded by Theoni Stathopoulou on Jan 16, 2020
Content may be subject to copyright.
Questionnaire Design and Translation for
Refugee Populations: Lessons Learned from
the REHEAL Study
THEONI STATHOPOULOU
National Centre for Social Research, Athens 10552, Greece
theosta@ekke.gr
ELICA KRAJC
ˇEVA
cApStAn Linguistic Quality Control, 1170 Brussels, Belgium
NATALJA MENOLD
Technische Universitaet Dresden, Dresden, Institute of Sociology, Technische
Universita
¨t Dresden, D-01062 Dresden
STEVE DEPT
cApStAn Linguistic Quality Control, 1170 Brussels, Belgium
MS received October 2018; revised MS received May 2019
Surveying the refugee population poses particular challenges: what measurement
and culture effects need to be taken into account? Are some of the constructs
related to refugees unique or can constructs used in other surveys be adapted?
Due to considerable variation in educational background, in trauma history or
in perception of ethnicity or gender roles in refugee populations, one needs to
raise the question whether a one-size-fits-all approach is suitable when designing
a questionnaire for refugee populations. Drawing upon the experience of the
REHEAL study conducted in 2016 during the early phase of refugees’ settle-
ment in Greek refugee camps, the article addresses the particular challenges
when designing and translating a questionnaire for multinational, multicultural
and potentially traumatized refugee populations, residing in refugee accommo-
dation settings. A post hoc scrutiny of the Arabic and Farsi versions of the
REHEAL questionnaire forms the basis of this article in order to empirically
inform best practices in designing and translating questionnaires for special
populations. The authors set the stage by summarizing key aspects of concept
and measurement equivalence in cross-cultural research, and then by expanding
on the fact that cross-cultural differences in response styles or response sets can
affect comparability. Translation and adaptation challenges are examined and
illustrated by examples. This article serves to empirically document the benefit
and pitfalls of appropriate measurements for use in refugee research.
Journal of Refugee Studies Vol. 32, Special Issue No. 1 ßThe Author(s) 2019. Published by Oxford University
Press. All rights reserved. For permissions, please email: journals.permissions@oup.com
doi:10.1093/jrs/fez045
Downloaded from https://academic.oup.com/jrs/article-abstract/32/Special_Issue_1/i105/5688806 by theosta@ekke.gr on 16 January 2020
Keywords: REHEAL, refugee, trauma, psychosocial, translatability, sensitive questions,
translation in cross-cultural surveys, research, translation of scales, questionnaire design,
questionnaire translation, questionnaire design, refugees, Greece
Introduction
Surveying the refugee population poses particular challenges: what measure-
ment effects and what culture effects need to be taken into account? Are some
of the constructs related to refugees unique or can constructs used in other
surveys be adapted? In the summer of 2016, data was collected in six Greek
refugee camps to identify factors that affect safety, health and trauma among
newly arrived refugees and asylum seekers. This was part of the REHEAL
(Refugees’ Healing) project. The fact that there is a considerable variation,
for example, in educational background, in trauma history or in perception
of ethnicity or gender roles in refugee populations raises an ancillary ques-
tion: does this variation rule out a one-size-fits-all approach when designing a
questionnaire for refugee populations? If best practice is to embed the ques-
tion-adaptation process in the questionnaire design, what factors should be
considered when preparing the data-collection instruments?
Drawing upon the experience of the REHEAL survey conducted by the
National Centre for Social Research in Greece in 2016 (Introduction, this
special issue), the article addresses the main challenges when designing and
translating a questionnaire for multinational and multicultural refugee popu-
lations, residing in refugee accommodation settings. The languages of interest
are Arabic and Farsi (spoken by the majority of the population residing in the
camps). Therefore, a post hoc scrutiny of the Arabic and Farsi versions of the
REHEAL questionnaire forms the basis of this article in order to empirically
inform best practices in designing and translating questionnaires for special
populations.
Concept and Measurement Equivalence in Cross-cultural Research
In a survey, researchers use questions to address a theoretical topic or a
concept, such as respondents’ attitudes towards gender roles or democracy,
trust in the government, health or life satisfaction. The measurement of these
topics may be more or less based on a particular theory from social or be-
havioural science, for example about specific opinions (e.g. liberal or conser-
vative), values (e.g. Schwartz’s Human Values, Schwartz 1992), behaviours
(e.g. helpful or aggressive behaviour) or individual personality characteristics,
such as masculinity or extraversion. Ideally, the measurement of a concept
under investigation should be grounded in a specific, well-evaluated theory
(e.g. Rammstedt et al. 2015). However, this theory should be adapted when
refugee populations are studied. Because the possible need for the adaptation
of theory is often not considered when collecting data from refugee
i106 Theoni Stathopoulou et al.
Downloaded from https://academic.oup.com/jrs/article-abstract/32/Special_Issue_1/i105/5688806 by theosta@ekke.gr on 16 January 2020
populations, the concepts measured may have a different theoretical structure
or even be absent from the cultures concerned.
Within psychometric measurement theory (e.g. Raykov and Marcoulides
2011), but also within the Total Survey Error framework (Groves et al. 2004),
validity is the most important quality characteristic of data collected with a
questionnaire. Thus, the collected data should make it possible to draw con-
clusions that are relevant to the concept under investigation (Kane 2013). In
a cross-cultural study, this requirement should be fulfilled in each language or
culture of the target population, so that comparability of the questions with
respect to the concept under investigation can be established. This is referred
to as measurement invariance. As proposed by Mellenbergh (1989),
measurement invariance (MI) requires that the association between the items (or
test scores) and ... the latent traits of individuals should not depend on group
membership or measurement occasion (i.e. time) (van de Schoot et al. 2015,
editorial).
With structural equation modelling and related methods, such as multi-group
confirmatory factor analysis (Byrne 2011), it is possible to empirically test
assumptions with respect to MI and data comparability between different
measurement occasions (e.g. different countries). While many researchers
find that data in cross-cultural surveys does not exhibit MI (and therefore
is not comparable, e.g. Davidov et al. 2008), few researchers analyse the effect
of questionnaire design on MI, showing, for example, that differences be-
tween rating scales, such as a different number of categories or different
degree of verbalization of the categories, violate MI (Menold and Tausch
2016). The aspects of concept comparability and MI should be taken into
account when adapting or translating questionnaires for refugee populations.
When piloting questionnaires, translations and procedures should ensure that
data is comparable, namely measurement-invariant. It is from this perspective
that the REHEAL study is examined here.
Response Sets and Sensitivity as Potential Sources of Incomparability
Comparability between the data gathered from different cultures can be af-
fected by cross-cultural differences in response styles or response sets. Response
sets are reactions of the respondents to a question that are not determined by
the content of the question (Paulhus 1991). Response sets can be of various
types, such as tending to agree regardless of the content of the question (ac-
quiescence) or showing a tendency to provide moderate or extreme responses
(van Vaerenbergh and Thomas 2013). Cross-cultural differences in response
sets can potentially limit cross-cultural comparability, thus posing a challenge
for questionnaire design for refugee populations. In addition, the sensitivity of
a question or of survey content can differ between different cultures
(Mneimneh et al. 2018). Tourangeau et al. (2000) differentiate between three
Questionnaire Design for Refugee Populations i107
Downloaded from https://academic.oup.com/jrs/article-abstract/32/Special_Issue_1/i105/5688806 by theosta@ekke.gr on 16 January 2020
forms of sensitivity: (i) social desirability, which is related to the violation of
cultural norms or norms accepted in a society or social group; (ii) questions
that are too private or address taboos; (iii) intrusive questions, to which a
truthful response would be associated with negative consequences such as pro-
secution. When respondents are faced with a sensitive question, deliberate mis-
reporting is the most likely reaction (Jones and Forrest 1992; Tourangeau and
Smith 1996). Sensitivity is an issue when surveying refugees, because questions
may be of different sensitivity in refugees’ countries of origin or their context-
ual (e.g. religious) groups with regard to all three forms described above. This
has particular consequences for questionnaire design, because questions that
may be considered as sensitive in a Western context may not be perceived as
such in non-Western contexts. As Derluyn et al. (2004: 863) point out ‘we must
acknowledge the risk of transcultural errors when using psychological measures
in other cultural contexts’.
To evaluate the extent to which response sets, sensitivity or other issues can
limit cross-cultural comparability and validity, cognitive pre-testing techniques
are essential (Miller et al. 2014; Willis, 2015; Mneimneh et al. 2016, 2018).
Mneimneh et al. (2018) stress the importance of cognitive pre-tests when adapt-
ing questionnaires to different cultures or translating them into other lan-
guages. Cognitive pre-testing can be supplemented with a translatability
assessment, in which experienced linguists representing the target language
and target culture analyse a draft questionnaire. They identify potential trans-
lation and cultural issues and use a framework of translatability categories to
report these. However, in emergency situations and humanitarian settings like
those under which REHEAL was conducted, time and resources are restricted,
preventing researchers from piloting survey tools. Translation ‘in tandem with
questionnaire development’ can be a sound compromise (Pennell et al. 2014:
127).
The REHEAL Questionnaire
The REHEAL questionnaire was designed by the National Centre for Social
Research in Greece in collaboration with the Norwegian University of
Science and Technology and comprised five sections: a. Socio-demographics;
b. Staying in another country after leaving home; c. Arriving in Greece
(feelings of safety, living conditions); d. Health issues and trauma (health
problems before and after fleeing the homeland, a shorter version of the
Harvard Program in Refugee Trauma questionnaire); and e. Social media
use. The source questionnaire was developed in English and translated into
Arabic and Farsi with the help of experienced translators. During fieldwork,
certified interpreters were hired to aid the researchers who were responsible
for inspecting and facilitating the self-completion procedure.
The main goal of the survey was to record refugees’ perceptions of living
conditions in Greece, their feelings of safety and loss, their discriminatory
and traumatic experiences and their self-assessed health status (Stathopoulou
i108 Theoni Stathopoulou et al.
Downloaded from https://academic.oup.com/jrs/article-abstract/32/Special_Issue_1/i105/5688806 by theosta@ekke.gr on 16 January 2020
et al. 2019). Fieldwork was conducted during the first phase of settlement in
Greece in autumn 2016, shortly after the 2015 peak in refugee flows, in six
refugee camps across Greece (see the ‘Introduction’ in this Special Issue). The
field observations collected during the survey (Stathopoulou 2019) were used
as a basis to address the main challenges in translating and developing a
questionnaire for refugees. To this end, a linguistic and translatability post-
assessment of certain questions of the REHEAL questionnaire was conducted.
Challenges in Questionnaire Design and Translation for Surveying Refugee
Populations
The two main types of challenge are linguistic and socio-demographic/cul-
tural and it is likely that most, if not all, of these challenges would exist in
non-refugee populations with the same cultural and linguistic background.
However, some of these factors are compounded by the complexity of po-
tentially traumatic experiences that the refugees have had to cope with during
their displacement and after their resettlement in accommodation facilities
(Steel et al. 2009; Mollica 2016). Thus, psycho-social challenges such as cog-
nitive capacity or perception of time emerge as a third component that has to
be taken into consideration (Figure 1).
Linguistic Challenges
Linguistic Diversity The issue of linguistic diversity is inherent to surveys in
3MC contexts and this is particularly true for the refugee population (Pan and
Fond 2014; Pennell et al. 2014). The vast majority of the refugees arriving in
Europe as of 2015 came from Syria, Eritrea, Iraq, Afghanistan and Iran.
1
Figure 2 below illustrates the complexity of the situation regarding the
main language of these countries. Arabic, Tigre
´and Tigrigna are Semitic
languages; Kurdish, Pashto, Persian and Dari are Indo-European languages;
Turkmen, Uzbek and Azerbaijani are Turkic languages; and Somali is a
Cushitic language (a branch of the Afroasiatic language family) spoken in
Eritrea and Somalia.
Semantic and conceptual differences across languages need to be taken into
account as well as different text reading and writing patterns (Wolf et al.
2016). Different scripts have an impact on the design of a questionnaire,
because the way it is read by respondents or interviewers differs. Right-to-
left script is the most salient example for part of the refugee population:
Arabic, Azerbaijani, Farsi/Dari (Persian), Sorani (Kurdish) and Urdu all
use the right-to-left script, but not the same alphabet. The REHEAL ques-
tionnaire was exposed to some of the challenges related to differences in
concept, culture and script in the Arabic and Farsi translations.
Translators’ Competence Translating a survey instrument such as the
REHEAL questionnaire requires background knowledge about questionnaires:
Questionnaire Design for Refugee Populations i109
Downloaded from https://academic.oup.com/jrs/article-abstract/32/Special_Issue_1/i105/5688806 by theosta@ekke.gr on 16 January 2020
Although good translation products do not assure the success of a survey, badly
translated questionnaires can ensure that an otherwise sound project fails be-
cause the poor quality of translation prevents researchers from collecting com-
parable data (Mohler et al., 2016, Translation: Overview, p. 233.).
The translators need to be (made) aware of the purpose and structure of the
questionnaire. Even high-quality translation can result in response bias due to
Linguisc challenges
Language diversity
Translators' competence
Translatability of key
terms
Socio-demographic -
cultural challenges
Percepons of me
Social status of women,
children, elderly
Educaon
Percepons of naonality
Sensive quesons (e.g Religion)
Psychosocial challenges
Trauma
Figure 1
Main Challenges in Questionnaire Design for Refugee Populations
i110 Theoni Stathopoulou et al.
Downloaded from https://academic.oup.com/jrs/article-abstract/32/Special_Issue_1/i105/5688806 by theosta@ekke.gr on 16 January 2020
errors in instructions, filtering, coding, scales and order of response options
or emphasis of text.
An accurate rendering of interviewer instructions in a face-to-face admin-
istration mode and of respondent instructions in a self-administered question-
naire is crucial to collecting the right data. For example, inappropriately
translated instructions may lead to questions being skipped or multiple re-
sponses being selected where only one is required, both of which result in a
non-response. A translator who is (made) aware of the importance of such
instructions will be more likely to pay due attention to them.
Translation of Key Terms Words can have multiple meanings and respond-
ents’ correct understanding of a question is based on asking the same
question in the target language as in the source language. To this end,
the translators need translation notes to help them focus on sensitive
terms or expressions that are crucial to the understanding of the survey
questions.
Figure 2
Languages of Main Nationalities with 1,000,000þNative Speakers among the
New Migrants
Source: www.ethnologue.com
Questionnaire Design for Refugee Populations i111
Downloaded from https://academic.oup.com/jrs/article-abstract/32/Special_Issue_1/i105/5688806 by theosta@ekke.gr on 16 January 2020
In REHEAL, conveying terms such as ‘deporting’, ‘fleeing’ or ‘displaced’
correctly in the translation is crucial with a view to ensuring the validity of
the responses (Figures 3 and 4). The word ‘displaced’ implies that respond-
ents have had to move against his or her will—that they have been expelled
or forced to flee their home or homeland, typically because of war, persecu-
tion or natural disaster. Conveying this dimension in the translation will
ensure that the distinction is preserved with the concept of moving for
other reasons such as work or marriage.
For example, in Farsi, the translation for ‘displaced’ needs the addition of
the adjective ‘forced’ to the word ‘move’ for clarity (Figure 3).
If ‘fleeing’ were rendered as ‘running away from’ rather than ‘abandoning
the country’, which is what the current translation reads, it would be closer to
the source (Figure 4).
In Arabic, ‘fleeing’ in this same question was translated as ‘leaving’ (Figure
5).
The translation of ‘internally displaced before leaving your home country’ as
‘displaced inside’ (Figure 6) could be understood in different ways by the re-
spondent. The confusion might result in non-response, or in the respondent
understanding the question as referring to moving and not deportation or
fleeing. Clarity could be improved either by adding a translation note for
the translator or by rewording the question along the lines of ‘Were you
already forced to move inside your country before you left your country?’.
Notes for the translators and definitions of key terms prepared by the
survey designers and provided to the translators before the translation—
Figure 4
REHEAL Questionnaire, Farsi version
Figure 3
REHEAL Questionnaire, Farsi version
i112 Theoni Stathopoulou et al.
Downloaded from https://academic.oup.com/jrs/article-abstract/32/Special_Issue_1/i105/5688806 by theosta@ekke.gr on 16 January 2020
and to reviewers after the translation—will help convey the key terms accur-
ately and fluently. The notes for the translators and the key term definitions
should ‘draw their content from the various methods employed, such as
expert review, advance translation, translatability assessment, or cross-cul-
tural pre-testing (e.g. Jowell 1998)’ (Behr and Scholz 2011: 157–179), which
are development procedures advocated by Harkness et al. (2010).
Translatability assessment, in particular, is one of the upstream processes
that, besides identifying potential translation and cultural issues in the
source items, result in translation and adaptation guidelines that help with
‘making the source material fit for adaptation’ (Dept et al. 2017: 168–191),
the aim being to ask the same question across languages and cultures. Time
and resource constraints did not allow a translatability assessment in the
REHEAL questionnaire to be made nor its translated versions to be cogni-
tively pretested (Willis 2004) to identify whether the intended meaning can be
conveyed to the respondents of different cultures. Instead, the few flaws de-
tected during the start of fieldwork were corrected ‘on-site’ in the first camp
and the corrections were incorporated in the questionnaire.
Translatability of Source Questionnaire The question ‘When did you first
leave your home country?’ might be understood to refer to respondents’ de-
parture that has led to them seeking refugee status or it might be understood
to refer to the first time they ever left their home country—for example,
10 years ago in order to work abroad for a limited time. This kind of
Figure 5
REHEAL Questionnaire, Arabic version
Figure 6
REHEAL Questionnaire, Arabic version
Questionnaire Design for Refugee Populations i113
Downloaded from https://academic.oup.com/jrs/article-abstract/32/Special_Issue_1/i105/5688806 by theosta@ekke.gr on 16 January 2020
ambiguity can be identified in the translatability assessment and rewording
the question or adding a specific reference to the reason for leaving may bring
the item closer to the intended construct and avoid answers that include
travelling abroad in the past for other reasons—for example: ‘When did
you first leave the country before you arrived here?’
Grammatical Gender Grammatical gender is a known hurdle in translation.
In languages where a masculine and a feminine form exist for ‘spouse’, omit-
ting the masculine or feminine form of the word is confusing. If the posses-
sive form is not correctly rendered in a complex phrase, then non-response is
more likely. For example, ‘spouse/partner’s health’ translates back to ‘spouse
(m)/ health of partner’ instead of ‘health of spouse (m/f)/partner’ (Figure 7).
Translation of Scales The difference in form, structure and concepts between
languages may affect perception of the scales and, consequently, challenge the
assumption of measurement equivalence (e.g. Menold and Kemper, 2015,
Menold and Tausch, 2016). Different linguistic and cultural components
come into play when maintaining semantic distance between the points on
the scale. It remains a challenge to strike the best possible balance between
accuracy and fluency, because one needs to take into account possible meas-
urement effects such as social desirability or acquiescence, but also culture
effects such as differences in conversational norms (Yang et al., 2010; Roberts
2016).
Figure 7
‘‘Spouse/Partner’s Health’’ Rendered as ‘‘Spouse (m)/ Health of Partner’’
i114 Theoni Stathopoulou et al.
Downloaded from https://academic.oup.com/jrs/article-abstract/32/Special_Issue_1/i105/5688806 by theosta@ekke.gr on 16 January 2020
For example, the word ‘safe’ in the scale ‘Very safe/Somewhat safe/Not
very safe/Not safe at all’ may mean ‘protected from danger or risk’ or ‘not
likely to cause harm or injury’. It will need a translation note to make sure it
translates as ‘protected’. The translation of the quantifiers (Very/Somewhat/
Not very/Not at all) will need to collocate with ‘safe/protected’.
In Arabic, ‘fair’ is difficult to render. It can be translated as ‘OK’ or ‘not
bad’, but linguists need to maintain the semantic distance between categories.
In the example shown in Figure 8, the translation of ‘fair’ is ‘poor condition’,
which introduces a bias in the scale.
Rating scales can range from a low to a high value, from disagreement to
agreement, from disliking to liking, but can be also reversely orientated,
starting with a high value and continuing to a lower one. This characteristic
of rating scales has been referred to as scale direction.
Scale direction has been found to impact measurement quality and differ-
ent directions lead to significant differences in responses (Krebs and
Hoffmeyer-Zlotnik 2010; Ho
¨hne and Krebs 2017). Furthermore, ‘Layout
and direction of response scales may be affected when different writing sys-
tems are involved’ (Behr and Shishido 2016: 277). For example, the script
may affect filters when a filtering direction appears before the response
option, as shown in Figure 9.
Figure 8
Rendition of ‘‘Fair’’ as ‘‘Poor Condition’’ alters the scale
Figure 9
Filtering Direction (Go to Q14) Appears before the Answer (Yes) in Right-to-
Left Script
Questionnaire Design for Refugee Populations i115
Downloaded from https://academic.oup.com/jrs/article-abstract/32/Special_Issue_1/i105/5688806 by theosta@ekke.gr on 16 January 2020
In a right-to-left script table, the filtering direction should appear in the left
column of the table, as shown in Figure 10, since the reader starts from the
right column.
Psycho-social Challenges
Emotions—Trauma A number of questions in surveys may be emotionally
difficult for refugees, as they are closely related to the potentially trau-
matic events that they have already experienced and often the actual
reason for fleeing. Questions about religion, nationality or even gender
may be particularly sensitive because the respondents may have been dis-
criminated against, displaced or tortured precisely on one or more of these
grounds.
In addition to possible mistrust and reluctance to answer as the result of
this, the cultural and social environment from which the respondent origin-
ates may appear as an obstacle to answering the questions. Conversely,
survey questions may have a cathartic function, as telling one’s traumatic
story is essential for healing (Mollica 2016, 2019) and strengthening the re-
silience of refugees towards the multiple adversities they have to cope with.
Emotional distress may have an impact on the cognitive capacity of respond-
ents in refugee accommodation settings (Mneimneh et al. 2014; Silove et al.,
2017).
Socio-demographic and Cultural Challenges
Adaptation of Calendars Questions about age and year of birth are typical
questions where a one-size-fits-all approach will not work. A question such as
‘In what year were you born?’ often needs to be adapted to the usual and
fluent form. In some countries, it is ‘Year of birth’; in others, it is ‘What is
your age?’. However, the questionnaire should take into account the fact that
some countries use a calendar other than the Gregorian, such as Iran. Iranian
respondents may use the Solar Hijri Calendar to answer such a question.
Also, if year of birth is partially prefilled, survey designers need to ensure
that this matches the sampled population.
Education Educational levels and school programmes also need adaptation
to different countries and cultures to obtain harmonized and comparable
data. Such issues may need to be addressed and monitored during the
Figure 10
Filtering Direction (Go to Q14) Appears Correctly after the Answer (Yes) in
Right-to-Left Script
i116 Theoni Stathopoulou et al.
Downloaded from https://academic.oup.com/jrs/article-abstract/32/Special_Issue_1/i105/5688806 by theosta@ekke.gr on 16 January 2020
entire process: while linguists can receive guidance for the translation of a
number of components, adaptation of the terms describing education levels
may need to comply with the International Standard Classification of
Education (ISCED). A review and harmonization process may be needed.
Existing tools like CAMCES (https://www.surveycodings.org/education/),
which provide classifications of education for different countries
2
and make
coding to the ISCED categories possible, can be used to verify translation or
as a substitute for translation.
Social Status of Women, Children, Elderly People The translation of ‘gender’
is known to be problematic in various respects. ‘Male/female’ will probably
translate into ‘man/woman’ in a number of languages, since the formally
equivalent words may be reserved for animals and/or sexual contexts.
In some countries (like Pakistan or Afghanistan), underage children are
considered to be responsible for the whole family when the father is absent
or deceased.
Among the cultural factors that require special attention when designing a
questionnaire is the concept of the respondent’s ‘own children’. Foster-chil-
dren, unborn or miscarried children, stepchildren or the gender of the chil-
dren may affect the number of children perceived as being the respondent’s
‘own’ children. When asked ‘How many children do you have?’, respondents
from rural Iran may not account for children with certain characteristics.
Usually, a note for the interviewer (in the case of face-to-face interviews) or
an instruction (for self-administered questionnaires) will resolve this issue
(‘Count male and female children’) or separate options for boys and girls.
In socially more conservative cultures, free mixing between men and
women may not be the norm. Thus, the survey administration needs to be
adapted to the circumstances and female interviewers may be needed in the
field for women respondents. A woman interviewer may still have difficulty in
obtaining a private interview with women, since it is considered that it is the
man’s role to represent the family. The position of the elderly is also to be
taken into consideration. Given their position as leaders of the community,
they are likely to be the interviewer’s contact and their approval, implicit or
otherwise, is likely to facilitate the interview.
Religion Questions about religion and faith are particularly sensitive, espe-
cially for refugees fleeing regions where religion may be a part of the conflict
or basis for discrimination.
Including religious sub-groups in the responses to the question ‘Do you
consider yourself as being a member of any of these religious communities?’
may lead to an aversion to declaring affiliation to a particular religious sect,
as observed in the case of Syrian refugees. For those of the Baha’i and non-
majority faiths of Iran, this question may lead to false identification, as this
question in Iran usually leads to exclusion and discrimination of non-Shiites.
Questionnaire Design for Refugee Populations i117
Downloaded from https://academic.oup.com/jrs/article-abstract/32/Special_Issue_1/i105/5688806 by theosta@ekke.gr on 16 January 2020
Nationality Concepts such as ethnicity, race or citizenship may be under-
stood differently, depending on the culture. The term ‘nationality’ may be
understood as the status of belonging to a particular nation or to an ethnic
group forming a part of one or more political nations or may be identified as
referring to affiliation to a particular church.
For example, many Arabs surveyed in the REHEAL study have not under-
stood the meaning of ‘ethnicity’, because of the concept of ‘Umma’—a supra-
national community with a common history equivalent to ‘nation’ and closely
related to the phrase ‘Ummah Wahidah’ in the Quran ( , ‘One nation’)
referring to all the Islamic world.
The response to the question ‘What is your nationality?’ may be ‘Syrian’
for Syrians and ‘Palestinian Syrian’ for Palestinians, who technically do not
have nationality in Syria.
3
Perceptions of Time Some notions, such as ‘pre-war’ and ‘post-war’, do not
make much sense for, Afghans, for example, who have been exposed to war
for the greater part of their lives (Hollifield et al. 2002). Time may also be
perceived differently by highly traumatized individuals who may not be able
to recall with accuracy the sequence of events they are asked about. Time
spent in camps and hot spots is usually characterized by high levels of anxiety
and distress, which can also affect respondents’ answers (Steel et al. 2009).
Perceptions of Intimacy Differences in perceptions of intimacy may also
affect the understanding of questions such as ‘Intimacy with one or more
family members’ as a response option to the question ‘To what extent have
you lost any of the following things because of fleeing your home?’. The
concept and definition of intimacy are understood differently in various cul-
tures and, in this respect, the translators need guidance about whether the
question refers to ‘intimacy between husband and wife’ or ‘close familiarity or
friendship’ to avoid misinterpretation. For example, the literal translation of
‘intimacy’ in the Arabic version of the response option ‘Intimacy with one or
more family members’ may carry connotations of more than just close family
relations and friendship, and the term needs to be translated as ‘close famil-
iarity or friendship’.
Conclusions
An analysis of the Arabic and Farsi versions of the REHEAL questionnaire
reinforces the idea that the traditional model of designing and piloting a master
questionnaire in a Western language and then adapting it into languages
spoken by the refugee population may not be the best approach to measuring
trauma or health outcomes in refugees. A path that should be explored and
calls for further research is to involve—at the questionnaire-design stage—pro-
fessionals who master the target language(s) and linguists who have expertise in
questionnaire adaptation. This would open up the discussion on construct
i118 Theoni Stathopoulou et al.
Downloaded from https://academic.oup.com/jrs/article-abstract/32/Special_Issue_1/i105/5688806 by theosta@ekke.gr on 16 January 2020
validity and would need to be supplemented with pre-testing techniques.
However, conducting research in emergency and trying conditions such as
those experienced in Greece does not always allow researchers to follow the
ideal survey life cycle. Instead, a careful trade-off between time and resources
has to be made. In light of the work with the REHEAL study, we have
outlined the major challenges in designing and translating a survey instrument
(questionnaire) for use in diverse, multicultural and multinational populations.
Our goal was to empirically document the benefits and pitfalls of appropriate
measurement instruments for use in refugee research.
1. http://ec.europa.eu/eurostat/statistics-explained/index.php/Asylum_statistics (ac-
cessed July 2019).
2. For example, in Iran, students receive their high-school diploma after Grade 11.
Grade 12 is considered pre-university and exists as an extension year that prepares
students for the college entrance exam.
3. When Israel was created, Palestinians were displaced. Many were displaced to
refugee camps in Syria, where they were treated well and had access to the
Syrian education system, but did not acquire Syrian nationality. This also applies
to Palestinians born in refugee camps in Syria.
BEHR, D. and SCHOLZ, E. (2011) ‘Questionnaire Translation in Cross-national Survey
Research’. Methoden—Daten—Analysen 5(2): 157–179
BEHR, D. and SHISHIDO, K. (2016) ‘The Translation of Measurement Instruments for Cross-
cultural Surveys’. In Wolf, C., Joye, D., Smith, T. W. and Fu Y. (eds) The Sage Handbook of
Survey Methodology. London: Sage, pp. 269–288.
BYRNE, B. (2011) Structural Equation Modeling with Mplus: Basic Concepts, Applications, and
Programming (Multivariate Applications). London: Taylor & Francis.
DAVIDOV, E., SCHMIDT, P. and SCHWARTZ, S. H. (2008) ‘Bringing Values Back In: The
Adequacy of the European Social Survey to Measure Values in 20 Countries’. Public Opinion
Quarterly 72(3): 420–445.
DEPT, S., FERRARI, A. and HALLEUX, B. (2017) ‘Translation and Cultural Appropriateness
of Survey Material in Large-scale Assessments’. In Lietz, O., Cresswell, J., Rust, K. and
Adams, J. (eds) Implementation of Large-scale Education Assessments. Chichester: John
Wiley & Sons, pp. 168–191.
DERLUYN, I., BROEKAERT, E., SCHUYTEN, G. and DE TEMMERMAN, E. (2004) ‘Post-
traumatic Stress in Former Ugandan Child Soldiers’. The Lancet 363(9412): 861–863.
GROVES, R. M., FOWLER, F. J., COUPER, M. P., LEPKOWSKI, J. M., SINGER, E. and
TOURANGEAU, R. (eds) (2004) Survey Methodology. Hoboken, NJ: Wiley.
HARKNESS, J. A., EDWARDS, B., HANSEN, S. E., MILLER, D. R. and VILLAR, A. (2010)
Designing Questionnaires for Multipopulation Research. Hoboken, NJ: John Wiley and Sons,
pp. 33–58.
HO
¨HNE, J. K. and KREBS, D. (2017) ‘Scale Direction Effects in Agree/Disagree and Item-
specific Questions: A Comparison of Question Formats’. International Journal of Social
Research Methodology 16: 1–13.
HOLLIFIELD, M., WARNER, T. D., LIAN, N., KRAKOW, B., JENKINS, J. H., KESLER, J.,
STEVENSON, J. and WESTERMEYER, J. (2002) ‘Measuring Trauma and Health Status in
Refugees: A Critical Review’. JAMA 288(5): 611–621.
JONES, E. F. and FORREST, J. D. (1992) ‘Underreporting of Abortion in Surveys of US
Women: 1976 to 1988’. Demography 29(1): 113–126.
Questionnaire Design for Refugee Populations i119
Downloaded from https://academic.oup.com/jrs/article-abstract/32/Special_Issue_1/i105/5688806 by theosta@ekke.gr on 16 January 2020
JOWELL, R. (1998) ‘How Comparative is Comparative Research?’ American Behavioral
Scientist 42(2), 168–177.
KANE, M. T. (2013) ‘Validating the Interpretations and Uses of Test Scores’. Journal of
Educational Measurement 50(1), 1–73, doi: 10.1111/jedm.12000.
KREBS, D. and HOFFMEYER-ZLOTNIK, J. H. P. (2010) ‘Positive First or Negative First?’.
Methodology: European Journal of Research Methods for the Behavioral and Social Sciences
6(3): 118–127.
MELLENBERGH, G. J. (1989) ‘Item Bias and Item Response Theory’. International Journal of
Educational Research 13(2): 127–143.
MENOLD, N. and KEMPER, C. J. (2015) ‘The Impact of Frequency Rating Scale Formats on
the Measurement of Latent Variables in Web Surveys – An Experimental Investigation using a
Measure of Affectivity as an Example’. Psihologija 48(4): 431–449.
MENOLD, N. and TAUSCH, A. (2016) ‘Measurement of Latent Variables with Different Rating
Scales: Testing Reliability and Measurement Equivalence by Varying the Verbalization and
Number of Categories’. Sociological Methods and Research 45(4): 678–699.
MILLER, K., CHEPP, V., WILLSON, S. and PADILLA, J. L. (eds) (2014) Cognitive
Interviewing Methodology. Hoboken: John Wiley & Sons.
MNEIMNEH, Z., AXINN, W. G., GHIMIRE, D., CIBELLI, K. L. and AL KAISY, M. S.
(2014) ‘Conducting Surveys in Areas of Armed Conflict’. In Tourangeau, R., Edwards, B.,
Johnson, T. P., Wolter, K. M. and Bates, N. (eds) Hard-to-Survey Populations. Cambridge:
Cambridge University Press, pp. 134–157.
MNEIMNEH, Z., CIBELLI, K. L., BILAL, L., HYDER, S., SHAHAB, M., BINMUAMMAR,
A. and ALTWAIJRIB, Y. (2018) ‘Probing for Sensitivity in Translated Survey Questions:
Differences in Respondent Feedback across Probe Types’. In Behr, D. and Sha, M. (eds)
Translation& Interpreting, Vol. 10, No. 2.
MNEIMNEH, Z., PENNELL, B.-E., KELLEY, J. and HIBBEN, K. C. (2016) ‘Surveys in
Societies in Turmoil’. In Wolf, C., Joye, D., Smith, T. W. and Fu, Y. (eds) The Sage
Handbook of Survey Methodology. London: Sage, pp. 178–189.
MOHLER, P., DORER, B., DE JONG, J. and MENGYAO, H. (2016) Cross-Cultural Survey
Guidelines, University of Michigan, Survey Research Center, Institute for Social Research,
http://ccsg.isr.umich.edu/index.php (accessed 10 August 2019).
MOLLICA, R. F. (2016) Healing Invisible Wounds, 3rd edn. Nashville, TN: Vanderbilt
University Press.
MOLLICA, R. F. (2019) ‘TheNewH5Model: Trauma and Recovery: A Summary’. In Suarez-
Orozco, M. (ed.) Humanitarianism and Mass Migration. Confronting the World Crisis.
Oakland, CA: California University Press.
PAN, Y. and FOND, M. (2014) ‘Evaluating Multilingual Questionnaires: A Sociolinguistic
Perspective’. Survey Research Methods, European Survey Research Association 8(3): 181–194.
PAULHUS, D. L. (1991) ‘Measurement and Control of Response Bias’. In Robinson, J. P.,
Shaver, P. R. and Wrightsman, L. S. (eds) Measures of Social Psychological Attitudes, Vol. 1.
Measures of Personality and Social Psychological Attitudes. San Diego, CA: Academic Press,
pp. 17–59.
PENNELL, B.-E., DESHMUKH, Y., KELLEY, J., MAHER, P., WAGNER, J. and TOMLIN,
D. (2014) ‘Disaster Research: Surveying Displaced Populations’. In Tourangeau, R., Edwards,
B., Johnson, T. P., Wolter, K. M. and Bates, N. (eds) Hard-to-Survey Populations Cambridge:
Cambridge University Press, pp. 111–134.
RAMMSTEDT, B., BEIERLEIN, C., BRA
¨HLER, E., EID, M., HARTIG, J., KERSTING, M.
and WEICHSELGARTNER, E. (2015) Quality Standards for the Development, Application,
and Evaluation of Measurement Instruments in Social Science Survey Research’; prepared and
written by the Quality Standards Working Group. RatSWD Working Papers 245, http://www.
ratswd.de/dl/RatSWD_WP_245.pdf (accessed July 2019).
RAYKOV, T. and MARCOULIDES, G. A. (2011) Introduction to Psychometric Theory. New
York: Taylor & Francis.
i120 Theoni Stathopoulou et al.
Downloaded from https://academic.oup.com/jrs/article-abstract/32/Special_Issue_1/i105/5688806 by theosta@ekke.gr on 16 January 2020
ROBERTS, C. (2016) ‘Response Styles in Surveys: Understanding their Causes and Mitigating
Their Impact on Data Quality’. In Wolf, C., Joye, D., Smith, T. W. and Fu, Y. (eds) The Sage
Handbook of Survey Methodology. London: Sage, pp. 579–597.
TOURANGEAU, R. and SMITH, T. W. (1996) ‘Asking Sensitive Questions: The Impact of Data
Collection Mode, Question Format, and Question Context’. Public Opinion Quarterly 60(2): S.
275–304.
TOURANGEAU, R., RIPS, L. J. and RASINSKI, K. (2000) The Psychology of Survey Response.
Cambridge, MA: Cambridge University Press.
SCHWARTZ, S. H. (1992) ‘Universals in the Content and Structure of Values: Theoretical
Advances and Empirical Tests in 20 Countries’. Advances in Experimental Social Psychology
25: 1–65.
SILOVE, D., VENTEVOGEL, P. and REES, S. (2017) ‘The Contemporary Refugee Crisis: An
Overview of Mental Health Challenges’. World Psychiatry 16(2): 130–139.
STATHOPOULOU, T. (2019) ‘Surveying the Hard-to-Survey. Refugees and Unaccompanied
Minors in Greece’. In Suarez-Orosco, M. (ed.) Humanitarianism and Mass Migration. Oakland,
CA: University of California Press, pp. 137–165.
STATHOPOULOU, T., AVRAMI, L., KOSTAKI, A., CAVOUNIDIS, J. and EIKEMO, T. A.
(2019) ‘Safety, Health and Trauma among Newly Arrived Refugees in Greece’. Journal of
Refugee Studies, doi: 10.1093/jrs/fez034.
STEEL, Z., CHEY, T., SILOVE, D., MARNANE, C., BRYANT, R. A. and VAN OMMEREN,
M. (2009) ‘Association of Torture and Other Potentially Traumatic Events with Mental Health
Outcomes among Populations Exposed to Mass Conflict and Displacement: A Systematic
Review and Meta-analysis’. JAMA 302(5): 537–549.l
VAN DE SCHOOT, R., SCHMIDT, P., DE BEUCKELAER, A., LEK, K. and ZONDERVAN-
ZWIJNENBURG, M. (2015) ‘Editorial: Measurement Invariance’. In Van de Schoot, R.,
Schmidt, P. and De Beuckelaer, A. (eds) Measurement Invariance. Lausanne: Frontiers
Media, p. 1064.
VAN VAERENBERGH, Y. and THOMAS, T. D. (2013) ‘Response Styles in Survey Research: A
Literature Review of Antecedents, Consequences, and Remedies.’International Journal of
Public Opinion Research 25(2): 195–217.
WILLIS, G. B. (2004) Cognitive Interviewing: A Tool for Improving Questionnaire Design.
London, New Delhi: Sage Publications, Inc.
WILLIS, G. B. (2015) ‘The Practice of Cross-cultural Cognitive Interviewing’. Public Opinion
Quarterly 79(1): 359–395.
WOLF, C., SCHNEIDER, S. L., BEHR, D. and JOYE, D. (2016) ‘Harmonizing Survey
Questions between Cultures and Over Time’. In Wolf, C., Joye, D., Smith, T. W. and Fu,
Y. (eds) The Sage Handbook of Survey Methodology. London: Sage, pp. 502–524.
YANG, Y., HARKNESS, J. A., CHIN, T. and VILLAR, A. (2010) Response Styles and Culture,
Vol. 2010 (203–223). Hoboken, NJ: Wiley.
Questionnaire Design for Refugee Populations i121
Downloaded from https://academic.oup.com/jrs/article-abstract/32/Special_Issue_1/i105/5688806 by theosta@ekke.gr on 16 January 2020
