Science method

Survey Design - Science method

Explore the latest questions and answers in Survey Design, and find Survey Design experts.
Questions related to Survey Design
  • asked a question related to Survey Design
Question
7 answers
Dear experts,
I am writing a research paper using quantitative descriptive survey design. That means I have my standard deviation to be reported in my research result section. However, I find that it can be confusing when it comes to standard deviation and mean score. I mean, I indeed analyse my mean score along this way, but how about my standard deviation? should I left it there or...???
Thank u for ur time.
Relevant answer
Answer
Except for nominal scales, descriptive statistics on dependent variables should report measurements on central tendency (e.g., mean) and variability (e.g., standard deviation) together—in table or text. For more insights, you could go through the following reader-friendly textbook.
Morgan, G. A., Barrett, K. C., Leech, N. L., & Gloeckner, G. W. (2020). IBM SPSS for introductory statistics: Use and interpretation (6th ed.). Routledge. https://www.routledge.com/IBM-SPSS-for-Introductory-Statistics-Use-and-Interpretation-Sixth-Edition/Morgan-Barrett-Leech-Gloeckner/p/book/9781138578210
Good luck,
  • asked a question related to Survey Design
Question
12 answers
If the original scale has one reverse-coded item, and if I want to adopt it in my study, is it possible to un-reverse it and use all the other items in the same order?
Relevant answer
Answer
Good catch Bruce Weaver , I assumed 1 as the minimum, and this despite two hours ago having colleagues change scales from 1-10 to 0-10 for a study we are about to send to IRB.
  • asked a question related to Survey Design
Question
5 answers
Hello,
Is there anyway to have the correlation coefficient of two categorical variables ,both ordinal and nominal (For example level of education and ethnicity)? I have checked the independency by chi square test, and now the strength of association is important for me.
I have heard about Cramer coefficient but my problem is that  it seems STATA doesn't provide it under survey design set (svy command). Any suggestion?
Thank you very much for your help.
Relevant answer
Answer
You could Cramer's V (according to my shared source) for measuring the strength of an association between two categorical variables when one is ordinal. However, STATA would then simply treat both variables as nominal, meaning that you will lose some basic information on one of your variables. Hence, it might not be the ideal test.
  • asked a question related to Survey Design
Question
1 answer
I'm triying to analyze the teleworker's labor market before and during the pandemic crisis but there aren´t any specific surveys designed to measure this kind of workers, do you know anyone who has managed to identify teleworkers as cases in the survey? (not as a proportion of specific sectors or activities like Dingel and Neimann or Monroy Gomez Franco already did).
Thanks in advance.
  • asked a question related to Survey Design
Question
2 answers
Dear all
Is there any free software or open source program for seismic survey designing?
Relevant answer
Answer
Dear Mohammed Farfour, try this website:
After registration, you can download a free version by a month, OMNI 3D Seismic Survey Design. Also, I suggest before that, check the input data that you need for free, and therefore, do not lost days of evaluation.
Best regards, Mario E. Sigismondi
  • asked a question related to Survey Design
Question
10 answers
I am going to conduct a survey among the experts who are working in power plant construction projects in my country. So far I know, construction of 30 mega projects are going on. The targeted experts are 7-categories, for example, contractors, sub-contractors, vendors, project director (PD), project manager (PM), site engineer, and consulting engineer (or consultant). The other variables are project size in terms of power generation capacity, budget, project location, experts' experience (year), and academic qualification. Please suggest me in a precise way to save my time? I am now in a critical moment. I have a presentation just after couple of weeks. Thank you for your patience and time. 
Relevant answer
Answer
Dear, Muhammad Saiful Islam, this link will help you in your research:
Determining the sample size in a quantitative research study is challenging. There are certain factors to consider, and there is no easy answer. Each experiment is different, with varying degrees of certainty and expectation. Typically, there are three factors, or variables, one must know about a given study, each with a certain numerical value. They are significance level, power and effect size. When these values are known, they are used with a table found in a statistician's manual or textbook or an online calculator to determine sample size.
Choose an appropriate significance level (alpha value). An alpha value of p = .05 is commonly used. This means that the probability that the results found are due to chance alone is .05, or 5%, and 95% of the time a difference found between the control group and the experimental group will be statistically significant and due to the manipulation or treatment.
Select the power level. Typically a power level of .8, or 80%, is chosen. This means that 80% of the time the experiment will detect a difference between the control and experimental groups if a difference actually exists.
Estimate the effect size. Generally, a moderate to large effect size of 0.5 or greater is acceptable for clinical research. This means that the difference resulting from the manipulation, or treatment, would account for about one half of a standard deviation in the outcome.
Organize your existing data. With the values for the three factors available, refer to the table in your statistician's manual or textbook; or enter the three values into an online calculator made for determining sample size.
  • asked a question related to Survey Design
Question
7 answers
Dear academy colleagues,
I'm looking for a truly comprehensive resource for teaching graduate students the elements of conducting robust survey research, including proper survey development, validation, distribution, confidentiality, data security, collation, statistical analysis, interpretation, and sense-making. I've seen elements of these in resources here and there, but not complete, and usually not written in a way that is accessible to a graduate student.
Do you have recommendations about a single resource or a progression of resources that really help a student get from zero to fairly strong (obviously with practice and some mentoring)?
Relevant answer
Answer
Progression. What resources to which learners must be exposed to, this must be carefully done otherwise learners get confused.
  • asked a question related to Survey Design
Question
10 answers
I was studying the following paper:  Park, D.H., Lee, J. and Han, I., 2007. The effect of on-line consumer reviews on consumer purchasing intention: The moderating role of involvement. International journal of electronic commerce11(4), pp.125-148.
My query is that first 3 items in the scale used to measure attitude in this paper are definitely about positive attitude while next 3 items seem to be about negative attitude. Is my understanding correct? If yes, are these items (from #4 to #6) reverse coded?  
Relevant answer
Answer
I agree that items 5 and 6 are reverse coded, but #4 essentially refers to "not worrying," so I would expect it to correlate positively with the first 3 items.
  • asked a question related to Survey Design
Question
5 answers
Asking this as Google forms doesn't allow multiple items in a semantic differential format.
Relevant answer
Answer
You can also try SoGoSurvey. They have a free student account.
  • asked a question related to Survey Design
Question
2 answers
I'm a bit new to these aspects of survey design and analysis. What should I read and what are some approaches to the following situation and question?
Suppose:
  • We've a population-of-interest based on an affiliation, certain actions, or a set of ideas; (e.g., 'vegetarians' or 'tea-party conservatives)... call it the "Movement"
  • There has never been a national representative survey nor a complete enumeration of this group. There is no 'gold standard'
  • For several years we've advertised a survey (with a donation reward) in several outlets (web pages, forums, listserves which we call 'referrers') associated with the 'movement'
  • We can track responses from each referrer. We suspect some referrers are more broadly representative of the movement as a whole than others, but of course there is no gold standard.
This is essentially a 'convenience sample', perhaps more specifically a 'river sample' (using the notation of Baker et al, 2013) or 'opt-in web-based sample'. It is probably non-representative because of
  • Exclusion/coverage bias: Some members of the movement will not be aware of the survey (they don't visit any of the outlets or they don't notice it)
  • Participation/non-response bias: Among those aware (through visiting the 'referrers') only a smallish share complete the survey (and these likely tend to be the more motivated and time rich individuals). Some outlets/referrers may also promote the survey more prominently than others.
We wish to measure:
  • The (changing) demographics (and size) of the movement
  • Measures of the demographics, beliefs, behavior, and attitudes of people in the movement (and how these have changed from year to year)
Our methodological questions
Analysis: Are there any approaches that would be better than 'reporting the unweighted raw results' (e.g., weighting, cross-validating something or other) to using this "convenience/river' sample to either:
i. Getting results (either levels or changes) likely to be more 'representative of the movement as a whole' then our unweighted raw measures of the responses in each year?
ii. Getting measures of the extent to which our reports are likely to be biased ... perhaps bounds on this bias.
Survey design: In designing future years' surveys, is there a better approach?
Brainstorming some responses...
Analysis
  • E.g., as we can separately measure demographics (as well as stated beliefs/attitudes) for respondents from each referrer, we could consider testing the sensitivity of the results to how we weight responses from each referrer.
  • Or we might consider using the demographics derived from some weighted estimate of surveys in all previous years to re-weight the survey data in the present year to be "more representative."
  • As noted we subjectively think that some referrers are more representative than others, sSo maybe we can do something with this using Bayesian tools
  • We may have some measures of the demographics of participants on some of the referrers, which might be used to consider weighting to deal with differential non-response
Survey design
  • Would 'probability sampling' within each outlet (randomly choosing a small share within each to actively recruit/incentivize, perhaps stratifying within each outlet if the outlet itself provides us demographics) somehow be likely to lead to a more representative sample?
It's not immediately obvious to me why this would improve things. The non-response within probability samples would seem to be an approximately equivalent problem to the limited participation rate in the convenience sample. The possible advantages I see would be:
i. We could offer somewhat-stronger incentives for the probability sample, and perhaps reduce this non-response/non-participation rate and consequent biases.
ii. If we can connect to an independent measure of participant demographics from the the outlets themselves this might allow us to get a better measure of the differential rates of non-participation by different demographics, and adjust for it.
Some references (what else should I read?)
Baker, R., Brick, J.M., Bates, N.A., Battaglia, M., Couper, M.P., Dever, J.A., Gile, K.J., Tourangeau, R., 2013. Summary report of the AAPOR task force on non-probability sampling. Journal of survey statistics and methodology 1, 90–143.
Salganik, M.J., Heckathorn, D.D., 2004. Sampling and estimation in hidden populations using respondent-driven sampling. Sociological methodology 34, 193–240.
Schwarcz, S., Spindler, H., Scheer, S., Valleroy, L., Lansky, A., 2007. Assessing Representativeness of Sampling Methods for Reaching Men Who Have Sex with Men: A Direct Comparison of Results Obtained from Convenience and Probability Samples. AIDS Behav 11, 596. https://doi.org/10.1007/s10461-007-9232-9
Relevant answer
Answer
For hard-to-reach populations you might find participant-guided-sampling useful. Explore
  • asked a question related to Survey Design
Question
6 answers
For example: Would using a 7 point scale of 0 - 3 in steps of 0.5 give you different results to using a 7 point scale of 0 - 6 in steps of 1? I'm aware that verbal labels are likely to be better but I'm interested in the possible differences between purely numerical scales that use 0.5 or 1 increments.
Thanks.
Relevant answer
Answer
Hello David,
As the 0-6 scale your describe is a simple, linear transformation of the 0-3 scale (by doubling the scores), there is no statistical or measurement advantage to either. Whether respondents react similarly to the different response scales is an empirical question.
Good luck with your work.
  • asked a question related to Survey Design
Question
8 answers
Does anyone know a good software that you can use to develop psychological experiments that are run on a smartphone? Thank you!
Relevant answer
Answer
Resultal.com is an affordable and reliable research platform. Resultal offers online questionnaires, tests and experiments for PC, tablet and smartphone for research on every location. For only 5,50. Check: https://www.resultal.com
  • asked a question related to Survey Design
Question
5 answers
Anderson et al. 2017 describe a method for minimising the limitations of PERMANOVA when data is heteroscedastic and with unbalanced designs. Im trying to compare community composition between populations in R using the 'adonis' function but I dont know what to do with regards to the structure of my data and survey design which dont meet the assumptions. I dont know if this is available in PRIMER 7 but ideally I would like to apply it in R.
Anderson et al. 2017. Some solutions to the multivariate Behrens–Fisher problem for dissimilarity-based analyses.
Aust. N. Z. J. Stat. 59(1), 2017, 57–79
Relevant answer
Answer
Paul Anstey As far as I know Adonis is the only function for running PERMANOVA in R. I ended up installing a Windows Boot Camp on my Mac and then downloading the trial version of PRIMER which lasts for 30 days. It gave different results to Adonis but Im not sure why
  • asked a question related to Survey Design
Question
4 answers
Hi. I am trying to help provide some rough advice for selecting amongst alternative sampling designs to measure bird density or abundance. The surveys are short-term in duration such as for impact assessments and habitat association studies. In this instance, I am looking at rough guidelines for how to select where to sample (survey design) and not how to sample (survey method). Attached is a rough first draft of a decision tree (see attachment) to select between various forms of study design ranging from conducting a census to spatially balanced sampling & stratified random designs. Do you have thoughts and suggestions? Is neglecting to include simple random sampling a fatal flaw for example? If so, where would you place it? Of course, the main advice will be to consult with a biostatistician, but hopefully this (with come accompanying text and references) can provide some rough guideance and be a starting point for that conversation. Literature, debate and suggestions welcome and appreciated.
Relevant answer
Answer
Steven L. Van Wilgenburg I agree with your last post. Have you tried searching the relevant literature?. If I were doing it I would do it as I advised in a recent post. Divide you total area up into strata and take a simple random sample from each stratum. My post here should help:
  • asked a question related to Survey Design
Question
4 answers
Dear esteemed colleagues,
I would love to hear your thoughts or opinions on the following statements posted. Thank you very much in advance
Example statement
1). Launching a research study to study why doing something as now become a cultural phenomenon.
OR
2). Launching a research study to study if doing something as now become a cultural phenomenon.
I am of the train of thought that with statement 1, there is an existence of a cultural phenomenon due to previous extant literature and this research study wants to study why this is the case as opposed to statement 2 which the study wants to study if there is one (Yes or No)
Relevant answer
Answer
Yes, statement 1 supposes there exists a certain cultural phenomenon and proposes to study reasons for it; statement 2 proposes to determine whether such a cultural phenomenon exists at all. But in what sense is existence "due to literature"? Is the cultural phenomenon in question a literary phenomenon? Or are you referring to research articles (the research literature) that alleges there is a cultural phenomenon? In the latter case, the existence may not be "due to literature" unless the research literature has become like a self-fulfilling prophecy.
  • asked a question related to Survey Design
Question
9 answers
My team and I have developed a prototype of an augmented reality mobile application for teaching primary school students human anatomy. We are going to do a usability testing and evaluation with the primary school students using FUN toolkit, and we are also going to conduct an expert review using heuristic evaluation and cognitive walkthrough. Furthermore, we also want the teachers to test the app, and to evaluate the usability in the context of their students' usage. However, the teachers are neither usability experts nor end-users so what is the most appropriate method for them regarding usability testing, survey design etc? Do you have any recommendations for usability testing methods, survey design/template etc.?
Relevant answer
Answer
I would let the teachers have a test run and then interview them afterwards about their pedagogical views. I see no real need for a CW with the teachers. I would include them in the test with children though. They will be of great help in verbalizing issues that the kids cannot identify themselves and they know the children and are expert about children’s behavior. Ideally you’d let the teachers play fist so they know the interface. My2Cents
  • asked a question related to Survey Design
Question
2 answers
We invite you to take a short survey about key concepts used in the field of speciation research. Through a series of questions, this survey is designed to capture people’s thoughts about concepts that are central to speciation research. We plan to use the answers to understand why researchers think about speciation differently, and to help people understand one another’s perspectives.  
The survey is anonymous, and open to anyone that would like to take it, though we are primarily targeting researchers that study speciation. The summarized results may be circulated and published and answers will ultimately be available to anyone that wants to use them. The country of origin will only be used to measure the spread of the survey.  Please circulate the link to anyone that you think may be interested in taking part!
The survey was written by Dr. Sean Stankowski, a research scientist from the department of Animal and Plant Sciences at the University of Sheffield (https://www.sheffield.ac.uk/aps), and Dr. Mark Ravinet, a research scientist from the Centre for Ecological and Evolutionary Synthesis (https://www.mn.uio.no/cees/english/). The survey has received ethical approval from the Department of Animal and Plant Sciences, University of Sheffield. In the event of any concern or complaint about this survey, please contact the Head of Department, of Animal and Plant Sciences, University of Sheffield. You can take the survey here:
Please forward it to any colleagues or students that you think would like to take part!
If you have any general questions about the survey, please email: s.stankowski@sheffield.ac.uk and/or mark.ravinet@ibv.uio.no
Sean Stankowski and Mark Ravinet
📷
Relevant answer
Answer
Maybe you can share some results on RG in due course.
  • asked a question related to Survey Design
Question
4 answers
Hey guys, the research question of my master thesis is " Effects of age on attitude towards information disclosure on e-commerce platforms". For testing my hypothesis, I gathered data through survey. Its a self constructed survey, with certain limitations and now I realize I should have tested more items mentioned in the literature review for certain variables which I didn't.
I plan to mention it in the limitations but not sure what valid reason to give for not testing certain items in contrast to the ones which I have tested in the survey. Please advice. Thanks in advance.
Relevant answer
Answer
RG colleagues
Have given valuable suggestions.
  • asked a question related to Survey Design
Question
3 answers
How do I refer to the fact that these two questions will be analysed using inductive content analysis?
Is it mixed -methods survey design?
Survey design using mixed methods to analyse the gathered data?
How would you refer to it? Thank you.
Relevant answer
Answer
An Explanatory Sequential design has more to do with how the data are used, rather than when they were collected. So, if you establish results with the quantitative data, and follow-up by using the qualitative data to help understand those results, then that is QUANT --> qual.
But, if you are not producing any quantitative results, then it sounds like your design is probably Complementary, in the form QUANT + qual. This is especially likely if the qualitative data help you go beyond the content covered in the quantitative data.
  • asked a question related to Survey Design
Question
5 answers
While developing a questionnaire to measure several personality traits in a somewhat unconventional way, I now seem to be facing a dilemma due to the size of my item pool. The questionnaire contains 240 items, theoretically deduced from 24 scales. Although 240 items isn't a "large item pool" per se, the processing time for each item is averages on ~25 seconds. This yields an overall processing time of over >1.5 hours - way to much, even for the bravest participants!
In short, this results in a presumably common dilemma: What aspects of the data from my item analysis sample to I have to jeopardize?
  • Splitting the questionnaire into parallel tests will reduce processing time, but hinder factor analyses.
  • Splitting the questionnaire into within-subject parallel tests over time will require unfeasible sample sizes due to a) drop-out rates and b) eventual noise generated by possibly low stability over time.
  • An average processing time over 30 minutes will tire participants, jeopardize data quality in general.
  • Randomizing the item order and tolerating the >1.5 hours of processing time will again require an unfeasible sample size, due to lower item-intercorrelations.
I'm aware that this probably has to be tackled by conducting multiple studies, but that doesn't solve most of the described problems.
This must be a very common practical obstacle and I am curious to know how other social scientists tackle it. Maybe there even is some best practice advise?
Many thanks!
Relevant answer
Answer
Sounds like you've created an instrument which is sound as it is based on theory - but this version should only be the pilot version, rather than the final instrument. As you see - there are too many items, which will affect the quality of the data collected. Have you conducted any sort of pilot study - factor analysis - and tested to see how those items relate? You'll probably find some items are redundant, and possibly even some scales ... use EFA to explore how the items load - then as you delete items - the number of items in each scale will decrease - you can then delete scales on the basis of insufficient items.
I find David De Vaus' work on survey design and validation very useful:
  • asked a question related to Survey Design
Question
7 answers
Should I use a pretest postest design or a survey design? Thank you very much
Relevant answer
Answer
Ideal will be qualitative research design
  • asked a question related to Survey Design
Question
5 answers
I have two different advertisements and two dependent variables, which are the same - 'how positive or negative is your attitude toward the advertisement?'. One ad is a 'femvertisement' (feminist advertisement) and I plan on doing an ordinal regression with 3 models.
Model 1 independent variables: interest, knowledge, etc. to determine how individuals process the ads
Model 2: processing variables and feminist attitudes
Model 3: processing variables, feminist attitudes, and socio-demographics
I'm wondering whether, from the two regressions, I will be able to infer how different processing levels (as inferred from the interest, knowledge, etc.) will effect attitudes toward the advertisements. Is there an analysis I can do to compare the effects or can I explain this with the two regressions?
Thanks in advance.
Relevant answer
Answer
Thank you all for your responses, they are greatly appreciated.
  • asked a question related to Survey Design
Question
14 answers
Hello everyone! 
I am conducting a survey on the faculty members of the universities in my country. My question is since I don't know the exact number of teachers in my my country how can I calculate sample size?
Relevant answer
Answer
Also you can use Power and Precision V4 Application for this purpose. Gook Luck !
  • asked a question related to Survey Design
Question
10 answers
Hi all,
I was wondering what is your opinion on converting data (themes and categories) into a survey in a mixed methods research design (exploratory sequential MMR). Any instructions or articles about this? Do you follow survey design instructions or is there something more in MMR?
Thank you
Relevant answer
Answer
What you are proposing makes perfect sense in terms of an exploratory sequential MMR design. the problem is that not much has been written that explicitly deals with procedures for how to convert themes and categories into operational survey research. Apparently people assume that it is simple and straightforward.
In general, each theme or theoretical category would be the equivalent of a multi-item scale in a survey. If you are not familiar with writing survey items and creating scales, a good introductory source is: Fowler, Survey Research Methods, from Sage.
  • asked a question related to Survey Design
Question
3 answers
I am designing a study looking at mental health outcomes of a population that had a certain exposure a year ago. While modeling the growth curve of the mental health outcome as related to the degree of initial exposure, I would also like to model the influences of some external factors that evolve/ take place in the next few years on the health outcome. So my study involves a panel survey to be administered a few times in the next few years, with an additional questionnaire for the 1st round that requires participants to recall their initial exposure. Should I call this a combined retrospective and prospective study? Can anyone recommend some texts that define/ describe/ comment on similar research designs (survey design, methodology, or empirical examples)? Thank you very much!
Relevant answer
Answer
Ambi-directional Cohort Study
  • asked a question related to Survey Design
Question
1 answer
I know of some research into priming effects, but usually these are presented to demonstrate that context changes/ distorts attitudes. I am wondering whether there is any research suggesting that asking students to recall an example first (i.e., before asking their belief/opinion) is good survey design practice because it ensures that the construct being probed is salient to them?
Relevant answer
Answer
Are you familiar with the hooligan-professor priming study?. It is not about surveys but is interesting. . I recommend you to also check the comment by Dijksterhuis in the same journal issue.
  • asked a question related to Survey Design
Question
4 answers
Thanks to everyone in advance for any assistance with this conundrum.
I recently completed the defense of my dissertation proposal and was asked by my committee to shift from an email-based survey to a web-based format. At first, this seemed straight-forward and a good way to collect and manage the data from my responses.
However, upon closer inspection, I am running into a challenge I had not encountered in my original design.
I plan on administering a 5-question survey to all business (and related) faculty in Ohio community colleges to measure the degree of curriculum internationalization in the courses they develop and/or instruct. The first two questions are simply requests for job titles and contact information (textbox entries). Question 3 asks each respondent to list every business-related course that they have taught at the institution in the past 5 years (again, a textbox seems logical here).
Question 4 then asks each respondent a series of four Likert-scale questions based on each course they identified in Question 3. While this seems simple enough (and probably would be easy for someone to complete in a simple Word document--as I had originally planned), I'm struggling to figure out how to incorporate this type of question on any of the major web-based survey websites.
Question 5 is similar to question 4 as it then asks the respondent to identify for how long each course has been utilized/deployed with the degree of internationalized curriculum identified in question 4.
My main question is how do I create a tool on any web-based survey platform that would allow respondents to volunteer course numbers/titles that would later have follow-up questions about each specific course? I've dabbled with Google Forms, SurveyMonkey, and Sogosurvey without much success as I keep running into this issue.
I had posited about using a comprehensive dropdown list of EVERY business-related course offered at EVERY institution to allow self-selection and logical page progression, but that doesn't seem to resolve the issue and just makes the survey look even more convoluted. The same would be said if I created a dropdown list on page 1 for the respondent to choose the specific institution, the list of courses on page 2, etc.
I am probably overthinking this entire process, but I can't seem to think of how to structure this survey using the web-based tools that are out there and wonder if simply asking respondents to just type out a word document with answers to the questions would be simpler and result in just as many responses.
Relevant answer
Answer
Thanks to everyone! I managed a workaround by changing the approach and specific questions being asked. Branching would have meant exhaustingly long surveys for faculty with many different preparations to analyze. I looked into several of the options with Qualtrics, SurveyMonkey, SurveyPlanet (and many others) and they do seem viable if there would not be such a workaround.
  • asked a question related to Survey Design
Question
50 answers
I have a very large survey dataset and am writing two distinctively different papers from the same dataset. One more focused on policy implications and the other a more traditional academic journal article. My question is how should I handle the methods section? For the most part, the methods section is the exact same, for obvious reasons. The statistical analyses differ. What do I do about the methods that are the same--survey design, implementation, etc? I have already written a descriptive methods section and would just like to use verbatim what I wrote for the second paper but then would I have copyright issues? Is it okay to just refer my audience for the second paper to the first paper for most of the methods? Thanks for your feedback!
Relevant answer
Answer
you can use the same data set but take care from the plagiarism
  • asked a question related to Survey Design
Question
6 answers
Hi Folks,
I am looking for a validated survey tool to track changes in participants' notions of power and inequality in society. Rather than reinventing the wheel, I am wondering if anyone has, or has used one that I may be able to draw from in an upcoming study.
Relevant answer
Answer
you might find the following helpful
The Gilens and Page Measure of Political Inequality
However, it's more likely that you will find measures if you define your variables of interest more as it's unlikely that one measure can capture all aspects of power and inequality.
  • asked a question related to Survey Design
Question
3 answers
I am currently trying to design a survey based small scale experiment for my dissertation. I have chosen behavioral economics and want to base my paper on prospect theory and heuristics but am struggling with the questions for my survey.
I wanted to first test for the obvious things such as people reacted more drastically to negatively 'framed' situations.
But then also wanted to test whether the object under uncertainty has an effect on someone's riskiness such as comparing the same situation but using money and then human lives and seeing if their response is the same or different. 
Can anyone offer any help? 
Regards
Joshua Isaacs
Relevant answer
Answer
Joshua Isaacs Did you end up with a good design for prospect theory?
  • asked a question related to Survey Design
Question
9 answers
As you are doubtless aware, paper-based survey has been known as one of the most common methods for gathering data relevant to people's behavior (either revealed preferences or stated preferences). I wanna make sure how much can we rely on new methods like Internet (Web)-based survey instead of traditional paper-based survey? In particular, my research's scope is related to travel behavior analysis. My research' sample should cover all socioeconomic groups and almost all geographical areas in a city.
I would be happy if somebody shared with me his/her opinion or the valid references.
Thanks in advance
Relevant answer
Answer
Another problem you have to consider is about respondent’s willingness to participate. You have to have a reliable database of lead contacts and be aware that response rate is very low, commonly around 5% to 10% so if you need a sample of 400 subjects, for example, you have to contact at least 8000 people.
Of course, never forget the main characteristics of your sample otherwise your results will be biased.
  • asked a question related to Survey Design
Question
7 answers
We are conducting a randomized study on the effect of Pterigopalatine fossa block (PPFB) during endoscopic sinus surgery on intraoperative bleeding.The number of endoscopic sinus surgeries performed the previous year were 86. What would be the valid sample size for this study?
Relevant answer
Answer
HI your sample size depends on the turnover of patients per year or month, based on your study duration.If your study duration is 1 year check the number of patients of last year treated on that particular case, then decide the sample size. For eg., if last year was 75 patients treated on that particular case,you can take sample size as 60 for a safer side.
  • asked a question related to Survey Design
Question
7 answers
I am planning to include "risk behaviour and time preference of smallholder farmers" as factors that affect their decision to adopt long-term improved soil and water conservation practices in Ethiopia. For this I need to construct simple questions which are easy to be understood by farmers. If you have any idea, please, forward it.
Relevant answer
  • asked a question related to Survey Design
Question
8 answers
Hello, folks:
Few days ago, I've received the following comments from one of the reviewers for my manuscript I submitted. His comment was as follows:
The authors' data source for their dependent variable, mentally unhealthy days, is the Behavioral Risk Factor Surveillance System (BRFSS) based on a complex sample survey design. Yet, the models the authors use (Ordinary Least Square- OLS) and their analyses do not take this survey design into account; specifically, their analyses do not account for the positive correlation among respondents in their replies to interviewers within State strata and primary sampling units. The variability of the authors' estimates (for example, regression coefficients of associations) are probably underestimated.
Do you guys know how to take the survey design into account? Honestly, I don't have a clue on how to address the reviewer's comment. I've used SPSS to run the model using BRFSS data. Please advise and any useful comments and tips (videos or references) will be greatly appreciated. Thanks in advance!
Relevant answer
Answer
I am only a student and thus may not have great insight in methods like other contributors here but, i would strongly advise to look through
"Applied Survey Data Analysis, Second Edition Steven G. Heeringa, Brady T. West, Patricia A. Berglund" Examples of similar problems are shown with interpretation of results and assessment of the standard errors. all you need to do is identify the design variables ( PSUs, Final weights, stratum variable) these are always provided by the data provider if you read the documentation. once you declare these, the computational models are the similar and depending on the software of choice, you will obtain design adjusted standard errors with adjusted degrees of freedom.
  • asked a question related to Survey Design
Question
3 answers
We are working in a survey design to youths, but we will include children are 12-14 year old. The classic measuring of time preferneces and risk will be complex to childrens. Likewise, some recomendation of general application of this question in survey experiment would be perfect. 
Relevant answer
Answer
Hi
Maybe this research can help you:
Subjective time discount rates among teenagers and adults: Evidence from Israel
E Lahav, U Benzion, T Shavit - The Journal of Socio-Economics, 2010
Best wishes,
Yaakov
  • asked a question related to Survey Design
Question
1 answer
I am carrying out research to better understand candidates' experiences of doing the IELTS test. I am seeking to measure the construct of test-takers' positivity towards their experiences of taking IELTS using Likert Scales and some open-ended comments. I have designed a draft questionnaire and am seeking to enlist the help of anyone with expertise in survey design, language testing, or the IELTS test to provide feedback on the content and construct validity of the survey.
If you can help, feel free to get in touch with me at my email address wsp202 'at' exeter.ac.uk
Thanks,
William
Relevant answer
Answer
Hi William,
I've taken quite a few tests including IELTS, paper and computer-based version. I also worked as a test evaluator for the SPEAK test quite a long time ago, though. I might qualify for my experience if it is useful to you... Is it a long survey?
Jorge E. Benavides B.
  • asked a question related to Survey Design
Question
6 answers
While working with a translated (into Lithuanian) version of the IMS (Investment Model Scale) I'm getting a sub-par Cronbach's alpha score of 0,629. The highest increase with an item deleted only goes up to 0,666.
However, when scoring each of the 4 sub scales separately I get good reliability:
,798
,826
,788
,721
I'm sure there's something fundamental I don't understand here, but at this point I'm clueless as to what to do to increase the reliability of the questionnaire. Any help would be appreciated.
Best regards.
Relevant answer
Answer
Dear Edgrasas
1. Yes, it is possible.
2. There are several measures. Indeed, reliability measures are: Cronbach’s alpha reliability, split-half reliability and Cronbach’s alpha if item deleted. They are essential and very important measures to assess the survey design.
3. For more details, please refer to:
i. Quantitative and qualitative indicators to assess the performance of higher education institutions. Int. J. of Information and Decision sciences, 2014, Vol.6, No. 4, pp.369-392 (Inderscience).
ii. Statistical characteristics of performance indicators. Int. J. of Quality and Innovation. Vol.2, No.3-4, pp. 385-309 (Inderscience).
Regards,
Zuhair
  • asked a question related to Survey Design
Question
2 answers
My mixed methods study is on the impact of feedback in the writing process and I have three questions (listed below). To resolve the first question, a repeated measures design was used to determine which form of feedback has the most impact. A survey design was used for the second question and a discourse analysis will be used to analyze the language for similarities and differences. After referring to Crewell and Plano Clark's book, I know that I can not use what I find in question three to explain question one and two. Based on my research questions and what I read in Crewell's book, I am thinking that this is not a true mixed methods and my study will not fall under any of the major mixed methods designs. Am I on the right track?? Please advise
1. What is the impact of feedback via teacher one-on-one writing conferences versus computer-based writing feedback on the writing skills of at-risk Pre-college students?
2. How do students who participated in the study perceive the effectiveness of teacher feedback versus computer-based feedback?
3. How is the discourse in teacher-student interactive feedback versus computer-based feedback alike and /or different
Relevant answer
Answer
What you are describing sounds like a QUANT --> qual design, where you use the QUANT results to design a qual study that will help you explain (and perhaps expand on) the original QUANT results.
This is known as an explanatory sequential design, so look for that term in the Cresell and Plano-Clark textbook.
  • asked a question related to Survey Design
Question
3 answers
I have two instruments for my research. I gained approval to use the 6 question instrument which was provided to me in a word document. The other one must be provided to the participant through the Center for Leadership Studies website for a 3 dollar fee for each survey. I will also need to create a demographic questionnaire and provide the participant with an appropriate consent form. I am wondering the best way to ensure my participants receive and take all of these assessments since they are in different formats. Does anyone have any best practices. This is for my dissertation so this is my first time doing a survey design.
Thank you in advance for any advice you can offer!
Relevant answer
Answer
There are many application for survey, paid and free, google forms is better.
  • asked a question related to Survey Design
Question
4 answers
I know how to set up distribution panels/survey invitations in Qualtrics, but how do I send a message to the contact list without sending the survey with the message? Specifically, how do I send a pre-communication so I can give advance notice to the contacts that they will be receiving a request to participate in a survey?
Thanks,
M. Justin Miller, M.S.
Relevant answer
Answer
Through the Contacts tab at the top of the page, select your target distribution list. Once selected, go to "List Options" on the right side of the screen. On the drop-down menu, click "Send Email to List..." and a message window should appear. Be sure to include both a "From" and "Reply to" email address, or Qualtrics will not allow you to send your email.
Hope this helps!
  • asked a question related to Survey Design
Question
7 answers
I am still collecting survey tools used by researchers who have used in studies of transgender men and women. The tools will be used to inform the development of a new survey tool that may be added to CDC's National HIV Behavioral Surveillance (NHBS) survey set. If you are willing to share your survey tool (and have not already done so) please do so.
All the best,
Stephen 
Relevant answer
Answer
I follow the question
  • asked a question related to Survey Design
Question
9 answers
I need a measure for biographical learning for my survey design assignment - I was searching for hours and couldn´t find one.
Relevant answer
Answer
The Reynolds and Bigler studies can be useful, especially the Memory of Learning (TOMAL) test:
Reynolds and Bigler, 1994a
C.R. Reynolds, E.D. BiglerTest of Memory and Learning (TOMAL)
Pro-Ed, Austin, Texas
Reynolds and Bigler, 1994b
C.R. Reynolds, E.D. BiglerTest of Memory and Learning (TOMAL) Examiner's manual
Pro-Ed, Austin, Texas
Reynolds and Bigler, 1996
C.R. Reynolds, E.D. BiglerFactor structure, factor indices, and other useful statistics for interpretation of the Test of Memory and Learning
Archives of Clinical Neuropsychology, 11 (1) (1996), pp. 29-43
  • asked a question related to Survey Design
Question
6 answers
When being asked about the number of partners they have had, sex workers and sexual assault survivors may not want to include their clients or their rapist. Does anyone have questions they have used or suggestions for how to word survey questions to take this into account? How do we give survey respondents permission to leave these sexual partners out of the total and do so in a sensitive way?
Relevant answer
Answer
One possibility is prefacing these questions with a brief explanation that the questions refer to consenting partners, since reaching age of majority, and excluding any sex work.
Your data will be noisier if you leave it up to the respondent to include or not. If you want to focus on consenting sexual behavior outside of sex work, then I would recommend specifying.
  • asked a question related to Survey Design
Question
4 answers
I want to develop a research project which compare the behavior of the same respondent before and after of an experiment. kindly help me by referring appropriate literature.  
Relevant answer
Answer
Thank you very much for your precious comments. I would appreciate if someone refers some good quality articles related to this technique.
  • asked a question related to Survey Design
Question
4 answers
As there is always some attrition in longitudinal research, then how can a researcher match the responses from same respondent at T1, T2 and T3...... in a longitudinal survey design? So that a researcher may carry out t-test later to check whether those who participated in all the surveys and those who left out earlier are significantly different.
Relevant answer
Answer
You will need to have a data file that contains information how to contact people and you might have a separate data file that only holds identifiers and research data. You  need to ensure the identifier are unique. I normally use a database to store the information on people and have other systems to collect the data. The database creates unique number to be used and these numbers can be used to connect to the data. SPSS (research data) and ACCESS (personal data) can be used to handle of this. It is very hard to get unique numbers. In the Netherlands I would like to use the BSN (civel status number), but I am no allowed to have it (I could get it, but am not allowed to). One unique identifier is the email address. At no point in time there will exist two identical email addresses. However I am not sure if this holds over time.
Reconstructing record or person identification is detective work and feasible for small sets of data, but very hard to do for large data sets. Tracking and tracing duplicates may take a lot of time and it will mess up you data.
Is the data already collected or are you still at the start of your project? 
  • asked a question related to Survey Design
Question
9 answers
Hello research experts!
I'm currently performing research into different types of survey designs, response coding and analyses. I came across a citation I was hoping someone might be able to decode:
"When creating response scales, it's important to take into consideration its orientation. While there are many caveats and exceptions when creating response items, one effect is that respondents tend to favor the left side of a response scale.  Take the following two response options:
Strongly-Disagree      Disagree        Undecided       Agree        Strongly-Agree
Strongly-Agree          Agree      Undecided        Disagree       Strongly-Disagree
If you code the values from 1 to 5 for the first scale and 5 to 1 on the second scale then you’ll have a higher average score on the second response option".
I'm just slightly confused as to how the average score would change in the second response item scale, since it's been inversely coded from the first, so my thinking is that the average shouldn't change, but there may be something crucial I'm missing.
Any help would be greatly appreciated. Thank you!
.
Relevant answer
Answer
Dear Darko,
you can also include several, slightly different questions around the same issue into your questionnaire,  and have the same scale running from 1 to 5 in all of them. Just change the perspective in the questions or statements into opposite ones. Let  me explain this with an example: You could have these two statements in your survey (a real-World example from our institution)
1. Larger municipalities are more cost-efficient   :    1-strongly disagree  5-strongly agree
2. Smaller municipalities listen better to the wishes of the inhabitants   1-strongly disagree  5-strongly agree
We tried to single out the preferences of the inhabitants to the municipal structure, so we had a myriad of questions constructed so that around half of them we pro-small and the other half pro-large municipalities.
Good luck,
Jokke
  • asked a question related to Survey Design
Question
8 answers
This quotation comes from the famous poem  “Morte d’Arthur” by Alfred, Lord Tennyson.
Compare, perhaps, scientific testing of the proposition:
“More patients suffering from disease D are cured by treatment T than doctors yet realise”
which seems analogous.
Relevant answer
Answer
This isn't an answer but I am thinking about bird ringing and disease surveillance. It all hinges on the nature of the disease (or the prayer). I guess what you are implying is that it cannot be the case that the disease/prayer can be definitively marked as heard/cured at some moment in time (when it is treated/made) but may be later. (I pray for money to keep my aged granny and a year later she wins the pools. I suppose Go has quite a long "To Do List".) I do not know if there are diseases like this though. The bird ringing comes in because you'd need, at later points, to be able to tell (surveillance) to what extent people "found cured" had previously been ringed. (Maybe depression is an example, You get better before you get well and saying when you were fully well again is pretty hard and often retrospective.) If the quote is really about inscrutability (we will never know if x is the answer to a prayer) then I think it is untestable as stated. Or this could be complete bosh and not what you mean at all! Of course you do not really need bird rings, unambiguous identifiers would do fine (National Insurance Numbers?) but disease trials are usually heavily anonymised so this would go against the culture. You can tell I am thinking as I am writing but this might be an interesting question for educators. At the time the students may hate your lecture. 20 years on they may wish they had listened more carefully.
  • asked a question related to Survey Design
Question
6 answers
Hello. I want to measure citizens' "perceptions of energy insecurity". Therefore I need to ask them several questions about different dimensions of this concept and then construct an index. The questions I have asked my respondents in the pre-test stage of my survey do not seem to correlate much, which is detrimental for the reliability of my index. I have a very low Cronbach's alpha coefficient. Can someone recommend me (a) some paper(s) which have used similiar approach in the survey design? Thank you. 
Relevant answer
Answer
Lala, you might even post preliminary questions here to get some feedback. In this particular case you need to know how scientific they are to cover various important components and meanings on energy (in)security. They also should be understandable for ordinary people (this is always true for surveys) and in the right sense.
Perhaps you should start from classification of components of energy security based on scientific literature. Then you should figure out typical perception of energy security in press. Sometimes public opinion may be non-scientific but quite persistent. It is good if your survey will allow to study both issues. But in any case you should start from theory.
  • asked a question related to Survey Design
Question
4 answers
Hi fellows,
I am working on a design of survey about job loss and EI. I intend to use the already available scales for Risk propensity by Meertens and Lion’s (2008) and "Fear of failure" with 5-item scale developed by Conroy, Willow, and Metzler (2002).
In social settings of Pakistan, India etc. I feel that instead of 7 point scale, its better to use 5-point scales to make things simpler and easier for the respondents. What is your experience and opinion about it?
Secondly I guess we need to use consistent scales for both like if one measurement uses 5-point scale the other should also be 5-points scale. Need your experience and opinion about that also.
Thanks.
Relevant answer
Answer
Hi Shahzad Hanif,
to use five or seven points Likert scale, Hair et al. (2011) recommend to observe the formal eductaion levels of the respondents. If you have to much respondents with few years in schools, you have to use five points (for childrens and olders peoples - use three points) and if you have to much repondents with many years in formal school, you shoud use seven points (for universitary people use eleven points - from zero to ten scores, like the school system of notes).
How much more points do you use, also more variability do you have, but more levels or points can to confuse the respondents. More variability in the answers are desired becose we increase the precision of statistics tests.
Best regards
Dirceu
Hair Jr, J.F., Wolfinbarger, M., Money, A.H., Samouel, S., & Page, M.J. (2011). Essentials of Business Research Methods. 2 ed. edition. New York: Routledge.
  • asked a question related to Survey Design
Question
11 answers
1. What are African American male’s perception of mental illness?
2. How much does stigmas, associated with mental illnesses impact African American men understanding of mental illness?
There is a need for additional studies to identify the perceptions of mental illness within the black population, specifically with African American men. The researcher will use qualitative research techniques to explore the African American male participants' perceptions of mental illness and mental health services. The focus of this study will examine 25 African American men, in Tallahassee, Florida, ranging between the ages of 18 and above. The selection of the specific geographical area is a convenience sample.
The purpose of this study is to explore African American adult men perceptions of mental illness. The researcher will explore stigmas and misconceptions of mental illness among the participants. Additionally, the researcher will attempt to identify if such stigmas and misconceptions influence the participants willingness to seek mental health services. The use of qualitative research is to identify any patterns and gain general knowledge of the phenomen, based on participant’s opinions.
Relevant answer
Answer
Inn general, case studies would treat each individual as a case and make comparisons across those case. I suggest you look at Robert Yin's work as a guide.
In contrast, phenomenology would look at the in-depth, lived experience of each individual. There are any number of divisions within phenomenology, ranging from those that are heavily influenced by philosophers to those that are similar to content analysis (e.g., interpretive phenomenological analysis).
If you are not already familiar with the more philosophical versions of phenomenology, it would probably take considerable effort to master one of them.
Also, there are any number of other approaches to collecting and analyzing qualitative data, so you might say something more about why you have chosen these two.
  • asked a question related to Survey Design
Question
13 answers
Though there are a lot of significant discussion about this issue but today I am confused when I am putting my data into data sheet from the questionnaire. 
I have variable name social capital and in my research, I have taken six dimensions of social capital (*e.g. trust, social cohesion etc.) and in every dimension, I have several questions. Data were collected by using 5 points scale (strongly agree to strongly disagree) and in the questions, I have both positive and negative questions equally (e.g. I have trust and I do not have trust). 
Now, I am confused. In the case of positive statement - should I have to give 5 for strongly agree and for negative statement should I have to give 1 for strongly agree? 
Thank you in advance for your anticipated cooperation. 
Relevant answer
Answer
You need to reverse the items if you are going to calculate Cronbach's alphas, which assumes all the correlations are positive. Beyond that, it would be wise to run at least an Exploratory Factor Analysis, and even better, a Confirmatory Factor Analysis.
  • asked a question related to Survey Design
Question
9 answers
I am an MA student in Dance Practises and I have to produce a research proposal poster for my final dissertation, I am going to 5 dance companies who teach disabled children to find out the methods they use and why (Physical guidance, tactile etc) I have looked at Social constructivist theory and using qualitative research methods, but would I use a mixed methods approach to analyse my information or stick with qualitative? Sorry its all very new to me!
Relevant answer
Answer
Hi, Julie. I recommend you do a content analysis - thematic analysis. The categorization process can be based on previously defined concepts (theory),  or you can build the category grid from the answers obtained in the questionnaires and interviews (empirical procedure)
  • asked a question related to Survey Design
Question
5 answers
I am struggling to install the software suggested by the FAO to analyse Food Insecurity Experience Scale and use
Relevant answer
Answer
You May Use Chi-Square to measure the association between the respective variables, also you may try using DEA approach for classification base on DMUs
  • asked a question related to Survey Design
Question
11 answers
I need to carry out some test on Spss but i am a bit confused what item from my questionnaire scale or variable items is most appropriate to add to my dependent variables? I have 17 items on a particular variable and they all likert scale items, but i am not sure if i can select  any item to carry out the test or use the whole items to carry out the test? Or is it alright if  only Cronbach alpha is use to test for the reliability of the items and then can use any of the variable items to carry out the test? 
Relevant answer
Answer
Cronbach's alpha assumes that all the items measure a single underlying concept, but it does not assess that assumption. That is why Peter Samuels recommends Exploratory Factor Analysis, because EFA shows how many separate scales may be necessary.
Because questions on this topic come up often here, I have collected a set of resources on the topic:
  • asked a question related to Survey Design
Question
3 answers
I am conducting a baseline audit of a large organizations that provides diverse services to a wide range of clients (housing, employment, etc.). We would like to capture participants perceptions and experiences with services prior to implementing a system wide Trauma Informed Practice intervention. Many of the clients may have low English literacy, significant trauma histories and some have cognitive challenges. I am interested in some quality resources for helping me design the survey instruments for these populations. 
Relevant answer
Answer
Based on the the sample characteristics of being illetrate, I would suggest to use the interview with well trained data collectors.
Then do pilot on similar population to figure out possible pitfalls.
Good luck
  • asked a question related to Survey Design
Question
5 answers
Am using the mix method approach, I just want to know if it is Necessary to include an observation sheet to an ethical clearance since My last email to my supervisor wasn’t attended to.
Relevant answer
Answer
Yes I agree - you need to be transparent in the tools you are using so that the ethical procedures can be assessed by an ethics committee and so that the participants are fully informed when they give their consent to be involved in the research.
  • asked a question related to Survey Design
Question
2 answers
We're looking for it several times and still there's no result related to it. We will be needing it for our research paper. Thank you.
Relevant answer
Answer
Did you see this? The link is more of what you seem to be looking for
Looks like things  will change in 2018:
Journal List
World Psychiatry
v.15(3); 2016 Oct
PMC5032513
Logo of worldpsych
World Psychiatry. 2016 Oct; 15(3): 291–292.
Published online 2016 Sep 22. doi: 10.1002/wps.20353
PMCID: PMC5032513
Bodily distress disorder in ICD‐11: problems and prospects
Oye Gureje 1 and Geoffrey M. Reed 2
Classifying the disorders associated with burdensome somatic concerns has been a challenging exercise in psychiatric nosology1. The classifications of these conditions in ICD‐10 and DSM‐IV have not fared much better than earlier attempts2. Even though not exactly identical, these classifications were broadly similar and criticisms of either system are therefore generally applicable to both. Among the most salient criticisms are those relating to their utility in routine clinical practice. These include the rarity of the major categories of the group, both in the community and in general clinical practice, as well as the evidence suggesting poor diagnostic reliability3.
A central feature of the definition of these disorders, that the symptoms are not due to physical or medical causes, has been criticized for being unreliable and for posing a fundamental nosological problem: defining a disorder on the basis of the absence of a feature rather than the presence of a problem4. Labels assigned to burdensome somatic preoccupations that have come to be seen as pejorative create another problem for clinical utility. Some patients object to the term “somatoform”, which they think may imply that their symptoms are of doubtful clinical importance and are “in their heads” or not real. Furthermore, the notion that the symptoms are medically unexplained is often rejected by patients as essentially an issue of detection.
As part of the activities designed to lead to the approval of ICD‐11 by the World Health Assembly in 2018, the World Health Organization, through its International Advisory Group5, constituted the Somatic Distress and Dissociative Disorders Working Group, which, among other tasks, was asked to propose changes to the section on somatoform disorders in ICD‐10. The Working Group has proposed a new and much simplified category of bodily distress disorder, which replaces all of ICD‐10 categories within the group of somatoform disorders (F45.0) and, to a large extent, neurasthenia (F48.0), bringing these together under a single category. The only ICD‐10 somatoform condition excluded from BDD is hypochondriasis (F45.2).
In the proposed new classification, bodily distress disorder is defined as “characterized by the presence of bodily symptoms that are distressing to the individual and excessive attention directed toward the symptoms, which may be manifest by repeated contact with health care providers. If a medical condition is causing or contributing to the symptoms, the degree of attention is clearly excessive in relation to its nature and progression. Excessive attention is not alleviated by appropriate clinical examination and investigations and appropriate reassurance. Bodily symptoms and associated distress are persistent, being present on most days for at least several months, and are associated with significant impairment in personal, family, social, educational, occupational or other important areas of functioning. Typically, the disorder involves multiple bodily symptoms that may vary over time. Occasionally there is a single symptom – usually pain or fatigue – that is associated with the other features of the disorder” (this is the proposed brief definition for bodily distress disorder; for the format of ICD‐11 diagnostic guidelines, see First et al6).
Responding to the same set of criticisms, the DSM‐5 created a new grouping called Somatic Symptom and Related Disorders, in which the prototypic condition is somatic symptom disorder. Even though this diagnosis can be given to a condition with “one or more somatic symptoms”, it nevertheless requires that “excessive thoughts, feelings, or behaviors are related to the somatic symptoms or associated health concerns”. Specifically, for a diagnosis of somatic symptom disorder, at least one of three psychological criteria should be present: health anxiety, disproportionate and persistent concerns about the medical seriousness of the symptoms, and excessive time and energy devoted to the symptoms or health concerns.
In both the proposed bodily distress disorder and somatic symptom disorder, the most fundamental revision has been the abolition of the distinction between medically explained and medically unexplained somatic complaints. On the other hand, there are now specific psychological criteria that need to be fulfilled before the diagnosis can be given. The revised classifications thus address the problem of defining somatoform disorders on the basis of the absence of a feature (a physical or medical cause) by specifying the features that must be present, such as distress and excessive thoughts and behaviors7.
Dropping the criterion of “medically unexplained” is not without its consequences and has been criticized in somatic symptom disorder. It has been argued that patients with medical conditions and with a justifiable reason for somatic complaints may receive an inappropriate psychiatric diagnosis, with the possibility of associated stigma8. The specification in bodily distress disorder that “if a medical condition is causing or contributing to the symptoms, the degree of attention is clearly excessive in relation to its nature and progression” is meant to address this concern.
A single somatic symptom may lead to a diagnosis of bodily distress disorder or somatic symptom disorder. A good justification for this revision is that a single symptom, for example pain, may sometimes be as bothersome as multiple somatic symptoms. However, the point has been made that this lowering of the threshold for the diagnosis may lead to an inappropriate labeling of apparently healthy persons as having a psychological disorder8. This concern is addressed in bodily distress disorder by the requirement that other features, in particular associated psychological features, as well as significant functional impairment, be present before the diagnosis is given. Also, further information is provided in the proposed diagnostic guidelines that seeks to delineate mild bodily distress disorder from normal somatic concerns which may exist in the community and do not require clinical attention.
One of the important differences between the proposed ICD‐11 and the DSM‐5 approaches is the name of the prototype disorder. While the DSM‐5 has retained the word “somatic”, the proposed ICD category has avoided this term altogether. While no label can prevent completely the risk of negative connotations and misinterpretations, a more descriptive label that avoids the term “somatic” might prove more acceptable to both patients and primary care clinicians.
While the DSM‐5 has retained hypochondriasis (or health anxiety) within the cluster of Somatic Symptom and Related Disorders, the current proposal for ICD‐11 has placed hypochondriasis within the grouping of Obsessive‐Compulsive and Related Disorders. The position of DSM‐5 is supported by evidence suggesting a high co‐occurrence of hypochondriasis with somatization disorder as well as shared cognitive perceptual styles between the two conditions. On the other hand, the position of the ICD‐11 Working Group is supported by findings associating repetitive cognition and behaviors as well as task‐related neural activation patterns on brain imaging with hypochondriasis1. Also, there is evidence that, unlike somatization disorders, hypochondriasis responds to some treatments used for obsessive‐compulsive and related disorders9.
The new proposals for bodily distress disorder are being systematically tested in the field studies conducted as part of the ICD revision process. These studies include Internet‐based approaches, in which a large number of clinicians participate through the Global Clinical Practice Network (http://gcp.network), as well field studies conducted in clinical settings. It is hoped that the findings from the field studies will provide opportunities for a further strengthening of the utility and validity of the classification of burdensome somatic concerns in ICD‐11 prior to its approval by the World Health Assembly.
Oye Gureje1, Geoffrey M. Reed2
1WHO Collaborating Centre for Research and Training in Mental Health, Neuroscience and Substance Abuse, Department of Psychiatry, University of Ibadan, Ibadan, Nigeria; 2World Health Organization, Geneva, Switzerland and School of Psychology, Universidad Nacional Autónoma de México, Mexico City, Mexico
O. Gureje is Chair of the WHO Working Group on Somatic Distress and Dissociative Disorders. G.M. Reed is a member of the WHO Secretariat, Department of Mental Health and Substance Abuse. The views expressed herein are those of the authors and, except as specifically indicated, do not represent the official positions of the WHO Working Group, the International Advisory Group for the Revision of ICD‐10 Mental and Behavioural Disorders or the WHO.
Go to:
References
1. Gureje O. Curr Opin Psychiatry 2015;5:345‐9. [PubMed]
2. Mayou R, Kirmayer LJ, Simon G et al. Am J Psychiatry 2005;162:847‐55. [PubMed]
3. Creed F, Gureje O. Int Rev Psychiatry 2012;24:556‐67. [PubMed]
4. Voigt K, Nagel A, Meyer B et al. J Psychosom Res 2010;68:403‐14. [PubMed]
5. International Advisory Group for the Revision of ICD‐10 Mental and Behavioural Disorders . World Psychiatry 2011;10:86‐92. [PubMed]
6. First MB, Reed GM, Saxena S et al. World Psychiatry 2015;14:82‐90. [PubMed]
7. Rief W, Martin A. Annu Rev Clin Psychol 2014;10:339‐67. [PubMed]
8. Frances A. J Nerv Ment Dis 2013;201:530‐1. [PubMed]
9. Greeven A, Van Balkom AJ, Visser S et al. Am J Psychiatry 2007;164:91‐9. [PubMed]
Articles from World Psychiatry are provided here courtesy of The World Psychiatric Association
  • asked a question related to Survey Design
Question
3 answers
I am interested in whether using AI biases the direction of an evaluation survey and, if so, whether this is a limitation of the methodology in terms of rigour.
Relevant answer
Answer
  • asked a question related to Survey Design
Question
2 answers
I have generally seen the scale used as a continuous scale. One recommendation for cut points (high hope, etc.) would be to use one or two standard deviations above or below mean, although this would be dependent on the characteristics of those who take the scale.
Substantively I am attempting to show change in the hope of individuals with serous mental illness at the time of onset of involvement in mental health services over time. I can use simple change scores but in presentation it would be clearer to be able to say that the individuals moved from low hope to ...
Many thanks
Relevant answer
Answer
I would stick with the continuous scale for analysis and modeling because of power issues and then turn the modeled results into caseness at the end for 'clinical' interpretation which you may get through conventions in the literature. And you should know the context of application better that anyone else - trust yourself on this.
  • asked a question related to Survey Design
Question
6 answers
I am conducting a study on autistic traits in typically developed subjects and I am wondering what would be a better approach to examine neural differences in this population:
1. looking at the scores (the AQ questionnaire for autistic traits) as a continuum
2. looking at the two extreme quartiles
Does anyone have a pro\cons suggestions for each option?
Can you please recommend a relevant article that deals with the topic of working with the extreme quartiles as opposed to the continuum?
Thank you all for helping,
Leehe.
Relevant answer
Answer
Are you using the AQ to screen subjects to choose which to use in the imagining phrase of the study?
btw, for the distribution of AQ for women and men, see end of the following paper https://www.researchgate.net/publication/221882649_Measuring_Empathizing_and_Systemizing_with_a_Large_US_Sample?ev=prf_pub
  • asked a question related to Survey Design
Question
4 answers
I’m doing an investigate of blood glucose level to see the pre-diabetes progression. I use a form that consist of demographic characteristics related to that disease. I’m really looking suggestions in order to make all data gathered valid.
Relevant answer
Answer
Usually, assessment of validity and reliability are only required for attitudinal or behavioural questions. We can assume that demographic items have face validity and are likely to be consistently answered.
  • asked a question related to Survey Design
Question
2 answers
Hi! My colleagues and I have to use a good scale of stressors, but we cannot find any precise information if the mentioned scales are publicly available or not. While some say SRRS (1967) can be used for professional purposes, others claim they asked for a permission. Also, we tried to contact the authors (specifically, authors of LEI) on mail addresses they provided in the article, but no one responded for a few weeks. Does anyone have any reference from which it could be clearly concluded that (at least one of) these scales can be used for research purposes? Or at least an idea who (else) to contact to get the permission? We wouldn't like to send a new round of mails and wait for another few weeks to get no response, since timing of our project is a bit limited.
Relevant answer
  • asked a question related to Survey Design
Question
6 answers
I am combining different scales in my questionnaire. These scales have been tested for their validation and their reliability. 1st. Do I still need to examine discriminant validity? 2nd. The questionnaire is long and I need to eliminate items, is it possible to do that based on my experience and the review of other experts in the field? Thanks
Relevant answer
Answer
Deleting items from a scale can affect its reliability and validity, so you would need to reassess those standards on the revised scale, once you collect it. Of course, that can be risky.
As for discriminant validity, if you are worried that some of your scales essentially measure the same thing, you might be better off shortening your questionnaires by dropping one of those overlapping measures.
  • asked a question related to Survey Design
Question
3 answers
I am intending to use a scaled survey from 1 to 6 (Not Probably to Definitely) as responses to the seven learning styles that my respondents would think a hand hygiene program approach should consider for it to become effective?
Though, there are several 16-20 item learning style questionnaires existing such as the VARK and LSQ, I would just want to make it as a short and simple survey. Could it be exempted in any condition?
Relevant answer
Answer
Dear Alixes 
Every scale has been constructed and administered on their own cultural settings, therefore, the way of receiving and processing the information is different from the other learners. I think the best way is  is check again the reliability and validity of the scale which you want to use. 
Mehraj 
  • asked a question related to Survey Design
Question
4 answers
whether any modification is needed?
Relevant answer
Answer
You included in the attached file "Observation checklist for Intramuscular Injection administration" 
this procedure is already validated and approved in its steps and technique in all the fundamental textbooks
  • asked a question related to Survey Design
Question
3 answers
There are no existing resources for my dissertation topic. I am studying the ability of an existing fellowship certification process to indicate professional capability and efficacy for healthcare organizations and administrators.
Relevant answer
Answer
Thank you both!
  • asked a question related to Survey Design
Question
4 answers
Hello everybody,
I'm working on a study about patients' perspectives on being treated by a resident. In my literature search I found a lot of studies with a cross-sectional survey design. I really want to check the quality (risk of bias) for these studies, however, I cannot find a good appraisal tool. Does anybody have any recommandaties for a appropriatie tool?
Relevant answer
Answer
Dear Sarah
                     As per the construct and context of your study,  you may design your  own appraisal tool.  To reduce the bias in the study, reliability as well as  validity of measuring tools to be tested properly before their actual use. The bias can also be reduced by increasing the sample size. proper selection of the respondents and creating an appropriate environment while interacting with them for eliciting the correct responses. 
  • asked a question related to Survey Design
Question
4 answers
Folks, I have used Qualtrics panels - not sure if you are familiar with this, Qualtrics have panels and charge a certain amount per survey depending on how scarce the panel respondents are - for example CEOs may be $50 a survey, whereas a middle manager might be $8-10 depending on criterion. It is a little difficult to get response rate data in this case. 
Another method some of my colleagues are using is  Amazon Mechanical Turk.  I have not tried this and am not familiar with the quality. It is definitely a less expensive option - I have heard it could be $1 per survey response. 
What are your views and experience on some good ways to get  quality survey data?
Relevant answer
Answer
Definitely i do recommend Qualtrics over MTurk for academic research;
Qualtrics will provide a dedicated project manager to help conduct, manage the survey access, distribute questionnaires and collect data from diverse demographic of the targeted sample, which will increase the quality of the collected data.
In contrast, MTurk is an academic research method being done online. However; this method may be leading to flawed or unreliable data.
 Dr. Kelly check the following video:
  • asked a question related to Survey Design
Question
4 answers
We are developing and validating a questionnaire regarding chronic disease and disaster preparedness. The questions are all categorical in nature ie: 
1) Which of the following items do you have as part of your household’s disaster preparedness? 
Check all that apply
 Water, two liters of water per person per day for at least three days
 Food, at minimum a three-day supply of non-perishable food
 First-aid kit
 Working flashlight
2) Do you require supplemental oxygen?
Mark only one box
If no go to question 5
 Yes
 No
3) Do you have 72 hours of supplemental oxygen cylinders (compressed gas, not liquid)?
Mark only one box
 Yes
 No
I am struggling to find the best way to examine test-retest. I think I should be using Kappa Stats but I'm a bit confused on it and there is little literature on the subject matter. 
Relevant answer
Answer
Hi Maddy
Steven is right in that the correct method of assessing test-retest reliability for categorical variables is kappa for Yes/No and nominal variables, and weighted kappa for ordinal variables such as Likert scales. You would like kappa to be at least 0.6 for each question.
Adrian 
  • asked a question related to Survey Design
Question
6 answers
I'm looking for two things basically:
1) Syllabi for sampling or survey data analysis courses you've taken or tought. Good and bad examples both wanted.
2) If you've take a course on this, we'd love your frank thoughts (again positive or negative) even if you can't share the syllabus. Direct message is fine if you don't feel comfortable commenting "out loud."
Colleague Stas Kolenikov and I are up to something.
Just so you don't think I'm doing my own work, this isn't for a class I need to prepare. :)
Thanks in advance! :) We can continue the conversation here. 
Relevant answer
Answer
I thought sampling was taught in a way that was not particularly helpful. If you could come up with a way to present it in such a manner that it was more intuitive than the standard presentation a la Kish (which is thorough but pretty dry), that would be great --
  • asked a question related to Survey Design
Question
9 answers
I'm looking for something like the O'Brien article in Academic Medicine 2014 (Standards for Reporting Qualitative Research:A Synthesis of Recommendations) or CONSORT or something similar?
Relevant answer
Answer
Try looking at a survey research text.  Many will have a chapter on reporting results.  You might also look at the author guidelines for Public Opinion Research which is the journal for the American Association for Public Opinion Research (AAPOR) or the AAPOR website.
  • asked a question related to Survey Design
Question
5 answers
I am starting my dissertation related to spiritual well being and compassion fatigue, and I plan to use three different measurement surveys, such as the Professional Quality of Life (ProQOL) measure. What is the best way or ways to make comparisons, statistically among the three scales?
Relevant answer
Answer
Hello Alan,
I am responding primarily to your most recent question:
It all depends on what you mean by "mixed methods".  By these words, many people assume that the distinction between methods refers to what are traditionally referred to as qualitative methods versus quantitative methods.
Given that your original question indicated that you were interested in analysing your data statistically, that essentially rules out qualitative methods.
In essence, if you use surveys / instruments that yield only quantitative data (numbers), as does the ProQOL, you have ruled out using mixed methods in the traditional sense. (i.e., you have ruled out qualitative methods.) However, it depends on what you mean by "methods".  There are different methods within the quantitative bundle.  Up above, Peter has referred to correlations.  They are related to other things such as regression analyses.  But there are other quantitative "methods" such as tests that assess differences. These include t-tests and ANOVAs. 
Much depends on what your surveys are like in terms of the methods you can use with them. Perhaps some of your surveys yield data (usually verbal responses) that permit qualitative analyses. However, even verbal data can be analysed in ways that are primarily quantitative - and that therefore permit subsequent statistical analyses.
Apart from that, ultimately, a lot depends on what your research questions are.  For example, are you wanting to make comparisons between the scales, or are you wanting to make comparisons between groups within your research based on their results on each of the scales?  These are slightly different focuses.
Robert.
  • asked a question related to Survey Design
Question
1 answer
Urban based population exposed to messages for 3 years.
Relevant answer
Answer
Yes wherever there is data, there is an insight . If the insights that you have collated over three years for similar messages then it is quite possible that you are using quasi-experimentation approach. This would, however imply that you have controlled the variable  (communication message) and have measured other related  variables in terms of age, education and other criteria (which are defined and measured) more than once in your study. Is communication campaign the cause and behavior change the effect?