Science topic

Online Surveys - Science topic

Explore the latest questions and answers in Online Surveys, and find Online Surveys experts.
Questions related to Online Surveys
  • asked a question related to Online Surveys
Question
8 answers
Dear researchers,
Recently i have conducted a online survey work through google form.I want to know what could be it's methodology?
Help me suggesting any good articles where authors used/mentioned this types of survey process.
Thanks in advance!
Relevant answer
Answer
I agree with the previous thoughtful responses. I want to stress that as an alternative questionnaire instrument you may prefer the "Qualtrics.com" over Google form (It requires a subscription though). I found it is easy to get a better response since this platform provides a wider possibility of presenting your survey questions. Non-response rate might affect this kind of survey very much.
About the methodology, in addition to other ideal procedures in any other research, please consider some points:
-Develop your research question wisely based on a review of existing literature in line with the question of your interest
-If there is a control group, the characteristics should be commonly shared between the treatment and control groups.
-Close-ended questions are very helpful to reduce data management efforts after completing the data collection part.
-The selection of the study population needs to be validated in a systematic way for example who you choose to participate in the research. Questionnaires should be randomly distributed over any network.
I hope it helps.
  • asked a question related to Online Surveys
Question
3 answers
Hello!
I have found my sample size based on the population of the city by using online sample size calculators like Survey Monkey. However, the number I get is about 200-300 and I am wondering is this the number of surveys I should distribute, or should I multiply this number by a number (if yes, what number?) or something to account for non-respondant?
I have heard a 5-10% response rate is common, and 10% of 300 is not much... so I am a bit confused about how to proceed.
I appreciate any help!
Relevant answer
Answer
Hi,
a sample size of 200-300 persons answering your survey seems to be ok if the city is not that big. Depending on how differentiated your analysis is intended to be (e.g. for living areas, districts and so on) you might want to have more participants.
A response rate of 5-10% is reasonable. So, you would have to contact about 3000 persons. If you want to do a online survey, this can be done by advertising on social media or by using some other way of dissemination, like e-mail lists.
If you do it by regular mail, response rate is also quite low.
If you want to do face to face, these might be less persons to be contacted, because the response rate is higher.
Best
R.
  • asked a question related to Online Surveys
Question
7 answers
Our lab is conducting a research project about Asian-American families. We are investigating Asian-American families' well-being during the COVID-19 pandemic, specifically, their experience of racial discrimination, their parent-child relationships, and their children's development. So far we manage to contact churches, organizations, and school communities, but we still need more Philippine and Korean teenagers (12-18 years old) and parents to participate. Any suggestions to help us recruit? I appreciate your help.
Relevant answer
Answer
One way is to go through the schools. A common approach is to reach out to the superintendent of the district with your IRB approval, summary of study, and that you are seeking support/permission. Once granted, you would then reach out to individual principals of schools in those districts to get their support. At that point, you provide the students at the school where you have support an informed consent and permission letter to take home for parents. If the parent consents, you may even schedule a call to address any questions or concerns they might have. Once you have consent, you then can proceed. Good luck.
  • asked a question related to Online Surveys
Question
9 answers
I am looking for online survey tools with the ability to upload a photo file and send your exact, present location (in terms of longitude and latitude).
The tool should work perfectly on mobile devices, and preferably be free of charge.
The online survey will be completed on mobiles while being outdoor, so sending location need to be easy and user friendly. I was considering just using LimeSurvey and ask participants to copy - paste their location url from Google Maps app, but it is inconvenient and inaccurate.
Thank you!
Relevant answer
Answer
You can use KoboToolbox. The platform is free and easy to use, suitable for mobile devices and enables to upload photos, GPS locations, etc.
  • asked a question related to Online Surveys
Question
14 answers
I'm searching for an online platform that, on top of setting up and collecting survey data for participants, would also allow automatically generate individual feedback profile unique for each participant based on the questionnaire they had just filled in. Such as, for example, pie charts generated with various online personality questionnaires.
If anyone knows anything, the advice would be much apprecited!
Kind regards,
Ugne
Relevant answer
Answer
Qualtrics can do this.
  • asked a question related to Online Surveys
Question
8 answers
Many scientists suggest that a good way to analyze the level of innovation in action, generate innovation in financial institutions, eg in banks, is conducting surveys among managers and department directors, departments in these institutions.
How should such surveys be carried out? What method of surveys is the most effective? Do online questionnaire forms are an effective instrument for carrying out surveys?
What other research techniques can be used to investigate the level of innovation in operation, generate innovation in financial institutions?
Please reply
Best wishes
Relevant answer
Answer
Dear Mohammed Jaafar Ali Alatabe,
Thanks and invite you to joint discussions on the subject of innovation in financial institutions.
Regards,
Dariusz Prokopowicz
  • asked a question related to Online Surveys
Question
4 answers
Dear community,
We are working with attention check items in several questionnaires to improve the data quality (compare ). We are also measuring several constructs (such as embodiment, immersion, presence, motivation, self-efficacy etc.) which established questionnaires recommend measuring with many items (sometimes >30). This length makes it infeasible given participants' limited attention and time. Thus, we have shortened some scales. I would like to justify why this is acceptable given the higher data quality due to the attention check items. Unfortunately, I could not identify any literature that indicates this. Are you aware of anything in this direction? Please also feel free to point out any literature regarding shortening scales or the trade-off of long scales and time.
Thank you!
Relevant answer
Answer
This paper and its reference list should help:
Marsh, H. W., Martin, A. J., & Jackson, S. (2010). Introducing a short version of the physical self description questionnaire: new strategies, short-form evaluative criteria, and applications of factor analyses. Journal of Sport and Exercise Psychology, 32(4), 438-482.
  • asked a question related to Online Surveys
Question
6 answers
Dear RG Colleagues,
Microsoft Forms is an online survey software which is part of Office 365. Has anyone seen an objective academic review of it? In particular, do you know whether it is secure for use with personal data? I am looking for something along the lines of criteria used in this review:
Farmer, R., Oakman, P. and Rice, P. (2016). A review of free online survey tools for undergraduate students. MSOR Connections, 15(1), pp. 71-78. Available at: https://journals.gre.ac.uk/index.php/msor/article/view/311.
Thanks for your help.
  • asked a question related to Online Surveys
Question
21 answers
Hi there,
Can anyone recommend a Delphi method online tool that I can use? 
Mesydel.com has been recommended but you are not able to download it. 
I have been recommended http://armstrong.wharton.upenn.edu/delphi2/ as well although the website is less user friendly than I would like.
Any other suggestions?
Thanks.
L
Relevant answer
Answer
We have recently launched a course on Delphi method, checkout at https://researchhub.org/course/delphi-method/
  • asked a question related to Online Surveys
Question
9 answers
Colletion of Online Survey
Relevant answer
Answer
It depends what you plan to do with your data. Are you just wanting to report descriptives? Also, are you conducting the survey using software? For some needs that software may suffice. Also (2), what software do you have available (or are you happy buying additional software)? Also (3), what are you and your colleagues most comfortable using?
  • asked a question related to Online Surveys
Question
21 answers
I am currently conducting research and using a survey which contains 2 parts, 10 closed ended (Yes/No, multiple choice) questions and 5 open ended (opinions, follow on comments). Not all participants are required to complete the 5 open ended questions. I am aware that I am only using one research method so would assume this is primarily classed as quantitative research. However, after some reading, if my analyse on both sets of data collected are analysed separately (as quantitative data and qualitative) then combined/ compared, would this be classed as a mixed method?
Relevant answer
Answer
Robert Trevethan I disagree with you statement "that appending some open-ended questions to a set of fixed-choice questions on a survey in no way constitutes mixed-methods research."
In particular, if the qualitative content can be integrated with the quantitative content in ways that add to the overall understanding of the research question, then that is precisely the goal of mixed methods research. As I mentioned above, this is quite possible by using a QUAN --> qual explanatory sequential design in a survey that uses open-ended questions.
Ultimately, what matters most for mixed mentors is the purpose that the two types of data serve within the overall design, not when or how they were collected.
  • asked a question related to Online Surveys
Question
6 answers
My PhD study is quantitative in nature on emotional abuse in the workplace in the public sector. Due to covid19 I am doing online survey. My worry is what can i do to encourage participants to participate, i am worried about low return percentage (%) for online survey
Relevant answer
Answer
Generally, people dislike spending over ten minutes responding to a survey. Therefore, as a good rule of thumb, you should keep your online survey precise and concise, avoiding language-related or discipline-related complexities. Here are two recent studies that could provide you with more insights.
Ball, H. L. (2019). Conducting online surveys. Journal of Human Lactation, 35(3), 413–417. https://doi.org/10.1177/0890334419848734
Brosnan, K., Kemperman, A., & Dolnicar, S. (2019). Maximizing participation from online survey panel members. International Journal of Market Research, 147078531988070. https://doi.org/10.1177/1470785319880704
Good luck,
  • asked a question related to Online Surveys
Question
6 answers
We want to continue the study of what drives people to play video games.
This new version of the study has been approved by the Instituto Superior Miguel Torga ethics committee on March 31st, 2021
Participant requirements
  • Age > 18
  • Video gamers (people that play any game available for computer, PDA, mobile phones, TV consoles, or portable devices)
Online survey / Taking part link
Instructions
The questionnaire takes about 15 minutes to complete, with no right or wrong answers. In no case will there be any sharing of the specific information understood under professional confidentiality. Participation in this study is voluntary, and you can withdraw at any time by closing your browser window.
Number of Participants
300
Relevant answer
Answer
Done
  • asked a question related to Online Surveys
Question
3 answers
Good morning everyone,
Due to the pandemic, I conducted several online surveys. It is quite agile, not at all expensive, but there are several problems: the sample's representativeness and its bias, the difficulty of accessing social profiles that do not have social networks and a certain distrust. What other problems would you highlight? What solutions would you recommend? Are the results - fundamentally the search for associations between variables - reliable given a certain level (n> 50) of observations for each variable? Anyway, what are your opinions about this tool and how can we improve the research design to incorporate online surveys in an appropriate way? Thank you very much
Relevant answer
Answer
Sample "coverage" is probably the main issue in online surveys, and journals such as Public Opinion Research have published articles on this topic for the past decade or more. One prominent option is "mixed mode" surveys where you collect data both online and by phone.
As for the association between variables, this kind of statistical issue is the same regardless of how the data are collected. Rather than informal criteria such as 50 observations for each variable, you should use a program such as G*Power to determine the necessary "power" of your test.
  • asked a question related to Online Surveys
Question
7 answers
Hi!
I need to perform an online experiment where short videos are presented and time response can be saved.
Any suggestions?
Relevant answer
Answer
  • asked a question related to Online Surveys
Question
2 answers
I'm wondering if anyone has a sample of online AHP surveys that can share with me? Since due to the COVID-19 lock-down it is not possible to physically access people, so are there any best online survey portals for researchers to collect data online that can support AHP kinds of questions as it's different from normal kind of survey.
Relevant answer
Answer
Dear, araneh Sahraiyan,
Can you see the following table See annex 1 pp. 141-142
  • asked a question related to Online Surveys
Question
8 answers
We are now working on the online survey and we want randomly assign to the participants one of three task conditions. Is there any tool to make this for free or almost for free? Thank you for all the suggestions! 
Relevant answer
Answer
I tested a few online survey platforms today, and most were terrible for this task. Many platforms use the language “A/B split testing” to refer to assignment of separate streams of stimuli (that detail is for Auneeb Hussain). I found two acceptable options:
https://freeonlinesurveys.com seems far better than many alternatives. You can randomize text, images, or videos, and doing so is very easy. A major weakness is that you cannot randomly display an entire question from a question bank. Nor can you randomly display a video with some text underneath (from a set of possible videos with paired text).
Soscisurvey is powerful and provides its services for free to some groups (non-profits, individuals, etc.). However, it is not a good option for casual use (for quick student projects, for instance) because the learning curve is steep and the documentation is difficult to follow. For example, to randomize showing a question, I had to set up a random variable TR01 that is either 1 or 2 and then add PHP code that would display question TC01 if TR01 took on a value of 1 and question TC02 if TR01 was not 1:
if (value('TR01') == 1) {
question('TC01');
} else {
question('TC02');
}
Paid versions of Survey Monkey will randomize images and text (and possibly videos) like freeonlinesurveys.com, but it will also (sort of) randomize questions. Strangely, I could only find an option to randomize the question stem text. This means that if you want respondents to face different multiple choice questions, both questions must have the same answer options. I have no idea why. I guess that’s just the way the monkey likes it. Pretty silly. The free version of Survey Monkey does not support this and does not even let you download survey responses as a spreadsheet, so I would avoid it.
Paid versions of Qualtrics are powerful and can certainly do what you want. However, this is nowhere near “mostly free”.
Other products I tested:
SoGoSurvey: Despite the positive words elsewhere in this thread, I could not find the option for A/B testing. Perhaps, it is completely hidden from non-pro users.
Typeform: Could not find the functionality (although the company frequently implies that the functionality exists)
Zoho Survey: Could not find the functionality
Microsoft Forms: I am almost certain the option exists, but my understanding is that implementation would require even more coding than with Soscisurvey
Google Forms: Same problem as Microsoft forms
Note that there are many other platforms that I did not test.
  • asked a question related to Online Surveys
Question
12 answers
If I want to collect data from the respondents using online survey then what should I need to do to get better responses.
Relevant answer
Answer
The following publications may further help:
  • Andrews, D., Nonnecke, B. and Preece, J. (2003) Conducting Research on the Internet: Online Survey Design, Development and Implementation Guidelines, International Journal of Human-Computer Interaction, 16, 2, pp. 185-210.
  • Bowers, D. K. (1998) FAQs on online research, Marketing Research, 10, 4, pp. 45-47.
  • Fricker Jr., R. D. (2017) Sampling Methods for Online Surveys, in Fielding, N.G., Lee, R.M. and Blank, G. (eds.) The SAGE Handbook of Online Research Methods. 2nd ed. London: SAGE Publications Ltd, pp. 162-183.
  • Nayak, M. S. D. P. and Narayan, K. A. (2019) Strengths and Weakness of Online Surveys, Journal of Humanities and Social Science, 24, 5, pp. 31-38.
  • Nulty, D. D. (2008) The adequacy of response rates to online and paper surveys: what can be done?, Assessment and Evaluation in Higher Education, 33, 3, pp. 301-314.
  • Sue, V. M. and Ritter, L. A. (2007) Conducting Online Surveys. Thousand Oaks, California: Sage Publications, Inc.
  • asked a question related to Online Surveys
Question
8 answers
If we collect data through online survey (i.e. survey monkey, google form & etc.) then what should we called for sampling method? What are the sampling methods can be used for collecting data through online survey?
Relevant answer
Answer
I agree with Dennis Njung'e and Aruditya Jasrotia totally. Convenience sampling would do as well.
  • asked a question related to Online Surveys
Question
3 answers
I plan to conduct an online survey via Survey Monkey. It is a quantitative study that will measure perceived stress and I plan to use purposive sampling. No correlations, just a simple survey research design on a specific sample. I endeavor to use the Statistical Package for Social Sciences (SPSS) I am not too sure what will be the best form of statistical analysis if my target population will be 100? Thank you in advance for your advice.
Relevant answer
Answer
What is the purpose of your study, research question, any hypotheses? Are you testing any model? Your statistical analysis should be the one that will enable you to test your hypothesis. Also, is your target population 100 (and you will be sampling from 100) or your target sample is 100? If your population size is 100 and the sample is lower, depending on a sample size you might face challenges in meeting some of the assumptions for the statistical test.
  • asked a question related to Online Surveys
Question
6 answers
Hi everyone,
I need to plan an online experiment in which each participant should watch a series of screens each containing an image (the screens/images must appear in a randomized order). During the whole series of screens, an audio file should be played, it should begin at the first image and it should end at the end of the session.
I have a Qualtrics account, but I'm not able to implement this kind of procedure. In general, as I build a page with the audio player, the audio won't be playing anymore as soon as the next screen is on. On the contrary, I need the audio to be playing in the background during the whole presentation.
Could I achieve my aim by programming something in Python / Java / Node JS / HTML? Or should I change software? Any suggestions?
thanks in advance for any help
all the best,
Alessandro
Relevant answer
Answer
Our lab at the University of Birmingham have just started using Gorilla.sc to build online experiments. So far, I have found Gorilla.sc really intuitive and easy to use and so I would definitely recommend it to other researchers. I'm not 100% sure whether you will be able to have the file playing in the background for the whole experiment, however, there is an option to have audio files playing in on each trial on Gorilla.sc so you might be able to?
If you do choose to use Gorilla.sc, some of the following webpages could be helpful to you! I referred to some of the following webpages to get an idea of how to build a task successfully:
Video walkthroughs that give a speedy introduction to using Gorilla.sc, some of which I found quite useful: https://gorilla.sc/support
Sample Code: https://gorilla.sc/support/script-samples#altercontent. I found the sample code really useful as there were examples for altering or adding content to the task via script (i.e. forcing participants to be in fullscreen when taking part, changing background text and colour, implementing live scoring etc.).
Hope this helps! Wishing you all the best with your research!
  • asked a question related to Online Surveys
Question
5 answers
I wish to assess the level of stress among a specific group of nurses redeployed to other hospital settings (i.e, research nurses) during the COVID pandemic in my research proposal.
May I ask for your thoughts as to which sampling method is best and may I ask why?
Relevant answer
Answer
Hello Daryl,
It depends on your research question, your population, and your measure(s). If you're talking about a modest number (say, 100) of nurses, it might be best to try to obtain a census sample (all of the cases). If the population is much larger, then the ideal approach would likely be a simple random sample or a stratified random sample. Of the two, I suspect stratification (for example, based on the type of reassignment venue) would likely be far better. Any probability sampling method pretty much requires that you have a way of identifying/locating the members of the target population so as to be able to: (a) make tentative selections; and (b) make the solicitation to participate in your study.
On the other hand, if your research aim is more along the lines of a qualitative inquiry, then perhaps a more purposive approach (and not a probability sample) would be better for you, as suggested by Dean Whitehead .
Good luck with your work.
  • asked a question related to Online Surveys
Question
6 answers
I'm conducting an anonymous quantitative survey on SurveyMonkey and some participants left the majority of the questions unanswered. How can we encourage participants to completely answer all questions in the survey?
Relevant answer
Answer
Its a tricky question. May be some incentive!
Did you do pilot testing before running the main survey? Many a time respondents find questions offending/ response anchors not appropriate etc. The demographics of people who left the survey mid way, may provide you some insight as why some kind of people are not responding to some specific questions.
all the best!
  • asked a question related to Online Surveys
Question
13 answers
Dear Friends & Colleagues,
Greetings!
Amidst the distress caused by the COVID-19 pandemic, we wish you're safe and healthy. As a research team, we're interested to capture your valuable responses on consumption pattern during the lockdown period. We appreciate your time and interest to answer the questions below (takes around 5 minutes) and it is assured that the individual responses will be confidential and solely intended for research purpose.
*Kindly fill the online form and please share with others* (only for RESIDENTS of India)
Thanks 🙏
Relevant answer
Answer
An interesting survey. good luck Dr. Sendhil Ramadas
  • asked a question related to Online Surveys
Question
3 answers
Due to the COVID-19 lock-down it is not possible to physically access people,
so are there any best online survey portals for researchers to collect data online? and reaching people at mass level?
I am from Pakistan and I have no Idea, whether there are any such portals for Pakistani researchers?
Relevant answer
Answer
For Quantitative data gethering, Qualtrics is a good option if your research institution has a license. There are many modalities that can be used and the Qualtrics support service offers swift replies on your struggles in programming surveys.
  • asked a question related to Online Surveys
Question
8 answers
Hi all,
I've been looking for a suitable online platform to host my Q-methodology survey. Most of the platforms now charge a LOT of money to host the survey (upwards of $500 a month!). I did look into QSortWare but don't think the interface is very user-friendly. I need something that is simple, clear and extremely easy to use.
I thought of using SurveyMonkey but this would have to be done so that each statement is ranked on a scale of -2 to +2 in an unforced structure. I know there's some debate about whether to have the forced or unforced distribution but some researchers seem to think it could be OK to allow participants to rank the statements in an unforced manner. I think this is what SurveyMonkey would allow.
Does anyone have any experience of running a Q on SurveyMonkey, or equally do you have suggestions for other platforms?
Cheers,
Niki
Relevant answer
Answer
Hi, I've now tested doing a Q-sort with Unipark, but I assume it would work with any other software that is not per se equipped to handle Q-methodology. I went this route because, like you Niki, I found all other programs challenging to deal with. It worked pretty well, I'd be interested to hear what you think about the approach! Here is how I went about it:
1. Participants rate all statements on a Likert scale (that corresponded to my ratings in the Q-sort, so in my case from -3 to +3)
2. After, each column of the Q-sort became a question to which participants responded by selecting as many statements as they would be allowed under that column in a Q-sort. In my case it looked like this:
- questions 1 and 2 (for +3 and -3, one statement each): Participants are asked to select the one statement that is most important / most important for them to decline. All statements are listed which were given a rating >0 (or +3) or <0 (for -3) on the Likert-scale.
- questions 3 and 4 (for +2 and -2, three statements each): Participants select 3 statements that are most important/ important for them to decline. All statements are listed which were given a rating >0 (or +2) or <0 (for -2) on the Likert-scale, except for the statement chosen in question 1 and 2.
- questions 5 and 6 (for +1 and -1, five statements each): Here, all statements are listed that have not been selected in questions 1-4, no matter their rating on the Likert-scale.
3. In cleaning the data afterwards, all statements that were never selected in questions 1 to 6 were interpreted to have been placed in the middle/ 0 column of the Q-sort.
There are two apparent limitations:
- the programming only works if participants rate at least 4 statements >0, and at least 4 statements <0 on the Likert-scale (otherwise there are not enough statements to do the Q-sort questions 1-4). While annoying for those participants who don't rate this way, the likelihood that people naturally rate this way is quite high. It didn't seem to be a huge issue for my testrun.
- the comparative nature of Q-sorting is of course "simplified" a bit, because participants no longer see all the statements when they select the statements to be placed on the extremes. This too I think is justifiable, but would be curious to hear if anyone thinks it isn't.
  • asked a question related to Online Surveys
Question
2 answers
What methods do you use in open, self-recruitment anonymous online-surveys?
We use the completer/Drop.Out comparison.
Do you have other suggestions?
Relevant answer
Answer
I would use Facebook and find some groups that pertain to your topic and ask the admin for permission. Outside of that, I wouldn't use any only internet surveys. The only other option I would choose personally is SurveyMonkey to see if they have potential participants.
  • asked a question related to Online Surveys
Question
7 answers
I am thinking to do an online questionnaire. However, I want to focus on young adults group only. How should I do my sampling procedure section? Which procedure is suitable for sampling my research?
Relevant answer
Answer
Random sampling technique
  • asked a question related to Online Surveys
Question
4 answers
Hello,
I am currently analyzing a large sample from an online survey distributed via e-mail and received a large number of out-of-office notifications.
Is it legimate to exclude those notifications from the total amount of mails, to get a more exact response rate? I could not find literature on this topic.
Thanks a lot!
Best,
Andreas Schühly
Relevant answer
Answer
Hello Andreas,
To my mind, those instances would represent non-response; you did not get agreement and completion of your survey from the auto-notification cases. The email address is obviously legitimate, as you did connect and receive verification.
However, do give such cases an adequate window of time so that you don't ignore submissions that do come, say, within a couple of weeks of your solicitation.
Good luck with your work.
  • asked a question related to Online Surveys
Question
5 answers
I'm working on an R&R and a reviewer is questioning our response rate. We e-mailed a link to an online survey to student-athletes on our campus. We targeted specific teams, offered an incentive (entered into a drawing for a $25.00 amazon.com gift card) and got a response rate of 37.6%. I thought that was decent, but the reviewer specifically asked "Why was the response rate so low?" Are there published/expected response rates that I can cite?
Thanks!
KS
Relevant answer
Answer
Are there rules of thumb or expected percentages for online survey response rates?
Not sure there is such a rule of thumb % but my personal experience shows the online survey response rate is ranging from 10% to 25% in which your 37.6% is considered very good. Suggest you refer to some journal articles that you'd reviewed relevant to your research field / industry then cite their response rates instead of looking for an industry average response rates.
  • asked a question related to Online Surveys
Question
5 answers
Has anyone used PsychData for their online surveys in the field of psychology? I was interested in using them because they are specifically tailored for the needs of research in this field but not many people seem to have heard of them - despite the long list of uni that subscribe. If anyone could provide an opinion, it would be appreciated.
Relevant answer
Answer
I haven't used PsychData in years. They advertise that they live up to the highest human subjects ethical standards. I had no problems when I used their services but it was years ago. I also had no problems with Google Sheets and Qualtrics.
  • asked a question related to Online Surveys
Question
6 answers
Dear network;
I am conducting a research about online-shopping adoption (drivers) in emerging countries. Due to time and (especially) budget constraints, we're thinking to conduct the survey online targeting people aged 25 to 65 years old.
We have some questions and doubts about the reliabality of online surveys especially when it comes to the publication of such research in indexed journals. My questions are :
1) We might mix online and offline data collection as the 55 to 65 years old might not be on the internet to participate in our study. What are the risks? and, Are there any papers about the (+) and (-) of this kind of multi-mode approach?
2) Are online-based research papers accepted in top journals (in the marketing/consumer field) ?
3) What are the common biases in online research? and how one can remove them (any recommandations from your own experience with onine surveys)?
Thank you in advance for your help.
Relevant answer
Dear Souad,
I will send you some papers about online surveys in marketing research, they are not recent (for example Bradley tried to give some solutions to sampling problems for internet studies in 1999!!!) but I think you could get some answers to your questions and particularly with this paper : .
You can also find a lot of online-based survey in ranked journals. good luck !
  • asked a question related to Online Surveys
Question
6 answers
I am working to finalise a questionnaire for an online survey. Currently I am not aware of any method that is published and acknowledged for its rigor which guides regarding how to incorporate comments provided by experts from different but related fields. To share a little more about it, it may happen like, multiple experts gave comments on a particular question and then I decide to take one/or merge two comments to edit the question.
Any help will be greatly appreciated!
Relevant answer
Answer
The process that you describe to develop a questionnaire falls under both modified delphi technique and content validity. If you are merely seeking expert opinions to develop the items of the questionnaire, you can provide them with an item pool and then select the items that they recommend/suggest. For this purpose, you can develop a Likert Scale 0-4 (not relevant to very relevant) and then select the items that are rated as 3 or 4. The same scale can be used to determine content validity. For content validity, you will have to calculate both Item Content Validity Index and Scale/Questionnaire Validity Index. For detail guidelines you can refer to the following texts.
Polit, D. F., & Yang, F. (2015). Measurement and the measurement of change: a primer for the health professions. Wolters Kluwer Health.
Polit, D. F., Beck, C. T., & Owen, S. V. (2007). Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in nursing & health, 30(4), 459-467.
Younas, A., & Porr, C. (2019). A step-by-step approach to developing scales for survey research. Nurse researcher, 27(1).
  • asked a question related to Online Surveys
Question
6 answers
Hi all,
I was wondering if any of you have recommendations for a good online survey platform for teaching research methods to students? We have been using SurveyMonkey, but the platform has recently narrowed licencing to allow only two users per account. As a result it is now far too expensive for us to use.
We are looking for something that costs around the same as SM ( around USD$350 for an account), and which allows us hierarchical access: ie, a superuser or two (lecturers) and a multitude of ordinary (student) users.
We have tried contacting LimeSurveys, but we have stalled at trying to generate an invoice for payment and after tearing our hair out for eight weeks have now given up on them.
Any other suggestions will be gratefully received!
Thanks & warmest regards
Sam Young
Relevant answer
Answer
QualTrics might be another option.
  • asked a question related to Online Surveys
Question
5 answers
Hi, I am planning a community survey in a remote community. I have designed it as a paper based survey but am considering transferring it to an online survey platform such as Survey Monkey, Survey Gizmo, Survey Sparrow (later two have offline functionality so are looking more appealing). However, since the community is remote and there is a mix of young and old, I would like to still have the option of paper based surveys for those who prefer. Is there any risks or considerations I should be aware of if I go down this route?
Relevant answer
Answer
In survey research, this is known as a "mixed mode" approach, and there is a small literature on this topic, which you can search on the internet.
  • asked a question related to Online Surveys
Question
9 answers
I have utilized Qualtrics and SurveyMonkey in the past, and I truly appreciate the look/feel both for researcher and participant found with Qualtrics. What do you use, and what would you recommend to do if funding is difficult to secure for a more expensive survey application (i.e. Qualtrics) for a three to six month study?
Relevant answer
Answer
I am agreed with vidyasri khadanga
  • asked a question related to Online Surveys
Question
8 answers
Hi everyone!
I am on the lookout for online survey tools that allow us to display Likert scales on mobile devices (mostly smartphones). All I found so far were tools that display the answer options (fully agree, partly agre, ...) in column style from top to bottom. I guess it'll be more intuitive for participants, if the answer options are side-to-side (left to right).
Any suggestions where the Likert scales actually look like Likert scales (and are not just work-around single-choice questions)?
Much obliged
Jürgen
Relevant answer
Answer
for anyone who is also on the search: I found "formr" (formr.org) to suit my purposes. It is very flexible and compatible with Markdown and R code.
  • asked a question related to Online Surveys
Question
7 answers
Dear professors, researchers and students, I am writing Master's thesis about consumers' innovation resistance behavior because certain innovative product is not successful in Germany compared to other countries.
I want to find out that what factors hinder consumers from adopting that innovation.
Based on research results, I want to suggest recommendation on marketing strategy for German firms providing that innovative product. Then, here is my question,
for my research survey, should I investigate only German people? or all consumers regardless of their nationality as long as they resist to this innovation and live in Germany ?
Thank you very much in advance for your any advice and help.
Sincerely yours,
Dahee Baek
Relevant answer
Answer
The key issue is to investigate whether the target group of customers of a given product or service offer functions in only one country or is also present in other countries. It seems to me that it is first necessary to examine the national and foreign target markets on which a specific range of products or services is sold. Only the answer to this question will enable defining a representative target group of clients.
I invite you to the discussion.
  • asked a question related to Online Surveys
Question
6 answers
I created one questionnaire with 8 questions on Insomnia with a 4 point Likert scale response. This was followed by 36 questions on job satisfaction using a 6 point Likert scale. 
 
I used the Bristol Online Survey tool.
Relevant answer
Answer
Dear Steve,
If you are looking at relationship between Insomnia and Job satisfaction, you can apply correlation and regression analysis, and based on the outcomes you can conclude how the two variables are related with each other.
On the other hand, If you are looking at comparing the mean scoring of Insomnia and Job satisfaction among different groups. The Likert scale, ideally comes under ordinal or interval scale of measurement, and the scorings obtained would usually not follow normal distribution and therefore, you may have to apply any non-parametric tests such as Krushal-Wallis test or Friedman test, based on the numbers of groups you have for comparison.
All the best for your analysis.
Regards
Vishwanatha
  • asked a question related to Online Surveys
Question
4 answers
I used the Bristol Online Survey tool to create one questionnaire: -
There are 8 questions on Insomnia using a 4-part Likert Scale, followed by 36 questions on Job Satisfaction using a 6-part Likert Scale.
Which statistical tests should I use?
Relevant answer
Answer
Dear Dr. Steve,
If you looking at relationship between insomnia and Job satisfaction, you can apply correlation analysis to see how these two factors are related with each other and derive conclusions based on the outcomes.
On the other hand, if you are looking at mean different in scoring of Insomnia and Job satisfaction among the different groups, Liker scale seems to be an ordinal or interval scale of measurement and the scorings obtained would rarely follow normal distributions, and hence in this scenario its better to use any non-parametric tests such as Krushal-Wallis test or Friedman test based on number of groups you have for comparison.
All the best for your analysis
Regards
Vishwanatha
  • asked a question related to Online Surveys
Question
21 answers
Hi,
I am looking for support or suggestions from those who have conducted research into sensitive areas.
For my masters dissertation, I am attempting to explore the experiences of male victims of "Sextortion".
The method has adopted an online approach and I have developed a qualitative survey. Participation is anonymous and I have distributed the survey on numerous sites including Facebook, Twitter, Linkedin, Grindr, Reddit, men's forums and others. I have only managed to obtain one response.
Has anyone conducted research in a similar way? If yes, how can get participants?
Any help will be greatly appreciated.
Relevant answer
Answer
Hi Isabel,
Per your definition, you may also place an announcement in some type of newspaper, other type of media outlet, or use social media to recruit potential participants.
Have a great day!
--Adrian
  • asked a question related to Online Surveys
Question
2 answers
We assume that the Desk-Crit -as a form of studio assessment- is considerably the most relevant, frequent and direct-contact tool especially when assessing students’ behavior on the outcomes level of person, product and process. 
what kind of attributes can we evaluate regularly with students?
a survey is attached to the question, I would appreciate if you provide me with your answer using this online survey.. regards
Relevant answer
Answer
I would be very interested to see the results of your survey. Would it be possible to get a copy of them?
  • asked a question related to Online Surveys
Question
9 answers
I am conducting a research on how an individual analyses the stocks when trading or investing. Why despite proven results of the complementing nature, majority of them resort to Technical analysis or Fundamental analysis, but only a few use both together. For my primary research, I need to circulate an online questionnaire to find the widely used techniques and suggest an idea for analyzing stocks for both short term trading and long term investing. I am particularly looking for a forum or discussion page of investors where people might fill up the questionnaire. Attached is the proposal draft for more information. Any kind of contribution would be appreciated.
Relevant answer
Answer
As a matter of behavior, these people (stockbrokers) are not so easy to get them answer a questionnaire. Try some other methods.
  • asked a question related to Online Surveys
Question
5 answers
I am conducting an online survey to know opinion from public health experts regarding selection of indicators to assess health profile of a city. What is the minimum sample to collect opinion in a online survey? 100 responses, 50 or 30 responses.
Relevant answer
Answer
You are asking opinions so some of the issues covered in most texts about power calculations may not apply. The issue here is actually simpler: it is a question of precision and you don't really need to refer to other studies to answer that - you need to ask yourself how precise you need to be. I think most statistical power programs will focus on calculating power for detecting differences (e.g. G*Power). What you need to do is to determine the sample size that gives you a 95% CI that is 'acceptable'. You are surveying opinion so you'll likely need to determine a % of the population that endorses a particular view with a CI that is (say) +/- 5%. https://www.surveysystem.com/sscalc.htm will do the calculations for you.
If there is some measured parameter to be estimated you will need to have some prior information to estimate the SD which would allow you to detemine n associated with a given width of CI (95% CI is =/- 1.96 SE and SE is SD/root n) [in vv simple terms]
Perhaps MORE important though is your sampling strategy. The precision of your estimate is of no importance if you are not providing an unbiased estimate of the population - this is entirely about sampling.
  • asked a question related to Online Surveys
Question
3 answers
I'd like to conduct an online survey through Facebook. I do not have access to a list-based sample, but will need to employ a sampling frame. Therefore, I am looking for a few references for researchers/research studies that have used an adequate sampling frame to complete a study utilizing an online survey.
Relevant answer
Answer
Hi, Aldenise,
My students conduct several online surveys each year using primarily facebook to solicit participation from family and friends. The largest samples seem to come from those who can get family and friends to also share the solicitation on facebook. Two articles have been published which you can find on my RG page. One is on attitudes toward same-sex marriage and one on advice for BSW students.
When using facebook to solicit participants, we have found that it is much better to employ a wide sampling frame (e.g., adults over 18 years), then collect demographic and other categorizing information on the questionnaire. When we try to screen respondents with early questions, we seem to get much lower rates of participation. I think people are less likely to share a solicitation message if they are unable to answer the questions themselves.
  • asked a question related to Online Surveys
Question
6 answers
Dear Colleagues,
After having some quite useful discussion with some colleagues (but not expert in this field) about two major methodological aspects of my PHD-Thesis I am still quite uncertain and still looking for useful references in order to justify my research design.
Currently, I examine the way language is used in “heated debates” / “crosstalk” charged with emotions due to a specific constellation, for examples during amateur football / soccer games.
Therefore, I choose a triangular (qualitative) approach (semi-structured interviews, participant observation and content analysis) in order to generate hypothesis which will be tested through a standardized online-survey (classic sequential mixed-methods approach).
The semi-structured interviews (one singular semi-standardized questionnaire) were being conducted by students from 2 different universities in 2 different cities, at 3 survey periods (autumn 2016, spring 2017 and autumn 2017), the participant observation was being conducted over one season likewise the content analysis of one chat protocol. Consequently, previous (even if superficial) results influences posterior findings (especially observation). However, I do not project to compare period 1 with period 2 or findings generated through observation with findings obtained through a content analyses of a chat protocol. It is rather about having a large amount of data in order to generate useful hypothesis which will be tested afterwards.
My questions are therefore:
1. Is it rather a longitudinal or a cross section study? Maybe something in between, like a (special) trend analysis? Do you know relevant and recent published literature?
2. Given that I chose to observe as a (secret) participant a group of football players (the idea to do so came by playing with them and hearing what they said) without having an “informed consensus”, I would like to know, if you have convincing arguments which “justifies” the procedure or if you know relevant literature dealing with this topic.
Many thanks.
Kinds Regards
Relevant answer
Answer
This would be a cross-sectional study.
In addition, informed "consent," where participants knowingly and willingly participate in research is usually required when: collecting sensitive information on individuals, collecting identifying information on individuals or collecting information that could be to the detriment of participants.
Since it seems your analysis is at the group level and I am assuming no personal or identifying information is being used in your research, I would say that informed consent is not necessarily mandatory.
Special exceptions can be granted by the ethics board (as was the case with my PhD) if the observed participants would act differently knowing that they were being observed for research purposes (Social Desirability Bias etc.) In my case I had to spy on illegal fisherman and wait for them to do something illegal, which they probably wouldn't have done had they known I was watching/recording.
In any case, you should run it by your ethics committee but I don't think it should be an issue.
Further reading:
  • asked a question related to Online Surveys
Question
2 answers
Is there another online survey to check the reading comprehension?
Relevant answer
Answer
Thank you Sir for giving me different sources of Online Survey but I want to know about the specific one Online Survey of Anderson (2003) Will have to follow specific rules to use OSORS (Online Survey of reading Strategies) to check the response of Online readers and Can we do modification in already designed survey according to our need?
Regards
  • asked a question related to Online Surveys
Question
4 answers
The system should generate an individual code for each participant.
Relevant answer
Answer
I've tried doing this.  One way is to create an Acrostic that is easy for the respondent to remember and reproduce (i.e. reliable) and very difficult to decode.  For example:
What day of the month were you born?  21
What is the last letter of your mother's maiden name?  Z
How many letters are there in your first name?  5
This person's acrostic = 21Z5.  There are unlikely to be too many duplicates unless the sample is very large.  Adding one other letter question (what is the second letter in your best friend's first name?) would reduce duplicates even further but implies that everyone has a best friend and requires that people will answer the same at T1 and T2 (i.e. good memory, and same best friend).  If you get a high perfect match rate, you might be able to deduce most of the imperfect ones.  For people with good cognition over short followup periods, four source questions combining numbers and letters with constant answers should produce pretty high perfect match rates.
  • asked a question related to Online Surveys
Question
1 answer
Dear Jan,
It was long time without any communications. I hope you are fine and doing well. I am in Uppsala for any help or meeting for our other remaining publications. I hope also if we could write any review together.
Kindest regards, Hesham 
Relevant answer
Answer
This should really have been sent as an email.
  • asked a question related to Online Surveys
Question
5 answers
I'm working on my Master's degree and I have this survey about Customer Experience which has about 45 questions. How many customers do you think I have to contact via email to get representative results? I have no restrictions in terms of demographics or other factors, because those are possible influencing factors so I need a large amount of responses to cluster them for the analysis. The survey is going to take place in the whole customer base of an online vendor.
Relevant answer
Answer
Thank you all for answering my question. This will definitely help me out.
  • asked a question related to Online Surveys
Question
2 answers
We are planning to do a fully remote clinical trial to test an online intervention for depression.
Recruitment flow:- Participants will see all the relevant information on a webpage. Participants may contact us to clarify any doubts regarding the study. Then they will complete two assessment forms to test the inclusion/exclusion criteria. Then, we will call the participants to validate their contact information (at this point we wouldn't know who will be assigned to which group). After that the participants will be asked to sign up to the intervention website. Then they will be randomized to the experimental or the control group. We will be using minimisation (using MinimPy) to randomize participants.
Blindedness:- The participants wouldn't know if they are assigned to the experimental or the control group (so it is definitely single-blind). Since there is no investigator who will be assessing the depression severity, there is no source of any human bias there (since double-blind trials are designed to eliminate investigator bias).
Intervention:- The intervention is fully stand-alone and online. However, there is a forum in the experimental group where everyone, participants and investigators, will be able to post questions and answer questions. But, that is an extra feature of the experimental group which we think will help participants with their problems. We will be in touch with the participants via telephone only once. All other correspondence, if any will be via email or through the website itself.
I was wondering if this can be called a double-blind study. I have seen other fully remote trials with a similar design but they haven't mentioned the term double-blind anywhere. Considering the fact that it is the gold standard, I am surprised that investigators did not mention it in their paper. That is the source of my confusion.
Any thoughts are welcome. Thanks in advance.
Relevant answer
Answer
@shuchi Thanks for your reply. I have added more information to the question.
  • asked a question related to Online Surveys
Question
11 answers
I'm conducting a qualitative study with a Delphi design. I want to demonstrate that my data has been properly triangulated. I've shared my data collection with a trusted associate (another researcher). In your experience, does that qualify as triangulation? If not, how have you approached triangulation? Thank-you in advance!
Relevant answer
Answer
Merely sharing your data in this manner does not in any form translate into triangulation. Triangulation of data have to involve the implementation and validation of data through  verification from two or more sources. It is a technique that applies and combines two or more research methods within the same study domain.
  • asked a question related to Online Surveys
Question
1 answer
Stack Overflow advertises several official (https://stackoverflow.com/help/how-to-ask) and de facto standard guidelines (http://tinyurl.com/stack-hints) for writing good questions -- that is, questions that have greater chances to be resolved.
We built a statistical model to empirically validate this question-writing netiquette. The last step we're missing in our research is: What does the SO community think of these recommendations?
If you ever used Stack Overflow before, please help us find out by taking this very short survey: https://goo.gl/EzS3eN
Thanks for your contribution! 
EFFORT
The time to completion for the survey is about 5 minutes only (before June 21, 2017).
INTENT
This is a purely academic research project with no commercial interests. We will openly publish the results so everyone can benefit from them, but will anonymize everything before doing so; your responses will be handled confidentially. Please, note that you are not obligated to participate in the survey. If at some point during the survey you want to stop, you are free to do so without any negative consequences. Incomplete survey data will not be used.
A NOTE ON PRIVACY
This survey is anonymous. The record of your survey responses does not contain any identifying information about you, unless a specific survey question explicitly asked for it.
Relevant answer
Answer
The survey is no longer accepting responses.
  • asked a question related to Online Surveys
Question
2 answers
I want to conduct a A/B test. I want to send one weblink to anonomous respondents. When respondents click on the web link it should randomly assign each respondent to either version A or version B of the survey. 
I thought there was a free programme available which does the trick but unfortunately I cannot find it.
Any help much appreciated . 
Relevant answer
Answer
Thank you for your information, Unfortunately it is not helpfull for me. Too complicated. 
  • asked a question related to Online Surveys
Question
1 answer
please take 5 minutes to complete this online survey
Relevant answer
Answer
I did it for you 
with my best wishes 
  • asked a question related to Online Surveys
Question
3 answers
Good references on sample size for quantitative study when sampling frame is big, say more than three hundred thousands and the listing of all individuals is not possible?
Relevant answer
Answer
Hello Bibek -
It isn't just how big the sample size may be that matters, it's how the sample is constructed.  If you want to be able to infer from a sample to a population, there are two basic ways: (1) by a randomized selection, where the inverse of the probability of selection is used to weight the sample up to the population, or (2) by use of regression modeling to relate the sample data (y) to regressor data (x), where the x are known for the population.  Or (3), a combination of (2) and (3).  Otherwise you can have seriously biased results. Even (2) alone can be risky. 
 
It is often good to stratify (categorize) your population, regardless, to reduce uncertainty. 
 
A sample size, from your reduced-bias sample, is then determined by how large it needs to be to reduce, typically, standard error of a mean or total, and this also depends upon the details of your sample design, and population or subpopulation standard deviation(s).  And this assumes you do not strain resources and lose data quality. 
 
With a large size for each subpopulation or stratum, you may not need to worry about a finite population correction factor. 
   
Because you cannot cleanly identify your population, I assume that presents a challenge in drawing a "representative" sample as described above.  Perhaps you are in a situation where you need an area frame, where your sampling unit is at a higher level, and you use a two-stage sample where you first choose among the larger units, and then sample or census at the level you want within the larger units chosen.  Therefore, you may want to research the following terms: "multistage sampling," "area sampling," and "cluster sampling." 
   
If you are doing a nonprobability sample with no regressor data, then results can be substantially biased, and there is no way to really measure standard error and no sample size requirement 'formulas.'  The best you might do is to divide your population into categories and be sure to sample from each, and repeatedly sample enough that your mean results in each category stop changing to an unacceptable degree ... meaning unacceptable to the certainty your subject matter and application requires. 
 
At any rate, it might be best to start with a pilot study. 
Best wishes - Jim
  • asked a question related to Online Surveys
Question
4 answers
Hello RG!
I'm trying to make an online survey which would measure the satisfaction and importance of specific elements in higher education. 
In order to maximize the response rate I'd like to make it so that respondents (students) can evaluate every variable as easily as possible, but I just can't seem to find the tools to make that happen.
As a picture is worth a thousand words, I'll just attach one, instead of trying to explain what I'm looking for.
If anyone knows a provider that allows (if possible, for free) such questionnaires, please link me.
GoogleForms, LimeSurvey, SurveyMonkey don't seem to have that option, but I might be wrong since I'm mostly familiar only with Google's services.
Thanks!
Relevant answer
Answer
You might want to check with qualtics.
If not successful, you can consider ‘importance-performance’ analysis. If that is the case, you should not perhaps measure it using a double response item; rather you should use two distinct Likert type items, something like the following:
E.g.,
How important it is to for a prof to be an expert in their field?
From “Very unimportant” (1) to “Very important” (5).
This prof is an expert in his field
From “Strongly Disagree” (1) to “Strongly Agree” (5).
There are various things you could do with importance-performance analysis.
Here are some references:
Martilla, J. and James, J. (1977), ‘Importance-Performance Analysis’, Journal of Marketing, 41(1), 77-79.
Bacon, D. (2003), ‘A Comparison of Approaches to Importance-Performance Analysis’, International Journal of Market Research, 45(1), 55-71.
Silva, F. H., & Fernandes, P. O. (2011). Importance-performance analysis as a tool in evaluating higher education service quality: the empirical results of ESTiG (IPB). In In the 17th International Business Information Management Association Conference (pp. 306-315). Khalid S. Soliman.
Best wishes
  • asked a question related to Online Surveys
Question
5 answers
I'm looking for a program that helps fill in questionnaires for statistical surveys and also helps in automating the sending and receiving of questionnaires themselves (a web-based tool). There are tools available i.e. Microsoft and goggle, but do not have all the features of a professional Surveys.
Relevant answer
Answer
Have you considered qualtrics?  www.qualtrics.com
Also, you could try survey monkey: www.surveymonkey.com
Best wishes!
  • asked a question related to Online Surveys
Question
9 answers
I want to conduct a field survey with real employees, and I'm curious what/how other researchers prepared and conducted such surveys.
If you have any experience, (e.g., how you contacted the firms, what difficulties were there, what kind of different ways there are in collecting field responses, etc.) I'm very curious about all of it.
Thanks a lot in advance!
Relevant answer
Answer
Hello.
I can recommend an excellent book which also gives access to online resources - it is entitled 'Social Research Methods' by Alan Bryman, published by Oxford University Press.
You will find many helpful tips and practical advice on areas of study for surveys, interviews, focus groups and mixed methods of research.
Hope this helps.
Best wishes.
  • asked a question related to Online Surveys
Question
10 answers
I am analyzing responses to surveys where people have written very different text responses to the "please explain" text fields from their Likert scale x-choice in the same question?  For example:
Q: How would you rate the responsiveness of home office to field office needs? 
  X choices=Excellent, fair, neutral, poor, terrible (respondent selected "fair")
*Same* respondent's text response in comment field =  "Home office has almost never responded to any of my requests as field office director. My understanding from other ODs [office directors] is that they have an equally hard time getting home office to reply to requests, let alone get the request filled."
This is a strong and broad pattern across many questions and many respondents. I'm beginning to think I did something wrong to break the platform (SurveyMonkey) or just did a really poor job of constructing questions...if it is not me, what causes people to answer this way?  Is this a real survey-based research thing or is it just me?
Relevant answer
Answer
Interesting point Vicki! Sometimes people are interesting. When we did a survey in 2015, a guy sent us a 5-page letter describing why he thinks that the survey was flawed. In essence, he hated the topic related to the survey.
  • asked a question related to Online Surveys
Question
8 answers
I'm interested in potentially measuring implicit attitudes (via implicit reaction time) toward head injury while participants complete survey items. Looking for a platform that will allow me to create/add my own survey items.
Relevant answer
Answer
The Psychtoolbox for MATLAB. It is free, flexible and accurate.
  • asked a question related to Online Surveys
Question
4 answers
I am interested in any personal experience you may have using this combination (MTurk and Qualtrics) for survey research, and any advice you may have. 
Specifically ease of use, ability to get a representative sample, the number of responses and the cost. Beyond that,  for what type of research would you feel comfortable using survey data obtained through this method?  Would it be best for only testing survey ideas or is the data quality sufficient for more than that?
Finally has anyone had experience with data obtained this way and IRBs?
Thank you!
Relevant answer
Answer
Thank you Mitch--that was great information.  We are developing a policy research project that includes some behavioral elements and we were trying to figure out if using this combo would be good to first test our ideas and maybe use those results to get funding to do a random sample survey so this was very helpful.  Thank you very much!
  • asked a question related to Online Surveys
Question
1 answer
I am considering to write a code to automatically check for careless answered items in digital questionnaires using the data that is naturally tracked by most platforms. So far I only found studies discussing the inclusion of test items, adding special instructions in the instruction text or similar stuff.
But I'd like to make use of the data that you get especially from the digital format.
Relevant answer
Answer
Hi,
I am curious as to what type of questionnaires you are thinking of focusing on.
CK
  • asked a question related to Online Surveys
Question
14 answers
I am doing a study on consumer engagement in Facebook brand pages. I am collecting the data using an online survey that was posted on a certain brand page on Facebook.  I posted the link of the online survey on the brand page where i was able to get 591 responses from consumers who are members in that page. 
My question is : am I using simple random sampling?  Some researchers told me that i am using that sampling technique because every member in the population of the brand community had an equal chance to appear in the sample?. I didnt choose the participants who answered but i sent an invitation asking consumers to answer my survey
I am not sure about the sampling technique that i have used. Please advice me.
Relevant answer
Answer
Technically speaking, it probably is a convenience sample, but that label has a negative stereotype. So, for any write up that you do, I would simply describe what you did, rather than trying to label it.
  • asked a question related to Online Surveys
Question
4 answers
Can anyone point to sources that explain what might be good, normal, or bad response rates for online questionnaires (evaluating training and development). We are being told different norms by different bodies involved.
Relevant answer
Answer
There are many factors.  If you are sending out messages trying to recruit strangers, your return rate may be low.  If you have an organized group and online is simply the way they complete the  survey, return may be high.
  • asked a question related to Online Surveys
Question
1 answer
These questions are not related directly to my research.
For those who asked for papers please access them via my website: http://www.eself-learning-arabic.cornell.edu/
Relevant answer
Answer
Absolutely!
  • asked a question related to Online Surveys
Question
4 answers
What can be done on the research design and questionnaire?
Relevant answer
Answer
Hello Sumiran,
Is this website of any help?
Or this paper:
Bartneck, C., Duenser, A., Moltchanova, E., & Zawieska, K. (2015). Comparing the similarity of responses received from studies in Amazon’s Mechanical Turk to studies conducted online and with direct recruitment. PloS one, 10(4), e0121595.
Very best wishes,
Mary
  • asked a question related to Online Surveys
Question
116 answers
Hello everyone
I have 3 online surveys that are comprised of a series of closed and open-ended questions. One survey in particular was heavy on the open-ended questions. Data collection was concurrent. I have some comprehensive written responses to these questions and some that are quite brief. 
My question is: is this combination of closed and open-ended questions enough for my research to be considered a mixed-method study? I have not carried out any interviews or collected any other qualitative data. Or do I need to describe my study as purely quantitative?
The data from the open-ended questions is very important and it will be triangulated in analysis.
I have been searching for an answer but I can't find anywhere where this issue is explicitly addressed. There does seem to be some contention as to whether open-ended questions can even be considered qualitative data.
Does anyone know of any literature that provides good commentary on this issue?
Or what are your thoughts?
Thank you.
Relevant answer
Answer
I think that Mixed Methods is much more about the use of the data, rather than the specific technique that was used to collect it. In particular, if you have research questions that amount to "how and why" and your open-ended data can address those questions, then I would certainly consider that qualitative research.
In your case, you mention triangulation, so that would imply that you could use your qualitative data and your quantitative data to produce results comparable that could be compared. That would certainly be an example of Mixed Methods research.
Think of it this way: if your questionnaire was exactly split 50-50 between close-ended and open-end questions, you probably wouldn't have any difficulty called that mixed methods. But what it were split 60-40, would that be good enough? Could you set the boundary at 25-75, is that enough? What if there was only one open-ended question, but it produced the most interesting and important results in the whole data set, would you deny the qualitative nature of that data and claim that it was quantitative merely because it came from a survey?
  • asked a question related to Online Surveys
Question
3 answers
I started looking at "standard" large datasets, like the GSS, but I can't seem to find any that have open-ended questions.
Relevant answer
Answer
Hello Richard,
See the attached link,
Hope you find the dataset.
Regards.
  • asked a question related to Online Surveys
Question
8 answers
We – a research group at the software engineering institute at the JKU – have been investigating the compliance of object oriented design best practices directly in the source code. Therefore, we implemented a tool that is used in practices and enhanced it at various points. In spite of the practical usage of the tool, we are now eager to systematically validate our approach.
For this validation, we want to figure out the impact of object-oriented design best practices for achieving good object-oriented design. This is conducted by using an online survey. I would like to ask you to participate in this survey (duration approx. 12 minutes):
Token: cctmvg
The survey is anonymous since the token is needed for security reasons but used from all invited participants.
At the end of the survey, you can sign up for receiving a summery showing the results and findings of the investigation (the anonymity of your answer is still ensured). We are happy to share our insights in this research area. Hopefully your support helps to answer this question.
Relevant answer
Answer
Having taken the survey - its results need considered with a pinch of salt because some principles (being condensed to a very few words) were not entirely clear, especially since they use terminology that can differ across multiple languages. But most of the principles were clear enough so the survey should have value.
  • asked a question related to Online Surveys
Question
4 answers
What are the best strategies and places to recruit a clinical sample for an online survey study? It can be difficult to find individuals online who are willing to participate in an online study.
Relevant answer
Answer
'Snowball' sampling (https://en.wikipedia.org/wiki/Snowball_sampling) is one method you might employ to reach out to eligible participants.  I presume your clinical sample refers to patients here, but I did wonder whether it might have been clinicians?  Either way, you could potentially create a study/project steering group, that would comprise some key individuals who have a core interest in the study and, crucially, contacts so that they could inform others of the survey.
The only issues with snowball sampling is that you can have biased data as it can be significantly influenced by the initial participants, who then inform others of the study.  Furthermore, it is a non-random method, so the results may not necessarily be representative of wider population.  This could be problematic if you are intending on conducting parametric analyses.  However, there would likely be non-parametric equivalents.
An idea would also be to take a look at what other similar surveys have done for their recruitment strategy, and in which areas geographically.  This may give you some useful insights into how you could structure a similar survey.
Thinking a little differently also, is there any way in which you could offer the survey both online and in paper form?  It might be that your clinical sample may genuinely have a preference for either method.  For example, some of your sample may prefer to complete the survey in paper form because of concerns over online security etc.
All the best.
Adam
  • asked a question related to Online Surveys
Question
3 answers
Background: I'm using an online survey company to launch a questionnaire nationally. The survey is to identify factors affecting online shopping intention, so there will be a lot of scale questions, as well as respondents' demographic information. I'm asking the company to get 50 responses before carrying on the full survey.
Question: Would you suggest some quick stat tests using the pilot data, to check if the responses are of high quality? (whether the profile is diversified / whether they answered the questions carefully / etc)
Many thanks for your help!
Li
Relevant answer
Answer
Would you suggest some quick stat tests using the pilot data, to check if the responses are of high quality? (whether the profile is diversified / whether they answered the questions carefully / etc)
You should always conduct a pilot study / test before going for "big bang" data collection.  The purpose of the pilot study include the following which is not an exhaustive list:
  1. checking the reliability of your survey questionnaire especially for those new questions you'd developed on your own to measure certain constructs / variables e.g. Cronbach Alpha Reliability / Composite Reliability scores.
  2. evaluating whether your future respondents can fully understand your survey questionnaire - if not these can be rectified before going for "big bang" data collection.
  3. give you initial glimpse on your tested results after initial data analyses based on your hypotheses formulated - so that you can adjust your research model, operationalization of your constructs / variables, hypotheses etc. as appropriate.
  • asked a question related to Online Surveys
Question
5 answers
Hello all,
I am a doctoral student who is seeking advice or any relevant references on best methods for recruiting a sociodemographically (race, ethnicity, SES) diverse adult sample to complete online surveys. My dissertation involves observation/examination of how mechanisms of socioeconomic and neighborhood disadvantage influence violence and related variables, so recruiting a sociodemographically diverse sample is critical. I was wondering if anyone out there had any words of wisdom for best ways or mechanisms for obtaining a diverse sample in completing online surveys.  
Any guidance would be very helpful. Thank you all so much
Relevant answer
Answer
Online samples tend to be younger and more affluent, so for the purposes of recruiting a diverse population you could consider offering a print version. Conducting written research will likely  exclude people who are not confident about their literacy. You could triangulate using different methods, e.g telephone or face to face interviews, but you also need to be realistic about what is achievable within your timeframe and resources. No research is ever perfect, you just need to be aware of your limitations so you don't make unwarranted claims about what your data means. 
You could snow-ball sample, from an original convenience sample by asking people you know to share a link to the research with everyone they know. You can use twitter and Facebook, and other social networking sites if you can get ethical approval. You could also seek out support groups for various populations and ask them to publicise your research. For my doctorate I recruited through online support groups, a specialist clinic and with posters with tear off slips with my email address on displayed in places I thought relevant participants might see them and could get the info discretely. These strategies don't lead to a representative population, but do increase the chances of recruiting a diverse population. If you are doing statistical analysis this could be a problem, but if you're doing exploratory qualitative research with the aim of generating rather than testing hypotheses then diversity is probably more  important than representativeness (although that depends on what qualitive approach you use) 
I hope that helps. Feel free to contact me for further information.
Kat Macey
  • asked a question related to Online Surveys
Question
5 answers
Social media can be used to recruit study participants to participate in online data collection. But how does one insure the participants meet the eligibility criteria?
  • asked a question related to Online Surveys
Question
8 answers
Colleagues,
Design and formatting of online survey questionnaires can (and do) result in poor response rates and/or poor quality data. An example of poor quality data might be lack of variability in the responses. What are some of the other response related problems that have been identified with online surveys, and what suggestions do you have for avoiding these problems? References to the scholarly literature appreciated. Thank you...
Ruth
Relevant answer