HP Inc.
Question
Asked 21 October 2016
Quick quality check techniques for survey pilot?
Background: I'm using an online survey company to launch a questionnaire nationally. The survey is to identify factors affecting online shopping intention, so there will be a lot of scale questions, as well as respondents' demographic information. I'm asking the company to get 50 responses before carrying on the full survey.
Question: Would you suggest some quick stat tests using the pilot data, to check if the responses are of high quality? (whether the profile is diversified / whether they answered the questions carefully / etc)
Many thanks for your help!
Li
All Answers (3)
Would you suggest some quick stat tests using the pilot data, to check if the responses are of high quality? (whether the profile is diversified / whether they answered the questions carefully / etc)
You should always conduct a pilot study / test before going for "big bang" data collection. The purpose of the pilot study include the following which is not an exhaustive list:
- checking the reliability of your survey questionnaire especially for those new questions you'd developed on your own to measure certain constructs / variables e.g. Cronbach Alpha Reliability / Composite Reliability scores.
- evaluating whether your future respondents can fully understand your survey questionnaire - if not these can be rectified before going for "big bang" data collection.
- give you initial glimpse on your tested results after initial data analyses based on your hypotheses formulated - so that you can adjust your research model, operationalization of your constructs / variables, hypotheses etc. as appropriate.
2 Recommendations
Northcentral University
You should also calculate the demographic profile of the pilot sample to see if it approximately the demographic profile of the population you are interested in surveying. Can the survey company reach the right proportion of men, women, age groups and racial groups reflective of the population. Although this may not be easy to determine with a test sample of only 50.
1 Recommendation
Mosaic Insights
Absolutely check it - you'll get to see how your data is going to be coming in, so you can work out how to handle it, and identify any issues you may not have foreseen (such as invalid responses). These checks are essential for ensuring high-quality data.
You will also be able to look at the type of statistics you want to run and check that it's all going to go according to plan. 50 is a good number to test these sorts of things on.
Most surveys are not perfect from the start, and it's impossible to predict every way that your questions might be interpreted by your target audience. Testing - and checking through the results of these tests - are crucial. It is also a good test of the survey company and the way you'll be interacting with them.
Good luck!
1 Recommendation
Similar questions and discussions
Need some suggestions on sentiment analysis?
I am working regarding sentiment analysis of about 500 comments over a blog. The blog states the career frustrations of professionals in a particular profession (i.e it itself reveals the negative aspects related to that profession).
Firstly, I have performed the content analysis (using the QDA Miner) of the blog to find the career aspects that are causing career frustrations among the professions.
Next, I have planned for sentiment analysis (using Python NLTK Text Classification/ http://text-processing.com/demo/sentiment/ ) of the comments (given exclusively by the professionals associated with that particular career) on the blog post . However, on moving further I have obtained that in some comments, there is fragmentation of sentiments. The first fragment which is related to the blog gives a positive sentiment whereas, the second one which is related to descriptions of career aspects gives a negative polarity.
For eg: Excellent article (+ve Sentiment related to the blog post). the problem is that you learn this once you have spent 15 yrs in this field trying to achieve what you dreamt of...... it's a complete waste going into this career field in this country. (-ve Sentiment wrt career aspect) it's better if you aim early and move out...to the US or any other place of your liking...(-ve Sentiment wrt career aspect).
Kindly give me some advice on how to conduct sentiment analysis of such type of dual-natured comments that reflects sentiments for the blog post and the blog aspects itself.
Recommendations
In recent decades various modes of data collection have emerged due to the implementation of computer methods. Especially in the case of multimedia enhancement, some arguments based on theoretic considerations can be raised for the special viability of these methods for children as respondents (e.g. overcoming problems regarding literacy skills, br...
Dirty Data in Online Surveys. How to improve data quality before and after the field phase. More and more data are being processed faster and faster. Customer needs, time pressure and ethical standards are driving up relevance of data quality. How can we optimize data quality even before the field phase. And what if data have already been collected...
