Content uploaded by Piotr Zaborek
Author content
All content in this area was uploaded by Piotr Zaborek on Jul 05, 2015
Content may be subject to copyright.
1
Elements of Marketing
Research
Piotr Zaborek
Warsaw School of Economics
Prepared as a part of the EU co-funded project “Youth design management”
2
Contents:
Chapter 1: Introduction to marketing research ...................................................................................................... 3
Definition of marketing research ........................................................................................................................ 3
Rationale for marketing research ....................................................................................................................... 4
Two basic types of marketing research .............................................................................................................. 6
Process of marketing research ........................................................................................................................... 8
Marketing Information System, Primary and Secondary Data ......................................................................... 11
Distinctions between data, data structures and information........................................................................... 13
Chapter 2: Conceptual framework of research study ........................................................................................... 15
Defining decision problem ................................................................................................................................ 15
Formulating research problem ......................................................................................................................... 20
Theory and Analytical Models in Marketing Research ..................................................................................... 22
Chapter 3: Choosing research design .................................................................................................................... 31
Qualitative and quantitative studies ................................................................................................................ 31
Deductive and inductive strategy of theory building ....................................................................................... 32
Generalizing results .......................................................................................................................................... 36
Other considerations in choosing between qualitative and quantitative approach ........................................ 38
Exploratory, Descriptive and Causal Research designs ..................................................................................... 40
Chapter 4: Conducting Qualitative Research with Focus Group and Depth Interviews ....................................... 44
Focus Group Interview ...................................................................................................................................... 44
Depth Interview ................................................................................................................................................ 52
Analysis of Textual Data .................................................................................................................................... 56
Chapter 5: Designing questionnaires for survey research .................................................................................... 59
Quality criteria of a survey questionnaire ........................................................................................................ 59
Closed and open ended questions .................................................................................................................... 60
Primary scales of measurement ....................................................................................................................... 62
Things to consider in designing individual scales ............................................................................................. 66
Overview of multiple measuring scales ............................................................................................................ 68
General guidelines for designing questionnaires ............................................................................................. 74
References ............................................................................................................................................................ 81
3
A note from the author
I wrote this volume as a teaching aid for a 30-hour course in Marketing Research that I run at the
Warsaw School of Economics. The book, just like my course, is dedicated to undergraduate students
in various management, marketing and economics majors, who completed courses in Principles of
Marketing and Introductory Statistics. The overall aims of the course, which are hopefully reflected in
the textbook, are to teach how to develop a proper conceptual framework for a study, select the right
research design and employ common research techniques, in particular focus group interviews and
cross-sectional surveys. The contents to not include statistical analysis, which is beyond the scope of
the elementary course in MR, but familiarity with the basics of statistics will be helpful in gaining a
deeper understanding the discussed topics.
Chapter 1: Introduction to marketing research
Definition of marketing research
It is possible to come across many definitions of marketing research in academic literature and
textbooks. More often than not, the differences among them are not critical as they rarely concern
substantive issues but rather the terms used to describe the concept as well as the scope and focus of
the definition. One of the most comprehensive and widely accepted is the conceptualization of
marketing research proposed by the American Marketing Association. On the web page of this
organization it can be found that:
1
Marketing research is the function that links the consumer, customer, and public to the marketer
through information - information used to identify and define marketing opportunities and problems;
generate, refine, and evaluate marketing actions; monitor marketing performance; and improve
understanding of marketing as a process. Marketing research specifies the information required to
address these issues, designs the method for collecting information, manages and implements the data
collection process, analyzes the results, and communicates the findings and their implications
The definition contains a dual but consistent view of marketing research: as an organizational function
and as a process of producing useful information for decision makers. The first perspective indicates
1
http://www.marketingpower.com/AboutAMA/Pages/DefinitionofMarketing.aspx retrieved on October 16,
2012.
4
that marketing research supports the market mechanism of allocating resources and integrating supply
and demand. As such, it serves the company to spot market occasions and threats, not apparent on
the surface, and once they have been identified it provides further insights to help find adequate
solutions. The marketing problems – including both threats and opportunities – can encompass a wide
range of topics calling for developing and shaping whole new marketing programs, or focusing on
specific elements of the marketing mix as well as various monitoring and probing activities.
The second approach to understanding marketing research paints it as a process of providing useful
information consisting of several interdependent sequential stages in which a research problem is
being specified, a research method designed, data collected, analyzed and interpreted and finally a
report written up and presented to the decision maker. It is not accidental that such a work-flow is
mentioned in the definition as following it is one of the critical prerequisites for obtaining good quality
actionable information, that can actually help managers solve their dilemmas.
Rationale for marketing research
What is the purpose of marketing research and what affects its utility to decision makers? Obviously it
all comes down to the nature of decision and its context.
Marketing research is essentially conducted to take out some of the risk from decision making. If no
risk was involved there would be no need for research since its value to the company would be
negligent. On the other hand, the utility of a research project is greater the less certain are the
outcomes of the decision and the more doubts managers have regarding the best course of action.
The factors that increase uncertainty and decision risk - and thus make research more useful - include:
importance of the decision to managers in terms of the amount of resources involved and the
potential negative and positive consequences for the company,
non-routine nature of the decision and hence insufficient knowledge of its background and
possible outcomes
complexity of the decision determined by the number of variables under manager’s control as
well as those factors that are non-controllable but potentially affecting final outcomes.
Given the above, it is understandable that decision makers stand to benefit the most from research
when faced with problems which concern substantial amounts of money, can potentially affect large
5
parts of the company, are only loosely related to routine business situations, are multifaceted and
encompass multiple variables that can be manipulated. As an example of such a situation could serve
the question of whether a new product should be introduced and if so what features should be offered
and what distribution, advertising and pricing policies ought to support it.
A significant consideration in deciding whether a marketing study is necessary is time. Some
management problems are about taking advantage of unexpected opportunities or remedying concern
situations when the best effects can be achieved the sooner the decision is made and its consequences
implemented. The concept of the relationship between risk, time and the amount of information
acquired in a research project was depicted in Figure 1.
Figure 1: Relationship between the risk of decision making, time and the amount of information
acquired in a research project
Source: Own elaboration based on Mazurek-Łopacińska K.: Badania marketingowe: podstawowe
metody i obszary zastosowań, Wydawnictwo Akademii Ekonomicznej w Poznaniu, Poznań,
1999.
Total risk
Risk of
decision
delay
Risk of
uncertainty
Risk involved in decison making
Time elapsed and amount of acquired information
6
In the circumstances illustrated in the picture, the risk of making decision is decreased by additional
insights from research studies and increased with passing time: the two opposing sources of risk
(uncertainty and delay) are represented by blue curves while the total risk is plotted in red. As such,
decision makers need to balance the two counteracting factors and determine how long they can
afford to wait for the research outcomes. Also, it is important to note, that usually when a new
management challenge is being faced the most valuable are initial research activities and each
consecutive study performed provides less and less new insights. Hence, it can be said that the
marginal utility of each additional piece of information from a research study is diminishing, as the
information garnered later contains more and more overlapping elements with what is already known.
Finally, although appropriate research can considerably decrease uncertainty involved in decision
making, thereby improving prospects for successful outcomes, it will never entirely eliminate the
whole risk. The business environment is basically too complex and dynamic, and capabilities of
research tools in social sciences too limited to make it happen. Besides, if marketing research was able
to provide certain and unequivocal recommendations managers will not be necessary to take
decisions. In reality, though, this is obviously not the case and an important part of managers’ role is
taking reasonable risks by making decisions based on information from research, their own experience
and subjective assessment of the situation.
Two basic types of marketing research
At the most general level, it is useful to distinguish between two types of studies: problem
identification research (also known as exploratory research) and problem solving research.
Problem identification research is undertaken to find out potential sources of threats and
opportunities to the company that are not apparent on the surface. The studies of this kind are
exploratory in nature (frequently they are termed “exploratory” instead of “problem-identification”)
and usually employ relatively unstructured research techniques such as desk research, focus-group
interviews and depth interviews. Studies of this type often attempt to identify trends and patterns in
marketing environment that could be used by managers as a basis for shaping marketing strategies. In
general, the recognition of economic, social or cultural trends, such as changes in consumer behavior,
may point to underlying problems or opportunities. Problem identification research typically is only an
introduction to further more detailed analyses, and as such rarely provides actionable conclusions that
could be readily used in decision making but rather indicates directions for further research. For
example, a study could alert to declining market potential, implying that a firm is likely to have
7
difficulties achieving its growth targets, however to obtain more detailed insights about how to best
avoid stagnating sales it is necessary to perform a follow-up problem solving research.
On the other hand, problem solving research is relevant when the essence of a decision problem is
known but decision makers are unable to solve the problem with their current state of knowledge. In
contrast to the problem identification research, this type of inquiry is characterized by precise
formulation of research objectives and usually calls for more formalized, quantitative methods
including surveys, experiments and observations. As a rule, the problem solving research is more
expensive and time consuming in comparison to exploratory studies.
Types of problem identification and problem solving research are given in the table.
Table 1: Categories of problem identification and problem solving research
Problem identification research
Problem solving research
Market potential research
Segmentation research
Market share research
Product research
Image research
Pricing research
Market characteristics research
Promotion research
Sales analysis research
Distribution research
Forecasting research
Business trends research
Source: Malhotra N.K., Briks D.F.: Marketing Research: an Applied Approach, Third European Edition,
Pearson Education Limited 2007, p. 11.
Bearing in mind that problem solving studies in comparison to exploratory research tend to be more
complex, more expensive, and in general more important for decision making, the following table
contains more specific examples of marketing research studies for each of the five categories from the
earlier table.
8
Table 2: Typical examples of problem solving research
Segmentation Research
determine basis of segmentation
establish market potential and responsiveness for various segments
select target markets and create lifestyle profiles, demography, media and product image
characteristics
Product Research
Promotional Research
concept tests
optimal promotional budget
optimal product design
sales promotion relationship
package tests
optimal promotional mix
product modification
copy decisions
brand positioning and repositioning
media decisions
market tests
creative advertising testing
control store tests
claim substantiation
evaluation of advertising effectiveness
Pricing Research
Distribution Research
importance of price in brand selection
type of distribution
pricing policies
attitudes of channel members
product line pricing
intensity of wholesale and retail coverage
price elasticity of demand
channel margins
response to price changes
location of retail and wholesale outlets
Source: Burns Alvin C., Bush Ronald F.: Marketing Research: Online Applications, 2005, Pearson
Education Inc., Upper Saddle River, New Jersey
From the table it is evident that the topics of the problem solving studies are linked to management
problems that arise in the four different areas of marketing mix and segmentation. However, before
problem solving studies are initiated the need for them is frequently demonstrated by exploratory
research involving various, more general market analyses.
Process of marketing research
As was indicated earlier, marketing research takes the form of a sequential process entailing several
distinctive steps or stages. To obtain high quality results, i.e. information that is relevant, accurate and
reliable, research specialists need to work on their project by following the specific stages in the right
order as well as spending the adequate amount of time on each one of them. Going through certain
stages – especially those at the beginning of the process - in a rush or omitting them entirely is a simple
recipe for a flawed research design producing meaningless or confusing data.
Even though the definition of marketing research lists only five steps in the research process: (1)
specifying the needed information, (2) designing methods for collecting information, (3) implementing
9
data collection process, (4) analyzing the results and (5) communicating the findings; for practical
purposes of effectively guiding work on a project it is beneficial to have a more detailed specification
of the process. For that reason the further discussion of marketing research will be based on a more
specific framework put forward by Naresh Malhotra.
2
As seen in the following chart it contains six
broad stages.
Figure 2: Stages in Marketing Research Process
Source: Malhotra N.K., Briks D.F.: Marketing Research: an Applied Approach, Third European Edition,
Pearson Education Limited 2007, p. 7.
The particular stages in the marketing research routine will be discussed more closely in the following
chapters of the textbook. Here only a short characteristic of each step in the process will be presented
to provide readers with a general understanding of what is the nature of a typical research project.
2
Malhotra N.K., Briks D.F.: Marketing Research: an Applied Approach, Third European Edition, Pearson
Education Limited 2007, p. 7.
Stage 6
Report preparation and presentation
Stage 5
Data preparation and analysis
Stage 4
Fieldwork or data collection
Stage 3
Research design formulation
Stage 2
Research approach development
Stage 1
Problem definition
10
1. Problem definition is about specifying first the decision problem and then the research
problem. It is critical for the success of the study since wrong problem definition results in
findings that may not be useful in decision making. Other stages in the sequence are directly
derived from the research problem definition.
2. Developing an approach to the problem involves identifying and specifying components of
the research problem, i.e. research questions and hypotheses. This is also where variables of
interest and their relationships are proposed in the form of an analytical model. The objective
here is to break down the research problem into manageable pieces that will allow for easier
selection of a research design and safeguard against omitting issues that could be critical for
resolving the research problem. The most important output from this phase is a detailed and
comprehensive list of all items of information that are to be collected if a satisfactory answer
to the problem is to be found.
3. Research design formulation. Out of a wide selection of different research methods and
techniques such must be chosen that best suit the purpose of the study and available
resources. In other words, the necessary data must be collected in the most efficient and
reliable way to meet quality criteria and not to exceed limits on time, workforce and financial
resources. At the end of this stage the researchers must not only have a detailed
understanding of how to proceed to obtain the required information but also all the research
tools at the ready as well as the specification of a general population and research sample. The
most typical examples of research tools employed in marketing research are questionnaires
and interview outlines.
4. Fieldwork or data collection. Depending on the research methods chosen in the preceding
phase, this is where trained personnel collects data from assorted secondary and primary
sources. It can take the form of desk research where already published information is
investigated or original data could be gathered through interviews conducted on the phone or
in person. It is also possible for the questionnaires to be filled out by respondents themselves
in one of self-administering modes, e.g. by using traditional mail or answering questions on a
web page. The major tasks of the researchers in this step are coordination and control of the
field force, so that they work honestly and thoroughly to generate accurate and complete data.
5. Data preparation and analysis. Data preparation involves such activities as editing, coding,
transcribing and verifying of raw, unprocessed material generated in the field. Depending on
the attributes of the data its analysis could employ statistical methods or various qualitative
techniques. As a result of the analysis, the data are transformed into meaningful, actionable
11
information. Usually data analysis is focused on finding answers to research questions and
verifying hypotheses put forward in step two of the process.
6. Report preparation and presentation: To satisfy requirements of decision makers who
commissioned the study the findings must be presented in a comprehensible and coherent
manner. To accomplish the objective of effectively communicating the study results a good
report must account for different educational backgrounds and preferences of report users.
Typically, aside from a written document, a PowerPoint presentation ought to be prepared
and delivered verbally.
An important thing to note about the process of marketing research is that for the best results,
regardless of the type of a research project, each stage should be covered in the exact sequence as
presented above. It is true that for more straightforward research activities on topics already familiar
to the researchers the preparatory part of the project (stages 1 through 3) can be shortened, but never
any phase should be entirely omitted. Even though the nature of the decision could be similar to the
ones from the past, the salient characteristics of market environment might be changed (e.g. a
different competitive structure of the market or a shift in consumer preferences) and the internal
situation of the company could be altered as well, precluding certain courses of action and making
earlier research problem definitions irrelevant. For these reasons, meticulous investigation of the
available background information and careful analysis of the research problem and its components is
essential for any new research undertaking. Otherwise the study could produce findings that may not
be very helpful in management problem solving or – if acted upon - could even result in wrong
decisions with potentially disastrous consequences.
Marketing Information System, Primary and Secondary Data
The concept of marketing information system (MIS) refers to the way an organization obtains,
processes and distributes information to managers and other employees who may find them helpful
in making decisions. A marketing information system is an array of people, equipment and procedures
to gather, sort, analyze, evaluate and distribute needed, timely and accurate information to decision
makers.
3
As such, the MIS organizes the flow and storage of information in the company. It is often
based on computer networks and software for semi-automatic processing of data according to
predefined rules and formulas. If its design and implementation fits the organization’s features and
3
Kotler P., Keller K.: Marketing Management, 14 Edition, Prentice Hall, 2012, p. 69
12
suits the managers’ decision making style it can save significant amounts of effort and money, as well
as improve quality and shorten the time for reaching decision.
4
An advanced computer-based MIS can typically include four subsystems: internal records, marketing
intelligence, marketing decision support and marketing research.
5
They are characterized shortly
below:
Internal records system gathers information generated inside the company, which includes
orders, billings, receivables, inventory levels, stock-outs and the like.
Marketing intelligence system is a set of procedures used by managers to obtain everyday
information about pertinent developments in the environment using publicly available data
sources such as newspapers, trade publications, reports and web pages.
Marketing decision support system is a computer system consisting of a database and a set
of methods for accessing and analyzing data using marketing models and other analytical
techniques.
Marketing research system collects information not gathered by the other MIS components.
Its function is accomplished through research projects, addressing specific information needs
of the decision maker.
It is worth noting that only one element of MIS is tasked with implementing or commissioning and
monitoring marketing research projects which yield original data. This original data that is generated
specifically for the research problem at hand through fieldwork is termed primary data. The fact that
its acquisition calls for fieldwork makes it expensive and time consuming and therefore only justified
when other more accessible sources do not satisfactorily fill in the information gaps of decision makers.
The other type of data, that should be examined first for answers to research questions is known as
secondary data. These are various facts and figures and other materials that already exist, since they
were originated for some other purpose then the research study. In case of companies, a great but
often underutilized source of secondary data are sales and costs figures, records of customer
complaints and realized warranties, reports from quality controls and others. Also very useful and
secondary in nature are information resources contained in trade and academic journals and
syndicated research reports sold by professional research agencies to all interested parties. There is a
4
Khodakarami F., Chan Y.: An Investigation of Factors Affecting Marketing Information Systems’ Use, Journal of
Marketing Development and Competitiveness, Vol. 7, No. 2, 2013
5
Kotler P., Keller K.: Marketing Management, 14 Edition, Prentice Hall, 2012, pp. 69-92
13
general practical rule in all research studies which states that before a decision is made to collect
primary data all available secondary sources should be thoroughly checked for relevant information
– this should at the very least allow to better prepare a research design for obtaining primary data and
at best could address research questions and make a follow-up primary data collection unnecessary.
In view of the above considerations it is fair to say that one of the fundamental objectives of a well-
functioning marketing information system is to optimize the use of secondary data, especially those
internal to the company, resulting from its day-to-day operations. This way the managers can often
achieve rich knowledge about market, customers and above all strong and weak points of their
company, at a low cost and in a time-efficient manner.
Distinctions between data, data structures and information
It is important to keep in mind that even though in colloquial language the terms data and information
are often used as synonyms they in fact have a different meaning. At the general level to successfully
address decision problems marketing research has to acquire raw data, apply some form of analysis to
create data structures and then interpret those data structures to arrive at practically usable
information. For a typical research the transformation process goes through the three phases:
6
Gather raw data → Create data structures → Provide interpretative meaning (information)
In such a framework data can be understood as actual firsthand responses that are obtained about an
object or subject of investigation by asking questions or observing actions. These initial responses have
not been analyzed or given an interpretive meaning. Some examples of raw data are (1) the actual
individual responses on a questionnaire, (2) the words recorded during a focus group interview, (3) the
number of customers pass through a specified aisle in a supermarket in an hour, (4) the list of
purchases, by product type, recorded on an electronic cash register at a local supermarket.
On the other hand, data structures represent the results of combining individual raw responses into
aggregates using some type of quantitative or qualitative analysis to reveal patterns, trends or other
regularities. Some examples are (1) the average number of times a specific demographic group of
consumers frequents a move theater in a given length of time, (2) the frequency distribution of the
number of times a certain product was bought in the previous month by 300 consumers, (3) a multiple
6
Hair J., Bush R., Ortinau D.: Marketing Research Within a Changing Information Environment, McGraw-Hill,
Second Edition, 2003, p.34
14
regression equation modeling factors influencing the inclination of consumers to get involved in a
charitable behavior.
Once data structures are obtained the task of researchers is to interpret them in terms of what they
actually mean for decision makers and how they contribute to answering research questions and
resolving the decision problem. To illustrate, once a regression equation was obtained that explains
what demographic, attitudinal and behavioral factors affect people’s tendency to get involved in
charity, a charitable organization may be interested in revising their promotional strategy to affect
different segments of consumers with changed advertising copy. These recommendations suggesting
changes to segmentation, promotional and other marketing policies for a specific organization are
information.
15
Chapter 2: Conceptual framework of research study
Defining decision problem
A problem can be described as a general question for which there is currently no answer. In the
common language the term “problem” has clear negative connotations but it should not be considered
synonymous with difficulties and troubles as it can also concern beneficial opportunities and unique
occasions – what is problematic here is the lack of knowledge about what solution to employ and what
course of action to follow.
Specifying decision and research problems are arguably the most important tasks in the marketing
research process. Even so, they are often treated with insufficient attention or entirely omitted by
inexperienced researchers. When faced with a problem, it is tempting to start searching for an answer
immediately. This tendency is commonplace and stems from the belief that decision makers
understand the source of a problem and the answer is rather self-evident. It is only natural that with
such an attitude the introductory, conceptual phases of the research process are treated very shortly,
or skipped altogether, and the main effort is focused on designing research tools, data collection and
analysis. However, such a manner is usually not efficient and does not save time nor money as the
first cursory diagnosis of the decision problem tends to be incorrect. Moreover, the true problem may
remain unsolved even if no more mistakes are made and a research design, data collection and
analysis are performed spotlessly - all because wrong questions were asked at the outset of the
project. In marketing research the old saying about a problem well defined being half solved is so very
true.
As a first step researchers are required to define a decision problem, which on the surface seems to
be easily achieved by just simply asking the decision maker. However, what is often obtained this way
may be superficial and not truly adequate to the actual challenge facing the company. Therefore the
important role of the researchers early in the project is to assist the decision makers in determining
whether what they are anxious about is really the problem or just a symptom of an as yet unidentified
problem. The difficulty involved in that task is aptly illustrated by the iceberg metaphor which is
presented in the following figure.
16
Figure 3: The iceberg principle in Marketing Research
Source: Hair J., Bush R., Ortinau D.: Marketing Research Within a Changing Information Environment, McGraw-
Hill, Second Edition, 2003, p.67
The iceberg principle implies that, just like only 10% of an iceberg is visible above water, it is often just
a small fraction of the true decision problem that managers are aware of. In fact, what managers
consider to be the real problem may merely be only observable, most apparent symptoms. The
symptoms could be direct causes of the managers’ concern about a market position and future of their
company and they may be determined to eliminate them altogether or at least reduce their severity.
The trouble is that, just like with symptoms of illnesses in humans, they are often signs of more
fundamental dysfunctions and to do away with them for good the managers should not try and
directly influence symptoms but rather affect their underlying causes. Those causes are often not
easily noticeable and could be likened to the submerged part of the iceberg. What the chart presents
as above-the-water signs can be treated as market performance metrics measuring various kinds of
competences of the company like product features, brand image, efficiency and effectiveness of the
distribution system and others corporate skills. As such, to improve marketing metrics the firm’s skills
and competences should be studied and refined. In view of the above, the role of the researcher is to
help the manager find the factors that probably provide the best opportunities for augmenting the
17
company’s competitive position, phrase them as a coherent research problem and design a study that
will most likely provide adequate information to guide management decisions in this area.
To avoid troubles with wrong assumptions about causes of problematic situations – just like the iceberg
analogy implies - it is useful to employ the so-called critical thinking. Critical thinking can be considered
as a process containing the following three steps:
7
1. Identifying the preexisting assumptions held by company employees regarding the cause of
a problem or a potential opportunity.
2. Using internal research data to examine whether these assumptions are accurate and based
on facts.
3. Exploring new ideas about the actual source of a problem and its possible solutions
Of the above, the most crucial is the second step where critical thinking process often stops and
assumptions are being uncritically accepted without being questioned. The reason for this automatic
acceptance lies in common patterns of thought among company employees and the natural desire
most people feel to comply in attitudes and behavior with accepted standards or norms that each
company possess.
If everyone in a company tends to view the company’s products, their consumers and the external
world in the same way, it is difficult for researchers to argue against these beliefs. However, it is these
common patterns of thought that can cause the company’s problem and its solution to be self-evident.
These shared thought patterns can also keep businesses from seeing opportunities that can be
explored by using research.
8
Applying critical thinking to find out the assumptions held in the company, validate them and
investigate new ideas about possible causes and solutions requires the researchers to collect and
analyze data even before the main part of the project is initiated. At the problem definition stage it is
vital to thoroughly look into secondary data, both internal and external to the company, and collect
some additional material from primary sources through interviews with decision makers and other
employees knowledgeable about the problem and its context.
With regard to the secondary data the following sources are usually of significant interest:
previous research results
sales records
7
Kolb B.: Marketing Research: A Practical Approach, SAGE Publications Ltd., 2008, p. 38.
8
Ibidem
18
customer complaint information
customer databases
product service requests
catalogue orders
website hits
The easiest available kind of experts are employees of the company which commissioned the study.
Involvement of internal experts is immensely valuable given that top managers’ knowledge about the
context of the decision problem may be incomplete and could be distorted as a result of information
flowing from the operations level, where it is usually originated, up through several layers of the
organizational structure. Thus, experts who should be interviewed by the researcher ought to include
lower level managers and non-management employees tasked with performing various functions close
to the area where the phenomenon that requires top management’s attention was first observed. The
interviews with them should provide information augmenting and verifying facts already obtained
from the executives. When the extra information is needed, beyond what was already revealed by the
decision makers or found in secondary sources, of particular benefit could be accounts provided by:
sales force employees
production managers
financial analysts
marketing personnel
It is crucial to make the contacts with decision makers as effective as possible. These are people whose
time is seriously constrained and it may well turn out that only one relatively short meeting is possible
at the initial phase of the project. So getting well prepared for the meeting is critical. A good practice
is to thoroughly analyze the company’s web page and those of its major competitors. If a trade portal
exists for the firm’s industry it should also be carefully studied, especially the competitive structure of
the market and major trends affecting the rivalry of incumbent businesses. Next, if the general theme
of the research is known before the meeting it is helpful to skim through 2 or 3 recent academic articles
available from one of web repositories (e.g. www.emeraldinsight.com). After such preparations it is
likely that the decision maker’s time will not be wasted with trite and obvious questions, and the
researchers will make a good impression as professional and knowledgeable individuals, which should
go a long way promoting harmonious and productive future cooperation.
19
When in a meeting with a decision maker, it is important to probe the following issues which
collectively make up what is called the problem audit:
9
1) The events that led to the decision that an action is needed or the history of the problem –
As was already pointed out, decision makers can often come to hasty conclusions so it is good
to know what were the actual circumstances when the marketing problem was first noticed.
Knowing the roots of the problem can result in suggesting insightful solutions, additional to
the ones proposed by the manager.
2) The alternative courses of action available to the DM – Researchers should collect only this
information which directly pertains to any of the pondered courses of action. If some solution
is apparently unacceptable to the DM it is counterproductive to try and collect data on its
feasibility - it is a good habit to keep the scope of the project as focused as possible to
investigate only a limited number of issues but with a great amount of detail. It is sometimes
not apparent if a given action plan – which might have some virtue for the company – will be
considered workable by the manager. It is often due to discrepancy between objectives of the
manager and those of the company, or rather shareholders. Shareholders are mostly
interested in the company increasing its value and managers may sometimes be unwilling to
take actions which will maximize this objective but will result in significant staff lay-offs or risk
alienating valued channel partners. Essentially, personal priorities and commitments may
make managers opt for an option that is suboptimal in light of increasing shareholders value
but allows them, for instance, to enhance or retain their own power and prestige, increase
their wealth or protect employees with whom they have friendly relationships.
3) Criteria that will be used to evaluate the alternative courses of action – Different action plans
can be chosen based on different sets of evaluation standards. As such, managers who prefer
maximizing short term profitability will likely take a different decision than managers who are
more concerned with increasing market share, enhancing prospects for long term profit or
increasing brand equity. It is a task of the researcher to design the study so that it provides
information on the metrics necessary to assess available alternatives in line with managers’
criteria. Disregarding this aspect of the decision problem is likely to produce findings that may
not be entirely useful.
4) The information that is needed to answer the DM’s questions – It may seem unnecessary to
ask directly about what information the manager requires after examining thoroughly the
9
Malhotra N.K., Briks D.F.: Marketing Research: an Applied Approach, Third European Edition, Pearson
Education Limited 2007
20
three earlier points. However, as it was already mentioned, some managers may not have a
clear vision of the possible ways of solving the problem. They may be unable or unwilling to
articulate all considered options. To avoid miscommunication it is worthwhile to ask straight
away about required insights, as it may help to verify whether the list of analyzed solutions is
complete
5) The manner in which the DM will use each item of information in making the decision – Once
researchers believe they compiled a more-or-less complete list of information items that the
study must provide they should ask the DM how each piece will help them in solving the
decision dilemma. Such an exercise may reveal that some variables are not necessary since
they are rather useless or provide similar information as some other variables. As a result the
research project may become more concise and coherent.
6) The corporate culture as it relates to decision making – It is about who will be using the
research findings: only a single executive or a wider group of managers, which depends on
whether the decision making process is rather centralized and authoritative or it follows along
more democratic, collaborative lines. If it is asserted that more people will be involved in
decision making it is quite a good idea to talk to them as well with a similar list of questions in
hand.
Formulating research problem
The research problem is a refined and usually narrowed down form of the decision problem. It is
expressed not in terms of what is to be done to resolve a management dilemma but rather by
stipulating types of information that should be collected to resolve the decision problem. As a rule,
working on a research problem statement involves doing away with parts of the decision problem that
in all likelihood do not contain a viable solution because the company lacks resources, time is scarce,
the manager dislikes certain options or there are some other obstacles. While developing a research
problem it is important not to frame it too broadly or too narrowly. An overly broad definition,
including areas of marketing policy that are unlikely to produce a satisfactory solution, forces
researchers to spread their resources to generate data over a large number of issues often preventing
an in-depth inquiry of them. As a result the analysis may be too superficial to provide conclusive
answers or – if resources are not an issue – the project may come out much more costly than it should
be. By contrast, a too narrow definition risks omitting less obvious but potentially very effective
marketing strategies.
21
The task of defining a research problem is best conducted in a systematic manner, with each
subsequent step allowing to outline more precisely the parts of the decision problem that are most
worthy of the researchers’ attention. The way to redefine a decision problem into a more researchable
statement was detailed in the table.
Table 3: Transformation of decision problem into research problem
Source: Proctor T.: Essentials of Marketing Research, Fourth Edition, Pearson Education Ltd., 2005, p.
47.
From the table it is evident that the original array of marketing actions to be taken by the company
was considerably narrowed down by taking into account relevant limitations and obstacles. The final
set of possible actions can be summarized by several variations of the research problem statement,
e.g. “What new markets and consumer segments should be targeted by the company with current
products to achieve the highest grow of sales and profits within one year period?” .
In light of the above it is worth noting that a single decision problem could be translated into numerous
research problems centered on different possible ways of improving the firm’s market position. The
specific formulation depends on many factors including business environment conditions, internal
resources and skills and preferences of researchers and decision makers. Below as an example there
is a set of decision problems and corresponding research problems. In line with the above discussion
22
it is possible to come up with alternative research statements that would be more adequate for
different external and internal circumstances.
Table 4: Examples of decision and research problems
Decision problem
Research problem
Should the company enter a new market
segment with its existing product?
How many customers in the segment will be
interested in buying the product at different
price levels?
Is it necessary to change the advertising
strategy?
What is the effectiveness and efficiency of the
current advertising instruments in comparison
to the ones utilized by competitors?
Should the firm change the positioning strategy
of its major brand?
What are the attitudes towards the brand of
customers in segments the company targets
relative to competitive products?
Source: Own elaboration
Theory and Analytical Models in Marketing Research
Once the decision problem has been properly formulated and before deciding on the final form of the
research problem, it is very useful for the researcher to familiarize themselves with previous studies
on the same or similar topics. The body of previous research, often found in peer reviewed academic
journals, constitute theoretical underpinnings of the research problem. It may seem that for non-
academic studies, which are most marketing research projects, knowing theory may not provide too
many benefits – after all theory is commonly considered impractical. In fact it could not be further
from the truth, since as the old saying goes “there is nothing more practical than a good theory”. For
the marketing researcher theory provides ample information about previous research methods and
designs employed to study the topics of interests, in particular:
1) Specification of variables accounted for in earlier research projects
2) Assumed and validated causal relationships between variables
3) Ways of operationalizing variables of interest with measurement scales if survey method is to
be used
4) Population definitions and sampling methods
5) Employed analysis methods, particularly statistical techniques in quantitative studies.
Familiarity with previous studies implies knowing their shortcomings and advantages, which in turn
allows to design a new study in a manner that avoids previous mistakes and leverages tried and good
solutions.
23
Apart from guiding a research design, the other area where theory may prove to be exceptionally
beneficial is the interpretation of data structures – for instance the claims about cause and effect links
between variables may be more believable if besides statistical correlation the researcher is able to
provide some proof of causality from previous projects.
A good source of theory in marketing are scholarly articles published in peer-reviewed journals, which
could be found in university libraries or on the Internet in special repositories; two of the most popular
are located at www.emeraldinsight.com and www.ebscohost.com.
Variables that were identified as relevant for the decision problem at hand, following talks with
decision makers, other employees and a theory review, should be arranged in an analytical model.
Analytical model, like any other kind of model, is an abstraction of the way we choose to perceive a
specific part, function, property or aspect of reality. It is a representation of a ‘system’ that is
intentionally constructed to study some aspect of that system or the system as a whole.
10
Models
are always simplified representations of a specific real-life phenomenon with some components
emphasized and some entirely omitted to make it easier to understand and analyze. In case of
marketing studies a model in its most basic form can include a set of variables pertinent to explaining
a certain phenomenon and their interrelationships. Usually, decision makers are most interested in
discovering relationships between variables allowing them to find levers that when manipulated
should enhance firm’s competitive position and improve various marketing and financial metrics. As a
result in many instances the general objective of a marketing study is to identify variables that could
serve as levers and verify relationships between them and other dependent variables so managers
could effectively and efficiently influence the market position of their companies. In truth, the task is
not easy and often the answers are inconclusive or relationships not too strong so to effect a
discernible change not one but several variables should be manipulated. With that in mind, researchers
can improve the chances of success of such a study if they develop a detailed and well though-out
analytical model of variables and their links. The popular form of the model is a graphical one with
circles representing variables and lines denoting associations. Later on, once the fieldwork is done and
data are collected, the graphical model can be further expanded in a mathematical form, for example
as a regression equation with one of many available statistical software. Below is an example of a
model depicting factors affecting consumer loyalty.
10
Joniker J., Pennink B.: The Essence of Research Methodology: A Concise Guide for Master and PhD Students in
Management Science, Springer-Verlag Berlin Heidelberg 2010, p. 43.
24
Figure 4: Graphical model depicting factors influencing loyalty behavior of customers
Source: Olsen S., Tudoran A., Brunso K., Verbeke W.: Extending the prevalent consumer loyalty
modeling: The role of habit strength, European Journal of Marketing, vol. 47, no. 1, 2012.
The model involves three different factors that may have direct or indirect effects on consumer loyalty
behavior understood as repeat shopping. If the study were to follow the model, all factors would have
to be operationalized by separate multi-item scales encompassing several individual variables
representing different aspects of each factor in question. Individual arrows signifying relationships
between pairs of factors could suggest the content of research questions and hypotheses with typically
one research question and corresponding hypothesis addressing one relationship.
Marketing researchers are well advised to draft up models of relationships early in working on their
project as besides ensuring that all suitable variables are included in the study, an analytical model is
instrumental in developing the right set of research questions and hypotheses.
Research questions attempt to capture the meaning of all important components of a research
problem. By answering a research question a solution is found to a part of the research problem. To
solve the research problem all the research questions must be addressed. Thus, the correct set of
research questions should stipulate every meaningful aspect of the research problem and if
researchers are in a hurry and work in a haphazard manner, it may turn out that the research questions
do not attend to all critical components of the problem; consequently, it may not be possible to solve
it conclusively and the effort spent on organizing the study will be wasted.
A hypothesis is a possible – usually the most likely – answer to a research question. As such, it can
be thought of as an educated guess about the answer to the pertinent research question. Hypotheses
are the researcher’s attempt to explain the phenomenon of interest, and that explanation should
involve prediction about variables being studied. These predictions are then tested by gathering and
25
analyzing data and the hypotheses can either be supported or falsified (refuted) on the basis of data.
11
A well formulated hypothesis should be testable, which means that it ought to be possible to specify
the conditions that must be met to consider the hypothesis false. If such conditions are not clear or it
is extremely unlikely that they will be met, such a hypothesis is not specified rightly. Another typical
mistake is forming a hypothesis using vague terms for variables: to avoid this error every term should
be precisely defined in a way that allows measuring the level of the pertinent variable, i.e. terms
should be operationalized. Providing clear and precise definitions for all important terms and variables
ensures that every person involved in the project understands its important elements in the same way
and it also improves the reliability of the study as it permits replication of the design by other
researchers.
If researchers decide to work out a set of hypotheses they must remember that each individual
hypothesis should be independent, that is if one statement is verified it should not follow that some
other proposition is automatically corroborated as a result. In applied marketing research decision
makers quite often do not require listing hypotheses: instead, they are well satisfied with research
questions. It is worth remembering that it is easier to define research questions than hypotheses, as
the latter requires more specific knowledge about the studied phenomenon so that it would be
possible to propose the most likely answer to the research questions. If a study is to have scientific
qualities hypotheses are necessary.
To make the discussion of correctly formulating the conceptual framework of a research study less
abstract and easier to comprehend, it is worthwhile to provide a few illustrative examples. The
subsequent table contains three combinations of decision problems, research problems, research
questions and hypotheses. It is important to note that each research problem was assigned just one
research question for the sake of conciseness – in reality more questions and matching hypotheses
should have been developed to address all aspects of the research problems.
11
Marczyk G., DeMatteo D., Festinger D.: Essentials of Research Design and Methodology, John Wiley & Sons,
Inc., 2005, p. 8
26
Table 5: Examples of conceptual frameworks of three research studies
Decision
problems
Research
problems
Research
questions
Hypotheses
Situation 1
What can be done
to energize new
markets and
create more
favorable attitude
towards
swimming
association?
What are the
reasons why
families may
choose to join or
not join a “swim
club”?
How do the type
of amenities and
pricing relate to
family attitudes
toward a swim
facility?
Child-friendly
pool designs are
positively related
to favorable
attitudes toward
the facility.
Flexible pricing
policies are
positively related
to favorable
attitudes toward
the facility.
Situation 2
What product
features can be
enhanced and
emphasized to
improve
competitive
positioning of the
new product in
B2B market?
What actions may
overcome the
objections
(switching costs)
of B2B customers
toward adoption
of the new
product?
What are the
strongest factors
that lead to the
perception of
high switching
costs?
Perceived
difficulty in
learning how to
use the new
device is related
to switching
costs.
Price is positively
related to
switching costs.
Knowledge of
new product is
positively related
to switching
costs.
Situation 3
How can more
consumers be
encouraged to
come to the
microbrewery
and try out its
products?
How situational
factors influence
beer
consumption and
consumer
attitudes toward
beer products?
Do situational
factors, including
time of day, food
pairings and
environmental
factors, relate to
taste perceptions
of beer?
Microbrewery
beer is preferred
when consumed
with food.
An exciting
atmosphere will
improve
consumer
attitudes toward
the
microbrewery.
Source: Based on Zikund W., Babin B.: Exploring Marketing Research, Ninth Edition, Thomson Higher
Education, 2007, p. 114.
27
It is worth noting that any of research approaches employed in marketing research, if done correctly,
follows what is known as the scientific method. In a very general sense, the scientific method is a way
of acquiring knowledge based on empirical evidence that relies on gathering data from direct,
systematic observation or experimentation and using it to test hypotheses. This idea is illustrated in
the below figure.
Figure 5: Traditional model of scientific inquiry
Source: Babbie E.: The Practice of Social Research, Thomson Wadsworth, Eleventh Edition, 2007, p. 39.
The above diagram depicts a model of the most typical way of scientific inquiry known as deduction,
which is also dominant in marketing and management sciences. It takes as a starting point existing
theory, which - supplemented by researcher’s own observations and experience - leads to formulating
a sometimes vague research problem or question. Then the question is fleshed out with further details
and all relevant concepts are defined in operational terms, so that it is known what variables must be
observed in what population to resolve the question. The next step is formulating hypothesis, usually
as a statement containing a prediction about the relationship between two variables. The hypothesis
is further specified and operationalized using mathematical notation, with lower case letters
representing measurable indicators of variables that are in turn denoted by upper case characters. The
28
sequence completes with the use of statistics to ascertain if the proposition should be rejected or
accepted.
The fact that marketing research adheres to the more general rules of conducting scientific studies is
of great practical importance as it permits application of most methods and techniques used in this
discipline also in other areas of social science, such as sociology or demography.
The notions of research problem, research questions, analytical model and hypotheses are best
illustrated with a comprehensive example. All elements of a research conceptual framework were
either stated explicitly or could be easily inferred in a study on compulsive consumer purchases by
Shohan and Brencic.
12
Compulsive buying can be described as “consumers’ repetitive shopping, at
times excessive, because of boredom, tension or anxiety”.
13
The research problem that was addressed
in the study was “What demographic and behavioral characteristics are antecedents of compulsive
buying in consumers”. Here, the term “antecedents” was used since it implies a relationship but not
necessary a casual one – purely statistical correlations were also of interest, as they could be used for
predicting consumer behavior. Moreover it was safer to talk about antecedents and not causes, since
survey design that was employed cannot conclusively prove causation – for that purpose a more
suitable technique are experiments. Seeing that the research is academic in nature its primary purpose
was not solving any specific management problem but rather obtaining findings that would fill in the
gaps in the theoretical knowledge of marketing. However, the study may also have important practical
consequences and as such it is possible to come up with propositions of practical decision problems
that could be helped with the research results and are coherent with the research problem. One such
decision problem statement could be: “How to identify consumers with aptitude for compulsive
shopping?”. This could be used to either target such demographics of consumers with tailored
marketing programs (not necessarily an ethical purpose) or – considering that compulsive shopping is
in fact a pathological behavior – better identify those who might require psychological assistance
(could be useful for a charity or governmental organization).
After reviewing the available academic literature on compulsive shopping the authors developed a
simple graphical model of variables and their likely interrelationships and then came up with a set of
research questions and hypotheses. The model, research questions and hypotheses are given next.
12
Shohan A., Brencic M.: Compulsive buying behavior, Journal of Consumer Marketing, vol. 20, no. 2, 2003, p.
127-138.
13
Ibidem
29
Figure 6: Graphical model of relationships between inclination for compulsive purchases, selected
behavioral characteristics and gender
Source: Shohan A., Brencic M.: Compulsive buying behavior, Journal of Consumer Marketing, vol. 20,
no. 2, 2003, p. 127-138.
Research question 1: How consumers’ inclination to make planned purchases is associated with
proclivity for compulsive buying?
H.1: The higher the individual’s frequency of unplanned purchases, the higher their compulsive
purchase tendency.
H.2: The higher the individual’s tendency to buy items off their product list, the higher their
compulsive purchase tendency
Research question 2: How consumers gender is linked with the tendency for compulsive buying?
H.3: Females will exhibit lower levels of compulsive purchase tendencies than males.
In this simple research framework several things are worth noting:
Analytical models do not need to be complex: depending on the scope of the study they may
only encompass several elements.
The model in graphical form was devised before hypotheses: hypotheses were formulated
afterwards to reflect the relationships mapped out in the model.
30
With the exception of gender, the circles in the model represent constructs rather than
individual variables, i.e. to measure each construct it is necessary to use a scale made up of
several individual variables.
Some research questions are more general and require more than one hypothesis to find
conclusive answers.
All hypotheses make predictions about relationships between pairs of observable variables or
constructs – this is the most common type of hypothesis.
Every term used in each hypothesis is specific enough and possible to operationalize, so
pertinent variables can be measured using the questionnaire and survey method and their
relationships can be analyzed with statistical tests; as such all hypotheses are testable and so
defined correctly.
There may be one minor problem with the hypotheses put forward by the authors of the
article: it is likely that H.1 and H.2 are not entirely independent. Common experience may
suggest, that consumers who more often use shopping lists probably are less likely to make
unplanned purchases. If that indeed were the case, falsifying H.1 increases the probability of
supporting H.2. Whenever possible, such relationships between hypotheses should be
avoided.
31
Chapter 3: Choosing research design
Once researchers have precisely worked out what information they are seeking to obtain, which is
accomplished when the complete conceptual framework of the study is developed including decision
and research problems, research questions and hypotheses, they next need to decide about specific
means to collect data. There is a wide selection of diverse methods for gathering empirical material -
or research designs as they are often termed in literature. Each type of research design is characterized
by the kinds of research questions it can best handle, amounts of resources it requires, analysis
techniques that can be utilized and also the nature and quality of data it can produce. Needless to say,
the decision to choose one specific research design instead of others should flow from these
considerations. In commercial marketing research it is especially vital to find such an approach that
allows answering all research questions pertinent to the research problem in a reliable and valid
manner and – at the same time – keeps to a minimum the use of resources (i.e. time, effort and money)
that will be spent to complete the inquiry.
The objective of this chapter is to provide an overview of the most common types of research designs
as they are used in marketing research to facilitate adopting the best data collection technique with
regard to research questions and resources assigned to the project.
The most common classification distinguishes between qualitative and quantitative studies and will be
discussed first. The other equally important taxonomy which comprises exploratory, descriptive and
casual designs will be considered later on.
Qualitative and quantitative studies
Qualitative and quantitative studies differ in many facets of research process from objectives and
nature of gathered data through the number of studied units, analysis techniques and the ways of
drawing conclusions. The most obvious characteristic that distinguish the two approaches is that
qualitative studies involve only small amount of units (i.e. people, organizations etc.) but the analysis
is very thorough, yielding rich, in-depth data on each case. By contrast, with quantitative studies
samples are numerous, running into hundreds of units, but insights about individual objects are limited
since the research instruments employed are rigidly structured, mostly made up of closed-ended
questions with predetermined answer categories. As a result, statistics can almost only be used for
analyzing quantitative data while qualitative empirical material needs to be processed in a much more
subjective manner aimed at finding patterns in predominantly textual output.
32
The crucial differences between the two methods are summarized in the table with a more thorough
discussion below.
Table 6: Major differences between qualitative and quantitative research methods
Qualitative research
Quantitative research
General assumptions about
research inquiry
Often follows
phenomenological paradigms
Almost always adheres to
positivist or neo-positivist
paradigm
Objective
Usually exploration and in more
academic oriented studies
theory development based on
inductive approach
Usually theory verification
based on deductive approach
Number of research units
One to several
Several dozen to several
hundred
Method of selecting research
units
Purposeful sampling
Representative sampling
Data collection
Flexible, usually multiple
sources
Rigid, usually one source
Data analysis
Rarely applying statistical
techniques
Relying on statistical
techniques
Generalization
Analytic
Statistical
Source: Own elaboration
Deductive and inductive strategy of theory building
Academic literature on research methodology in social sciences often bring up the issue of research
paradigms which are a kind of philosophical perspectives attached to specific methodologies and
techniques. More specifically research paradigms are defined as very general conceptions of the
nature of scientific enquiry representing coherent sets of beliefs about the reality (ontological aspect)
and the process of acquiring knowledge (epistemological aspect).
14
14
For a more extended definition of research paradigm see Creswell J.: Qualitative Inquiry and Research
Design; Choosing Among Five Traditions, SAGE Publications, Inc., Thousand Oaks, California, USA, 1998, p. 74
33
Probably the most important issues which distinguish research paradigms are:
1) Whether reality and – by consequence – truth are subjective or objective?
2) What is the dominant way of theory building (induction or deduction)?
These relatively simple criteria allow to identify in social sciences a positivist paradigm – which is
equivalent to paradigm of natural sciences like physics or biology - and a group of non-positivist
research philosophies including constructivism, grounded theory, realism and others. This broad
selection of non-positivist paradigms is known by the general label of phenomenological paradigm, as
– despite sometimes considerable differences - they share a significant number of common
characteristics. The positivist researcher conducting their studies will be striving to discover some kind
of objective truth and will tend to use deductive strategy in theory building. In contrast, non-positivist
scholars will be convinced of subjective and multiple nature of reality, in which there are as many
truths as there are studied persons. Thus, the researcher “needs to report these realities, rely on voices
and interpretations of informants through extensive quotes, present themes that reflect words used
by informants, and advance evidence of different perspectives on each theme.”
15
Such a view also calls
for the informants being studied in-depth and up-close, which makes it more natural to rely on
qualitative rather than quantitative methods in gathering and analyzing empirical data.
In the previous table among many differences between paradigms it was asserted that under positivist
paradigm researchers are expected to “formulate hypotheses and then test them”, while in
phenomenological paradigm ideas are developed through induction of data. These two attributes refer
to deductive and inductive strategy of theory building. Typical for positivism deductive strategy relies
strongly on the previous theory, which is subject to reviewing to find gaps in scientific knowledge and
formulate a set of hypotheses, which – when tested – will allow to fill in these gaps with new insights.
In this sense, deduction means that empirical data are not used to generate hypotheses but to check
if they are true or not. It is quite opposite with the inductive strategy, which - in its extreme mode
characteristic for grounded theory – may involve analyzing some phenomena without any theory
whatsoever and then, step-by-step, building the pertinent theory using only empirical data.
The above description does not necessarily mean that a researcher have to choose only one way of
theory building. It is possible - and often necessary, or at least advisable - to merge both strategies
in one research project. This approach is consistent with the concept called “methodological
15
Creswell J.: Qualitative Inquiry and Research Design; Choosing Among Five Traditions, SAGE Publications,
Inc., Thousand Oaks, California, USA, 1998, p. 76
34
pluralism”.
16
The figure below illustrates the idea of using multiple case study method first to
inductively generate hypotheses and then verify them in a deductive manner. The message contained
in the graph is more universal if the term “case study” is considered in a wide sense to represent an
independent research unit like a person, company, group of consumers etc.
The figure suggests that case studies (but also individual focus groups or depth interviews) can be
used, in inductive mode, to better explore research problem. As can be seen, every successive case
increases theoretical knowledge of the researcher, by making it possible to elaborate preliminary
research questions and hypotheses. In other words, lines of inquiry of every case are based on research
results from all previous cases. At some point – in the graph after the fourth case – the researcher may
decide that theoretical proposals are developed well enough to proceed to the second phase:
verification. It must be stressed that at the end of inductive phase a resulting theory should explain
16
Gill J., Johnson P.: Research Methods for Managers, SAGE Publications, Paul Chapman Publishing, London,
1991, p. 168-169
Extent to which a
prior theory was
used
Number of studied cases
Inductive part of a
research project
Deductive part of a
research project
Figure 7: Example of case study usage in inductive and deductive theory building in a
single research project
Source: Based on Perry C.: Processes of a Case Study Methodology for Postgraduate Research in Marketing,
European Journal of Marketing, vol. 32, no. 9/10, 1998, p. 785-802.
35
investigated phenomena in all studied cases. Verification is based on comparing data from additional
cases to a set of theoretical proposals – hypotheses – generated beforehand. Now it is quite easy to
falsify a hypothesis – just one opposing case will be enough to make the researcher to either modify
the hypothesis to accommodate new data or to discard it altogether. Even if every case provides
affirmative data, which confirm the set of hypotheses, the researcher cannot be absolutely certain that
their theoretical propositions are universally true; however the more cases substantiating the theory
the more likely it is to hold for the larger group of observations. It is, however, impossible to provide
any estimate of this likelihood – for this it would be necessary to move on to the deductive phase and
employ one of quantitative methods with their inferential statistics and generalizability potential, such
a survey, experiment or observation.
The inductive orientation of qualitative research may be useful even for positivism oriented scholars.
Jacques Hamel, for instance, asserts that “before a theory can be validated, it must be constructed. In
other words, a theory or theoretical framework first emerges through the inductive approach of
studying an empirical case or object, not through a deductive process”
17
. It follows then that theories
are at first based on a specific case or object. It is then rational to employ qualitative research to help
generate a set of theoretical propositions and then put them to the test on a large sample of
observations using a quantitative technique.
It must be stressed that not only qualitative method can utilize both inductive and deductive approach.
Quantitative research is usually structured according to deductive strategy but when an objective is
exploration of some problematic area, elements of theory will be constructed in an inductive way, in
which case survey may contain a significant proportion of open ended questions and be conducted on
smaller sample sizes.
18
However, in the opinion of this textbook’s author the use of quantitative
approach for exploratory purposes is suboptimal in comparison to what qualitative methods have
to offer, as the findings probably won’t be so rich an revealing and expended costs and time most likely
higher. Hence, it stands to reason that surveys and other quantitative techniques should only be used
in deductive capacity, with proper representative samples permitting statistical generalizations.
17
Hamel J.: Case Study Methods, SAGE Publications, USA, 1993.
18
Gill J., Johnson P.: Research Methods for Managers, SAGE Publications, Paul Chapman Publishing, London,
1991, p. 97-98.
36
Generalizing results
Among the most distinctive differences between the two types of methodologies is the manner in
which results are generalized. Quantitative research studies involve numerous samples which are
supposed to permit formulating conclusions about the general population from which they were
drawn by the way of statistical generalization. To make it possible, however, two crucial conditions
must be met: the sample size must be large enough and selection procedure must be random. In
literature there can be found many suggestions of random sampling plans which accommodate
characteristics of different sampling frameworks (lists with population elements from which
observation are drawn). Their common feature is the fact that each population member must have a
known (preferably the same) probability of being selected. In this type of generalization conclusions
are made with the help of inferential statistics procedures. They always give a specific measure of error
and accuracy of verifying any given hypothesis, which are known as a level of significance and
confidence level. They indicate the likelihood of the results obtained for the sample not being true for
the whole population (level of significance), and probability that the results from the sample are also
valid for the population (level of confidence). Although this kind of generalization is being rarely
questioned by scientists, the mechanical nature of statistical procedures and typically short and
superficial contact with respondents makes it prone to accuracy errors and - by consequence - drawing
erroneous conclusions about the population. This problem is rather aptly encapsulated by Mark
Easterby-Smith et al. in the following words:
“However any results that show relationships should be examined carefully before claiming causality,
because there are many ways that spurious associations can be produced. And it is worth remembering
that statistical significance does not necessarily imply practical significance or causality, it merely
indicates that there is a strong pattern between the variables under consideration.”
19
Besides errors of judgments, which the above quotation refers to, quality of generalization, or external
validity, can be further diminished by various mistakes, especially in sampling procedure. One such
mistake would be the selection in which not all members of the population have a possibility to be
drawn; this will cause the research results to be biased in favor of opinions and facts provided by those
groups of participants who were most likely to land in the sample. Other typical example of faulty
sampling would refer to a situation of using phone directory as a sampling frame in a telephone survey.
In this case the results could be projected only to the population comprised of the people listed in the
19
Easterby-Smith M., Thorpe R., Lowe A.: Management Research: An Introduction, SAGE Publishing, London,
2002, p. 144.
37
phone directory and not all those who live in the specific area. In this way findings which are - for
example - supposed to describe public support for political parties may very weakly correspond to real
public election results in the given area. What is even more troublesome, this kind of mistake cannot
be easily quantified using statistical techniques.
Entirely different type of generalization is applicable to case studies, and other kinds of qualitative
research. Robert Yin calls it analytic generalization,
20
while Robert Stake labels it petite
generalization,
21
as opposed to grand generalization, which is a different name for statistical
generalization. Analytic generalization has as its goal “to expand and generalize theories and not
enumerate frequencies”.
22
It is a very similar approach to experiments, in which “scientific facts are
rarely based on a single experiment; they are usually based on a multiple set of experiments that have
replicated the same phenomenon under different conditions”.
23
To use replication logic in qualitative
research it is necessary to start with a thorough review of existing theory, which should be used to
construct a pattern of theoretical propositions (hypotheses) to which empirical results will be
compared. If one or more investigated cases (i.e. groups, objects or individuals) provide insights
consistent with the pattern it can be claimed that replication took place. The more cases produce
replications the stronger is the confidence in the new theory elements expressed in the pattern as
hypotheses.
24
Using replication logic quite naturally calls for purposeful (as opposed to random) case
selection with respect to information value of potential research units for the whole project.
Analytic generalization can also change a theory validated by statistical generalization. It happens
when a qualitative study provides a counter-example to the incumbent theory. If the theory cannot
accommodate the exception, then it may be necessary to modify the previous statistical
generalization.
25
However, a positive example will not be able to change existing theory; it can only
increase confidence of scholars in this generalization.
26
In the extreme situation, this type of
generalizing can produce what was called by Karl Popper falsification, which can be illustrated with
the well-known “black swan example”: if there were a theory asserting that “all swans are white” it
20
Yin R.: Case Study Research: Design and Methods, SAGE Publications, Inc., Thousand Oaks, California, USA,
2003, p. 10.
21
Stake R.: The Art of Case Study Research, SAGE Publications, Inc., Thousand Oaks, California, USA, 1995, p. 7.
22
Yin R.: Case Study Research: Design and Methods, SAGE Publications, Inc., Thousand Oaks, California, USA,
2003, p. 10.
23
Ibidem
24
Rowley J.: Using Case Studies in Research, Management Research News, Vol. 25, No. 1, 2002, pp. 16-27.
25
Stake R.: The Art of Case Study Research, SAGE Publications, Inc., Thousand Oaks, California, USA, 1995, p. 8.
26
Ibidem
38
would be enough to have just one observation of a single black swan to falsify this proposition.
27
This
points at how useful can be generalization conducted with only one well selected case.
Some scholars believe that the important benefit of theory created with analytical generalization is its
practical usefulness and understandability for managers. In this extent it is thought to be better than
quantitative approach, which is often blamed for widening the gap between what science has to offer
and what practice really needs.
28
Other considerations in choosing between qualitative and quantitative approach
Regardless of personal preferences for specific scientific paradigm and inquiry process, in my opinion
the two most important issues in determining strengths and weaknesses of research methods are
results validity (internal, external and construct validity)
29
and resources requirements. These
research quality and practicality criteria are strongly context dependent, which basically means that
the nature of research problem and conditions in which the inquiry is to be undertaken will make
alternatively qualitative or quantitative approach, or some kind of mix thereof, the best way to conduct
the investigation.
The complexity of research problem, in terms of the number of variables in studied relationships, is
one of defining conditions for selecting research method. It is quite common practice for quantitative
research methods to be applied where the objective is to search for cause and effect relationship
among a small number of variables; in comparison qualitative approach strives “for understanding the
complex interrelationships among all elements present in a particular case”.
30
When conducting
survey, research problem, hypotheses and research questions have to be translated into a
questionnaire that serves as a research tool. A single question in a questionnaire, containing a single
measuring scale, attempts to gather data about values of only one variable. When a question contains
multiple scales, such as Likert or Staple scale, as many variables are studied as there are scales used.
In my experience, a questionnaire to be an effective data collection tool should not contain more than
27
Flyvbjerg B.: Five Misunderstandings About Case-Study Research, Qualitative Inquiry, Vol. 12, No. 2, 2006, pp.
219-245.
28
Patton E., Appelbaum S.: The Case for Case Studies in Management Research, Management Research News,
Vol. 26, No. 5, 2003, pp. 60-71.
29
A detailed discussion of various aspects of validity can be found in Yin R.: Case Study Research: Design and
Methods, SAGE Publications, Inc., Thousand Oaks, California, USA, 2003, p.36.
30
Patton E., Appelbaum S.: The Case for Case Studies in Management Research, Management Research News,
Vol. 26, No. 5, 2003, pp. 60-71.
39
100 variables, which usually takes up to 6 pages of paper. This rough rule of thumb applies to many
typical research situation when respondents are not particularly interested in answering questions and
treat the interview as a kind of intrusion into their private life. If one were to use a very long
questionnaire it would most likely diminish quality of gathered data, as a significant number of most
uninterested respondents would try to finish interview as quickly as possible by providing the most
straightforward and not thoughtful answers. This may introduce a significant error into collected
empirical material, which will be difficult to estimate by any statistical or other techniques available to
a quantitative researcher. Therefore, I would suggest that when a research problem is considered
complex with more than 100 variables and a sizable portion of potential respondents are likely to be
uninterested and even bored by the research subject it may be a reasonable idea to conduct a few
qualitative investigations relying on focus group interviews, depth interviews or some other adequate
qualitative methodology instead of a survey. Qualitative studies may help reduce large array of
variables by excluding those least pertinent attributes which may make the subsequent questionnaire-
based research more feasible.
The resources factor can be particularly important for researchers catering to business clients,
particularly small and mid-sized companies, where funds and other means that owners or managers
are willing to allocate to the research project are rather scare while the expectation of extraordinary
results is high. Contrary to the all-to-often heard but erroneous opinion, qualitative studies do not
have to be either cheap or quick to complete. If the research entails studying customers and the
company operates in multiple market segments comprising persons with different lifestyles and
socioeconomic statuses it may turn out that each of those divergent customer groups requires a
separate qualitative study, which may be well beyond the allocated budget and deadline. Quite
unexpectedly, in terms of costs and speed a better alternative may be – for example - a CATI survey
(CATI stands for Computer Assisted Telephone Interviewing). However, if research objectives require
formulating creative concepts for product modifications, new promotional campaign or any other
marketing related strategy then quantitative study may be disappointing. The superiority of qualitative
methods in yielding creative insights is beyond doubt, but they lack external validity offered by surveys.
All in all, deciding between qualitative and quantitative designs or some combination thereof is a
critical and sometimes difficult decision, requiring making compromises and trade-offs between costs
and quality always keeping in mind contextual, firm related factors.
40
Exploratory, Descriptive and Causal Research designs
The second line of classification goes across principal objectives of research studies. As such, it is
possible to distinguish three groups of techniques: exploratory, descriptive and causal.
Typically research designs are categorized according to the classification presented in the figure.
Figure 8: Basic classification of research designs
Source: Malhotra N.: Marketing Research: An Applied Orientation, 6th Edition, Prentice Hall, Upper
Sadle River, New Jersey 2010, p. 71
Exploratory studies are undertaken to increase an understanding of obscure research problems and
are often the first step in a wider research initiative including in the later stage either descriptive or
causal inquiries. Serving as a preliminary study, exploratory research is expected to assist in
formulating hypotheses, designing research tools such as questionnaires with particular role in
constructing measuring scales and more precisely defining general population for representative
studies. Exploratory studies are encompassing all qualitative techniques, secondary sources analyses
not involving statistical processing of earlier collected quantitative data (e.g. review of academic and
trade literature on the topic of interest), plus those quantitate primary studies that are based on
samples that are non-representative and preclude statistical generalization. As stand-alone projects
exploratory designs can bring promising innovative solutions to research problems, though without
quantitative validation of the feasibility of possible solutions it is quite precarious to pick one and
implement it in an organization. However, some managers are willing to take the risk of acting upon
exploratory findings, often due to time restraints or insufficient funds and sometimes because of
compatibility of findings with their own hunches or convictions.
41
The category of conclusive research designs informs that embedded research techniques are fit to
provide “conclusive” answers to research questions as well as definite tests of hypotheses. To fulfill
this purpose all conclusive studies ought to be representative of some larger population. It is worth
noting that the term “conclusive” as used here, do not denote absolute certainty but only high
probability of truth. This is the consequence of using statistical tests in analysis that are admittedly not
perfect but better than many other options for providing reliable knowledge.
The most common group of research under the category of descriptive designs in marketing are those
studies that rely on structured interviews with large samples of respondents. The questionnaire used
here as a research tool is characterized by prevalence of closed ended questions, that is such items
that offer the respondents only a set of predefined categories to choose as an answer. The purpose of
predefined replay categories is to standardize answers and thus make possible application of statistics
in analysis. The questionnaire based research comes in many shapes and sizes, with distinctive
advantages and weaker points. Some examples of specific subcategories may include cross-sectional
surveys, that examine one sample of respondents only once and longitudinal studies, which utilize the
same sample for multiple individual studies, often to examine how certain phenomena evolved in time.
Other division can go along the lines of administration methods in collecting responses with surveys
conducted in person by interviewers and those where questions are asked through telephone, Internet
or post – those and other kinds of choices in survey design will be looked at closer in the next chapter.
Besides surveys, descriptive designs also include other methods that entail analyzing large samples of
subjects but not necessarily adopting questionnaire to record findings – one relatively popular
technique would be observation, that can utilize video cameras and other tracking devices to collect
data on consumer behavior, for example, in a supermarket, on the internet or in front of the TV set.
As the category name implies, the descriptive designs should describe the populations of interest in
terms of counts and percentages of population members displaying certain characteristics, opinions,
attitudes and behavioral patterns. For practical purposes this type of research is used in marketing for
collecting information on attitudes and behavioral patterns of consumers, sizes and characteristics of
different homogeneous groups of consumers (i.e. market segments), assessing performance of
marketing policies (e.g. a telephone survey gauging effectiveness and efficiency of an ongoing
advertising campaign) and collecting data on competitors.
Causal research involves situations where it is possible to directly observe how one set of variables,
considered to constitute causes, affects the other variables, regarded as effects. It implies, that to
examine causal relations, researchers should be able to manipulate independent variables (i.e. causes)
and measure the resulting changes in dependent variables (i.e. effects). Such conditions are afforded
42
by experiments, which can be conducted in artificial, laboratory-like environment or in real market
conditions, as a field experiments or market test. Laboratory experiments allow for a better control of
extraneous variables that could “contaminate” the studied relationship but market tests give
outcomes more reliable in terms of their predictive power of later real live developments if tested
policies are actually implemented. As examples of laboratory experiments could serve blind tests,
where new flavors of edible products are examined or tests of alternative advertising executions on
groups of consumers brought together in cinema auditoriums. On the other hand, market tests might
be illustrated by introduction of a new product or service on a limited scale in a few retail outlets
controlled by a larger company (like new, unproven menu items in a few McDonald’s restaurants) or a
change in a limited number of parameters of existing marketing programs for a specified geographical
area or selected, narrowly defined groups of customers (e.g. introducing several price points for the
same product sold in several separated local markets, to see how they will affect the demand levels).
Concerning the nature of data collected in experiments, they are usually of quantitative type and are
accordingly analyzed with statistical techniques. Based on observation of the Polish research market,
it is plain that the experiments are among the least utilized marketing research techniques. The reason
for that being not lack of potential applications or quality of insights yielded but the very high costs
particularly when compared to surveys. As such, causal conclusions are often developed based on
descriptive and exploratory studies, which is not impossible but should be done with exceptional
caution as it is easy to find statistical relationships that either represent very weak causal links or are
entirely spurious. The suitability of experiments for building causal explanations stems from its
congruence with the four conditions for causality which comprise:
31
1) Statistical correlation of cause and effect variables (though, it should be emphasized that
correlation in itself does not imply causation)
2) The required time order – the effect cannot come before the cause
3) Control for other factors – it must be possible to demonstrate that other variables are not
responsible for the observed effects instead of what the researchers are considering to be the
causes
4) There must be a plausible explanation for the actual mechanism through which the
independent variables induce changes in dependent variables.
31
Mooi E., Sarstedt M.: A Concise Guide to Market Research, Springer Verlag, Berlin Heidelberg, 2011, p.17-18.
43
Considering the above requirements for causality, it is obvious that that descriptive designs are not
able to directly detect case-and-effect relationships. However such associations can be implied with
appropriate statistical tests, reference to substantive theory (e.g. marketing or consumer behavior
literature) and – of course – the application of common sense. There is still some risk involved that
what we may consider causal relationship, can in fact be brought about by chance or some other
unobserved variables, but in absence of experimental data using descriptive studies for that purpose
seems to be the next best option in terms of explaining various phenomena in marketing and other
social disciplines.
44
Chapter 4: Conducting Qualitative Research with Focus Group and
Depth Interviews
In the current chapter the discussion will be focused on the two most popular and versatile qualitative
techniques in market studies. Beyond focus interviews and depth interviews there is a plethora of
alternative less well known options, though they are usually less useful to marketers and often require
specialist, rather arcane knowledge to implement them correctly. In most practical applications in
marketing, whenever qualitative data are required, group or individual interviews are likely to prove
adequate either as a stand-alone technique or possibly combined in a more complex research designs.
Below we start with focus groups, then move on to depth interviews and conclude with a short
overview of practicalities of qualitative data analysis.
Focus Group Interview
The focus group interview, according to the definition offered in the American Marketing Association
Dictionary, is a personal interview conducted among a small number of individuals simultaneously
which relies more on group discussion than on a series of directed questions to generate data. It is a
method of gathering qualitative data on the preferences and beliefs of consumers through group
interaction and discussion usually focused on a specific topic or product.
32
It is definitely the most popular qualitative technique in commercial marketing research to such an
extent that it is considered by many professionals as synonymous with qualitative research, who when
referring to qualitative research may in fact think mostly of focus groups. Its popularity in business
applications can be to a large extent ascribed to the high cost efficiency in comparison with other
qualitative techniques such, as depth interviews, and considerable effectiveness in studying groups,
which are frequently equivalent to market segments targeted by companies commissioning the
research.
The goal of a focus group is to elicit ideas, feelings and experiences about a certain issue that would
be obscured or stifled by more structured methods of data collection. The use of a small group allows
for the operation of group dynamics and aids in making participants feel comfortable in an unfamiliar
32
http://www.marketingpower.com/_layouts/Dictionary.aspx?dLetter=F accessed on December 10th, 2012.
45
environment. It is called a “focus” group because the moderator serves to focus the discussion on the
topic and does not let the group wander off onto irrelevant points.
33
The use of focus groups can be differentiated according to the desired outcome and the type of
research questions.
34
With regard to the type of outcome, focus groups just like many other qualitative
techniques can be self-contained or can be used as ancillary method to other types of studies, usually
quantitative ones such as surveys or experiments. When built into a multi-method research design,
focus groups can be instrumental in the initial exploratory or hypothesis-generating phase to provide
a contextual basis for survey design and in the final phase to provide interpretive aid to survey
findings. The latter application is less common, though it has a great potential for providing unique
insights to the research project. Personal opinions expressed via survey questions may not be entirely
reliable as they can oversimplify respondents’ understanding and attitudes to more complex issues.
Focus groups can show that personal opinions are provisional and developmental, meaning that group
members can possibly extend, change or qualify (i.e. provide exceptions or limitations) their ideas in
the process of discussion and interaction with other participants.
35
The survey research in most
instances is unable to show such subtleties thus it may be beneficial to follow up a quantitative study
with a series of focus interviews.
Considering typical research questions addressed with focus groups in marketing the common
applications include studies of perceptions, beliefs, attitudes towards a new product concept, product
modification ideas, brand positioning, advertising concepts, ideas for packaging and distribution
arrangements and changes to pricing policies. In this context, it is critical to understand that focus
interviews are suitable for examining group norms and attitudes but they do not work well in
documenting behavioral, cognitive and affective characteristics of individuals. The reason for that
being that the group setting and its dynamics promotes tendency for atypical behaviors to be
unreported or underreported. When one of the tasks of focus groups is to arrive at a group consensus
on a given topic, it is to be expected that divergent beliefs, attitudes and experiences will be silenced.
36
Understandably, in those research situations that center on detecting and analyzing atypical
individuals across potentially sensitive topics focus groups are best replaced with depth interviews.
33
Burns C.A., Bush R. F.: Marketing Research: Online Research Applications, Fourth Edition, Pearson Prentice
Hall, Upper Saddle River, New Jersey, 2005, p. 211.
34
Puchta C., Potter J.: Focus Group Practice, Sage Publications, London, 2004, p. 7
35
Bloor M., Frankland J., Thomas M., Robson K.: Focus Groups in Social Research, Sage Publications, London,
2001, p. 11.
36
Ibidem, p. 8.
46
A good example that aptly illustrates what insights can be brought by focus groups (and to a large
extent also depth interviews) and which will be beyond the reach of survey method is given by G.
Bonnie and H. Rosental:
37
“A simple survey will only identify which brand of aspirin, ibuprofen (most consumers just call it all
“Advil”), or acetaminophen (and they call all this “Tylenol”) consumers buy. What you won’t find out is
what goes through any consumer’s mind when he’s in a drug store standing in front of that shelf. You
could set up a video camera and tape him at the scene. I think you get more useful information if you
just ask. That’s when you’ll hear: “It’s so confusing now. There’s aspirin and Advil and Tylenol. But,
what’s Aleve? Then there’s gel caps and tablets. And this brand mentions headaches, that one says
muscle pain, and here’s one that suggests it will work for fevers.” Listen to him. What he’s wondering
is what’s going to relieve his wife’s tennis elbow, which is why he’s standing bewildered at the shelf in
the first place. Just listen to consumers talk about the analgesics area of mass retail outlets, with nary
a pharmacist in sight. And bend a little closer when those over 50, with less than perfect vision and
arms that can’t reach far enough, tell you the frustration of not being able to get an item off the shelf
or read the miniscule print on the label that might actually shed some light on which brand of analgesic
to purchase. You may have all the data but you haven’t heard what consumers are really saying, which
is that the proliferation of similar painkillers is driving them nuts. And, if they can’t find what they want
in the supermarket within 30 seconds they’ll just grab whatever is in easy reach, pay the asking price,
and go about their business.”
Here, the example shows what was already stated about consumer behaviors and attitudes: they can
be extremely nuanced and sometimes averages and generalizations that surveys can deliver are not
sufficient for solving research problems at hand. Serious limitations of survey studies pertinent to the
given example which are not shared by focus groups, arise from the artificial nature of the standard
interview procedures, the influence of an interviewer on respondent’s comments and the prevalence
of rigid pre-determined questions.
38
Those constraints all tend to stifle spontaneous responses and
identifying new issues. As such, at times like that, it is highly advisable to extend research design to
include supplementary qualitative techniques, possibly focus groups for their cost effectiveness and
wide acceptance.
37
Goebert B., Rosenthal H. M.: Beyond Listening: Learning the Secret Language of Focus Groups, John Wiley &
Sons, New York, 2002, p. 5.
38
Flick U.: An Introduction to Qualitative Research, Second Edition, Sage Publications, London, 2002, p. 13.
47
Following the author’s experience the success of a focus group study strongly hinges on the three
criteria:
Thorough planning resulting in adequate research objectives and a suitable moderator outline
Setting up thoughtful eligibility criteria for participants and selecting productive and
harmonious focus groups
Engaging moderator with sufficient skills and appropriate personal characteristics
I will discuss briefly each item below.
Moderator outline is sometimes likened to a questionnaire in quantitative research. It is obviously a
plain misconception as their structure, content and function are quite different. It is essential to keep
in mind that any qualitative research interview is not based on a formal schedule of questions to be
asked word-for-word in a set order as it is in surveys. Nor questions have predefined answer categories,
which is also characteristic of quantitative approach. Instead, what is used as a research tool in focus
groups is more like a list of topics which the moderator should try to cover in the course of the
interview. It also includes more specific issues that could be used to ask probing questions after too
general responses to tease out greater detail if participants will not provide it spontaneously.
The characteristics of the outline closely match the way it is used during group discussions. First of all,
moderators in most studies treat it more like a set of guidelines rather than an algorithm, which they
would have to follow tightly. They need the list of issues to make sure that nothing important for
solving the research problem would be omitted but they don’t need to ask each question in the same
way with each interview or even maintain the same order. Of course, if wording and order is the same
from interview to interview then the subsequent analysis is somewhat easier as the transcripts (if
transcripts are used) have similar structure, but much more important for the final success is the
possibility to uncover unexpected, creative insights. And for that reason the process of asking
questions is secondary to the need of properly managing group dynamics and can be tweaked from
what was originally planned if the moderator decides that it could support fostering more intense
creative interactions between group members or he/she notices some new emerging topic which is
worth pursuing.
The development of the moderator outline does not end at the start of the first interview. It may be
modified through use by adding probes or even whole topics which had originally not been included,
but have emerged spontaneously in previous interviews. It is also possible to drop or reformulate those
48
parts which are incomprehensible to participants or consistently fail to evoke responses in a way
relevant to the research problem.
39
Moderator outlines vary in the level of detail as well as the form of items listed on it. The more detailed
ones can contain relatively specific fully formed questions, both general and probing (not just themes)
The questions ought to be divided into several parts each having designated time limits which
moderator is obliged to adhere to (an example of a form of this type is given in an appendix at the end
of the textbook). At the other end of the spectrum, there may be a very simple research tool comprising
only a list of major topics to be discussed, without probing questions and timing information. In the
author’s experience both approaches can be correct: the choice depends mostly on the scope of the
project. If only a few interviews are to be completed and a person to run them is a part of the research
team with intimate knowledge of the project intricacies, the outline can be only loosely structured,
similar to the latter type. If, on the other hand, the research involves greater number of interviews
(not uncommon when several distinctive customer segments are involved, which have to be studied
separately, i.e. its members should not be included in the same focus groups) then it is justified –
recommended even – to apply moderator outline with a more rigid structure. More formal and
ordered focus groups lose some of the potential for bringing out very creative insights but they enable
a more meaningful comparison of different interviews carried out by several moderators. In a large
project with tight deadlines it is unlikely that only one moderator will be involved, so a more strict
format of interview guides ensures that different moderators proceed in a similar manner, which, in
turn, lends itself to obtaining more comparable results. To sum up, there are no strict guidelines on
how to prepare a good moderator outline, however from my previous projects I can suggest that it
may be beneficial to keep in mind the following hints:
research objectives must be carefully reflected in the outline’s contents,
enough of flexibility must be built in to allow the moderator to follow up on unexpected but
promising topics,
style, skills and knowledge of the moderator should determine the level of detail,
characteristics of respondents may require including more entertaining exercises (e.g.
projective techniques) to maintain participants’ involvement and concentration,
large scope of the project and further analytical requirements may call for a more formal
structure,
it is usually useful to provide some timing guidelines keeping in mind that under normal
circumstances the whole interview should not exceed 1.5 hours.
39
Cassel C.: Essential Guide to Qualitative Methods in Organizational Research, Sage Publishing, London, 2004,
p. 15.
49
Seeing how critical for a productive focus interview are constructive group dynamics it is worthwhile
to carefully consider the group’s size and makeup. With regard to the number of participants the group
should be neither too small nor too numerous. Most authors agree that the preferred size should be
between 8 and 12, though currently there is a tendency for smaller groups with the upper limit of 8,
more often endorsed both in European and American literature, even though until recently American
authors were inclined to suggest 12 as the reasonable maximum.
40
The recommendation of 8
participants usually work well in practice, though it should be remembered that the specific research
situation may dictate a different number. Specifically when experts are studied or other individuals
who will be interested in the topic and have ample experience and knowledge to contribute it may be
justified to limit the group size to 6 to allow each participant more time to talk. In general when group
turns out to be too small participants will not have a feeling of safety and anonymity from being a part
of the crowd and some interpersonal interactions may be not triggered, e.g. snowballing effect, when
one person’s comment starts a chain of reactions from other group members building on contributions
of previous speakers. On the other hand, a group that is too large may be difficult to control by the
moderator and some members could be frustrated at not being given enough opportunity to express
themselves. According to author’s observations, the groups of 10 to 12 are still quite often used in
business environment as they are considered more cost efficient and quite adequate for simpler
marketing topics.
When deciding on focus group composition it is vital to allocate to one research unit (i.e., single
discussion group) only participants with similar views on the topic of studies. In this case the
homogeneity promotes harmonious discussion and enhances comfort of respondents, which usually
stimulates openness and creativity. Conversely, groups with multitudes of points of view and lifestyles
tend to be dysfunctional, prone to focusing on irrelevant topics and apt to slip into squabbles. For that
reason it is often practiced not to mix up males and females, different age groups and people with
various education levels though in some research situations this characteristics may not be linked to
different opinions on studied topics and so be irrelevant. Also in marketing research there is a standard
rule that the recruited respondents should be strangers. It is believed that in such a way discussion is
more dynamic, participants share more information on themselves and their experiences and more of
them tend to disclose potentially sensitive information. However, groups made up of friends, work
colleagues, acquaintances or family members can also have their benefits. Among them I would
highlight as the most prominent: (1) more reliable information, as group members who know each
40
Catteral M., Maclaran P.: Focus Groups in Marketing Research in Handbook of Qualitative Research in
Marketing, edited by Belk R.W, Edward Elgar Publishing, Chaltenham, UK, 2006, p. 263.
50
other are inclined to check credibility of the statements, (2) more complete statements on many topics,
when other participants add what was left out from original speaker’s comment, (3) easier recruitment
and higher turnout than in a focus group of strangers. Beyond methodological grounds, which do not
always seem to be sufficient, the policy of almost always recruiting strangers for commercial projects
could be additionally explained by possible negative reaction from clients commissioning the study
when they start to become suspicious of recruitment policies on noticing the involvement of
acquaintances.
41
The summary of the strong and weak points of familiar and stranger groups are given in the table.
Table 7: Advantages and disadvantages of using acquaintance and stranger groups in focus interviews
Acquaintance groups
Advantages
Disadvantages
Observe naturally-occurring debate
Shared knowledge provides greater
detail
Group memory leads to greater
accuracy and group reminding of
experiences
Familiarity enriches the discussion
Secure discussion of stigmatizing issues
Easier recruitment
Lower attendance attrition
Less threatening for participants
Shorter “warm-up” time
Familiarity may inhibit disclosure
Existing hierarchy remains
Taken-for-granted information
Information may lack depth
Risk of over-disclosure
Confidentiality difficult to enforce
More challenging to moderate
Stranger groups
Advantages
Disadvantages
Confidential and anonymous
Participants speak freely
Avoids taken for granted assumptions
Detailed information (descriptions,
justifications, explanations, etc.)
Ethical issues controlled by researchers
Longer “warm-up” time
More complex recruitment
Attendance failure
Source: Hennik M. M.:International Focus Group Research: A Handbook for the Health and Social
Sciences, Cambridge University Press, 2007, p. 117.
Moderator’s role is critical since he/she is mainly responsible for creating a permissive, non-
threatening environment which promotes beneficial group dynamics that must operate for the study
to produce useful insights. Moderator should strive to induce such an atmosphere whereby
41
Gordon W.: Goodthinking: A Guide to Qualitative Research, Admap Publications, Henley-on-Thames, UK,
1999, p. 65.
51
participants feel comfortable to share their views and experiences without the fear of judgment or
ridicule from others. A common misconception is that the moderator must dominate the group and
“do most of the talking”. Such approach will not be very different from regular interviews where
respondents receive specific questions and attempt to answer them concisely. In focus groups it is the
interactions between group members that are most valuable as they can make respondents to trigger
reactions from each other, affect their attitudes and through collective effort stimulate creative and
innovative answers. However the respondents need to stay on topic and the moderator have to make
sure that they do not get involved too much in digressions that are unlikely to lead to any useful finding.
As such, it is possibly the most difficult task for the moderator to balance permissiveness with strictness
to support vivid group interactions and keep them on topic. Moderator is also the first analyst of the
research results: here the analysis runs in parallel to the interview and if done right allows the
interviewer to change the preplanned guidelines to follow an emergent topic that might contribute to
solving research problem. However to be able to do that – to know which diversions from the original
interview scenario are permissible and even should be supported and which need to be quickly
suppressed – requires a thorough knowledge of research objectives and the nature of studied
phenomena combined with capability to make quick accurate decisions. For that reason it is often a
better choice to have one of the research team members moderate than to bring in an outsider even
with considerable skills and experience.
The role of the moderator in progressive stages of the interview is described more thoroughly in the
table.
Table 8: Moderator’s role by stages of a focus group interview
Stage
Underlying processes
What happens
Moderator task
Forming
Participants feel
separate, dependent,
anxious, relatively
powerless
Awkwardness, caution,
light social chit-chat,
testing behaviors
Empower and make
safe by explaining the
task, behave as you
would want them to,
encourage interaction
Storming
Share of voice,
demanding attention,
challenges to
moderator and others,
opting out or rebelling
Challenge moderator
or each other,
question the task,
emphasize
individuality, dominant
and passive
participants emerge
Signal all opinions are
equally valued, accept
negative views but
look also for positive,
stop potentially
dominant respondents
becoming overbearing
Norming
Sense of harmony,
cohesion and support,
norms emerge and
group takes off
People take turns
without moderator
having to ask, the
energy feels more
Notice and enforce
norms, deal with any
implicit rule breaking;
time to make plans and
52
positive and
harmonious
set agendas, keep
communication
channels flexible
Performing
Individuals are
subservient to the
group, roles are
flexible and task
oriented
Sense of concentration
and flow, everything
seems easy, high
energy, group works
without being asked
Time to introduce
difficult issues,
stimulus material,
projective techniques
Re-adjustment:
Performing uses energy so after a while the group slips back into one of the other stages before it
can perform again
Mourning
Completion of task and
disbanding the group;
may be sense of loss
and anxiety, need for
closure
If task is incomplete
people may not want
to leave
Signal that the end is
coming, summarize to
give sense of
achievement, ask if
anything else they
want to say, thank
everyone
Source: Chrzanowska J.: Interviewing Groups and Individuals in Qualitative Market Research, Sage
Publishing, London, 2002, p. 53
The concept of group dynamics development depicted in the table requires of the moderator
performing different tasks to move the participants through the subsequent stages, which is a
necessary condition for productive interview. The varying tasks in different phases of the interview
should be reflected in moderator outline, particularly by including a warm-up phase at the beginning
and appropriate placing the tasks requiring the highest level of concentration and effort. However, if
any specific group behaves atypically the moderator should be always free to make any necessary
modifications to the schedule of the interview so that positive dynamics could be maintained and
valuable data gathered.
Depth Interview
Depth interviews, even though markedly less common than focus interviews, are all the same the
second most popular qualitative technique in marketing research. Depth interview is defined as “an
unstructured, direct, personal interview in which a single respondent is probed by a highly skilled
interviewer to uncover motivations, beliefs, attitudes and feelings on a topic”.
42
42
Malhotra N.: Marketing Research: An Applied Orientation, 6th Edition, Prentice Hall, Upper Sadle River, New
Jersey 2010, p. 153.
53
Similarly to focus groups, depth interviews are an unstructured technique for gathering information
that is often less formalized and more flexible than the former. The most striking difference lies in the
fact that it involves only two persons: interviewer and interviewee. Because one-on-one interview is a
more strenuous experience from the point of view of both sides of the conversation it is typical that it
lasts shorter than the focus group, often between 30 minutes and 1 hour. In many contexts, depth
interviews could be applied to address similar research questions as focus groups, though when their
utility is comparable focus interviews are picked more often in part due to lower costs per participant
and easier access to skilled moderators than qualified depth interviewers. There are, however, some
subject areas and applications where depth interviews are apt to provide more reliable and valid
outcomes. As such, the higher quality of data in certain research situations stems from several
distinctive features of this qualitative technique, which are often presented as its unique advantages:
43
1. In depth interviews group pressure is eliminated so that each respondent reveals more honest
feelings.
2. The personal one-to-one situation gives the respondent the feeling of being the focus of
attention, whose personal thoughts and feelings are important and genuinely wanted.
3. The respondent attains a heightened state of awareness in a personal interview because they
are in constant rapport with the interviewer and there are no group members to hide behind.
4. The extra time devoted to individual respondents encourages the disclosure of new
information.
5. Respondents can be probed at length to reveal the feelings and motivations that underlie their
statements.
6. Without the restrictions of cultivating the group process, new directions of questioning can be
improvised more easily. Individual interviews allow greater flexibility in exploring casual
remarks and side issues, which may provide critical insights into the main issue.
7. The closeness of the one-to-one relationship allows the interviewer to become more sensitive
to non-verbal feedback.
Considering the above strengths of depth interviews it follows that they may be preferable when:
studied issues are complex and the interlocutor is an expert or other knowledgeable person
topics are sensitive, potentially threatening to self-esteem
the researcher is interested in revealing hidden, subconscious motives or attitudes
43
Proctor T.: Essentials of Marketing Research, Fourth Edition, Pearson Education Ltd., London, 2005, p. 234-
235.
54
participants in the study are competitors and as such would be reluctant to speak openly in a
group setting.
On the other hand, focus groups are deemed more beneficial when the main interests are:
generating new, creative ideas
studying group interactions and topics that are shaped by interpersonal contacts
collecting quickly and relatively cheaply large volume of information on up to several dozen
research subjects (if large number of persons were to be studied, it may be better to employ
survey method).
Conducting depth interviews is admittedly more difficult than moderating focus groups and requires
more skilled and experienced interviewers. As was already discussed, with focus groups the role of the
moderator is essentially about managing group of participants, stimulating productive atmosphere and
maintaining the discussion on the right track as set by the research objectives. However, as was also
stated, the true value of focus groups is in interactions among participants and to the lesser extent
between the moderator and participants. If the group is made up of people with similar views on the
topic of interest it is likely that the job of the moderator will be not be overly demanding. In
comparison, as was already stated, depth interview engages only two participants who often are to
remain in conversation for one hour. In that time there are no group dynamics that could put the
participant at ease and encourage him/her to talk openly about topical issues. The role of the group to
make the subject comfortable enough to open up and speak freely here is entirely fulfilled by the
moderator. Considering how demanding the function of interviewer can be, it is worthwhile to
contemplate a few advises from Robert Yin – one of the foremost authors on qualitative research
methodology (note that most of the guidelines also apply to focus interviews). According to Yin the
following practices should assist in conducting effective interview:
44
1. Speaking in modest amounts. Interviewer should speak only when necessary allowing the
interviewee to present freely his/her thoughts. Accordingly, it is not desirable to pose lengthy
questions to which a person can satisfactorily respond with a short answer (e.g. yes/no
questions). Also, multiple questions should be avoided (the kind which contains two or more
queries in one sentence) and asking questions in quick succession without giving the other side
enough time to answer. Depth interview should not resemble interrogation but rather natural
everyday dialog. Yin advices that one key to keeping a conversation going with a minimum of
44
Yin R. K.: Qualitative Research from Start to Finish, The Guilford Press, New York, 2011, p. 136-139.
55
interviewer’s own words is to master the use of probing questions (or probes in short) and
follow-up questions. After a participant has made an insightful comment but one that is
apparently shorter than desired, thoughtful use of probes and follow up questions can prompt
the interviewee to elaborate. The effective probes may take the form of brief utterances, such
as: uh-huh, say more, why?, how is it so?. A very useful way of probing, but rarely used by
inexperienced researchers, is also a deliberate, long, silent pause.
2. Being non directive. The researcher should have a list of topics but it would be a mistake to try
and impose that on the participant. Rather, the participant should be allowed to follow freely
their own sequence of topics within the broader theme of the research study. Such manner of
interviewing provides extra clues about subject’s perspective and allows to control for possible
interviewer bias. For instance, the order in which the topics were brought up could reveal how
important they were to the participant and interviewed in this way participants are more likely
to use their own word for objects, notions and phenomena instead of simply repeating those
used by the researcher, which could also enrich the findings.
3. Staying neutral. The recommendation not only concerns avoiding leading questions, which can
suggest the answer by revealing our attitude to the topic, but also pertains to voice, body
language and facial expressions. None of these should reveal the researcher’s personal
opinions about the studied issues at the serious risk of the participant sensing it and
attempting to respond in a manner that in his/her opinion would please the interviewer. This
propensity is often of subconscious nature but nevertheless with very real detrimental effects
on research results. Even though the researcher may go to great lengths to avoid expressing
their opinions it may not be possible to stay clear of it entirely. In Robert Yin’s words “…you
will inevitably bring a point of view to all your conversations (…). The desired remedy is to
avoid blatant biases but also to be sensitive to those that remain. Later, you should do your
best to reveal and discuss how they might affect your findings”.
45
4. Maintaining rapport. Interviewer should maintain friendly relationship with the subject
throughout the whole interview and do their utmost not to irritate or displease the participant.
5. Using an interview protocol. Interview protocol serves very much the same purpose as
moderator outline in focus groups. It is useful not only because it contains a list of topics that
help the researcher to make sure that all important issues were covered but it can have a
surprising effect on the participant by giving him/her the impression of being a part of wider
formalized process, which may enhance subject’s self-esteem and increase willingness and
45
Ibidem, p. 138.
56
quality of answers. As was already stressed, the protocol (just like moderator outline) should
in no sense be considered a questionnaire – departure from original wording and sequence is
fully admissible.
6. Analyzing when interviewing. The final reminder is about a unique feature of qualitative
research in comparison to quantitative studies, of analysis and data gathering running in
parallel. The interviewer must assess gathered data in real time to take decision if attempt to
change the topic should made or to inquire deeper with probing questions for more details.
Also, some developments during the interview may warrant changing the original protocol to
accommodate new insights.
Analysis of Textual Data
Data analysis in qualitative research is notoriously troublesome. Part of the difficulty lies in the lack of
widely accepted and sufficiently precise guidelines on how to process raw data and interpret findings.
If fact, it remains in stark contrast to what is the case in quantitative research where statistics provides
extensive set of tools, which are readily available through several popular and easy-to-use software
packages. When faced with empirical material expressed as numbers the analysis procedure is
relatively simple: after transferring our database to the statistical software we select appropriate
statistical tests and procedures through a user-friendly point-and-click interface, run the chosen
analyses and interpret the results if they turn out to be statistically significant. Such approach is not
possible with qualitative data, which mostly involves text descriptions instead of numbers, and
comprises too small and non-random samples to permit statistical generalizations to a larger general
population of research subjects. What is similar for the two research approaches is that data analysis
is a search for patterns though methods for detecting them are different.
Analyzing data is a sequential process that can possibly take several distinct forms depending on the
resource constraints and the researcher’s epistemological preferences. The first step, which aims to
prepare raw data for further processing, requires transcription of taped interviews (i.e. putting them
in a written form) and combining all written materials into a single, ordered database. Transcription
could to some extent be automated with voice recognition software but given the still unreliable
nature of such aids each transcript needs to be carefully proofread by the researcher and each piece
of text needs to have ascribed a label representing the interview participant who originally made that
statement. The significance of building a comprehensive database is particularly critical when later
analytic tasks are performed with the help of software such as NVivo, which can considerably
streamline the process and allow for successful searching of more complex data structures, but only if
57
all relevant data are properly introduced into the program. The second stage in analysis – called coding
or indexing – demands bringing together all extracts of data that are pertinent to a particular theme,
topic or hypothesis to make numerous pages of text more manageable for interpretation.
46
Codes or
indexes could represent topics or themes relevant to the research problem that were identified in any
item of the empirical material. If researchers hypothesized analytical models containing variables and
their interrelationships, codes should be used to highlight the statements referring to particular
variables. If researchers do not use any specialized software dedicated to qualitative data processing
the number of different codes should be limited to a bare minimum to make the interpretation
feasible. With software, much more complex and detailed indexing systems can be employed without
causing unnecessary confusion and bogging down the further analysis. Of course, given the iterative
nature of qualitative research process (i.e. overlapping phases of data gathering and analysis and the
possibility of going back to the earlier steps of the process without adversely affecting its outcomes),
the coding performed in this step will not be final and will almost surely be modified and elaborated
when interpretation is attempted, though it is advisable here to do as thorough a job as possible,
accounting for all relevant themes and variables.
When it comes to locating data structures, discovering patterns and formulating interpretations,
researchers are faced with several major choices. For depth interviews and focus groups the
compromise between the quality of analysis and costs incurred will lead to choosing from the four
analysis strategies, which are progressively cheaper at the expense of a diminished quality of findings.
47
The most rigorous approach, yielding the most reliable results though at the relatively highest expense,
entails transcript preparation and examination. Next comes tape-based analysis, involving careful
listening to the tapes containing recorded interviews, but not the study of transcripts. The quickest
and the least detailed and systematic are notes-based analysis and memory based analysis. From the
author’s experience, the standard practice for most commercial research in Poland does not rely on
transcripts but rather utilizes notes from moderators, interviewers and observers supplemented by
memorized observations from witnessing of the interviews. Such an approach is facilitated by the
interview moderator acting as a sole writer or co-writer and relatively uncomplicated research
objectives. If a study is to have academic quality or just be more precise and thorough, then the use of
transcripts seems to be necessary.
46
Coffey A., Atkinson P.: Making Sense of Qualitative Data, Sage, London, 1996.
47
Krueger, R.A.: Focus Groups: A Practival Guide for Applied Research, Second Edition, Sage Publications,
Thousand Oaks, 1994, pp. 143-144
58
When it comes to data interpretation there are several methods for building explanations though the
most common and possibly most practical approach is known as analytical induction or deviant cases
analysis.
48
It is most suitable for more formal academic research but it can also inform analytical
procedures in commercial studies. Analytic induction is a means to derive explanatory hypotheses
which apply to all the data available on all the phenomena or the whole research problem. A
hypothesis or a set of hypotheses is constructed through sequential analysis of all the cases at hand,
one at a time, and - whenever it is necessary - modifying the hypothesis to fit data from new cases and
cover them with their explanatory capabilities. Here a case means a separate unit of analysis, which
can be a person (as in depth interviews), a group (in focus groups) or a whole organization (in a case
study). The analytical induction process starts with the researcher formulating an initial hypothesis
based on his/her experience, intuition and previous studies. Then, once the data from the field become
accessible, the hypothesis is compared to the cases, one by one, to see whether they support or refute
it. If a “deviant case” (one which provides data that does not yet fit the hypothesis) is found it is
recognized that the hypothesis does not fit the collected evidence. There are two ways to remedy this
problem: (1) to modify the hypothesis to make it consistent with the evidence from the new case (as
a result the hypothesis becomes more precise) or (2) to narrow down the population to which the
hypothesis applies so as it excludes the incoherent case together with other similar cases.
49
It is usually
more preferable to try and amend the hypothesis at first before resorting to the latter option of limiting
the scope of the proposition. This process should be repeated until a new case do not bring any data
that would be at odds with the current version of the hypothesis.
48
Bloor M., Frankland J., Thomas M., Robson K.: Focus Groups in Social Research, Sage Publications, London,
2001, p. 66.
49
Ibidem, p. 67.
59
Chapter 5: Designing questionnaires for survey research
The present section centers on practical aspects of designing questionnaires for survey studies in
marketing research. The first topic of interest is an overview of primary measuring scales including
those with nominal, ordinal, interval and ratio characteristics. As another point, the essential principles
of constructing individual scales are considered. Next, the subject of multiple scales is brought up and
the chapter completes with a short presentation of the general do’s and don’ts in preparing
questionnaires.
Quality criteria of a survey questionnaire
The questionnaire is typically the only measurement tool that is employed in survey research. Hence
the quality of its design reflects directly on the quality of data it generates and this in turn determines
the extent of success of the whole project. A good survey form is supposed to provide measurements
that are as close as possible to the true values of variables of interest. This problem is far from trivial
as interviews are artificially created situations in which respondents may not find themselves feeling
comfortable. They may also be in a hurry, distracted, tired or in a multitude of other states of mind
that are apt to affect they capacity and willingness to give accurate answers. In addition, studies in
marketing research are not only dedicated to measuring facts about consumers but also such abstract
and subjective characteristics as feelings, opinions, attitudes and various behavioral patterns. It is the
latter group of variables that is particularly troublesome in translating into adequate questions and
scales.
In simple terms, there are two criteria or aspects of quality that one should have in mind while
designing a questionnaire: reliability and validity. While there are complex and formal ways of defining
and evaluating both terms they go beyond the scope of this work as well as the basic course in
marketing research.
50
Here instead a simple and intuitive understanding will be given:
51
Reliability is the extent to which respondents in comparable situations answer questions in a
similar way. In other words, if two respondents are in the same situation (i.e. have the same
true value of a certain characteristic of interest, such as income, age or view on a product
50
Readers interested in an enhanced discussion of the quality of questionnaires and measurement scales may
find useful the following book: DeVellis R.: Scale Development: Theory and Applications, (3rd ed.), SAGE
Publications Inc., Thousand Oaks, CA, 2012.
51
Based on Fowler F.: Survey Research Methods, (5th ed.), SAGE Publications Inc., Thousand Oaks, CA, 2014, pp.
76-86.
60
quality) they should answer the question in exactly the same way. The inconsistences among
respondents that are not explained by actual differences in true variable values are interpreted
as the random error and are thus a measure of the lack of reliability. Problems with reliability
in survey research stem from not all respondents interpreting the items of the questionnaire
in the same way. Issues with wrong interpretations are often caused by faulty question
wording, incompetent interviewers or environmental factors which produce distractions.
Validity is the extent to which the answers given by respondents are the true measure of an
actual characteristic and mean what the researcher wants and expects them to mean. Validity
is particularly difficult to achieve in measuring constructs, or variables that cannot be assessed
directly but instead have to be evaluated through multiple indicators (see Chapter 1 for a more
extensive overview of constructs or latent variables). Such constructs include loyalty,
perceptions of brand image, evaluation of sales service quality, lifestyle of customers etc. Here,
errors made in selecting or developing multiple measurement scale may result in the
researchers measuring some other construct rather than the one intended, e.g. brand
perceptions instead of attitudes, which are often considered to be distinctive concepts.
52
Reliability and validity are closely related concepts. A measurement scale to be valid must also be
reliable. On the other hand, a reliable scale may or may not be valid: the fact that the measurements
are similar among the same types of respondents in one sample, or from sample to sample, is not
sufficient evidence that they actually measure what they were intended to.
Closed and open ended questions
At the most general level, the items for collecting information that form the questionnaire can be
classified as closed or open ended questions. For practical reasons, a survey questionnaire is
predominantly made up of closed ended questions that offer respondents a set of predefined answers.
The other terms for this type of items are categorized or scale questions. The latter name refers to the
fact that those preset answer categories form measurement scales that can use numbers to represent
each possible reply option to any given question. Later in the research project, when a computer data
file is created for further processing with statistical software, numerals turn out to be very useful as
52
For more on measuring constructs see Singh J.: Measurement Approaches For Consumer Behavior Constructs:
a Multidimensional Perspective", in Advances in Consumer Research Volume 15, Houston. M. (ed.), Provo, UT :
Association for Consumer Research, 1988, pp. 487-492.
61
they serve as concise codes to record answers provided by respondents. Indeed, depending on the
software employed, the use of numbers instead of other symbols or textual labels is either necessary
for conducting statistical analysis or, at a minimum, greatly facilitates analytical possibilities.
On the other hand, open ended questions permit survey participants to reply in their own words. This
freedom of articulating opinions has some benefits, but mostly is linked with difficulties in data
collection and analysis. As such, this type of questions tends to be avoided while crafting
questionnaires. One of the most compelling reasons for that has to do with the time consuming
preparation of this type of data before they could be examined with statistical software. Here is how
J. Krosnick and S. Presser describe the tasks involved in this process : “In order to analyze the answers
to open questions, they must be grouped into a relatively small number of categories. This requires
the development of a coding scheme, its application by more than one person; and the attainment of
a high level of agreement between coders. The cost of these procedures, coupled with both the
difficulties interviewers confront in recording open answers and the longer interviewing time taken by
the open questions, are responsible for the widespread use of closed questions.”
53
The coding mentioned there involves reducing a large variety of answers to an open ended question
into a smaller set of categories by merging similar replies given by different respondents. Due to a
possibly significant amount of subjectivity involved in identifying similar answers and combining them
in groups it is recommended to assign to the task at least two researchers who will have to be in
agreement to categorize an answer in any particular way.
The open ended format also results in longer interviewing times as compared to its categorized
alternative, because of additional time required by respondents to recall the information asked by a
question if the memory is not supported by preset options. In addition, open ended items tend to yield
lower response rates, since some respondents may refuse to answer because of the unwillingness to
exert the effort required for remembering the necessary facts or to avoid what they could perceive as
embarrassment if the answer is not right (event thought there are no right or wrong answers in a
marketing research questionnaire).
Some researchers may also decide to use open ended format in hopes of taping the creativity of
respondents to produce some unique insights. However, as the author’s experience shows, most of
the interviewees typically reply in a short, unimaginative way that is far from being useful as a source
of new ideas. This is explained by the fact that creativity is unleased only if the respondent is in the
53
Krosnick, J.R., Presser, S.: Question and Questionnaire Design, in Marsden, P. V, Wright, J.D. (Eds.): Handbook
of Survey Research (2nd ed.), Emerald Group Publishing Ltd., Bingley, UK, 2010, pp. 263-313.
62
right frame of mind, that is relaxed and concentrated, which requires a conducive environment and a
proper interviewing technique that is much easier to achieve with a qualitative approach such as depth
or focus group interviews.
Among a few truly beneficial applications of the open-ended format is summarizing blocks of questions
on the same topic (e.g. asking about a single most important strong and weak point of a product) or
obtaining unaided recall, like in studying the power and image of consumer brands (e.g. respondents
could be asked to indicate which brand comes first to mind when they think of a certain product, such
as a carbonated soft drink or a mobile phone). However, when using open questions in this manner
they must be placed in the right part of the questionnaire: summaries at the end of the questionnaire
or a thematic block of questions and prompts for unaided recall at the beginning of the form or a group
of questions.
Due to the above reasons, as a general rule, open questions ought to be avoided in designing survey
questionnaires.
Primary scales of measurement
As was already noted, closed ended questions use measurement scales, which are made up of
predefined answer options. There are four basic types of measurement scales - each with different
properties affecting the scope of permitted statistical analysis. The most crucial characteristics of each
scale format with some examples were given in the table.
63
Table 9: Primary scales of measurement: characteristics and examples
Source: Malhotra N.: Marketing Research: An Applied Orientation, 6th Edition, Prentice Hall, Upper
Saddle River, New Jersey 2010, p. 253.
The above classification was formed following the four basic characteristics of any comprehensive
measurement system, not only that used in marketing or social sciences. The full set of characteristics
includes description, order, distance and origin, and it defines the level of measurement of a scale,
which in turn determines what numeric properties are retained and thus what arithmetic operations
and transformations can be accomplished with the numbers representing answers collected from
respondents.
54
To understand the differences between the types of scales it is necessary to recognize
that the numerals assigned to response categories may not permit the full array of arithmetic
operations, like division, multiplication and even subtraction and addition, as would be the case in
mathematics. To give an example, the question about gender – present in almost every questionnaire
– uses only two numbers to designate the two possible answers: it can be a 1 for females and a 2 for
males. However, those numbers are used only as labels – to distinguish between respondents with one
54
Mooi E., Sarstedt M.: A Concise Guide to Market Research: The Process, Data and Methods Using
IBM SPSS Statistics, Springer Verlag, 2011, pp. 32-33.
Scale
Basic
Characteristics
Common
Examples
Marketing
Examples
Nominal
Numbers identify
& classify objects
Social Security
numbers.,
numbering of
football players
Brand names.,
store types
Percentages,
mode
Chi-square,
binomial test
Ordinal
Numbers indicate
the relative
positions of
objects but not
the magnitude of
differences
between them
Quality rankings,
rankings of teams
in a tournament
Preference
rankings, market
position, social
class
Percentile,
median
Rank-order
correlation,
Friedman
ANOVA
Ratio
Zero point is fixed,
ratios of scale
values can be
compared
Length, weight
Age, sales,
income, costs
Geometric
mean, harmonic
mean
Coefficient of
variation
Permissible Statistics
Descriptive Inferential
Interval
Differences
between objects
can be compared,
zero point is
arbitrary
Temperature
(Fahrenheit)
Celsius)
Attitudes,
opinions, index
nos.
Range, mean,
standard
deviation
Product-
moment
correlation, t
tests,
regression
64
or the other kind of gender characteristic. It is not possible to say that they contain any information
about the intensity of the characteristic, as “1” is in no way worse or better than “2”. For the same
reason it does not make sense to compute average gender. All these limitations are the consequence
of gender being measured on a nominal scale that has only the property of description. To distinguish
nominal scales from the more advanced measuring systems, and to avoid making errors in data
analysis, it is useful to examine if the answers could be ordered from low to high in a meaningful
fashion. If that is not possible, the scale is clearly nominal and enables only the most basic analyses:
presenting counted answers in frequency charts or tables and performing those non-parametric
statistical tests that do not rely on the order of observations, like the chi square test of independence.
One step up in the hierarchy is ordinal scale which besides the property of description also has order,
thus allowing to sort the observations from low to high or conversely. What this scale is lacking are
equal differences between adjacent points on the scale, since ordinal scales do not have a measuring
unit that would make such an assumption valid. An example of a typical ordinal scale is one about
completed education. In Poland, four different levels of that characteristic are typically used: 1 –
completed primary education, 2 – vocational education, 3 – secondary education, 4 – tertiary
education (i.e. a bachelor’s or higher academic degree). It is obvious that the scale has description and
order as every level of the variable is represented by a different number and the higher numbers
designate the more advanced educational accomplishments. It would be problematic, though, to
assume that the difference between 1 and 2 is the same as between 2 and 3 or 3 and 4. Similarly, it
would not make much sense to say that the university education (4) is four times better than primary
education (1) and twice as good as vocational (2). There is simply no reason to do that because the
same differences between neighboring points on the scale do not represent the same dissimilarities in
the intensity of the characteristic levels. In marketing ordinal scales are very commonplace as they are
typically used for measuring attitudes. The constraining problem of unequal differences, that do not
permit calculating the arithmetic average and other similar statistics, is commonly sidestepped by
making the assumption that ordinal scales for measuring attitudes with at least five different levels
of intensity have so negligible differences between different pairs of adjacent intervals that they can
be treated as equal. Consequently, if one wants to use more advanced statistical methods it is
important to make sure that in the designed questionnaire all attitudinal and other pertinent scales
without a natural measuring unit have at least five values. Thus, the statistical processing could utilize
both non-parametric and parametric tests. The prominent exception to this rule of thumb are
comparative ranking scales, where objects (e.g. 10 competing brand names of a certain product
category) are arranged in order of preference by allocating numbers from 1 to 10. In such instances,
65
because numbers represent relative and not absolute preference, the scale cannot be considered
interval.
Coming next, interval scale has the properties of both preceding scales (i.e. description for nominal
scale and description and order for ordinal scale) and its own characteristic of distance which entails
equal intervals between any neighboring points on the scale. Such scales may possess natural units of
measurement (euros, dollars, number of people, etc.) or they can be assumed to be interval based on
the number of levels in excess of 5. For interval scales the only constraint arises from the lack of natural
zero point that makes it impossible to calculate harmonic and geometric mean or express differences
between observations in terms of percentages or multiples (e.g. if temperature is measured in degrees
Celsius it is not meaningful to say that – for example –when the temperature is 10 degrees it is twice
as hot as when it is 5 because the zero point is arbitrary and do not represent the absolute lack of the
measured characteristic). To see that such a description of difference is erroneous it should be enough
to express the thermometer readings in degrees Fahrenheit, where that proportions are not held. The
only temperature scale that has natural zero point and permits such comparisons is Kelvin scale, where
zero represents the lowest possible temperature in nature and is equal to approximately – 273 degrees
Celsius. The problem with an arbitrary zero point is not particularly troublesome as almost every
advanced statistical test and procedure is applicable. Of course, there are often other conditions that
must be met for using specific tests, such as a sufficient sample size or the normal distribution of
variables, but the nature of the scale is not limiting so long as it is interval.
The fourth and most advanced is ratio scale and is distinguished by possessing all four characteristics
of the complete measuring system, i.e. description, order, distance and origin. The examples of
characteristics measured in this way include income, age, height, weight and temperature measured
with the Kelvin system, and they enable conducting any arithmetic operation permitted in
mathematics.
To sum up, it is worth remembering that description, order, distance, and origin represent successively
higher-level attributes, with origin being the most advanced property. Description is the most basic
characteristic that is present in all scales. If a scale has order, it also has description. If a scale has
distance, it also has order and description. Finally, a scale that has origin also has distance, order, and
description. Thus, if a scale has a higher-level characteristic, it must have all the lower-level
66
characteristics. However, the reverse may not be true, i.e. if a scale has a lower-level characteristic, it
may or may not have a higher level characteristic.
55
Things to consider in designing individual scales
Constructing individual rating scales involves making choices in several aspects of design. To end up
with items that provide trustworthy readings, those decisions must be thoughtful of respondents’
characteristics and other salient elements of the research context. Specifically, the following issues
should be the object of particularly careful consideration.
Number of scale categories: typically the number of categories ought to be between five and ten;
however, there is no single, optimal number of categories that would be applicable for all scaling
situations. What matters here is taking into account participants’ ability to precisely discern among
different levels of a variable, and choose such a number of units on a scale so that respondent replies
are not accidental – i.e. they would be able to justify why they picked such a point on the scale and not
any of the nearby points. Also, for the sake of not confusing participants unnecessarily, it is advisable
– after having decided on a particular number of categories – to use the same solution for all questions
of a similar type throughout the whole questionnaire.
Many researchers believe that the 10-point rating scale is excessive. However recently, there was a
study demonstrating that 10 repose options may work better than a more concise, five-point scale by
capturing more of the available information without increasing response error.
56
This contrasts with
some of the earlier research, which led the authors to hypothesize that nowadays respondents are
more used to 10-point scales and are able to better handle them than 20 years earlier.
Balanced versus unbalanced scale: a balanced scale has an equal number of favorable and unfavorable
categories; otherwise, the scale is unbalanced. The scale should be balanced in order to obtain
unbiased data; however, if the distribution of responses is likely to be skewed, an unbalanced scale
with more categories in the opposite direction of the skewness may be appropriate. In other words,
where we expect higher concentration and density of answers it may be reasonable to use more
categories with narrower intervals. However, if researchers have any doubts they should hold on to a
55
Malhotra N.: Marketing Research: An Applied Orientation, 6th Edition, Prentice Hall, Upper Sadle River, New
Jersey 2010, p. 252
56
Coehlo P., Esteves S.: The choice between a five-point and a ten-point scale in the framework of customer
satisfaction measurement, International Journal of Market Research, 49 (3), 2007, pp. 313-339.
67
balanced format, which is safer in part because it avoids evoking adverse reactions from some of the
respondents, who may interpret using unbalanced scale as an attempt at data manipulation.
Odd or even number of categories: with an odd number of categories the middle scale position is
generally designated as neutral. If a neutral stance or attitude is a possibility for at least some of the
respondents, an odd number of categories should be used. Some researchers, though, including the
author of the textbook, are more inclined to use rating scales without neutral option assuming that
respondents’ position cannot be truly neutral and given a precise enough scale (with at least 6 values)
those who would opt for the middle answer could very often find their opinions accommodated in
either a slightly negative or slightly positive answer. It can be observed that a neutral answer is often
treated as a “safe choice” taken by those who are not interested in the study. Therefore, it can be
argued that without such an option survey participants may be forced to think more carefully about
their response, which can likely result in higher quality data.
57
On the other hand, odd numbered scales
tend to have distributions more akin to the normal distribution, which is beneficial for some statistical
analyses.
Forced versus non-forced scales: a forced rating scale does not have a “no opinion” or “no knowledge”
option. Thus, the respondents without an opinion may mark the middle scale position. If a sufficient
proportion of the respondents actually do not have an opinion on the topic, marking the middle
position in this manner will distort measures of central tendency and variance. In situations where the
respondents are expected to have no opinion, the accuracy of the data may be improved by having a
non-forced rating scale which includes a “no opinion” or “no knowledge” category.
Nature and degree of verbal description: Every rating scale has the so-called anchors which are
descriptive labels placed at both ends of the scale. The researcher has a choice of making them sound
more decisively or mildly; for instance in a scale measuring level of agreement with some statement
the extreme options could by tagged as “definitely disagree - definitely agree” or “disagree – agree”,
with the latter descriptors being noticeably milder and thus acceptable to more respondents. In
general, the less forceful expressions with the same number of points on the scale tend to result in
more respondents falling under the first and last categories yielding the frequency distributions more
dispersed and less tall. As with other choices there are no hard and fast rules here, but what should
be recognized is that overly strong descriptors tend to be more problematic as they could cause
57
Brace I.: How to plan, structure and write survey material for effective market research, Second edition,
Kogan Page, London 2008, pp. 72-73.
68
interviewees to avoid the outermost choices, thus decreasing the variance of answers and making
them seem more uniform than they truly are.
Overview of multiple measuring scales
Multiple measuring scales are those that consist of several individual scales of the same type (in terms
of the number of categories, inclusion of neutral answer, presence of no-opinion replies and other
choices) for measuring a set of related variables (e.g. statements on some topic, different brand names
of the same type of product etc.). They can be further classified into:
Comparative scales—where a direct comparison of stimulus objects is elicited. Thus, two brands may
be compared along a dimension such as quality.
Noncomparative scales—respondent is presented with a list of objects – only one at a time – and is
expected to evaluate each of them separately according to some specified criterion. For example when
a scale for evaluating different brands is considered, one brand is rated on the scale measuring its
perceived quality independently of other brands.
The most often used comparative scales in marketing research include:
69
Paired comparison scale. Here a respondent is presented with two objects at a time and asked to
select one object in the pair according to some standard. The data obtained are ordinal in nature. This
is frequently used in marketing when comparisons of products or brands are being made.
Table 10: Example of paired comparison scale
In the table below there are given different pairs of hypermarket chains operating in Poland. We
would like to know which ones you prefer the most for larger weekly shopping. Please consider all
of the pairs by filling 1 in a cell if you prefer the store whose name was placed at the top of the
column or 0 if you would rather go shopping to the store with the name at the beginning