Using video-based observation research methods in primary care health encounters to evaluate complex interactions

Abstract

The purpose of this paper is to describe the use of video-based observation research methods in primary care environment and highlight important methodological considerations and provide practical guidance for primary care and human factors researchers conducting video studies to understand patient-clinician interaction in primary care settings.
We reviewed studies in the literature which used video methods in health care research, and we also used our own experience based on the video studies we conducted in primary care settings.
This paper highlighted the benefits of using video techniques, such as multi-channel recording and video coding, and compared "unmanned" video recording with the traditional observation method in primary care research. We proposed a list that can be followed step by step to conduct an effective video study in a primary care setting for a given problem. This paper also described obstacles, researchers should anticipate when using video recording methods in future studies.
With the new technological improvements, video-based observation research is becoming a promising method in primary care and HFE research. Video recording has been under-utilised as a data collection tool because of confidentiality and privacy issues. However, it has many benefits as opposed to traditional observations, and recent studies using video recording methods have introduced new research areas and approaches.

Full text

Informatics in Primary Care Vol 21, No 4 (2014)
Using video-based observation research
methods in primary care health encounters
to evaluate complex interactions
Onur Asan
Division of General Internal Medicine, Center for Patient Care and Outcomes Research, Medical College of
Wisconsin, Milwaukee, WI, USA
Enid Montague
Division of General Internal Medicine and Geriatrics, Feinberg School of Medicine, Northwestern University,
Chicago, IL, USA
ABSTRACT
Objective The purpose of this paper is to describe the use of video-based obser-
vation research methods in primary care environment and highlight important
methodological considerations and provide practical guidance for primary care and
human factors researchers conducting video studies to understand patient–clini-
cian interaction in primary care settings.
Methods We reviewed studies in the literature which used video methods in
health care research, and we also used our own experience based on the video
studies we conducted in primary care settings.
Results This paper highlighted the benets of using video techniques, such as
multi-channel recording and video coding, and compared “unmanned” video record-
ing with the traditional observation method in primary care research. We proposed a
list that can be followed step by step to conduct an effective video study in a primary
care setting for a given problem. This paper also described obstacles, researchers
should anticipate when using video recording methods in future studies.
Conclusion With the new technological improvements, video-based observation
research is becoming a promising method in primary care and HFE research. Video
recording has been under-utilised as a data collection tool because of condenti-
ality and privacy issues. However, it has many benets as opposed to traditional
observations, and recent studies using video recording methods have introduced
new research areas and approaches.
Keywords: observations, primary care research, video recording
Research article
INTRODUCTION
The health care system is complex and involves a range of
people from various backgrounds and perspectives who com-
municate, interact and collaborate. Several US Institute of
Medicine (IOM) reports have addressed major problems in
healthcare delivery, such as medical errors, poorly designed
medical technologies and poorly designed work environ-
ments.1 To this end, an IOM report proposed a partnership
between health care and industrial and system engineering,
including human factors engineering (HFE), to create solu-
tions for these problems.2 HFE is the study of interactions
of humans with the systems, products and environment and
takes a system approach to study interactions.3 Primary care
is one of the main components of the health care system and
involves the widest scope of health care, including a variety of
Cite this article: Asan O, Montague E. Using video-
based observation research methods in primary care
health encounters to evaluate complex interactions.
Inform Prim Care. 2014;21(4):161–170.
http://dx.doi.org/10.14236/jhi.v21i4.72
Copyright © 2014 The Author(s). Published by
BCS, The Chartered Institute for IT under Creative
Commons license http://creativecommons.org/
licenses/by/4.0/
Author address for correspondence:
Enid Montague
Division of General Internal Medicine and Geriatrics
Feinberg School of Medicine,
Northwestern University,
750 North Lake Shore Drive,
Chicago, IL 60611, USA
Email: enid.montague@northwestern.edu

Informatics in Primary Care Vol 21, No 4 (2014)
Asan and Montague Video-based observation research methods in primary care health encounters 162
demographics, such as patients of different ages and socio-
economic backgrounds, as well as patients with different kinds
of chronic and acute health problems.4 There are several HFE
issues specic to the primary care environment, which human
factors researchers can address with various methods. Some
of them are related to information processing, standardisation,
simplication, work pressure and work load, organisational
design, information access, technology acceptance, usability
and the effect of EHR use on doctor–patient interaction.5 a,b
Depending on the context, HFE researchers are tasked with
determining which components of the system are likely to inu-
ence patient outcome measures (for example, satisfaction,
trust and adherence to treatment). Therefore, the HFE disci-
pline can play a major role in improving overall primary care
health systems leading to better health outcomes.4
Observational research is a commonly used method in pri-
mary care studies. However, direct observation is not always
the best choice for analysing primary care encounters,6 as it is
difcult for researchers to capture all details in a live setting, par-
ticularly when components occur simultaneously.7 Video record-
ing may eliminate some of the challenges that occur in direct
observation research in a primary care setting,8,9 since video
recording accurately records clinical events, allows research-
ers to verify their observations and allows for the collection of
systematic feedback by means of strategic participant review.10
Video data can also give researchers insight into the consis-
tency between self-assessment and observable behaviour.
Finally, the video recording of subjects’ ongoing activities in their
natural setting11 can also be a particularly useful way to employ
ethnographic studies in a complex primary care environment.
However, using video effectively requires the determination
of appropriate research questions and identication of types of
data required beforehand to inform study design. Video record-
ing research also requires technical knowledge to ensure the
appropriate selection of cameras, video quality adjustment and
positioning of cameras.12,13 Currently, enhanced video tech-
nology allows for richer data and facilitates the data collection
process with alternatives such as multi-channel streams and
remote-controlled cameras.14,15 It is essential to note that the
research purpose may affect the type of technology used in
the study design.
This paper outlines the steps for using video methods in
a primary care setting. This paper also addresses potential
benets of using video observation and video analysis meth-
ods, which can be used by human factors and health care
researchers in primary care settings.
Background on the use of video recording in
primary care research
Primary care researchers began using video recordings to
study consultations in the late 1970s.16 In one early study, a
communication analyst videotaped primary care consulta-
tions with a single video camera and subsequently analysed
the communication patterns between doctors and patients to
improve doctors’ communication skills.17 The results showed
that doctors’ communication styles affected patient satisfaction.
Recent studies have used video data to analyse non-verbal
communication cues to inform more effective doctor–patient
interactions.18–20 Video data were also utilised to train doctors
to improve their interactions with patients.16 In addition, studies
have used video recordings to explore doctor–patient–computer
interactions.21–29 These studies were instrumental in identifying
the best spatial organisation of an exam room, better design of
exam-room computers, impact of computer use on communica-
tion and effective use of the computer by the doctor during the
clinical visit. Several studies also utilised video elicitation inter-
views (which are basically interviews done after the recording,
asking the doctors or patients to reect on what they see on
the video) to analyse doctor–patient interaction in the visits for
teaching purposes.30,31 Video elicitation allowed researchers
to integrate the data from the video recording and participants’
related thoughts, beliefs and emotions obtained from the elici-
tation interviews.32 Although traditional observation can provide
a range of interesting and insightful information about primary
care encounters, the encounter occurs through complex and
multiple interactions that can be explored by video data better.
Finally, video data have also been used in health care settings
in addition to primary care consultation for various purposes.33
CONSIDERATIONS FOR COLLECTING VIDEO
DATA IN PRIMARY CARE
Video-recording methods require careful planning in order to
gather data that effectively answer potential research ques-
tions. Table 1, which is derived from our experience of several
studies,26–28 summarizes the steps to conduct a video obser-
vation study in a primary care setting for a given problem.
Some of the elements listed in different categories in Table 1
have inter-dependent nature, for instance, number of partici-
pants, time frame of the study, time needed for ethical approval
and the instruments may all have mutual effect. Furthermore,
video data might have ‘identiable private information’ and
involve human subject data, therefore require some additional
requirements for IRB review.34 In video data collection, com-
pared with traditional observation, studies conducted in US
showed that physicians might have concerns about potential
liability.35 Therefore, there should be a consensus between
administrators and investigators about the purpose of the
research and the methods used. Studies in US reported that
it can also be effective to have some strategies to overcome
doctors’ concerns with condentiality and liability, such as
obtaining certicates of condentiality36 or becoming familiar
with the liability coverage at the clinic where data will be col-
lected.37 As added protection, a previous study reported that
patients were generally less worried than doctors about being
videotaped.32 However, it is still essential to get certicates of
condentiality to protect the participants’ identiable informa-
tion from forced disclosure. IRB approval requires conden-
tiality, but in the case of some sort of legal case (such as a
malpractice case), the court might be able to force research-
ers to reveal this information. Certicates of condentiality –
which allow the investigator and others who have access to

Informatics in Primary Care Vol 21, No 4 (2014)
Asan and Montague Video-based observation research methods in primary care health encounters 163
Table 1 Steps followed to conduct this video study
1. Conceptualising the study
a. Choose an appropriate research question which can be answered by video data
b. Identify potential time frame of the study
c. Decide on the scope of the data collection
d. Decide on any additional data collection instruments, such as interviews and surveys
e. Decide on the required number of personnel for data collection
f. Decide how to link the data from video recording with the other interview and survey data
g. Choose method to analyse the data (quantitative, qualitative or mixed methods)
2. Legal and ethical issues
a. Ensure the study meets with ethical guidelines for human subjects research
b. Describe all details of the procedure of the study
c. Comply with all legal requirements for recording in real environments
d. Obtain legal consent for video recording
e. Ensure all privacy and condentiality issues related to participants’ ID preservation and identiable video data storage
f. Complete and comply with all local regulations, such as online HIPAA training in US to be eligible for human subject research
g. IRB application and nal approval in order to start the project
3. Participants and sampling
a. Determine the number of participants you need
b. Determine the unit of analysis and sampling frame that will most effectively help answer your research question (for example, do you need
a certain number of patients in general or a certain number per physician?
Will you recruit physicians or patients rst? Will you randomly recruit the physicians or have certain eligibility requirements, such as people
within a certain age range? Will participants be paid?)
c. Inform all participants about the benets and risks of your study
d. Conduct the recruitment as planned in the IRB
e. Get informed consent of all people who agreed to participate in the study
4. Data collection and management
a. Decide on all technical specications of the equipment you need
b. Choose an appropriate high quality camera or cameras
c. Choose the best audio recording style (built into camera or separate)
d. Determine the camera layout of the room; get the best angle to ensure a clear view of the patient and doctor
e. Establish a protocol for recording the interactions
f. Maximise the captured area by adjusting the camera angle
g. Create protocols to link the data
h. Sync the audio and video data for the analysis
i. Determine protocols for storing video recordings
j. Secure the hard drives for privacy protection
k. Back up the data
l. Train all researchers, camera persons, interviewers, and so on
5. Data analysis
a. Review the quality of all data
b. Identify the software you will be using to analyse the data
c. Clearly distinguish the research questions and analyse accordingly
d. Create coding schemes to analyse the video based on the variable of interest
e. A pilot run/trial analysis after collecting the data from a smaller sample to prevent potential mismatch
research records to refuse to disclose identifying information
on research participants in any civil, criminal, administrative,
legislative or other proceeding, whether at the federal, state or
local level – might prevent this potential conict between IRB
and legal jurisdictions with respect to discoverability.38
With technological advancements, some researchers
have started to use more complex video methods for data
collection to capture all interactions in detail – such as
body language and gazing direction.9,14,39 A multi-channel
video might be a superior method to single-channel video
depending on research question as it collects a greater
amount of information, allowing the research to see both
the care-provider and the patient simultaneously from dif-
ferent angles.14 For instance, some researches created a
multi-channel video technique and software to capture all
the computer use (including screen-capture, key stroke and

Informatics in Primary Care Vol 21, No 4 (2014)
Asan and Montague Video-based observation research methods in primary care health encounters 164
mouse movement), and doctor–patient interaction in detail,
which enabled them to view simultaneously all data relating
to any time or activity.25 Another study used multi-channel
video recording focussing on the patient’s face, the physi-
cian’s face and the overall interaction to capture eye gaze
patterns.27,28
Furthermore, as video recording technology becomes more
complex, researchers are faced with a wide variety of options,
so it is important to choose the methods and equipment best
suited to a given study. Researchers should standardise the
camera operation protocols and have back up cameras in
case of malfunctioning. In addition, multi-channel video and
audio recording can collect so much data that the process
of analysis becomes more complicated and time consuming.
Therefore, it is essential to determine the specic research
problems to minimise data collection and analysis time.
Table 2 The benets and drawbacks of video method and traditional observational method
Pros Cons
Traditional observational method
Enables rich data Researcher may be intrusive
Can capture events before and after the consultations Aspects of interactions may be missed
Allows researcher to ask follow up questions during the observation Does not allow for data validation
through cross-coding
More effective while shadowing a specic person
in multiple locations Prior work is necessary to prepare organised and
standard observation tools
Researcher is able to see all space in the room Hard to catch non-verbal cues during the encounter
Gives opportunity to concentrate on one individual continuously Cannot capture all interactions in a complex
clinical environment, such as a surgical room
Effective for medical students for training purposes Possibility of Hawthorne effect
Prior training of observers necessary
Cognitive workload for observers
Low inter-rater reliability
Video method
Less intrusive method for data collection (avoiding the observer effect) Reviewing and coding video data is labour intensive
Provides enough detail to analyse the work environment and human
interactions qualitatively and quantitatively Requires additional IRB procedures
Allows researchers to analyse events retrospectively Raises concerns about the discoverability and condentiality
of participants
Allows researchers to capture simultaneous complex interactions Additional equipment cost
Allows researchers to review consultations repeatedly Additional data management concerns
Creates a permanent and complete record Aggregation can be difcult and intrusive
Potential for multiple viewing/reviewing It can limit range of settings
Higher inter-rater reliability (with the help of practice coding) Possibility of Hawthorne effect
Can be used to establish connections between perceptions
and the observed activities during the visit Higher overall cost
Retains the captured data with no loss of its richness for reviewing
Enables self-evaluation and reection
Generates a large amount of data
Allows researchers to capture activities in much of their complexity in their
natural settings over an extended period of time
Allows for scientic rigour when conducted by trained researchers
Can be reviewed by both researchers and participants,
increasing the scope of interpretation
THE BENEFITS AND DRAWBACKS OF VIDEO
METHODS
Table 2 shows the pros and cons of traditional human obser-
vation method and video recording by ‘unmanned’ cameras.
This table was established based on our own experience and
previous studies.6,7,36,37,39,40–42
Video methods can be effective for research that can be
conducted in a single room (for example, the patient exam
room in a primary care clinic), since the cameras can be set
up in a xed position, specically focussing on the interaction
in the exam room. In addition, cameras can also be used in
various ways based on research questions because cameras
can be carried, placed in multiple rooms or cameras’ angle
can be changed in real-time by remote control. When the
required conditions are met, the video method can provide

Informatics in Primary Care Vol 21, No 4 (2014)
Asan and Montague Video-based observation research methods in primary care health encounters 165
a rich collection of data. For instance, in one study, we used
multiple small cameras with sufcient battery time and SD
cards and hooked them on to the walls or side of the desks
in the room. Remote control was utilised to start and stop
the camera, and a remote control was left with the doctor so
the doctor could stop the recording if the patients did not feel
comfortable or the conversation topic becomes highly con-
dential, such as drug use or suicide.
Furthermore, video method also limits the Hawthorne effect –
which is the possibility of altering the behaviour of partici-
pants – since video cameras have been shown to inuence
participant behaviour far less than a human observer.43
However, some people may be less willing to be videotaped
as opposed to live observation and feel there is more risk
involved in video data due to several reasons: a) video record-
ings may be viewed by multiple people over time, b) outsiders
may gain access to video data that are improperly stored,
and c) a person’s identity may be more readily determined
from a video recording than from written data. On the other
hand, video data might improve ecological validity, since the
video data give more complete (and visual) information about
the real environment rather than traditional observers’ obser-
vation notes.44
VIDEO DATA MANAGEMENT AND ANALYSIS
Observation data, including both video and non-video data, are
condential. However, video data introduce more risk to overall
condentiality because video data keep all interaction in a high
delity format for several years and might be accessed by mul-
tiple people for research or non-research purposes unless suf-
cient precautions are taken. Video data should be stored on
a secure storage without links to other identiable information,
such as address, name and social security number.32
Coding is a standard procedure to analyse the video data.
Coding is an established procedure that facilitates analys-
ing the video by identifying the tasks and interactions in the
video.19 A coding scheme classies variables of interest in the
video according to the purpose of the analysis, and it speeds
up the coding process. Development of coding scheme should
be informed by the literature.45 Each variable in the coding
scheme should be well dened, and the start and stop time of
all variables should be standardised. This may help to improve
the reliability of data coding and decrease biases of different
coders. For example, in one study, coders were interested in
the gaze direction of the doctor and patient46 and created a
coding scheme including the subject (patient or doctor) and the
object of the gaze (patient, care provider, computer, chart, and
so on.). This scheme allowed for a thorough and specic analy-
sis of gaze based on subject, object and duration, such as total
duration of doctor’s gaze at computer and patient during a visit.
Video data can be coded both quantitatively and qualita-
tively depending on the purpose of the research. Quantitative
data might include the duration of specic behaviours in the
visit. Software packages can help quantify all continuous
behaviour (such as gazing or typing) to obtain relevant data
with respective time frames.27 It is also possible to visualise
the sequence of the behaviours using software. Qualitative
analysis might be a thematic description of a practitioner’s
behaviour during the entire visit, such as patient-focussed or
computer-focussed. Qualitative data might also be gathered
based on verbal communication, such as analysing turn tak-
ings and sequence of utterances.18 Some studies also used
tools such as check lists (physicians' behaviour checklist) to
capture human performance data from the video recording,47
such as counting the occurrence of specic doctors' behav-
iours during the doctor–patient encounter in the video data.48
Video analysis tools
Several computer programs have been used to analyse vid-
eos effectively and accurately. These programs comprise dif-
ferent features to capture and analyse video and audio and
can produce different types of results, such as numerical
and visual. A few of these programs used in previous stud-
ies27,44,49,50 are listed in Table 3.
POTENTIAL USES OF VIDEO DATA IN
PRIMARY CARE RESEARCH
Evaluating complex constructs and interactions in real, com-
plex and dynamic clinical environments plays an important
role in improving health care system; and thus, it is a prior-
ity for HFE researchers. Effective functioning of the health
care system depends on the interactions among people
(patients, physicians and other medical staff) and the interac-
tion between people and technology.4 Therefore, their inter-
actions should be explored in detail to improve overall health
care systems. Video data can contribute to studies exploring
doctor–patient interaction for different research purposes,
such as analysing the decision-making process between
doctor and patient,30 determining the effects of non-verbal
behaviours between patient and doctor that inuence their
decisions,31 exploring factors which yield misunderstanding
and disagreement during the interactions51 and investigat-
ing patients’ responsiveness to specic doctor behaviours.52
One study also reported a list of seven different goals to use
video-recorded consultations.39 Furthermore, video data can
also contribute to the analysis of people–technology interac-
tion in primary care settings.53 For instance, it is critical to
capture accurately both the pathways users take and the
errors users commit while conducting a usability test of a
mobile device. The traditional observation method might fail
to obtain all data related to pathways and errors during real
patient encounters, so video recording could record all nec-
essary data from the screen to be analysed. In addition, with
the integration of an eye gaze tracker, video data can pro-
vide rich information about eye gaze pathways to analyse the
usability of medical software programs.
Video data have also been used to create and test a num-
ber of different interactions models in the primary care envi-
ronment. Provided below is a list of several studies that used
video data along with the various methods and models they
used to analyse verbal, non-verbal and technology interac-
tions in the clinical environment (Table 4).

Informatics in Primary Care Vol 21, No 4 (2014)
Asan and Montague Video-based observation research methods in primary care health encounters 166
Video data can also contribute to doctors’ training since
it provides an opportunity for doctors to review their own
activities.40 Multiple studies have recorded consultations in
the primary care environment to assess clinical competence
and design educational interventions.14 Video data were also
used with simulations for medical education.67 Clinicians’
interaction style with patient and computer during the visit
can inuence patient outcomes such as satisfaction, trust
and adherence,68 so video data analysis can also contribute
to teaching medical students better ways of interacting with
patients and EHRs during the encounter.
Video data and sociotechnical design
The components of a sociotechnical system include the individ-
ual (such as health care workers), tasks, tools and technologies,
the physical environment and organisational conditions.69 It is
essential to understand users of the system and interactions
among these users in real settings to address sociotechnical
design concerns.70 It is also necessary to better understand
the impact of boundaries on sociotechnical systems and their
implications for physical, cognitive and psychosocial ergonom-
ics. Furthermore, effective design, implementation and use
of newly introduced technologies into the overall system is
strongly related to the fundamentals of human factors ergo-
nomics.71 A number of studies have focussed on the concept
of sociotechnical factors that complicate health information
systems deployment,72 including the interaction between the
technical features of a health information system and the social
features of a health care work environment.73 After a new sys-
tem implementation, sociotechnical interactions have a direct
effect on the success of the process. In the future, many new
medical technologies will be introduced into the system. Video
recording might also be a strong tool to explore technology
interventions, which can make sociotechnical systems more
effective and efcient. For instance, video data can be used
to analyse the current medical technology, such as electronic
health records (EHR) and to inform how new EHR can be inte-
grated into the sociotechnical system more effectively.
Table 3 Video analysis computer programs utilised in several studies- partially adapted from (4, 43)
Programs Features
Observer/Noldus
(www.noldus.com) Allows users to annotate and log video data and analyse time line
MacSHAPA
(http://acs.ist.psu.edu/dismal/macshapa.html Integrated with VCR (video cassette recorder)
control, annotation and coding and post-coding analysis function
A.C.T Touch coding (that is, one key stroke input)
for reviewing videotapes in real-time observations
OCS tools
(http://trctech.com/send.php?ocs.php) Set of tools that enables VCR control,
time code reading, input of annotation and coding
Vanna This can display multiple video sources along with other time-stamped information on a single
computer monitor
VINA Manual and scripted VCR control
VCR control by pointing
Touch coding of events and activities
Temporal graphic representation
Data synchronisation with VCR
Tagging software Specically to capture several behaviours
Computer-assisted time and event recorder
(CATER) This computer program has been used to help record extensive observational data
from consultations
The ALFA
(Activity Log Files Aggregation) toolkit A method for precise observation of the consultation with multiple video channels
Atlas.ti
(www.atlasti.com) Organise text, graphic, audio and visual data les,
along with coding, memos and ndings into a project
QSR Nvivo
(www.qsrinternational.com) Analyse, manage, shape and store qualitative data
HyperRESEARCH
(www.researchware.com) Easy to use qualitative software package enables researchers to code and retrieve, build
theories, and conduct analysis of the data

Informatics in Primary Care Vol 21, No 4 (2014)
Asan and Montague Video-based observation research methods in primary care health encounters 167
Table 4 Type of analysis used by video observation studies
Type of analysis and methods
(corresponding reference) Explanations of what to measure
Observation
(Hermansson et al., 1988)54
The authors observed positive behaviours
such as gazing, body directions and gestures to see if the patient was
satised with the behaviours of the doctors
Roter interaction analysis system (RIAS)
(Roter, 1977)55 A content analysis system for verbal communication
Lag-sequential analysis
(Connor, Fletcher, & Salmon, 2009)56
Two-way analysis of non-verbal cues or verbal communication cues between
doctor and patient
Gender-based observation studies
(Hall, Irish, Roter, Ehrlich, & Miller, 1994)57
Specic correlation of doctor's gender's effect on verbal
and non-verbal communication
Bales Interaction analysis system
(Ong et al., 1995)47
Analyses interaction and information exchange between
doctor and patient; focusses on instrumental behaviours
Interpersonal skill evaluation
(Burchard & Rowland-Morin, 1990)58
Analyses surgeon's interpersonal skills and the appropriateness of the
physician's behaviour for a clinical visit
Maastricht history-taking
and advice checklist
(Kraan et al.,1989)59
Analyses physician’s interview skills during initial interviews
in the primary care units
Observer checklist
(Ong et al., 1995)47 Analyses specic interactions between doctor and patient
Factor analysis
(Duggan & Parrott, 2001)60
Based on coding of non-verbal behaviours from videos. The mean scores for use
of each type of non-verbal and verbal behaviour were computed separately
for the introduction and diagnosis segments to allow comparisons between
these interaction events
Retrospective approach
(Als, 1997)61
The videos were watched with doctors to analyse their behaviours
in the consultation together
Correlational analysis
(Collins, Schrimmer, Diamond, & Burke, 2010)62 Analyses the relationship between verbal and non-verbal communication skills
Non-verbal accommodation analysis system (NAAS)
(D'Agostino & Bylund, 2010)63
The NAAS enables researchers to investigate the ways in which physicians
and patients manage social distance through non-verbal behaviours within
medical interactions from a theoretically
informed perspective
Conversational analysis
(Newman, Button, & Cairns, 2010)64
Specically, turn taking in the communication of the
doctor and patient in the clinic
Goffman’s dramaturgical methodology
(Pearce et al., 2008)65
Dramaturgy analyses the consultation as though it were a dramatic play where
the consulting room is the stage and the participants are actors playing roles
Observational quantitative
(Mast, Hall, Klöckner, & Choi, 2008)66
Quanties non-verbal behaviours in
the patient visits using a special software tool
CONCLUSION
Video-based observation research is a promising method in
primary care and HFE research. Video recording has been
under-utilised as a data collection tool because of conden-
tiality and privacy issues. However, it has many benets,
and recent studies using video recording methods have
introduced new research areas and approaches. There are
several possible applications of video recording in HFE and
sociotechnical research as well as in traditional clinician
training, such as performance evaluation and analysing
clinician–patient interactions. This paper is intended to pre-
pare researchers for using video-based observation studies
in primary care settings by evaluating the necessary steps
involved, including the legal and condentiality processes,
technical aspects, data collection and data analysis and by
describing its contribution to human factors research.
A systematic analysis of video recordings gives research-
ers opportunities to nd solutions for human factors-related
problems, as well as a sociotechnical systems analysis of

Informatics in Primary Care Vol 21, No 4 (2014)
Asan and Montague Video-based observation research methods in primary care health encounters 168
interventions in primary care. Video recording method will be
increasingly used in future research not only in the health
care domain but also in other domains, such as usability and
social interaction. Video recording observation studies in pri-
mary care environment will continue helping to answer a vari-
ety of emerging research questions in primary care.
Acknowledgements
The project described was supported by the Clinical and
Translational Science Award (CTSA) programme, through
the NIH National Center for Advancing Translational Sciences
(NCATS), grant UL1TR000427.
REFERENCES
1. Kohn LT, Corrigan JM and Donaldson MS. To Err is Human:
Building a Safer Health System. Atlanta, GA: National
Academies Press, 2000.
2. Reid PP. Building a Better Delivery System: A New Engineering/
Health Care Partnership. Atlanta, GA: National Academy Press,
2005.
3. Dul J, Bruder R, Buckle P, Carayon P, Falzon P, Marras WS et al.
A strategy for human factors/ergonomics: developing the disci-
pline and profession. Ergonomics 2012;55(4):377–395. http://
dx.doi.org/10.1080/00140139.2012.661087. PMid:22332611.
4. Carayon P. Handbook of Human Factors and Ergonomics in
Health Care and Patient Safety, Boca Raton, FL: CRC, 2007.
5a. Beasley JW, Hankey TH, Erickson R, Stange KC, Mundt M,
Elliott M et al. How many problems do family physicians manage
at each encounter? A WReN study. The Annals of Family
Medicine 2004;2(5):405–10. http://dx.doi.org/10.1370/afm.94.
PMCid:PMC1466713.
5b. Carayon P. Emerging role of human factors and ergonomics
in healthcare delivery–A new eld of application and inuence
for the IEA. Work. A Journal of Prevention, Assessment and
Rehabilitation 2012;41:5037–40.
6. Carthey J. The role of structured observational research in health
care. Quality and Safety in Health Care 2003;12(suppl 2):ii13–6.
http://dx.doi.org/10.1136/qhc.12.suppl_2.ii13. PMid:14645890;
PMCid:PMC1765776.
7. Wears RL. Beyond error. Academic Emergency Medicine
2000;7(11):1175–6. http://dx.doi.org/10.1111/j.1553-2712.2000.
tb00457.x. PMid:11073460.
8. Jeffcott SA and Mackenzie CF. Measuring team performance
in healthcare: Review of research and implications for
patient safety. Journal of Critical Care 2008;23(2):188–96.
http://dx.doi.org/10.1016/j.jcrc.2007.12.005. PMid:18538211.
9. Kumarapeli P and de Lusignan S. Using the computer in the clini-
cal consultation; setting the stage, reviewing, recording, and tak-
ing actions: multi-channel video study. Journal of the American
Medical Informatics Association 2013;20(e1):67–e75. http://
dx.doi.org/10.1136/amiajnl-2012-001081. PMid:23242763;
PMCid:PMC3715353.
10. Seagull FJ and Guerlain S (Eds). Observation Measure of Team
Process and Performance in Health Care, Human Factors and
Ergonomics Society, 2003.
11. Schaeff JH. Videotape: New techniques of observation and anal-
ysis in anthropology. Principles of Visual Anthropology 2009:255.
12. Theadom A, De Lusignan S, Wilson E and Chan T. Using
three-channel video to evaluate the impact of the use of the
computer on the patient-centredness of the general practice
consultation. Informatics in Primary Care 2003;11(3):149–56.
PMid:14680538.
13. Sheeler I, Koczan P, Wallage W and de Lusignan S. Low-cost
three-channel video for assessment of the clinical consultation.
Informatics in Primary Care 2007;15(1):25–31. PMid:17612472.
14. Leong A, Koczan P, De Lusignan S and Sheeler I. A framework
for comparing video methods used to assess the clinical consul-
tation: a qualitative study. Informatics for Health and Social Care
2006;31(4):255–65. http://dx.doi.org/10.1080/14639230600991668.
PMid:17135058.
15. Refsum C, Kumarapeli P, Gunaratne A, Dodds R, Hasan A
and De Lusignan S. Measuring the impact of different brands
of computer systems on the clinical consultation: a pilot study.
Informatics in Primary Care 2008;16(2):119–27. PMid:18713528.
16. Pilnick A, Hindmarsh J and Gill V. Beyond ‘doctor and
patient’: developments in the study of healthcare interac-
tions. Sociology of Health & Illness 2009;31(6):787–802.
http://dx.doi.org/10.1111/j.1467-9566.2009.01194.x.
PMid:19843267.
17. Frankel R. The laying on of hands: aspects of the organization
of gaze, touch, and talk in a medical encounter. The social orga-
nization of doctor-patient communication. 1983:19–54.
18. Bensing JM, Verheul W and van Dulmen AM. Patient anxiety in the
medical encounter: a study of verbal and nonverbal communica-
tion in general practice. Health Education 2008;108(5):373–83.
http://dx.doi.org/10.1108/09654280810899993.
19. Zandbelt L, Smets E, Oort F, Godfried M and de Haes H.
Patient participation in the medical specialist encounter:
does physicians' patient-centred communication matter?
Patient Education and Counseling 2007;65(3):396–406.
http://dx.doi.org/10.1016/j.pec.2006.09.011. PMid:17085006.
20. Bensing J, Tromp F, Van Dulmen S, van den Brink-Muinen A,
Verheul W and Schellevis F. Shifts in doctor-patient communi-
cation between 1986 and 2002: a study of videotaped general
practice consultations with hypertension patients. BMC Family
Practice 2006;7(1):62. http://dx.doi.org/10.1186/1471-2296-7-
62. PMid:17064407; PMCid:PMC1630692.
21. Pearce C, Kumarpeli P and de Lusignan S. Getting seam-
less care right from the beginning-integrating computers into
the human interaction. Studies in Health Technology and
Informatics 2010;155:196. PMid:20543329.
22. Frankel R, Altschuler A, George S, Kinsman J, Jimison H,
Robertson NR et al. Effects of exam-room computing on
clinician-patient communication: a longitudinal qualitative study.
Journal of General Internal Medicine 2005;20(8):677–82. http://
dx.doi.org/10.1111/j.1525-1497.2005.0163.x. PMid:16050873;
PMCid:PMC1490186.
23. Pearce C, Dwan K, Arnold M and Phillips C. Analysing the doctor
patient computer relationship: the use of video data. Informatics
in Primary Care 2006;14(4):221–6. PMid:17504573.

Informatics in Primary Care Vol 21, No 4 (2014)
Asan and Montague Video-based observation research methods in primary care health encounters 169
24. Pug B, Kumarapeli P, van Vlymen J, Chan T, Ammenwerth E
and de Lusignan S (Eds). Measuring the impact of the computer
on the consultation: An application to synchronise multi-channel
video, automated monitoring, and rating scales, 2008.
25. De Lusignan S, Kumarapeli P, Chan T, Pug B, Van Vlymen J,
Jones B et al. The ALFA (Activity Log Files Aggregation)
toolkit: a method for precise observation of the consulta-
tion. Journal of Medical Internet Research 2008;10(4).
http://dx.doi.org/10.2196/jmir.1080. PMid:18812313;
PMCid:PMC2629369.
26. Asan O and Montague E. Technology-mediated information
sharing between patients and clinicians in primary care encoun-
ters. Behaviour & Information Technology 2013(Epub ahead of
print):1–12.
27. Asan O and Montague E. Physician interactions with electronic
health records in primary care. Health Systems 2012;1(2):
96–103. http://dx.doi.org/10.1057/hs.2012.11. PMid:24009982;
PMCid:PMC3760434.
28. Montague E and Asan O. Dynamic modeling of patient and
physician eye gaze to understand the effects of electronic
health records on doctor–patient communication and attention.
International Journal of Medical Informatics 2014;83(3):225–34.
http://dx.doi.org/10.1016/j.ijmedinf.2013.11.003. PMid:24380671.
29. Kazmi Z. Effects of exam room EHR use on doctor-patient
communication: a systematic literature review. Informatics in Primary
Care 2014;21(1):30–9. http://dx.doi.org/10.14236/jhi.v21i1.37.
PMid:24629654.
30. Saba GW, Wong ST, Schillinger D, Fernandez A, Somkin CP, Wilson
CC et al. Shared decision making and the experience of partnership
in primary care. The Annals of Family Medicine 2006;4(1):54–62.
http://dx.doi.org/10.1370/afm.393. PMid:16449397;
PMCid:PMC1466999.
31. Henry SG, Forman JH, and Fetters MD. ‘How do you
know what Aunt Martha looks like?’A video elicitation study
exploring tacit clues in doctor–patient interactions. Journal
of Evaluation in Clinical Practice 2011;17(5):933–9.
http://dx.doi.org/10.1111/j.1365-2753.2010.01628.x.
PMid:21210905.
32. Henry SG and Fetters MD. Video elicitation interviews: a
qualitative research method for investigating physician-patient
interactions. The Annals of Family Medicine 2012;10(2):118–25.
http://dx.doi.org/10.1370/afm.1339. PMid:22412003;
PMCid:PMC3315127.
33. Iedema R, Forsyth R, Georgiou A, Braithwaite J and Westbrook
J. Video research in health. Qualitative Research Journal 2006;
6(2):15–30. http://dx.doi.org/10.3316/QRJ0602015.
34. Menikoff J. Where's the Law-Uncovering the Truth about IRBs and
Censorship. Northwestern University Law Review 2007;101:791.
35. Guerlain S, Turrentine B, Adams R and Calland JF. Using
video data for the analysis and training of medical person-
nel. Cognition, Technology and Work 2004;6(3):131–8.
http://dx.doi.org/10.1007/s10111-004-0154-2.
36. Broyles L, Tate J and Happ M. Videorecording in clinical research:
mapping the ethical terrain. Nursing Research 2008;57(1):59.
37. Weinger M, Gonzales D, Slagle J and Syeed M. Video capture
of clinical care to enhance patient safety. Quality and Safety in
Health Care 2004;13(2):136.
38. Wolf LE, Zandecki J and Lo B. The certicate of condentiality
application: a view from the NIH Institutes. IRB: Ethics and Human
Research 2004;26(1):14–8. http://dx.doi.org/10.2307/3563582.
PMid:15281213.
39. Coleman T. Using video-recorded consultations for research
in primary care: advantages and limitations. Family Practice
2000;17(5):422. http://dx.doi.org/10.1093/fampra/17.5.422.
PMid:11021903.
40. Mackenzie C and Xiao Y. Video techniques and data
compared with observation in emergency trauma care.
Quality and Safety in Health Care 2003;12(suppl 2):ii51.
http://dx.doi.org/10.1136/qhc.12.suppl_2.ii51. PMid:14645896;
PMCid:PMC1765775.
41. Grimshaw AD. Sound-Image Data Records for Research on
Social Interaction. Sociological Methods & Research. Special
Issue on Sound-Image Records in social Interaction Research
1982; 11(2): 121–44.
42. Pearce C, Arnold M, Phillips, C and Dwan K. Methodological
considerations of digital video observation: beyond conversation
analysis. International Journal of Multiple Research Approaches
2010; 4(2): 90–9. http://dx.doi.org/10.5172/mra. 2010.4.2.090.
43. Pringle M and Stewart-Evans C. Does awareness of being video
recorded affect doctors' consultation behaviour? The British
Journal of General Practice 1990;40(340):455. PMid:2271278;
PMCid:PMC1371415.
44. Brewer MB. Research design and issues of validity. In Handbook
of Research Methods in Social and Personality Psychology.
Cambridge: Cambridge University Press, 2000:3–16.
45. Creswell JW. Qualitative Inquiry & Research Design: Choosing
Among Five Approaches. Thousand Oaks, CA: Sage 2007.
46. Montague E, Xu J, Chen P, Asan O, Barrett BP and Chewning B.
Modeling eye gaze patterns in clinician–patient interac-
tion with lag sequential analysis. human factors. The
Journal of the Human Factors and Ergonomics Society
2011;53(5):502–16.
47. Ong L, De Haes J, Hoos A and Lammes F. Doctor-
patient communication: a review of the literature.
Social Science and Medicine 1995;40(7):903–18.
http://dx.doi.org/10.1016/0277-9536(94)00155-M.
48. Blanchard CG, Labrecque MS, Ruckdeschel JC and Blanchard
EB. Information and decision-making preferences of hospitalized
adult cancer patients. Social Science and Medicine 1988;27(11):
1139–45. http://dx.doi.org/10.1016/0277-9536(88)90343-7.
49. Harrison BL (Ed) Video annotation and multimedia interfaces:
from theory to practice. Proceedings of the Human Factors
and Ergonomics Society Annual Meeting. Thousand Oaks, CA:
Sage, 1991.
50. De Lusignan S, Kumarapeli P, Debar S, Kushniruk A and
Pearce C. Using an open source observational tool to measure
the inuence of the doctor’s consulting style and the computer
system on the outcomes of the clinical consultation. Studies
in Health Technology and Informatics 2009;150:1017–21.
PMid:19745467.
51. Cegala DJ, McGee DS and McNeilis KS. Components of patients'
and doctors' perceptions of communication competence dur-
ing a primary care medical interview. Health Communication
1996;8(1):1–27. http://dx.doi.org/10.1207/s15327027hc0801_1.

Informatics in Primary Care Vol 21, No 4 (2014)
Asan and Montague Video-based observation research methods in primary care health encounters 170
52. Cromarty I. What do patients think about during their consulta-
tions? A qualitative study. The British Journal of General Practice
1996;46(410):525. PMid:8917871; PMCid:PMC1239747.
53. Pearce C, Dwan K, Arnold M, Phillips C and Trumble S. Doctor,
patient and computer--a framework for the new consultation.
International Journal of Medical Informatics 2009;78(1):32–8.
http://dx.doi.org/10.1016/j.ijmedinf.2008.07.002. PMid:18752989.
54. Hermansson GL, Webster AC and McFarland K. Counselor
deliberate postural lean and communication of facilitative con-
ditions. Journal of Counseling Psychology 1988;35(2):149–53.
http://dx.doi.org/10.1037/0022-0167.35.2.149.
55. Roter DL. Patient participation in the patient-provider interaction:
the effects of patient question asking on the quality of interac-
tion, satisfaction and compliance. Health Education and Behavior
1977;5(4):281. http://dx.doi.org/10.1177/109019817700500402.
56. Connor M, Fletcher I and Salmon P. The analysis of verbal inter-
action sequences in dyadic clinical communication: A review of
methods. Patient Education and Counseling 2009;75(2):169–77.
http://dx.doi.org/10.1016/j.pec.2008.10.006. PMid:19097842.
57. Hall JA, Irish JT, Roter DL, Ehrlich CM and Miller LH. Gender in med-
ical encounters: An analysis of physician and patient communica-
tion in a primary care setting. Health Psychology 1994;13(5):384.
http://dx.doi.org/10.1037/0278-6133.13.5.384. PMid:7805632.
58. Burchard KW and Rowland-Morin PA. A new
method of assessing the interpersonal skills of sur-
geons. Academic Medicine 1990;65(4):274–6.
http://dx.doi.org/10.1097/00001888-199004000-00012.
PMid:2334511.
59. Kraan H, Crijnen A, Zuidweg J, Van der Vleuten C and Imbos T.
Evaluating Undergraduate Training –A Checklist for Medical
Interviewing Skills. In: Communicating With Medical Patients.
Newbury Park, CA: Sage, 1989, pp. 167–77.
60. Duggan P and Parrott L. Physicians' nonverbal rapport
building and patients' talk about the subjective compo-
nent of illness. Human Communication Research 2001;
27(2):299–311. http://dx.doi.org/10.1093/hcr/27.2.299.
http://dx.doi.org/10.1111/j.1468-2958.2001.tb00783.x.
61. Als AB. The desk-top computer as a magic box: patterns of
behaviour connected with the desk-top computer; GPs' and
patients' perceptions. Family Practice 1997;14(1):17. http://
dx.doi.org/10.1093/fampra/14.1.17. PMid:9061339.
62. Collins L, Schrimmer A, Diamond J and Burke J. Evaluating ver-
bal and non-verbal communication skills, in an ethnogeriatric
OSCE. Patient Education and Counseling 2011;83(2):158–62.
PMid:20561763.
63. D'Agostino TA and Bylund CL. The nonverbal accommoda-
tion analysis system (NAAS): initial application and evalu-
ation. Patient Education and Counseling. 2011;85(1):33–9.
PMid:20851559.
64. Newman W, Button G and Cairns P. Pauses in doctor–patient
conversation during computer use: The design signicance of
their durations and accompanying topic changes. International
Journal of Human–Computer Studies 2010;68(6):398–409.
http://dx.doi.org/10.1016/j.ijhcs.2009.09.001.
65. Pearce C, Trumble S, Arnold M, Dwan K and Phillips C. Computers
in the new consultation: Within the rst minute. Family Practice
2008;25(3):202–8. http://dx.doi.org/10.1093/fampra/cmn018.
PMid:18504254.
66. Mast MS, Hall JA, Klöckner C and Choi E. Physician gender
affects how physician nonverbal behavior is related to
patient satisfaction. Medical Care 2008;46(12):1212–8.
http://dx.doi.org/10.1097/MLR.0b013e31817e1877.
PMid:19300310.
67. Cooper JB, Barron D, Blum R, Davison JK, Feinstein D, Halasz J
et al. Video teleconferencing with realistic simulation for medical
education* 1. Journal of Clinical Anesthesia 2000;12(3):256–61.
http://dx.doi.org/10.1016/S0952-8180(00)00148-3.
68. Roter D, Frankel R, Hall J and Sluyter D. The expression of
emotion through nonverbal behavior in medical visits. Journal
of General Internal Medicine 2006;21(S1):S28–34. http://
dx.doi.org/10.1111/j.1525-1497.2006.00306.x. PMid:16405706;
PMCid:PMC1484830.
69. Carayon P. Human factors of complex sociotechnical
systems. Applied Ergonomics 2006;37(4):525–35.
http://dx.doi.org/10.1016/j.apergo.2006.04.011. PMid:16756937.
70. Wilson JR. Fundamentals of ergonomics in theory and
practice. Applied ergonomics 2000;31(6):557–67.
http://dx.doi.org/10.1016/S0003-6870(00)00034-X.
71. Lawler EK, Hedge A and Pavlovic-Veselinovic S. Cognitive ergo-
nomics, socio-technical systems, and the impact of healthcare infor-
mation technologies. International Journal of Industrial Ergonomics.
2011;41(4):336–44. http://dx.doi.org/10.1016/j.ergon.2011.02.006.
72. Berg M. Implementing information systems in health
care organizations: myths and challenges. International
Journal of Medical Informatics. 2001;64(2–3):143–56.
http://dx.doi.org/10.1016/S1386-5056(01)00200-3.
73. Ludwick D and Doucette J. Adopting electronic medical records
in primary care: lessons learned from health information
systems implementation experience in seven countries.
International Journal of Medical Informatics 2009;78(1):
22–31. http://dx.doi.org/10.1016/j.ijmedinf.2008.06.005.
PMid:18644745.