ArticlePDF Available

Measuring exposure to bullying and harassment in health professional students in a clinical workplace environment: Evaluating the psychometric properties of the clinical workplace learning NAQ-R scale

Authors:

Abstract and Figures

Background: Instruments that measure exposure to bullying and harassment of students learning in a clinical workplace environment (CWE) that contain validity evidence are scarce. The aim of this study was to develop such a measure and provide some validity evidence for its use. Method: We took an instrument for detecting bullying of employees in the workplace, called the Negative Acts Questionnaire – Revised (NAQ-R). Items on the NAQ-R were adapted to align with our context of health professional students learning in a CWE and added two new factors of sexual and ethnic harassment. This new instrument, named the Clinical Workplace Learning NAQ-R, was distributed to 540 medical and nursing undergraduate students and we undertook a Confirmatory Factor Analysis (CFA) to investigate its construct validity and factorial structure. Results: The results provided support for the construct validity and factorial structure of the new scale comprising five factors: workplace learning-related bullying (WLRB), person-related bullying (PRB), physically intimidating bullying (PIB), sexual harassment (SH), and ethnic harassment (EH). The reliability estimates for all factors ranged from 0.79 to 0.94. Conclusion: This study provides a tool to measure the exposure to bullying and harassment in health professional students learning in a CWE.
Content may be subject to copyright.
Full Terms & Conditions of access and use can be found at
https://www.tandfonline.com/action/journalInformation?journalCode=imte20
Medical Teacher
ISSN: 0142-159X (Print) 1466-187X (Online) Journal homepage: https://www.tandfonline.com/loi/imte20
Measuring exposure to bullying and harassment
in health professional students in a clinical
workplace environment: Evaluating the
psychometric properties of the clinical workplace
learning NAQ-R scale
Kelby Smith-Han, Emma Collins, Mustafa Asil, Althea Gamble Blakey, Lynley
Anderson, Elizabeth Berryman & Tim J. Wilkinson
To cite this article: Kelby Smith-Han, Emma Collins, Mustafa Asil, Althea Gamble Blakey, Lynley
Anderson, Elizabeth Berryman & Tim J. Wilkinson (2020) Measuring exposure to bullying and
harassment in health professional students in a clinical workplace environment: Evaluating the
psychometric properties of the clinical workplace learning NAQ-R scale, Medical Teacher, 42:7,
813-821, DOI: 10.1080/0142159X.2020.1746249
To link to this article: https://doi.org/10.1080/0142159X.2020.1746249
View supplementary material Published online: 14 Apr 2020.
Submit your article to this journal Article views: 394
View related articles View Crossmark data
Measuring exposure to bullying and harassment in health professional
students in a clinical workplace environment: Evaluating the psychometric
properties of the clinical workplace learning NAQ-R scale
Kelby Smith-Han
a
, Emma Collins
b
, Mustafa Asil
c
, Althea Gamble Blakey
d
, Lynley Anderson
e
,
Elizabeth Berryman
f
and Tim J. Wilkinson
g
a
Department of Anatomy and Otago Medical School, University of Otago, Dunedin, New Zealand;
b
School of Nursing, Otago
Polytechnic, Dunedin, New Zealand;
c
Educational Assessment Research Unit (EARU), College of Education, University of Otago, Dunedin,
New Zealand;
d
Otago Medical School, University of Otago, Dunedin, New Zealand;
e
Department of Bioethics Centre, University of
Otago, Dunedin, New Zealand;
f
Waitemata District Health Board, Auckland, New Zealand;
g
Deans Department and Department of
Medicine, Otago Medical School, University of Otago, Christchurch, New Zealand
ABSTRACT
Background: Instruments that measure exposure to bullying and harassment of students learning
in a clinical workplace environment (CWE) that contain validity evidence are scarce. The aim of this
study was to develop such a measure and provide some validity evidence for its use.
Method: We took an instrument for detecting bullying of employees in the workplace, called the
Negative Acts Questionnaire Revised (NAQ-R). Items on the NAQ-R were adapted to align with
our context of health professional students learning in a CWE and added two new factors of sexual
and ethnic harassment. This new instrument, named the Clinical Workplace Learning NAQ-R, was
distributed to 540 medical and nursing undergraduate students and we undertook a Confirmatory
Factor Analysis (CFA) to investigate its construct validity and factorial structure.
Results: The results provided support for the construct validity and factorial structure of the new
scale comprising five factors: workplace learning-related bullying (WLRB), person-related bullying
(PRB), physically intimidating bullying (PIB), sexual harassment (SH), and ethnic harassment (EH).
The reliability estimates for all factors ranged from 0.79 to 0.94.
Conclusion: This study provides a tool to measure the exposure to bullying and harassment in
health professional students learning in a CWE.
KEYWORDS
Education environment;
undergraduate; work-based
Introduction
Bullying and harassment of students in health professional
education is a significant, ongoing and widespread prob-
lem, with evidence from medicine (Fnais et al. 2014), nurs-
ing (Clarke et al. 2012), dentistry (Rowland et al. 2010),
physiotherapy (Whiteside et al. 2014), and pharmacy
(Knapp et al. 2014). In addressing bullying and harassment
in a clinical workplace environment (CWE), it is likely to be
useful to examine the extent of the exposure both to
establish the extent, and to also measure the impact of
any intervention. In health professional education, the
prevalence of bullying and harassment is highly variable
ranging from 6.3% (Wolf et al. 1998) to 87.4% (Owoaje
et al. 2012) in medical students; 45.1% (Ferns and
Meerabeau 2008) to 90% (Foster et al. 2004) in nursing stu-
dents; 34.6% in dentistry students (Rowland et al. 2010);
and 25% in physiotherapy students (Stubbs and Soundy
2013). Reasons for this variability are due to the different
definitions of what constitutes bullying and harassment,
along with different instruments used to measure the
phenomena (Einarsen et al. 2011).
A significant amount of learning in health professional
education occurs in the CWE, which may account for
Practice points
An instrument to measure the prevalence of bul-
lying and harassment of health professional stu-
dents in a clinical workplace environment (CWE)
that has validity evidence is scarce.
This study provides some validity evidence for an
instrument that measures the prevalence of bully-
ing and harassment of health professional stu-
dents learning in a CWE. The instrument contains
five factors of workplace learning-related bullying
(WLRB), person-related bullying (PRB), physically
intimidating bullying (PIB), sexual harassment
(SH), and ethnic harassment (EH).
The Clinical Workplace Learning NAQ-R scale is
quickly administered and provides health profes-
sional educators information into the negative
experiences students may be facing learning in a
CWE. Information may also assist educators in
developing specific interventions to target par-
ticular negative experiences faced by health pro-
fessional students learning in a CWE.
CONTACT Kelby Smith-Han kelby.smith-han@otago.ac.nz Department of Anatomy, School of Biomedical Sciences, University of Otago, PO Box 56,
Dunedin 9054, New Zealand
ß2020 Informa UK Limited, trading as Taylor & Francis Group
MEDICAL TEACHER
2020, VOL. 42, NO. 7, 813821
https://doi.org/10.1080/0142159X.2020.1746249
students in the health professions reporting more bullying
and harassment than other higher education students
(Rautio et al. 2005). Health professional students are also
placed in a setting which is not just a learning environ-
ment, but also a working environment delivering health-
care to patients who are in need of care and treatment. It
is in this environment that learning about the profession
interacts with the provision of the health service, and is
also where abuse of health professional students occurs.
This is illustrated in a study by Rees et al. (2015) on clinical
workplace abuse narratives of students in a variety of
health professional education institutions, where many
examples of verbal and physical abuse along with sexual
and ethnic harassment were described.
When looking at addressing bullying and harassment of
health professional students, the leaders of health profes-
sional education institutions first need to ask the questions:
do we have bullying and harassment occurring at our insti-
tution when students are learning in CWEs? If so, to what
extent? And lastly, what specific forms of bullying and har-
assment are occurring? Answering these questions is key to
informing the development of a response to bullying and
harassment of health professional students. Furthermore, as
effective interventions are developed, for example the
Creating a Positive Learning Environment (CAPLE) initiative
(Gamble Blakey et al. 2019a,2019b,2019c), we need reli-
able and valid measures to determine impact.
To answer these questions, reliability and validity evi-
dence is needed for an instrument used to measure the
exposure to bullying and harassment of health professional
students in a CWE. To our knowledge, no studies have
been published explicitly demonstrating an instruments
validity and psychometric properties measuring the preva-
lence and type of bullying and harassment (that includes
sexual and ethnic harassment), in health professional edu-
cation specifically within a CWE. In order to address this
issue, we took the following approach to develop such
an instrument.
The NAQ-R
An instrument already developed to investigate bullying in
the workplace is the Negative Acts Questionnaire Revised
(NAQ-R). The questionnaire has been previously researched
to provide validity and reliability evidence for its use and is
widely used in measuring exposure to workplace bullying
of employees (Einarsen et al. 2009). The NAQ-R contains 22
questions with three factors: work-related bullying, person-
related bullying, and physically intimidating bullying.
Previous literature has illustrated that sexual and ethnic
harassment are significant factors associated with the bully-
ing and harassment of health professional students
(Gaughran et al. 1997; Richardson et al. 1997; White 2000;
Rautio et al. 2005; Witte et al. 2006; Wilkinson et al. 2006;
Garbin et al. 2010; Premadasa et al. 2011; Rees and
Monrouxe 2012; Bruce et al. 2015; Rees et al. 2015), how-
ever, the NAQ-R does not include behaviours associated
with these factors. Therefore, it was determined appropri-
ate to include two additional factors of sexual and ethnic
harassment, to provide a more comprehensive account of
bullying and harassment.
Purpose of the study
We have called the modified questionnaire the Clinical
Workplace Learning NAQ-R scale. Specifically, this study
attempts to answer the research question of:
To what extent are the factors present in the modified version
of the questionnaire, applicable to a CWE; and what is the
effect of adding two new factors on the validity of
the instrument? To this end, the psychometric properties of the
Clinical Workplace Learning NAQ-R scale were evaluated.
Method
Participants and procedures
The Clinical Workplace Learning NAQ-R questionnaire was
distributed to all undergraduate medical students who
were in their clinical years (years 46) at the University of
Otagos six-year medical degree, and all undergraduate
nursing students in their final two years of the Otago
Polytechnics Bachelor of Nursing degree. Years 46 of the
medical curriculum entails the immersion of medical stu-
dents in different CWEs for their learning, predominately
hospital and general practice learning environments. Year 2
of the nursing degree is when nursing students begin to
learn in CWEs, with final year students (year 3) spending
the largest amount of time learning in CWEs. The CWEs for
nursing students consist of Primary Healthcare settings,
hospitals and residential care facilities, with the predomin-
ant amount of clinical learning conducted in the hospital
environment.
For the medical students, hardcopy questionnaires for
year 4 and 5 were administered during whole class ses-
sions. Year 6 students are more geographically dispersed
so on-line questionnaires were used. For the nursing stu-
dents hardcopy questionnaires were distributed during a
whole class teaching session. Ethical approval of the study
was obtained from the University of Otago Human Ethics
Committee and by the Otago Polytechnic
Ethics Committee.
Theoretical underpinnings of the original NAQ-R
In the original NAQ-R bullying is defined as a situation in
which hostile and aggressive actions are systematically
directed at one or more persons in such a way that they are
stigmatized and victimized(Mikkelsen and Einarsen 2001,p.
394). Additionally, describing the dimensions of what con-
stitutes bullying in the workplace in which the NAQ-R is sit-
uated is necessary to understand its construction. Bullying
in the workplace constitutes negative and unwanted
behaviours (Einarsen et al. 2011) along with ‘…evolving
and often escalating hostile workplace relationships rather
than discrete and disconnected events and is associated
with repetition (frequency), duration (over a period of
time), and patterning (of a variety of behaviours involved)
as its most salient features(Einarsen et al. 2009, p. 25).
The NAQ-R contains three factors: work-related bullying,
person-related bullying, and physically intimidating bully-
ing. Work-related bullying consists of behaviours targeted
at an individuals working role and activities such as being
given unreasonable deadlines, or meaningless tasks
(Einarsen et al. 2011). Person-related bullying consists of
814 K. SMITH-HAN ET AL.
behaviours that are targeted at the individual themselves
for example, spreading gossip or rumours about you, hav-
ing insulting or offensive remarks made about your person
(Einarsen et al. 2011). As stated by Einarsen et al. (2011),
the behaviours associated with person-related bullying
‘…are by and large, independent of the work organisation
(p. 13). Physically intimidating bullying is associated with
behaviours targeting the individual with explicit acts of
physical aggression or violence or threats of violence
(Einarsen et al. 2011).
Instrument development
Demographic data collected in the survey included age,
ethnicity, sex and sexual orientation. Sexual orientation was
included purposively as previous literature indicates that
students who are in a minority regarding sexual orientation
are more likely to be bullied and/or harassed (Przedworski
et al. 2015).
To develop the instrument, we undertook the following
processes. The original concept underpinning the NAQ-R
(bullying and harassment of employees in a workplace)
was modified in order to fit our context (bullying and har-
assment of students learning in a CWE). Items used in the
original NAQ-R were then edited to align with our new
context. Finally, we added two new factors of sexual and
ethnic harassment. Both processes were conducted while
maintaining the original instruments behavioural design.
Modifying the concept of bullying and harassment in
the workplace
The NAQ-R consists of a three factor model of workplace
related bullying, person-related bullying, and physically
intimidating bullying. The NAQ-R was originally designed in
the context of workplace bullying and harassment of
employees and therefore, the items in the questionnaire
related to the definition of bullying and harassment in the
context of working as an employee. In order to make the
NAQ-R effective for measuring exposure of bullying and
harassment of students learning in a CWE we undertook the
following conceptual modifications.
Firstly, we modified the existing definition of bullying
used for the NAQ-R of a situation in which hostile and
aggressive actions are systematically directed at one or more
persons in such a way that they are stigmatized and victi-
mized(Mikkelsen and Einarsen 2001, p. 394) to fit our con-
text a situation in which hostile and aggressive actions are
systematically directed at one or more students in such a
way that they are stigmatized and victimized in a clinical
workplace learning environment.
Then, we edited specific items in the original question-
naire so they would align with this new modified concept
of bullying and harassment, from employees working, to stu-
dents learning in a CWE. Two statements that did not fit
the context of students learning in a clinical environment
were removed. One statement from the work-related bully-
ing factor: Pressure not to claim something to which by
right you are entitled (e.g. sick leave, holiday entitlement,
travel expenses),as it relates to the role of being
employed which is not relevant to students. The second
statement that was omitted was from the person-related
bullying factor: Practical jokes carried out by people you
dont get along with.As health professional students gen-
erally move around different clinical workplaces to experi-
ence different areas of healthcare practice, we concluded
that the behaviour indicated in the statement may occur
less frequently. We also re-analysed the remaining 20 state-
ments in the original NAQ-R associated with its three fac-
tors, and re-worded 10 statements to align with our
modified concept. The changes to the wording of particular
items can be seen in Table 1:
Table 1. Re-worded statements from the NAQ-R used in the clinical workplace learning NAQ-R scale.
Original statement in NAQ-R Wording change/omissionRationale for change
Work-related bullying factor
Someone withholding information which
affects your performance
Someone withholding information which affects
your learning
To fit our context of students learning in a
workplace environment
Being ordered to do work below your level
of competence
Being ordered to do tasks above your level
of competence
This reflects the importance of students learning
tasks to be within their level of competence in
order to keep themselves safe, and their
patients safe during the learning process.
Having your opinions ignored Having your opinions and views ignored Aides understanding of the statement to broaden
it out to more than just having your
opiniated commentsignored, but whatever
your contribution happens to be at the
time ignored.
Being given tasks with unreasonable deadlines Being given tasks with unreasonable or
impossible targets or deadlines
Aides understanding of the statement.
Person-related bullying factor
Being humiliated or ridiculed in connection
with your work
Being humiliated or ridiculed in connection with
your learning
To fit our context of students learning in a
workplace environment
Having key areas of responsibility removed or
replaced with more trivial or
unpleasant tasks
Having key areas of your student role removed or
replaced with more trivial or unpleasant tasks
Highlights significant change in what the student
should be doing at the level they are currently
at, which changes as they progress.
Being ignored or excluded Being ignored or excluded from the clinical team To fit our context of students learning in a
workplace environment
Having insulting or offensive remarks made
about your person, attitudes or your
private life
Having insulting or offensive remarks made
about your person (i.e. habits and background),
attitudes or your private life
Aides understanding of the statement.
Hints or signals from others that you should
quit your job
Hints or signals from others that you should quit
studying your profession
To fit our context of students learning in a
workplace environment
Persistent criticism of your errors or mistakes Persistent criticism of your work and effort To make sure students didnt get this confused
with patient safety literature (where errors etc.
have specific definitions).
Change/addition of wording shown in italics.
MEDICAL TEACHER 815
Two statements obtained from previous literature, were
also added to more accurately and thoroughly reflect the
context of experiencing bullying and harassment by stu-
dents learning in a clinical environment. Items added were:
Being assigned work for punishment rather than for educa-
tional value; and having learning opportunities blocked or
withheld by others. These statements were added to the
workplace related bullying factor and was re-named the
workplace learning-related bullying factor to fit our new
modified concept, (see Supplement Appendix, Table A1, for
all Clinical Workplace Learning NAQ-R Scale Items).
Adding two new factors
Two new factors were introduced in our modified version
of the questionnaire: sexual harassment and ethnic harass-
ment (Supplement Appendix, Table A1).
To include sexual harassment in our CWE context we
adopted Tills definition of academic sexual harassmentas
the use of authority to emphasise the sexuality or sexual
identity of a student in a manner which prevents or impairs
that students full enjoyment of educational benefits, climate,
or opportunities(Till 1980, p. 7). The items used for this fac-
tor that aligned with this definition drew on the general
sexual harassment literature such as the Sexual Experiences
Questionnaire (SEQ) (Fitzgerald et al. 1995); along with
questionnaires used in the bullying and harassment litera-
ture focused on professional working environments
(Crebbin et al. 2015); and health professional students
(Sheehan et al. 1990; Baldwin et al. 1991; Uhari et al. 1994;
Wilkinson et al. 2006; Woolley et al. 2006; Rowland et al.
2010; Clarke et al. 2012; Knapp et al. 2014; Whiteside et al.
2014; AAMC 2018).
We adopted the conceptualisation of ethnic harassment
as described by Schneider et al, defined as threatening ver-
bal conduct or exclusionary behaviour that has an ethnic
component and is directed at a target because of his or her
ethnicity(Schneider et al. 2000, p. 3). The items that were
used for this factor aligning with this definition were
derived from a combination of the Ethnic Harassment
Experiences scale (EHE) (Schneider et al. 2000); and the bul-
lying and harassment literature in the workplace (Keashly
1997; Einarsen et al. 2011; Crebbin et al. 2015) and of
health professional students (Sheehan et al. 1990; Baldwin
et al. 1991; Uhari et al. 1994; Wilkinson et al. 2006; Woolley
et al. 2006; Rowland et al. 2010; Clarke et al. 2012; Knapp
et al. 2014; Whiteside et al. 2014; AAMC 2018).
Maintaining instrument design
Two main aspects in the design of the original NAQ-R
ensures all items are written using specific behavioural
statements, and there are no definitions given about or
words mentioning bullyingor harassmentwhen partici-
pants undertake the questionnaire (Einarsen et al. 2009).
For example, the questionnaire asks participants how often
they have been subjected to the following negative acts
and gives a list of the terms such as Having insulting or
offensive remarks made about your person (i.e. habits and
background), your attitudes or your private life,as
opposed to asking how the participant feels about the
behaviour. Asking about specific behaviours without pro-
viding terms or definitions assists in minimising
misinterpretations by participants, specifically when devel-
oping an instrument investigating incidence and/or preva-
lence (Arvey and Cavanaugh 1995), such as is the focus of
this research. Taking this behavioural approach to the
design of the instrument is considered to provide a more
objective estimate of exposure to bullying behaviours than
self-labelling approaches, as respondentsneed for cognitive
and emotional processing of information would be reduced
(Einarsen et al. 2009, p. 27). Therefore, when modifying the
original NAQ-R, we kept the same approach of using
behavioural terminology and not mentioning the words
bullyingor harassment.
The frequency rating of behaviours in the original NAQ-
R was kept, because we determined that it would still fit
our context of students learning in a CWE. For this meas-
urement, participants are asked to rate how often they
experienced the behaviours listed on a 15 point Likert
scale where 1 ¼Never,2¼Now and then,3¼Monthly,
4¼Weekly,5¼Daily.
While in the NAQ-R, participants are asked to rate their
experiences over the past 6 months, we modified this to
8 months for the current questionnaire, as this gives the
students the opportunity to reflect on their experiences for
the majority of their clinical year.
In summary, in light of these modifications the newly
formed 31-item Clinical Workplace Learning NAQ-R scale
has been developed to measure health professional stu-
dentsexposure to negative inter-personal interactions in a
CWE. It is comprised of five hypothesised factors: workplace
learning-related bullying (WLRB), person-related bullying
(PRB), physically intimidating bullying (PIB), sexual harass-
ment (SH), and ethnic harassment (EH) (the Clinical
Workplace Learning NAQ-R questionnaire is located in
Supplement Appendix A2).
Statistical analyses
We developed and examined the psychometric characteris-
tics of the Clinical Workplace Learning NAQ-R scale using a
structural equation modelling approach to explore the val-
idity evidence. A Structural Equation Modelling (SEM)
approach, specifically Confirmatory Factor Analysis (CFA),
was carried out using MPlus 7 (Muth
en and Muth
en 2012).
We conducted the data analyses in two stages. Initially, we
examined the data for outliers and missing cases. Then, the
factorial structure of the Clinical Workplace Learning NAQ-R
scale was examined by means of Confirmatory Factor
Analysis (CFA). To provide support for the validity evidence
for the hypothesised factor structure of the Clinical
Workplace Learning NAQ-R scale, we investigated and com-
pared the goodness-of-fit of different competing models as
suggested by Noar (2003) and Strauss and Smith (2009).
Confirmatory factor analysis (CFA)
The hypothesized model (five-factor) model was compared
to three other alternative competing models. The hypothe-
sized model included three factors reported by Einarsen
et al. (2009) as well as two new factors that are sexual har-
assment (SH) and ethnic harassment (EH) all of which are
related to each other. The competing models included: (a)
a one-factor (unidimensional) model that assumed all
816 K. SMITH-HAN ET AL.
manifest variables loaded on a single factor, (b) a three-fac-
tor model that suggests WLRB, PRB, and PIB items loaded
on a single factor, and (c) a second-order (higher-order)
factor model with the five scale factors subordinating to a
single second-order factor.
One-factor model means that what we are measuring is
a unidimensional construct and university students are not
differentiating the bullying and harassment factors.
Evidence for a three-factor model indicates that all three
factors reported by Einarsen et al. (2009) are not distinct
from each other. Support for second-order model would
suggest that these related five factors can be accounted
for by an underlying higher order construct. Support for
the hypothesized five-factor correlated (oblique) model
would suggest that medical and nursing students differen-
tiate between the five bullying and harassment factors that
are related to each other.
Since the data were ordered-categorical, the weighted
least squares mean and variance adjusted (WLSMV) estima-
tion procedure was used for CFA analyses. The WLSMV is a
robust estimation technique that is suggested for model-
ling ordinal data (Flora and Curran 2004; Brown 2006). The
consequences of treating ordinal responses as continuous
which may lead to reporting inaccurate results are well-
established in the literature (Lubke and Muth
en 2004).
A number of different indices were examined to com-
pare the different models and evaluate model-data fit
(Cheung and Rensvold 2002; Fan and Sivo 2005,2007).
Each of these measures reflects a different aspect of model
fit and may not perform equally well under different types
of model conditions (Fan and Sivo 2007). Thus, it is import-
ant to use multiple indices rather than relying on one
measure (Hair et al. 2010). Indices reported in this study
included the Root Mean Square Error of Approximation
(RMSEA), the Comparative Fit Index (CFI), the Tucker-Lewis
index (TLI), and the Weighted Root Mean Square Residual
(WRMR). The chi-square (v
2
) values were also reported but
not used for model fit decisions as this statistic and its sig-
nificance are inflated with large sample sizes. The com-
monly accepted cut-offs for acceptableor goodfit
(Browne and Cudeck 1992; MacCallum et al. 1996; Hu and
Bentler 1998;Yu2002; Hair et al. 2010) included: a non-sig-
nificant chi-square (v
2
), RMSEA with values <0.08 indicating
an acceptable fit and values <0.05 indicating a good fit,
CFI and TLI with values >0.90 being indicative of reason-
able fit and values >0.95 indicating a good fit, and WRMR
with values being close to 1. The limitations of coefficient
alpha (a) as a measure of reliability estimate is well docu-
mented in the literature (Sijtsma 2009; Teo and Fan 2013).
Therefore, using polychoric correlations, we calculated and
reported McDonalds omega (x) (McDonald 1999) as a bet-
ter estimate of reliability.
Results
Participant and demographic information
A total of 428 from an eligible 852 medical students com-
pleted the questionnaire giving a response rate of 50%
(428/852). A total of 69 nursing students in year 2 and
43 year 3 completed the questionnaire, from an eligible
212 nursing students, a giving a response rate of 53%
(112/212). The questionnaire took approximately 5 minutes
to complete.
Therefore, 540 medical and nursing students completed
the questionnaire giving an overall response rate of 51%
(540/1064). Of the participants, 65.2% (n¼352) were
females and 34.4% (n¼186) were males. Only one student
did not report their sex and one student identified them-
selves as transgender. Mean age was 23.7 years (range
1953 years, SD ¼4.35). Self-reported ethnic composition
was reported as; New Zealand European (67.8%), M
aori
(9.4%), Chinese (8.3%), Indian (3.0%), Samoan (1.3%), Cook
Island M
aori (0.7%), Tongan (0.4%), other ethnicities (20%),
and not stated (0.7%). Because individuals can be of more
than one ethnicity, these totals are greater than 100%.
Sexual orientation was reported as heterosexual (91.7%),
homosexual (3.3%), bisexual (2.4%), other (1.5%), and not
stated (1.1%).
Descriptive statistics
No univariate outliers were identified to have an effect on
the results. The proportion of missing cases for each item
was trivial ranging from mostly zero to one percent. The
Expectation Maximization (EM) algorithm which assumes
that observations are missing at random (MAR) was utilized
to impute the missing cases rather than listwise deletion.
The means and standard deviations for the five factors of
the Clinical Workplace Learning NAQ-R scale are summar-
ized in Table 2.
Factor means ranged from 1.15 to 1.44, suggesting that
most students endorsed neveror now and thenwith the
statements. However, examination of the item means most
of which ranged from 1.00 to 5.00 revealed that there were
some students who had been subjected to negative acts
on a daily basis. The standard deviations ranged from 0.28
to 0.44 indicating that the dispersion of responses for each
factor were somewhat similar.
Confirmatory factor analysis (CFA)
The goodness-of-fit measures of hypothesized and alterna-
tive models are summarized in Table 3.
Evaluation of the unidimensional model revealed that
this model was not representing the sample data
Table 2. Descriptive statistics of clinical workplace learning NAQ-R scale factors.
Number of items M SD Min Max
Workplace learning-related bullying (WLRB) 8 1.44 0.44 1.00 3.75
Person-related bullying (PRB) 11 1.43 0.43 1.00 4.09
Physically intimidating bullying (PIB) 3 1.18 0.35 1.00 3.67
Sexual harassment (SH) 5 1.15 0.28 1.00 3.20
Ethnic harassment (EH) 4 1.15 0.38 1.00 4.00
MEDICAL TEACHER 817
sufficiently. The RMSEA, CFI, TLI and WRMR values did not
meet the commonly acceptable fit criteria. The chi-square
statistic and fit indices (RMSEA ¼0.045; CFI ¼0.955;
TLI ¼0.955; and WRMR ¼1.232) suggested that the
hypothesized five-factor correlated model provided the
best model fit with the data. The three-factor and second-
order models had also good fit close to the hypothesized
model. However, the chi-square difference test (DIFFTEST)
between the second-order and the hypothesized model
indicated that adding a higher-order dimension signifi-
cantly worsened the fit. Thus, the hypothesized model was
retained as the model of best fit with the five factors of
workplace learning-related bullying (WLRB), person-related
bullying (PRB), physically intimidating bullying (PIB), sexual
harassment (SH), and ethnic harassment (EH).
For this model, factor correlations and reliabilities are
provided in Table 4.
The correlations between the Clinical Workplace
Learning NAQ-R factors ranged from 0.48 to 0.95. The high-
est correlations were found between workplace learning-
related bullying (WRB), person-related bullying (PRB), and
physically intimidating bullying (PIB). These results were
closely consistent with findings of the Einarsen et al. (2009)
study. However, we observed moderate correlations
between the new factors sexual harassment (SH), ethnic
harassment (EH) - and other factors indicating that the evi-
dence of discriminant validity improved with the addition
of SH and EH factors.
All of the omega reliability estimates were greater than
the recommended level (0.70). Standardized factor loadings
for the hypothesized model are provided in Table 5.
All the unstandardized factor loadings were significant.
Standardized factor loadings for the hypothesized model
ranged from 0.63 to 0.96 providing support for convergent
validity. All of the items were strong indicators of the fac-
tors they were related to.
In summary, the CFA analyses and reliability estimates
provided support for the validity evidence of the factorial
structure of the Clinical Workplace Learning NAQ-R.
Discussion
The aim of this research was to construct and provide val-
idity evidence for a self-report instrument to measure
health professional studentsexperience of bullying and
harassment in a clinical workplace learning environment.
Previous research looking into the prevalence and type of
bullying and harassment that health professional students
face were from a variety of instruments lacking validity evi-
dence to support the intended purpose.
This study has the potential to contribute to the litera-
ture on measuring exposure to bullying and harassment of
health professional students by providing: empirical evi-
dence through supporting the validation of a new instru-
ment; an instrument that is specific to a clinical workplace
learning environment; and an instrument that contains
more relevant factors associated with literature on bullying
and harassment of health professional students.
Using an instrument (NAQ-R) that was developed for a
different context (workplace bullying among employees),
and with evidence supporting its validity, we modified it to
Table 3. Confirmatory factor analysis of alternative models.
Model v
2
df RMSEA CFI TLI WRMR
Alternative one-factor (unidimensional) 1995.244434 0.082 0.845 0.834 2.136
Alternative three-factor 906.221431 0.045 0.953 0.949 1.266
Hypothesized five-factor 878.706424 0.045 0.955 0.951 1.232
Alternative second-order 906.289429 0.045 0.953 0.949 1.280
Note. RMSEA: Root Mean Square Error of Approximation; CFI: Comparative Fit Index; TLI: Tucker-Lewis Index; WRMR: Weighted Root Mean Square Residual.
p<.01
Table 4. Clinical workplace learning NAQ-R scale factor correlations and reliabilities.
WLRB PRB PIB SH EH
Workplace Learning-Related Bullying (WLRB)
Person-Related Bullying (PRB) 0.95
Physically Intimidating Bullying (PIB) 0.80 0.89
Sexual Harassment (SH) 0.59 0.54 0.52
Ethnic Harassment (EH) 0.48 0.58 0.49 0.48
Reliability (x) 0.89 0.93 0.79 0.90 0.94
Table 5. Standardized factor loadings of the Clinical Workplace Learning
NAQ-R.
Item Standardized estimate
WLRB 1 0.65
WLRB 2 0.63
WLRB 3 0.72
WLRB 4 0.75
WLRB 5 0.69
WLRB 6 0.71
WLRB 7 0.80
WLRB 8 0.70
PRB 1 0.75
PRB 2 0.66
PRB 3 0.71
PRB 4 0.69
PRB 5 0.73
PRB 6 0.73
PRB 7 0.83
PRB 8 0.65
PRB 9 0.82
PRB 10 0.83
PRB 11 0.80
PIB 1 0.79
PIB 2 0.76
PIB 3 0.76
SH 1 0.91
SH 2 0.92
SH 3 0.65
SH 4 0.66
SH 5 0.93
EH 1 0.93
EH 2 0.96
EH 3 0.91
EH 4 0.91
818 K. SMITH-HAN ET AL.
investigate if it would fit in our context of (clinical work-
place learning environments for students, not employees).
The original NAQ-R contained three factors related to bully-
ing in the workplace: workplace learning-related bullying,
person-related bullying and physical related bullying. We
modified these three factors and also added two new fac-
tors not in the original NAQ-R related to sexual harassment
and ethnic harassment, because of the reported prevalence
of these types of experiences by students learning in a
CWE. Our analyses provide evidence for the validity of
these two new factors.
We suggest naming this instrument the Clinical
Workplace Learning NAQ-R scale, as this acknowledges the
significant body of work by Einarsen et al. (2009) in devel-
oping and analysing the psychometric properties of the ori-
ginal NAQ-R designed for workplace bullying of employees.
The results indicated that the original three factors of
the NAQ-R that we modified to fit a clinical workplace
learning (workplace learning-related bullying, person-
related bullying and physical related bullying) had relatively
high factor correlations. This indicates the modifications we
made to the original NAQ-R (Table 1) to reflect the new
context of measuring exposure to clinical workplace learn-
ing bullying, did not change the strength of the associa-
tions among factors we borrowed from the original NAQ-R.
The magnitude of the factor loadings indicated that all
items were strong indicators of the factors they were
related to. The factor loadings of the items of two new fac-
tors of sexual harassment and ethnic harassment were
even higher than the original NAQ-R items which provided
further convergent validity evidence. The correlations
between the two novel factors and the original ones were
moderate which revealed discriminant validity evidence.
Both convergent and discriminant validity are important
components of construct validity. The CFA analysis sug-
gests that adding these two new factors support that med-
ical and nursing students differentiate between the five
factors that are also inter-related to each other for the
overall construct of clinical workplace learning bullying
and harassment.
Implications
Having a comprehensive five factor model that includes
sexual and ethnic harassment among the bullying behav-
iours experienced provides a more comprehensive instru-
ment that aligns with the definition of bullying offered
earlier, and more accurately reflects the varied negative
experiences of health professional students learning in the
clinical workplace described in the literature (Dineen et al.
1997; Richardson et al. 1997; White 2000; Rautio et al.
2005; Witte et al. 2006; Wilkinson et al. 2006; Garbin et al.
2010; Premadasa et al. 2011; Rees and Monrouxe 2012;
Bruce et al. 2015; Przedworski et al. 2015; Rees et al. 2015).
During the development of modifying the original NAQ-
R and adding the two factors of sexual harassment and
ethnic/racial harassment we were mindful of trying to keep
the instrument short. The final questionnaire is a 31 item
instrument, which is only nine items longer the original
NAQ-R questionnaire. The length of a questionnaire is
important to consider, as practically implementing long
questionnaire in large organisations, or student groups can
be difficult to administer, and run the risk of larger attrition
of participant responses. When implemented, the 31 item
questionnaire takes participants approximately 5 minutes
to complete.
The Clinical Workplace Learning NAQ-R scale, as a five
factor model, would be useful for health professional edu-
cation institutions who would like to measure their stu-
dentsexposure to bullying and harassment in clinical
workplace learning environments. Developing a scale spe-
cifically for student learners in the workplace and extend-
ing the original NAQ-R to a five factor model could assist
institutions in identifying particular problematic areas (if
any) that their students maybe experiencing. For example,
are students experiencing bullying behaviours that reflect
the workplace learning aspects of their student learning
role (e.g. being asked to do something above their level of
competence); or are they experiencing more person-related
bullying (e.g. Being ignored or excluded from the clinical
team); or experiencing sexual harassment (e.g. inappropri-
ate physical contact); or ethnic harassment (e.g. made
derogatory comments about your racial or ethnic group).
Identifying these specific areas of bullying and harassment
would significantly benefit institutions in planning any
interventions to address the negative behaviours experi-
enced by health professional students.
Limitations
The questionnaire was delivered to only two health profes-
sional groups (medical students and nursing students) yet
these two groups would represent the largest health pro-
fessional groups in New Zealand. Although these two
groups occupy a variety of settings that include and repre-
sent various primary and secondary and community clinical
environments, we acknowledge they may not exactly mir-
ror all health professional CWEs. Additionally, using these
groups in our study could also be viewed as a strength,
given many validation studies only include more homoge-
neous populations.
Moreover, even though we view the statements used in
the Clinical Workplace Learning NAQ-R scale are generic
enough to apply to many clinical workplace learning set-
tings, further testing to look at how the instrument works
with other health professional student groups to confirm
this would be worthwhile.
We also understand that using this instrument design of
measuring only behaviours and their frequency does not
provide answers to other specific questions institutions
may be wanting. For example, this method does not exam-
ine what Einarsen et al. (2009, p. 40) describes as who did
what to whom.However, this issue could be addressed by
adding a self-labelling method. For example, after adminis-
tering the Clinical Workplace Learning NAQ-R scale, a defin-
ition of bullying and harassment is offered to participants
and then asked if they view themselves as victims accord-
ing to this definition and to describe what happened to
them and by who (Einarsen et al. 2009).
Although the five-factor model yielded the best fit, hav-
ing acceptable fit for three-factor and second-order models
suggests that there is still room for improvement on the
psychometric properties of the NAQ-R scale. We agree with
Einarsen et al. (2009) that even though the original three
MEDICAL TEACHER 819
dimensions of reported workplace bullying can be distin-
guished, yet they do not discriminate well between differ-
ent types of bullying behaviours, suggesting co-occurrence
of these different types of bullying (p. 31).Also, results
from the second-order model may support that idea that
NAQ-R constructs are correlated reflecting the presence of
a more general construct at a higher conceptual level and
can be considered together to create a composite score. In
this research, we wanted to maintain the integrity and
structure of the original NAQ-R as much as possible but
future research may consider improving the constructs by
refining the item wording or shortening the scale especially
for the WLRB and PRB factors.
The Clinical Workplace Learning NAQ-R was developed
based in a New Zealand cultural context which shares
some similarities with Einarsens et al. (2009) Anglo-
American context, yet also maintains its own cultural con-
text. The literature reports many behaviours that are similar
between many cultural contexts in relation to bullying and
harassment behaviours at medical and nursing schools.
However, it would be pertinent to assume that there would
be different beliefs, values and practices specific to certain
cultures that may inform the concepts of bullying and har-
assment. This would influence the wording of the state-
ments used in the instrument along with the meaning that
is attributed to the statements as well. Therefore, further
work needs to be conducted in making the instrument
applicable in different cultural contexts. Finally, future val-
idity research could explore how sensitive the scores are to
change over time, for example following an intervention.
Conclusion
In this paper we have shown the development of the
Clinical Workplace Learning NAQ-R scale and analysis of its
factor structure and provided supporting validity evidence
for its use. The Clinical Workplace Learning NAQ-R scale is
a quickly administered instrument in order to measure
exposure to bullying and harassment experienced by
health professional students in a clinical workplace learning
environment. Its structure may assist health professional
leadership to obtain vital information into the negative
experiences students may be facing, including what spe-
cific experiences may be occurring more frequently than
others in regards to the bullying and harassment of
their students. In turn, this information may assist in devel-
oping specific interventions to target the particular experi-
ences faced by health professional students learning in
a CWE.
Acknowledgments
The authors would like to thank all of the Associate Deans of Medical
Education for Otago Medical School who assisted with the data collec-
tion for this questionnaire, as well as Faculty in The School of Nursing
at Otago Polytechnic. We would also like to thank Dr Ella Iosua and
Michel de Lange for their initial advice on the analysis of this paper.
We would also like to thank the students for their time and effort in
completing this questionnaire.
Disclosure statement
The authors report no conflicts of interest. The authors alone are
responsible for the content and writing of the article.
Glossary
Confirmatory factor analysis (CFA): Is a type of structural
equation modeling (SEM) that deals specifically with measure-
ment models that is, the relationships between observed meas-
ures or indicators (e.g., test items, test scores, behavioral
observation ratings) and latent variables or factors(Brown &
Little, 2015, p.1).
Funding
The present research was financially supported by Division of Health
Sciences, University of Otago.
Notes on contributors
Kelby Smith-Han, MHealSc, PhD, is a Postdoctoral and Teaching
Fellow, Department of Anatomy and Otago Medical School, University
of Otago, Dunedin, New Zealand.
Emma Collins, MN, is Principal Lecturer, School of Nursing, Otago
Polytechnic, Dunedin, New Zealand.
Mustafa Asil, MA, PhD, is a Senior Research Fellow, Educational
Assessment Research Unit (EARU), College of Education, University of
Otago, Dunedin, New Zealand.
Althea Gamble Blakey, MHealSc, PhD, is a Research Fellow, Otago
Medical School, University of Otago, Dunedin, New Zealand.
Lynley Anderson, MHealSc, PhD, is an Associate Professor, Bioethics
Centre, University of Otago, Dunedin, New Zealand.
Elizabeth Berryman, MHSc, MBChB, is a Resident Medical Officer,
Waitemata District Health Board, Auckland, New Zealand.
Tim J. Wilkinson, MBChB, MClinEd, PhD, MD, is Professor, Deans
Department and Department of Medicine, Otago Medical School,
University of Otago, Christchurch, New Zealand.
ORCID
Kelby Smith-Han http://orcid.org/0000-0003-2105-5062
Mustafa Asil http://orcid.org/0000-0002-7827-6686
Althea Gamble Blakey http://orcid.org/0000-0002-8373-5816
Lynley Anderson http://orcid.org/0000-0002-7329-7058
Tim J. Wilkinson http://orcid.org/0000-0002-4080-4164
References
Arvey RD, Cavanaugh MA. 1995. Using surveys to assess the preva-
lence of sexual harassment: some methodological problems. J Soc
Issues. 51(1):3952.
Association of American Medical Colleges (AAMC). 2018. Medical
School Graduation Questionnaire; [accessed 2018 Nov 23]. https://
www.aamc.org/download/490454/data/2018gqallschoolssummaryre-
port.pdf.
Baldwin DC, Jr, Daugherty SR, Eckenfel EJ. 1991. Student perceptions
of mistreatment and harassment during medical school: a survey of
ten United States schools. West J Med. 155(2):140145.
Brown T. 2006. Confirmatory factor analysis for applied research. New
York: Guildford press.
Browne MW, Cudeck R. 1992. Alternative ways of assessing model fit.
Sociol Methods Res. 21(2):230258.
Brown, T. and Little, T. 2015. Confirmatory factor analysis for applied
research. 2nd ed. New York: Guilford Press.
820 K. SMITH-HAN ET AL.
Bruce AN, Battista A, Plankey MW, Johnson LB, Marshall MB. 2015.
Perceptions of gender-based discrimination during surgical training
and practice. Med Educ Online. 20:25923.
Cheung GW, Rensvold RB. 2002. Evaluating goodness-of-fit indexes for
testing measurement invariance. Struct Equ Modeling. 9(2):233255.
Clarke C, Kane D, Rajacich D, Lafreniere K. 2012. Bullying in under-
graduate clinical nursing education. J Nurs Educ. 51(5):269276.
Crebbin W, Campbell G, Hillis DA, Watters DA. 2015. Prevalence of bul-
lying, discrimination and sexual harassment in surgery in
Australasia. ANZ J Surg. 85(12):905909.
Dineen M, Cole M, Dally RJ, Dinned S, Gaughran F. 1997. Stress in
medical students. Ir Med J. 90(5):184185.
Gaughran F, Dineen S, Dineen M, Cole M, Daly RJ. 1997. Stress in med-
ical students. Ir Med J. 90(5):184185.
Einarsen S, Hoel H, Notelaers G. 2009. Measuring exposure to bullying
and harassment at work: validity, factor structure and psychometric
properties of the Negative Acts Questionnaire-Revised. Work Stress.
23(1):2444.
Einarsen S, Hoel H, Zapf D, Cooper C, ed. 2011. Bullying and harass-
ment in the workplace: developments in theory, research, and prac-
tice (2nd ed.). Boca Raton: CRC Press Taylor & Francis Group.
Fan X, Sivo SA. 2005. Sensitivity of fit indexes to misspecified structural
or measurement model components: rationale of two-index strat-
egy revisited. Struct Equ Modeling. 12(3):343367.
Fan X, Sivo SA. 2007. Sensitivity of fit indices to model misspecification
and model types. Multivariate Behav Res. 42(3):509529.
Ferns T, Meerabeau L. 2008. Verbal abuse experienced by nursing stu-
dents. J Adv Nurs. 61(4):436444.
Fitzgerald LF, Gelfand MJ, Drasgow F. 1995. Measuring sexual harass-
ment: theoretical and psychometric advances. Basic Appl Soc Psych.
17(4):425445.
Flora DB, Curran PJ. 2004. An empirical evaluation of alternative meth-
ods of estimation for confirmatory factor analysis with ordinal data.
Psychol Methods. 9(4):466491.
Fnais N, Soobiah C, Chen M, Lillie E, Perrier L, Tashkhandi M, Straus S,
Mamdani M, Al-Omran M, Tricco A. 2014. Harassment and discrimin-
ation in medical training: a systematic review and meta-analysis.
Acad Med. 89(5):817827.
Foster B, Mackie B, Barnett N. 2004. Bullying in the health sector: a
study of bullying of nursing students. N Z J Employ Relat. 29:6783.
Gamble Blakey A, Smith-Han K, Anderson L, Collins E, Berryman E,
Wilkinson T. 2019a. Interventions addressing student bullying in the
clinical workplace: a narrative review. BMC Med Educ. 19(1):220.
Gamble Blakey A, Smith-Han K, Anderson L, Collins E, Berryman E,
Wilkinson T. 2019b. “‘They cared about us students:learning from
exemplar clinical teaching environments.BMC Med Educ. 19(1):119.
Gamble Blakey A, Smith-Han K, Anderson L, Collins E, Berryman E,
Wilkinson T. 2019c. Itsprobably the teacher!A strategic frame-
work for clinical staff engagement in clinical student bullying inter-
vention. BMC Med Educ. 19(1):116.
Garbin C, Zina L, Garbin A, Moimaz S. 2010. Sexual harassment in
Dentistry: prevalence in dental school. J Appl Oral Sci. 18(5):
447452.
Hair JF, Black WC, Babin BJ, Anderson RE. 2010. Multivariate data ana-
lysis (7th ed.). Englewood Cliffs: Prentice Hall.
Hu L, Bentler PM. 1998. Fit indices in covariance structure modeling:
sensitivity to underparameterized model misspecification. Psychol
Methods. 3(4):424453.
Keashly L. 1997. Emotional abuse in the workplace. J Aggress Maltreat
Trauma. 1(1):85117.
Knapp K, Shane P, Sasaki-Hill D, Yoshizuka K, Chan P, Vo T. 2014.
Bullying in the clinical training of pharmacy students. Am J Pharm
Educ. 78(6):117.
Lubke GH, Muth
en BO. 2004. Applying multigroup confirmatory factor
models for continuous outcomes to Likert scale data complicates
meaningful group comparisons. Struct Equ Modeling. 11(4):
514534.
MacCallum RC, Browne MW, Sugawara HM. 1996. Power analysis and
determination of sample size for covariance structure modeling.
Psychol Methods. 1(2):130149.
McDonald R. 1999. Test theory: a unified treatment. Mahwah, NJ:
Lawrence Erlbaum Associates.
Mikkelsen EG, Einarsen S. 2001. Bullying in danish work-life: prevalence
and health correlates. Eur J Work Organ Psychol. 10(4):393413.
Muth
en LK, Muth
en BO. 2012. Mplus: statistical analysis with latent
variables: users guide. Los Angeles, CA: Muth
en & Muth
en.
Noar SM. 2003. The role of structural equation modeling in scale
development. Struct Equ Modeling. 10(4):622647.
Owoaje ET, Uchendu OC, Ige OK. 2012. Experiences of mistreatment
among medical students in a university in south west Nigeria. Niger
J Clin Pract. 15(2):214219.
Premadasa IG, Wanigasooriya NC, Thalib L, Ellepola A. 2011.
Harassment of newly admitted undergraduates by senior students
in a Faculty of Dentistry in Sri Lanka. Med Teach. 33:556563.
Przedworski JM, Dovidio JF, Hardeman RR, Phelan SM, Burke SE, Ruben
MA, Perry S, Burgess DJ, Nelson DB, Yeazel MW, et al. 2015. A com-
parison of the mental health and well-being of sexual minority and
heterosexual first-year medical students: a report from the medical
student change study. Acad Med. 90(5):652659.
Rautio A, Sunnari V, Nuutinen M, Laitala M. 2005. Mistreatment of uni-
versity students most common during medical studies. BMC Med
Educ. 5:36.
Rees CE, Monrouxe LV. 2012. A morning since eight of just pure grill:
a multischool qualitative study of student abuse. Acad Med. 86:
13741382.
Rees CE, Monrouxe LV, Ternan E, Endacott R. 2015. Workplace abuse
narratives from dentistry, nursing, pharmacy and physiotherapy stu-
dents: a multi-school qualitative study. Eur J Dent Educ. 19(2):
95106.
Richardson DA, Becker M, Frank RR, Sokol RJ. 1997. Assessing medical
studentsperceptions of mistreatment in their second and third
years. Acad Med. 72(8):728730.
Rowland ML, Naidoo S, AbdulKadir R, Moraru R, Huang B, Pau A. 2010.
Perceptions of intimidation and bullying in dental schools: a multi-
national study. Int Dent J. 60(2):106112.
Schneider KT, Hitlan RT, Radhakrishnan P. 2000. An examination of the
nature and correlates of ethnic harassment experiences in multiple
contexts. J Appl Psychol. 85(1):312.
Sheehan KH, Sheehan DV, White K, Leibowitz A, Baldwin DC. 1990. A
pilot study of medical student abuse: student perceptions of mis-
treatment and misconduct in medical school. JAMA. 263(4):
533537.
Sijtsma K. 2009. On the use, the misuse, and the very limited useful-
ness of Cronbachs Alpha. Psychometrika. 74(1):107120.
Strauss ME, Smith GT. 2009. Construct validity: advances in theory and
methodology. Annu Rev Clin Psychol. 5:125.
Stubbs B, Soundy A. 2013. Physiotherapy studentsexperiences of bul-
lying on clinical internships: an exploratory study. Physiotherapy.
99(2):178180.
Teo T, Fan X. 2013. Coefficient alpha and beyond: issues and
alternatives for educational research. Asia-Pacific Edu Res. 22(2):
209213.
Till F. 1980. Sexual harassment: a report on the sexual harassment of
students. Washington, DC: National Advisory Council on Womens
Educational Programs.
Uhari M, Kokkonen J, Nuutinen M, Vainionpaa L, Rantala H, Lautala P,
V
ayrynen M. 1994. Medical student abuse: an international
phenomenon. JAMA. 271(13):10491051.
White GE. 2000. Sexual harassment during medical training: the per-
ceptions of medical students at a university medical school in
Australia. Med Educ. 34(12):980986.
Whiteside D, Stubbs B, Soundy A. 2014. Physiotherapy studentsexpe-
riences of bullying on clinical internships: a qualitative study.
Physiotherapy. 100(1):4146.
Wilkinson T, Gill DJ, Fitzjohn J, Palmer CL, Mulder RT. 2006. The impact
on students of adverse experiences during medical school. Med
Teach. 28(2):129135.
Witte FM, Stratton TD, Nora LM. 2006. Stories from the field: students
descriptions of gender discrimination and sexual harassment during
medical school. Acad Med. 81(7):648654.
Wolf T, Scurria PL, Webster MG. 1998. A four-year study of anxiety,
depression, loneliness, social support, and perceived mistreatment
in medical students. J Health Psychol. 3(1):125136.
Woolley DC, Paolo AM, Bonaminio GA, Moser SE. 2006. Student treat-
ment on clerkships based on their specialty interests. Teach Learn
Med. 18(3):237243.
Yu CY. 2002. Evaluating cut-off criteria of model fit indices for latent
variable models with binary and continuous outcomes [Doctoral
dissertation]. University of California Los Angeles.
MEDICAL TEACHER 821
... We did not find substantial differences between male and female students, which is encouraging. These results are consistent with those of Strand et al. [30] and Roberts et al. [31], but contradictory findings in a comprehensive study among nursing and medical students from New Zeeland show that bullying and harassment of students in health professional education are widespread problems [46]. However, the prevalence of bullying and harassment seems to vary, ranging, for instance, from 25% among physiotherapy students [47] to 91% among medical students [48] and 90% among nursing students [49]. ...
... However, the prevalence of bullying and harassment seems to vary, ranging, for instance, from 25% among physiotherapy students [47] to 91% among medical students [48] and 90% among nursing students [49]. This inconsistency may be due to several factors, such as the different definitions of bullying and harassment and the use of different instruments [46]. To our knowledge, with the exception of the UCEEM, few instruments measure equal treatment aspects in the CLE. ...
Article
Full-text available
Background The clinical learning environment (CLE) influences students’ achievement of learning outcomes and the development of their professional behaviors. However, CLEs are not always optimal for learning because of clinical productivity expectations and a lack of support from supervisors. The purpose of this study was to describe and compare students’ perceptions of their CLEs across four undergraduate programs. Methods This study is cross-sectional. In total, 735 students who were registered in the medical, nursing, physiotherapy, and speech-language pathology (SLP) programs were invited to participate. Data were collected using an online survey, which included demographics and the Undergraduate Clinical Education Environment Measure (UCEEM). The UCEEM consists of 26 items congregated into two overarching dimensions—experiential learning and social participation—with four subscales: opportunities to learn in and through work and quality of supervision, preparedness for student entry, workplace interaction patterns and student inclusion, and equal treatment. Results In total 280 students (median age 28; range: 20–52; 72% females) returned the questionnaire. The mean total UCEEM score was 98.3 (SD 18.4; range: 91–130), with physiotherapy students giving the highest scores and medical students the lowest. The mean scores for the dimensions experiential learning and social participation for all the students were 62.8 (SD 13.6; range 59–85) and 35.5 (SD 6.2; range 13–45), respectively. Medical students rated the lowest for all subscales. The items receiving the highest ratings concerned equal treatment, whereas those receiving the lowest ratings concerned supervisors’ familiarity with the learning objectives. There were few statistically significant differences between the semesters within each program. Conclusions The students generally hold positive perceptions toward their CLEs. However, the students from the medical and nursing programs rated their learning environment lower than did the students from the physiotherapy and SLP programs. Importantly, in several aspects, the medical students provided significantly lower ratings for their CLE compared with the students from the other programs. The medical students’ low ratings for their supervisors’ familiarity with the learning objectives underscore the need to ensure that the prerequisites for optimal supervision are met.
... In case of degree holder nurses' work distractions, interruptions and nurse-patient ratio are main issues. Mental workload and physical workload initiated by delegation of trifling tasks impaired their cognition (Smith-Han, et al., 2020). The organizational positive work management according to PSYCHOLOGY AND EDUCATION (2021) 58(5): 4656-4677 ISSN: 1533 www.psychologyandeducation.net competence levels of workers can minimize these problems. ...
Article
Full-text available
The negative workplace behaviors are relevant to understand qualified nurses' job satisfaction and their advance level roles. The focus of this study is to reveal the impact of role conflict and workload on job satisfaction of degree holder nurses and moderating role of perceived organizational support. The data collected through adopted questionnaire from clinical degree holder nurses and nurse educators of public hospitals and nursing colleges and both are degree holder nurses working in clinical setting and academia. The questionnaires were 350 in numbers but only 300 were responded. SPSS software used to analyze the respondents' feedback. The results showed that workload and role conflict are negatively significantly correlated with job satisfaction. The study findings also provide the evidence that perceived organizational support moderates the negative impact of role conflict on job satisfaction of degree holder nurses. The strength of this study is to provide implications for medical and nursing leadership to enhance organizational support and to promote the advance roles such as Advance Nurse Practitioner (ANP) for qualified nurses in health sector.
... Workplace bullying is a pervasive problem in the nursing profession and threatens nurses' health and ability to work safely (Jones, 2017). Workplace bullying can be described as 'situations where an employee is persistently exposed to negative and aggressive verbal or behaviors at work' (Smith-Han et al., 2020). It is estimated that 22.0%~44.0% of nurses experience bullying at some point in their professional lives (Esfahani & Shahbazi, 2014;Schlitzkus et al., 2014). ...
Article
Aim To identify the effect of workplace bullying on the relationship between occupational burnout and turnover intentions among clinical nurses. Background Recent evidence showed that a sense of burnout may cause workplace bullying; nevertheless; few studies have explored the effects of occupational burnout on workplace bullying. Furthermore, whether the experience of workplace bullying can aggregate the effect of occupational burnout on turnover intentions remains unclear. Methods A cross-sectional study was conducted to recruit nursing staff from two general hospitals in Taiwan. Data measurements comprised demographic characteristics, workplace bullying (Negative Acts Questionnaire-Revised), occupational burnout (occupational burnout inventory), and turnover intentions (employee turnover intentions and job destination choice). A hierarchical linear regression model and indirect effect-test were conducted to examine the effect of workplace bullying on the relationship between occupational burnout and turnover intentions. Results An indirect effect-test confirmed that workplace bullying can exacerbate the effect of occupational burnout on turnover intentions. Nearly one in ten nurses with occupational burnout may have experienced bullying at work, which increased their turnover intentions. Conclusions Reducing workplace bullying should be considered an important strategy for lowering turnover rates in nursing environments. Nursing mangers should develop appropriate strategies and establish mandatory regulations to create a respectful work environment. Moreover, continuous education and training to empower nursing staff to confront and eliminate workplace bullying are required in healthcare institutions.
Article
Full-text available
Introduction: Bullying is a form of interpersonal aggressive behavior that may lead to alarming growth. Nursing students entering clinical practice are at risk of injury and bullying, which can have unpleasant consequences. This study endeavored to identify the frequency, type, source of bullying, method of report, and the undesirable effect of bullying on students. Methods: This descriptive study was performed on 193 of third and fourth year nursing students of Medical Sciences Universities of Yazd, Iran, who experienced the clinical setting. Sampling was done through stratified random sampling. Data collection instruments were questionnaires of demographic characteristics and Clark et al.'s bullying behaviors. Data were analyzed both descriptively and inferentially. Results: According to the findings, 61.85% (n=118) of the participants experienced bullying, of which 32% (n=769) were bullied by the clinical instructor and 24.8% (n=597) by the nurse and 16.72% (n = 402) by the patient and family. The most bullying behaviors (15.93%, n = 383) were frozen out, ignored and excluded. 10.9% (n=21) decided to leave the field due to bullying. 67.9% (n=131) did not report bullying behavior, of which 36.8% (n=55) expressed that the reason was fear of poor evaluation. Conclusion: The results indicated that more than half of nursing students are exposed to a variety of bullying behaviors that are often performed by a clinical instructor and are not reported for fear of poor evaluation which can be a factor of intentions to leave the nursing program. Therefore, clinical educational authorities should teach nursing students some ways to prevent bullying and how to react to such behaviors and appropriate measures should be taken into account to improve clinical learning contexts.
Article
Full-text available
Background: Student bullying in the clinical environment continues to have a substantial impact, despite numerous attempts to rectify the situation. However, there are significant gaps in the literature about interventions to help students, particularly a lack of specific guidance around which to formulate an intervention program likely to be effective. With this narrative review about student bullying interventions in the clinical learning environment, we examine and draw together the available, but patchy, information about 'what works' to inform better practice and further research. Methods: We initially followed a PICO approach to obtain and analyse data from 38 articles from seven databases. We then used a general inductive approach to form themes about effective student bullying intervention practice, and potential unintended consequences of some of these, which we further developed into six final themes. Results: The diverse literature presents difficulties in comparison of intervention efficacy and substantive guidance is sparse and inconsistently reported. The final analytical approach we employed was challenging but useful because it enabled us to reveal the more effective elements of bullying interventions, as well as information about what to avoid: an interventionist and institution need to, together, 1. understand bullying catalysts, 2. address staff needs, 3. have, but not rely on policy or reporting process about behaviour, 4. avoid targeting specific staff groups, but aim for saturation, 5. frame the intervention to encourage good behaviour, not target poor behaviour, and 6. possess specific knowledge and specialised teaching and facilitation skills. We present the themed evidence pragmatically to help practitioners and institutions design an effective program and avoid instigating practices which have now been found to be ineffective or deleterious. Conclusions: Despite challenges with the complexity of the literature and in determining a useful approach for analysis and reporting, results are important and ideas about practice useful. These inform a way forward for further, more effective student bullying intervention and research: an active learning approach addressing staff needs, which is non-targeted and positively and skilfully administered. (331w).
Article
Full-text available
Purpose: In order to foster positive student experiences in the clinical learning environment, we wanted to better understand which teaching practices they regard highly. Methods: In 2016, the authors undertook a paper 'exemplar' survey (ES) of all fifth year medical students at one tertiary teaching site. Students had experienced all assigned clinical rotations over a two year period. Following a 66% response rate, we identified two clear exemplar clinical areas (ECAs). Over 2016-7, six focus groups with multidisciplinary staff members from these clinical areas were held, with the aim to identify, discuss and understand their specific teaching practices in more detail. Results: The authors present descriptions of positive student experiences and related staff practices, in five themes. Themes emerged around foundational logistic and personal factors: central to student and staff data is that 'welcome' on a daily, and ongoing basis, can be foundational to learning. Central to ECA staff data are universal practices by which all staff purposefully work to develop a functional staff-student relationship and play a part in organising/teaching students. Students and ECA staff groups both understood teacher values to be central to student learning and that cultivating a student's values is one of their major educational tasks. Conclusions: The framework formed by this thematic analysis is useful, clear and transferrable to other clinical teaching contexts. It also aligns with current thinking about best supporting student learning and cultivating student values as part of developing professionalism. Instigating such practices might help to optimise clinical teaching. We also tentatively suggest that such practices might help where resources are scarce, and perhaps also help ameliorate student bullying.
Article
Full-text available
Background Student bullying in clinical practice persists, and poor outcomes continue: for learning, academic achievement and career goals, for their mental and physical health and potentially affecting all staff and patients in a clinical workplace. We describe an emergent framework for the strategic design of a bullying intervention, presented as a staff development opportunity. Methods CAPLE (Creating A Positive Learning Environment) was a bullying intervention designed around current best evidence about ameliorating student bullying in the clinical environment. CAPLE was also an action research project delivered in two eight- week cycles, one in 2016 & another in 2017. CAPLE’s primary practical foci were to offer clinical staff in two separate hospital wards an opportunity to develop their clinical teaching skills and to guide them in reflection and cultivation of values around students and learning. Research foci were: 1. to gain insight into staff experiences of CAPLE as a development process and 2. to evaluate how CAPLE might best help staff reflect on, discuss and develop values around student learning, to include bullying. Staff undertook five active learning workshops combined with supportive contact with one researcher over the research period. Data include individual interviews, staff and researchers’ reflective journals and a paper survey about staff experiences of the 2017 intervention. Results We confirm the effectiveness of best evidence from the literature and also that a strategic four-part framework of approach, process, content and person can further enhance a bullying intervention by increasing the likelihood of participant engagement, learning and values change. Conclusions This research aggregates and adds weight to the current literature about student bullying and adds important pragmatic detail about best practice for bullying intervention design and delivery. Ultimately, this emergent framework offers insight to help move past some persistent barriers encountered by those wishing to improve workplace behaviour.
Article
Full-text available
Background: The topic of discrimination, bullying and sexual harassment in surgery was raised in the Australian media earlier in 2015. This led the Royal Australasian College of Surgeons (RACS) to commission an Expert Advisory Group to investigate and advise the College on their prevalence in surgery in Australia and New Zealand. This paper reports the findings with respect to prevalence of these inappropriate behaviours. Methods: The data in this paper were drawn from the published results of two quantitative surveys. One was an online survey sent to all RACS members. The other was an invited survey of hospitals, medical institutions and other related professional organizations including surgical societies. Results: The prevalence survey achieved a 47.8% response rate, representing 3516 individuals. Almost half of the respondents 1516 (49.2%) indicated that they had experienced one or more of the behaviours. This proportion was consistent across every specialty. Male surgical consultants were identified as the most likely perpetrators. More than 70% of the hospitals reported that they had instances in their organization of discrimination, bullying or sexual harassment by a surgeon within the last 5 years. Surgical directors or surgical consultants were by far the most frequently reported perpetrators (in 50% of hospitals). Conclusions: Discrimination, bullying and sexual harassment are common in surgical practice and training in Australia and New Zealand. RACS needs to urgently address these behaviours in surgery. This will involve a change in culture, more education for fellows and trainees, and better processes around complaints including support for those who have suffered.
Article
Full-text available
Women represent 15% of practicing general surgeons. Gender-based discrimination has been implicated as discouraging women from surgery. We sought to determine women's perceptions of gender-based discrimination in the surgical training and working environment. Following IRB approval, we fielded a pilot survey measuring perceptions and impact of gender-based discrimination in medical school, residency training, and surgical practice. It was sent electronically to 1,065 individual members of the Association of Women Surgeons. We received 334 responses from medical students, residents, and practicing physicians with a response rate of 31%. Eighty-seven percent experienced gender-based discrimination in medical school, 88% in residency, and 91% in practice. Perceived sources of gender-based discrimination included superiors, physician peers, clinical support staff, and patients, with 40% emanating from women and 60% from men. The majority of responses indicated perceived gender-based discrimination during medical school, residency, and practice. Gender-based discrimination comes from both sexes and has a significant impact on women surgeons.
Article
Research is lacking on psychological distress and disorder among sexual minority medical students (students who identify as nonheterosexual). If left unaddressed, distress may result in academic and professional difficulties and undermine workforce diversity goals. The authors compared depression, anxiety, and self-rated health among sexual minority and heterosexual medical students. This study included 4,673 first-year students who self-reported sexual orientation in the fall 2010 baseline survey of the Medical Student Cognitive Habits and Growth Evaluation Study, a national longitudinal cohort study. The authors used items from published scales to measure depression, anxiety, self-rated health, and social stressors. They conducted bivariate and multivariate analyses to estimate the association between sexual identity and depression, anxiety, and self-rated health. Of 4,673 students, 232 (5.0%) identified as a sexual minority. Compared with heterosexual students, after adjusting for relevant covariates, sexual minority students had greater risk of depressive symptoms (adjusted relative risk [ARR] = 1.59 [95% confidence interval, 1.24-2.04]), anxiety symptoms (ARR = 1.64 [1.08-2.49]), and low self-rated health (ARR = 1.77 [1.15-2.60]). Sexual minority students were more likely to report social stressors, including harassment (22.7% versus 12.7%, P < .001) and isolation (53.7% versus 42.8%, P = .001). Exposure to social stressors attenuated but did not eliminate the observed associations between minority sexual identity and mental and self-reported health measures. First-year sexual minority students experience significantly greater risk of depression, anxiety, and low self-rated health than heterosexual students. Targeted interventions are needed to improve mental health and well-being.