Procedure-based assessments in trauma and orthopaedic training – The trainees’ perspective

Abstract

Introduction:
The study aimed to gain an understanding of the attitudes of trauma and orthopaedic (T&O) trainees regarding procedure-based assessments (PBAs) and identify factors that influence any perceived educational benefit.

Methods and materials:
A questionnaire was emailed to all T&O trainees in the UK via an established e-mail communication tool after an initial pilot exercise. The data were analysed using the online survey software.

Results:
Of the 616 trainees included 53% found PBAs useful as a learning tool for delivery of feedback. Trainees agreed that there were barriers to the successful use of PBAs (61%). Completing the PBA at the time of the procedure (p < 0.001) and the trainer delivering quality feedback with PBAs (p < 0.001) significantly increased the number of trainees perceiving an improvement in their practice. Completing higher numbers of PBAs did not have this effect (p = 0.26). There was wide geographical variation in the use of PBAs by trainees.

Conclusions:
This is the first nationwide study offering a deeper insight into factors influencing T&O trainees' perceptions of the educational benefit gained from using PBAs. This study informs the debate on how to improve the effective use of PBAs in T&O training, and generally, of workplace-based assessments in surgical training.

Full text

2015, 37: 444–449
Procedure-based assessments in trauma and
orthopaedic training – The trainees’ perspective
ALISTAIR R. HUNTER, EMILY J. BAIRD & MIKE R. REED
University College London Hospitals NHS Foundation Trust, UK
Abstract
Introduction: The study aimed to gain an understanding of the attitudes of trauma and orthopaedic (T&O) trainees regarding
procedure-based assessments (PBAs) and identify factors that influence any perceived educational benefit.
Methods and materials: A questionnaire was emailed to all T&O trainees in the UK via an established e-mail communication
tool after an initial pilot exercise. The data were analysed using the online survey software.
Results: Of the 616 trainees included 53% found PBAs useful as a learning tool for delivery of feedback. Trainees agreed that there
were barriers to the successful use of PBAs (61%). Completing the PBA at the time of the procedure (p50.001) and the trainer
delivering quality feedback with PBAs (p50.001) significantly increased the number of trainees perceiving an improvement in
their practice. Completing higher numbers of PBAs did not have this effect ( p¼0.26). There was wide geographical variation in the
use of PBAs by trainees.
Conclusions: This is the first nationwide study offering a deeper insight into factors influencing T&O trainees’ perceptions of the
educational benefit gained from using PBAs. This study informs the debate on how to improve the effective use of PBAs in T&O
training, and generally, of workplace-based assessments in surgical training.
Introduction
Procedure-based assessments (PBAs) are one form of work-
place-based assessment used in postgraduate medical training
and are the principal method of assessing a trainee’s surgical
skills in the operating theatre in the UK (Beard et al. 2011). The
trainee is assessed against a six-domain competency checklist,
consisting predominantly of generic competencies (e.g., pre-
operative planning and preparation) with some specific
procedure. A global summary score is divided into four
levels, with the lowest rating being unable to perform the
procedure and the highest rating the ability to perform the
procedure unsupervised at the level expected of a specialist in
practice (Intercollegiate Surgical Curriculum Programme 2014)
A PBA provides the trainee with formative assessment in
the form of constructive feedback from their trainer on their
performance in a particular operation (Pitts et al. 2005). Over
time, PBAs are completed with a variety of trainers and for a
range of operations. When collated, they form a summative
assessment of competence and progression of a trainee in
learning surgical procedures. In addition, PBAs are used in
decisions regarding suitability of a trainee for progression or
completion of training at their annual review of competence
progression (ARCP) meeting. This review is led by the regional
lead for each specialty Training Programme, known as the
Training Programme Director (TPD).
PBAs are a recent development in surgical assessment. The
introduction of competence-based curricula (PMETB 2005;
Scheele et al. 2008; Modernising Medical Careers 2010) has led
to the development of these formalised assessments of
technical skills and professional behaviours as one method
of workplace-based assessment in surgical training. PBAs were
introduced to trauma and orthopaedic (T&O) surgical training
in 2005 (Pitts et al. 2005) and into the surgical specialties
curricula by the Intercollegiate Surgical Curriculum Project
(ISCP) in 2007. These changes represented a major shift away
from the previous lengthy apprenticeship model (Galasko &
Mackay 1999; Thornton et al. 2003; Pitts & Rowley 2009),
where technical and non-technical skills were not formally
assessed in the workplace, towards shorter working hours
(Department of Health 2003), increased objective assessment
of doctors (Darzi et al. 1999) and emphasis on supervised
training opportunities.
Practice points
Trainees value the feedback from PBAs more than the
summative assessment gained.
Increased use of the free text feedback is encouraged
in improving the quality of the feedback given by
trainers.
Lessons can be learned from regions using PBAs
effectively in standardising good practice.
Focusing on quality rather than the number of
assessment events may improve the educational
benefit gained by trainees.
Correspondence: Alistair R. Hunter, Specialist Registrar, Department of Trauma and Orthopaedics, University College Hospital 235 Euston Road,
London NW1 2BU, UK. Tel: 44 20 3447 9418; Fax: 44 20 3447 9081; E-mail: hunteralistair@hotmail.com
444 ISSN 0142-159X print/ISSN 1466-187X online/14/050444–6 ß2014 Informa UK Ltd.
DOI: 10.3109/0142159X.2014.956055
Med Teach Downloaded from informahealthcare.com by University College London on 05/18/15
For personal use only.

The attitudes of surgical trainees to PBAs and the educa-
tional impact of PBAs on surgical trainees (Miller & Archer
2010) are not well understood. There is known to be some
dissatisfaction among surgical trainees with the general use of
the workplace-based assessment components of ISCP (Pereira
& Dean 2013). Previous research by Marriott et al. (2011)
concluded that PBAs had good acceptability among trainees
across several surgical specialties. However, analysis of the
relatively small sample of trainees was not broken down by
specialty and the study was conducted in only one region,
limiting generalisability of the results.
Trauma and orthopaedic (T&O) trainees make up 24%
of all UK surgical trainees (JCST, personal communication)
and have been using PBAs for the longest period of time
(Pitts et al. 2005). Given the degree of trainee and trainer
investment in PBAs and the weight given to them by the
governing regional and national training bodies (Joint
Committee on Surgical Training Quality Indicators for
Surgical Training 2012), investigating the attitudes of T&O
trainees towards PBAs and assessment of their perceived
educational impact seems to be both important and worth-
while. The aim of the study was to gain an understanding of
the attitudes of T&O trainees across the UK regarding their use
of PBAs and identify factors influencing any perceived
educational benefit. Educational benefit is defined for the
purpose of this study as an improvement to a trainee’s learning
of surgical skills, or as being valuable in providing evidence of
surgical competence.
Materials and methods
All T&O registrars holding a National Training Number
(‘‘trainees’’) in the UK are required to use the eLogbook
system to submit data relating to their operative experience
(Sher et al. 2005). An Internet-based questionnaire was sent to
all T&O trainees via the eLogbook e-mail system. The study
was approved by the eLogbook Validation and Audit
Committee.
In designing the questionnaire, an item pool of attitude
statements was constructed according to established methods
(Oppenheim 1992). In order to establish the intensity of
attitude to a statement, a five-point Likert-type scale was used,
ranging from ‘‘strongly disagree’’ to ‘‘strongly agree’’ and free
text responses. A pilot questionnaire of 12 trainees attending
their ARCP was conducted focussing on question phrasing and
clarity. The questionnaire was then further refined and
discussions with the then Educational Advisor to the British
Orthopaedic Association. Further questions were included to
provide more information on the changes in behaviour of
trainees in response to using PBAs, and which aspects trainees
found most useful, before conversion to an online format.
An e-mail with a link to the online questionnaire was sent to
all T&O trainees via eLogbook. A reminder e-mail was sent 1
week later in order to maximise response rate (Solomon 2001).
Responding trainees with an NTN who had previously used
PBAs were included. Analysis of results was conducted using
the online survey software and Microsoft Office Excel 2003
(Microsoft Corp, Redmond, Washington, DC). All tests
of significance were made using the Chi-squared test with a
pvalue of 50.05 considered significant.
Results
Of 668 responses 616 met inclusion criteria. This represents
54% of the total 1144 trainees in the UK at that time (JCST,
personal communication). There was a broad, representative
spread of seniority from ST3 to post-CCT trainees, and
of geographical location (defined by Regional Training
Programme) within the UK. The mean duration of use of
PBAs was 3.75 years (range 1–5 years, n¼614). Per six-month
post, trainees completed a median minimum number of four
PBAs (interquartile range (IQR) 3–6), and a median maximum
number of eight PBAs (IQR 6–10) (n¼601), with 35.9% of
trainees completing a maximum of 10 or more.
Which aspects of PBAs do trainees find
educationally beneficial?
More trainees found value in verbal feedback at the time of the
procedure than in using PBAs as evidence of competence
(Figure 1). The role of feedback was emphasised, with 53%
perceiving verbal feedback to be a useful learning tool
(Table 1). Overall 29.3% of trainees reported that they had
Figure 1. The attitudes of trainees to the most valuable aspects of PBAs (n¼592).
PBAs in T&O training – Trainees’ perspective
445
Med Teach Downloaded from informahealthcare.com by University College London on 05/18/15
For personal use only.

improved their practice as a result of using PBAs (Table 1).
41.3% of the respondents did not agree that they had improved
their practice as a result of using PBAs, underlining the
importance of improving PBAs as a learning tool for
trainees. On analysing the 50.4% of trainees who agreed
they had received good quality feedback when using PBAs
49.3% (143/290) of this group agreed they improved
their practice as a result of using PBAs. Of the trainees who
did not receive quality feedback (28.9%), only 9% (15/167)
agreed their practice was improved a significant difference
compared with the group reporting high-quality feedback
(p50.001).
Opinion was divided among trainees as to whether PBAs
should be used at their ARCP as evidence of the surgical
competence of the trainee (Table 1). About 42% thought that
PBAs were a valid assessment, but 30.2% disagreed, some
trainees giving examples in the free text of PBAs being
routinely completed retrospectively just prior to their ARCP.
Furthermore, trainees reported that the reliability of a PBA was
sometimes undermined by the different interpretation of
standards among trainers. These factors may explain the
51.4% who perceived completion of PBAs to be nothing but a
‘form-filling exercise’. However, trainees were more positive
about the role of PBAs in measuring progression in their
surgical competence over time (55.9%).
Effective implementation is a key to achieving
educational benefit
About 60.7% of trainees perceived there are barriers to the
successful use of PBAs (Table 1). Trainees highlighted several
factors, which can be subdivided into individual and organ-
isation factors.
Individual factors
The majority of trainees (77.8%) felt that the enthusiasm of
their trainer for PBAs determined the extent of benefit to be
gained from using them (Table 2). Trainer behaviours
identified in the free text as beneficial include identifying
cases on operating lists for PBAs and taking time to go through
the PBA with the trainee immediately after the procedure.
A similar high proportion of trainees (63.7%) felt that their own
role was important in determining the benefit gained from
PBAs (Table 2).
An analysis was performed to determine the behaviours
which had the most significant effect on the satisfaction of
trainees with PBAs. The completion of PBAs at the time of the
procedure was found to be most important, with significant
increases in their perception of PBAs as an effective learning
tool (p50.001), an effective tool in the annual progress review
(p50.001) and in improving their practice (p50.001).
Table 1. Trainee responses to attitude statements.
Statement N
Strongly
disagree (%) Disagree (%) Uncertain (%) Agree (%)
Strongly
agree (%)
Feedback from PBAs is a useful learning tool 588 5.1 19.0 22.8 49.3 3.7
In the past six months, your trainer has provided quality
feedback when using PBAs
580 4.8 24.1 20.7 45.7 4.7
On most occasions you complete PBA assessments and
gain feedback around the time of the procedure
581 6.2 27.9 11.7 51.6 2.6
Personally, using PBAs motivates you to talk about your
surgical skills with your trainer
582 10.0 27.5 22.2 36.4 4.0
You have improved your practice as a result of the feedback
from PBAs
588 9.5 31.8 29.4 27.4 1.9
There are barriers to the successful use of PBAs by trainees 584 1.6 12.8 24.9 48.4 12.3
Completing PBAs is nothing but a ‘form-filling exercise’ 593 2.2 22.6 23.8 32.0 19.4
PBAs should be used as evidence of a trainee’s surgical
competence in their ARCP/RITA
591 13.2 20.5 34.5 30.3 1.5
A PBA makes a valid assessment of a trainee’s surgical
competence in a particular operation
593 9.3 20.9 27.8 40.3 1.7
Over time, a series of PBAs can reflect a change in the level
of your surgical competence
592 6.3 16.7 21.1 52.4 3.5
Table 2. The role of the trainer, trainee, and TPD.
Statement N
Strongly
disagree (%) Disagree (%) Uncertain (%) Agree (%)
Strongly
agree (%)
The enthusiasm of your trainer towards PBAs
dictates the benefit gained from them
586 2.4 7.5 12.3 52.2 25.6
The enthusiasm of the trainee towards PBAs
dictates the benefit gained from them
584 2.7 14.6 19.0 52.2 11.5
Your Training Programme Director strongly
encourages your trainers to use PBAs as
an important aspect of your training
585 1.4 6.3 25.0 49.9 17.4
A. R. Hunter et al.
446
Med Teach Downloaded from informahealthcare.com by University College London on 05/18/15
For personal use only.

We analysed how the number of PBAs completed relates to
trainee satisfaction. The subgroup that had completed a
maximum of 10 or more PBAs in a six-month post (35.9% of
trainees) were no more likely to complete PBAs at the time
of the procedure (p¼0.19), find the feedback as a useful
learning tool (p¼0.34) or improve their practice due to PBAs
(p¼0.26).
Organisation factors
There was wide regional variability between Training
Programmes in the experience of trainees using PBAs. There
was no obvious geographical pattern, but the example in
Table 3 shows how important the influence of the Training
Programme can be. TPDs were perceived as being broadly in
favour of the role of PBAs in training (Table 2). Trainees who
strongly agreed that they had supportive TPDs were signifi-
cantly more likely to agree with the role of PBAs in their
annual review of progress (p¼0.012) and that feedback from
PBAs had improved their practice (p¼0.014).
Discussion
Understanding the attitudes of trainees regarding PBAs, the
perceived educational benefit derived from their use and the
factors influencing the trainees in these perceptions are
important in identifying how to improve the use of PBAs in
surgical training. Without a perceived educational benefit
experienced by trainees, PBAs simply become a paper
exercise. This is the first nationwide study investigating the
attitudes of T&O trainees towards PBAs in the UK, and has
relevance given the current shift towards increasing work-
place-based assessment of trainees and revalidation of
surgeons.
A major evaluation of ISCP (Joint Committee on Surgical
Training 2012) highlighted that outside pressures, such as
external administrative or structural training agendas, can
mean PBAs (and WBAs in general) are not used effectively and
lose their utility as drivers of experiential learning and
development. Our results and recommendations in this study
mirror these and other findings in the evaluation, such as the
constructive use of WBAs by some trainers and the develop-
ment, in some quarters, of a detrimental ‘tick-box’ culture.
A good response rate was achieved, indicating that this is
an issue of importance for trainees. A range of opinions were
expressed about the use of PBAs in training but trainees were
broadly supportive. When used effectively, trainees found
PBAs to be educationally beneficial. The formative role of
PBAs as a learning tool was felt to be more valuable than their
summative role as an assessment tool in their ARCP. This may
reflect the unease of some trainees regarding the validity and
reliability of the assessment event when PBAs are not used
effectively. Recent GMC guidance (GMC 2011) has empha-
sised the need for trainees and trainers to differentiate between
formative and summative WBAs, addressing one of the main
concerns about the misuse of WBAs in general.
Trainees felt that barriers exist to the effective use of PBAs.
We have identified factors that trainees thought influenced the
effectiveness of PBAs and their perceived educational benefit.
The most important individual factor was the enthusiasm of the
trainer; without this, trainees felt the value of the process was
limited. The importance of the trainer in maximising the
benefit gained from using PBAs has been cited in other studies
(Norcini & Burch 2007; Beard 2008). Assessor training for
WBAs has been shown to reduce the variability among trainers
when provide feedback and assessment (Holmboe et al. 2004).
It may be worth considering inclusion of compulsory training
for trainers in the use of PBAs and WBAs.
Another important factor was the quality of the feedback
received. In our study, trainees receiving good quality feed-
back when using PBAs felt that they were significantly more
likely to change their surgical practice as a result. This
underlines that well-implemented feedback from WBAs can
lead to a perceived positive effect on practice (Saedon et al.
2012). The reasons for failure of trainers to deliver quality
feedback are known (Ali 2013) and include focusing on the
assessment of performance at the expense of providing
adequate feedback. Inclusion of compulsory written feedback
in PBAs as a record of the verbal feedback conversation may
focus the feedback and improve its quality.
The enthusiasm of the trainee was perceived to have an
important role in gaining benefit from PBAs, which can be
manifested in behaviours such as contemporaneous comple-
tion of PBAs. This was the most important behaviour in
improving the perceived educational benefit to trainees.
Encouraging this practice appears crucial in maximising
trainee satisfaction and educational benefit, and could be
achieved with a limit of 72 h within which to complete the PBA
online.
This study showed that completing a high number of PBAs
does not improve the perception of trainees regarding the
educational benefit gained. Furthermore, there is an absence
of any published evidence that a higher number of WBAs
completed in a post signifies a higher quality of training in that
Table 3. Example of the variation in the use of and attitudes to PBAs across Training Programmes in the UK.
Training Programme 1,
N¼11 (%)
Training Programme 2,
N¼32 (%)
Training Programme 3,
N¼38 (%)
Completed a maximum of more than 10 in a
six-month post
100 6.25 36.8
Feedback from PBAs is a useful learning tool 54.5 50.0 83.8
Your Training Programme Director strongly
encourages your trainers to use PBAs as
an important aspect of your training
100 40.6 78.4
PBAs in T&O training – Trainees’ perspective
447
Med Teach Downloaded from informahealthcare.com by University College London on 05/18/15
For personal use only.

post. A fixed required number of WBAs at 40 per year (Joint
Committee on Surgical Training 2012) could undermine the
effective use of PBAs by encouraging the completion of the
assessment as a form-filling exercise. Our study adds to
concerns about how the current implementation of workplace-
based assessment is at odds with the intended use (Ali 2013).
Simply increasing the number of WBAs required per annum is
not regarded by the authors as a way to improve the
educational benefit to trainees. Instead, the emphasis should
be on how the PBAs are used and the quality of the feedback
and assessment.
As this study was conducted at a national level, we were
able to identify a great deal of variability between Training
Programmes, thus highlighting the importance of regional
factors in the attitudes of trainees and in their use of PBAs. Our
study shows that TPDs have an important influence in the
implementation of PBAs in their training regions and on
the educational benefit gained by trainees. Perhaps, then, it is
the ‘‘culture’’ surrounding PBAs in that region which shapes
the attitudes and behaviour of trainees. A culture in which
PBAs are valued as an educational tool and are conducted in a
way to ensure optimal benefit is likely to be important in
ensuring trainee satisfaction and delivering educational bene-
fit. Finding ways to implement good practice across Training
Programmes would help to standardise their use across the
UK.
As a questionnaire-based study, there are limitations in the
interpretation of the data. How far respondents had a shared
understanding of the meaning of terms used in the question-
naire is difficult to determine. In addition, separating out the
attitudes relating to the PBA assessment event from those of
WBAs in general is not easy. In piloting the study and using
clear phrasing, we hope to have mitigated against these issues
as far as possible. Likely responder bias is also acknowledged,
with those trainees more enthusiastic or disgruntled with PBAs
in general perhaps more likely to reply. Despite these
limitations, a questionnaire-based study was chosen in order
to include the greatest possible number of T&O trainees.
In conclusion, this study has examined the attitudes and
experiences of a sizeable cohort of T&O trainees across the
UK. Based on an enhanced understanding of the individual
and organisational level factors that influence both the
attitudes of trainees and the perceived educational benefit
derived from PBAs, possible improvements in their use and
implementation have been identified. Further work should
seek to identify whether the factors identified here apply to the
use of PBAs in other surgical specialities. Our findings may
also offer insights into the use of other, similar workplace-
based assessments, used outside the UK. These findings
should inform the debate on how to advance the effective use
of PBAs in T&O training and the wider use of PBAs and
workplace-based assessments in surgical training in general.
Notes on contributors
ALISTAIR R. HUNTER, MA, MSc, FRCS (Tr & Orth), FHEA, is a Trauma and
Orthopaedics Registrar at Chelsea and Westminster Hospital, London. He
has a strong interest in workplace-based assessment and curriculum design
for surgical specialties. He co-authored the national trainee position
statement on the use of PBAs in Orthopaedic Training.
EMILY J. BAIRD,FRCS (Tr & Orth) MFSTEd, is a Paediatric Orthopaedic
Fellow at RHSC, Edinburgh, and Past President of the British Orthopaedic
Trainees’ Association. She has a keen interest in education in orthopaedics,
especially work-based assessments, the development of clinical scenarios
for national selection interviews and OSCEs for paediatric orthopaedics.
MIKE R. REED,FRCS (T&O), MD, is a UK Training Programme Director and
a co-author of the trauma and orthopaedic curriculum. He has previously
been a member of the speciality advisory committee and the training
standards committee. He now chairs the education committee of the British
Orthopaedic Association.
Glossary
Performance-Based Assessment: Procedure-based assess-
ment is one form of Performance-Based Assessment
(Wojtczak 2003). This is an evaluation that demands
trainees be engaged in specified clinical activities. This
permits evaluation of an ability to perform clinical tasks and
not merely the recitation of medical knowledge. Typical
measurement tools for this form of testing are checklists,
observation logs, and anecdotal reports.
Wojtczak A. 2003. Glossary of medical education terms.
AMEE occasional paper no. 3. Dundee: AMEE.
Acknowledgements
The authors thank David Pitts for his contribution in discus-
sions regarding the design of the questionnaire and the design
of PBAs in T&O training and Lesley Pugsley for supervising the
inception of the study and its design.
Declaration of interest: The authors report that they have
no conflicts of interest.
References
Ali JM. 2013. Getting lost in translation? Workplace based assessments in
surgical training. Surgeon 11:286–289.
Beard JD. 2008. Assessment of surgical skills of trainees in the UK. Ann R
Coll Surg Engl 90:282–285.
Beard JD, Marriott J, Purdie H, Crossley J. 2011. Assessing the surgical skills
of trainees in the operating theatre: A prospective observational study.
Health Technol Assess 15:1–162.
Darzi A, Smith S, Taffinder N. 1999. Assessing operative skill. Needs to
become more objective. BMJ 318:887–888.
Department of Health. 2003. HSC 2003/001 –Protecting staff; delivering
services: Implementing the European Working Time Directive for
doctors in training. [Accessed 18 January 2014] Available from http://
www.dh.gov.uk/en/publicationsandstatistics/lettersandcirculars/
healthservicecirculars/DH_4003588.
Galasko C, Mackay C. 1999. Unsupervised surgical training: Logbooks are
essential for assessing progress. BMJ 315:1306–1307.
General Medical Council. 2011. Learning and assessment in the clinical
environment: The way forward. London: GMC.
Holmboe ES, Hawkins RE, Huot SJ. 2004. Direct observation of compe-
tence training: A randomized controlled trial. Ann Intern Med 140:
874–881.
Intercollegiate Surgical Curriculum Programme Website [Accessed 29 July
2014] Available from http://www.iscp.ac.uk/surgical/assessment_
A. R. Hunter et al.
448
Med Teach Downloaded from informahealthcare.com by University College London on 05/18/15
For personal use only.

pba.aspx https://www.iscp.ac.uk/assessments/pbaassessmentv2.aspx?
enc¼OwfD2eHNStF6zLw4þxaN9A¼¼.
Joint Committee on Surgical Training ISCP Evaluation Report. 2012.
[Accessed 31 July 2014] Available from http://www/jsct.org/
ISCP%20Evaluation%20/full_evaluation_report.
Joint Committee on Surgical Training Quality Indicators for Surgical
Training – Trauma & Orthopaedic Surgery. 2012. [Accessed 18
January 2014] Available from http://www.jcst.org/quality_assurance/
Docs/TandO _QI_Training.
Marriott J, Purdie H, Crossley J, Beard JD. 2011. Evaluation of procedure-
based assessment for assessing trainees’ skills in the operating theatre.
Br J Surg 98(3):450–457.
Miller A, Archer J. 2010. Impact of workplace based assessment on doctors’
education and performance: A systematic review. BMJ 341:c5064.
Modernising Medical Careers. 2010. Medical specialty training (England).
[Accessed 2 January 2014] Available from www.mmc.nhs.uk.
Norcini J, Burch V. 2007. Workplace-based assessment as an educational
tool: AMEE Guide No. 31. Med Teach 29(9):855–871.
Oppenheim AN. 1992. Questionnaire design, interviewing and attitude
measurement. London, New York: Continuum.
Pereira EA, Dean BJ. 2013. British surgeons’ experiences of a mandatory
online workplace based assessment portfolio resurveyed three years
on. J Surg Educ 70(1):59–67.
Pitts D, Rowley D. 2009. Competence evaluation in orthopaedics –
A ‘Bottom Up’ approach. In: Flin R, Mitchell L, editors. Safer surgery:
Analysing behaviour in the operating theatre. Farnham, England:
Ashgate Publishing Ltd. pp 27–46.
Pitts D, Rowley DI, Sher JL. 2005. Assessment of performance in
orthopaedic training. J Bone Joint Surg [Br] 87-B:1187–1191.
Postgraduate Medical Education and Training Board. 2005. Strategy
document: 2006–2010. London: PMETB 2005. [Accessed 18 December
2013] Available from http://www.gmcuk.org/PMETB_Strategy_2006_
2010_FULL_DOCUMENT__2005_.pdf_30377101.pdf.
Saedon H, Salleh S, Balakrishnan A, Imray CH, Saedon M. 2012. The role of
feedback in improving the effectiveness of workplace based assess-
ments: A systematic review. BMC Med Educ 2:12–25.
Scheele F, Teunissen P, Van Luijk S, Heineman E, Fluit L, Mulder H,
Meininger A, Wijnen-Meijer M, Glas G, Sluiter H, Hummel T. 2008.
Introducing competency-based postgraduate medical education in the
Netherlands. Med Teach 30:248–253.
Sher JL, Reed MR, Calvert P, Wallace WA, Lamb A. 2005. Influencing the
national training agenda. The UK & Ireland orthopaedic eLogbook.
J Bone Joint Surg [Br] 87(9):1182–1186.
Solomon DJ. 2001. Conducting web-based surveys. Pract Assess Res Eval
7(19). [Accessed 24 August 2014] Available from http://PAREonline.net/
getvn.asp?v¼7&n¼19.
Thornton M, Donlon M, Beard JD. 2003. The operative skills of higher
surgical trainees: Measuring competence rather than experience
undertaken. Ann R Coll Surg Engl 85:190–193.
Wojtczak A. 2003. Glossary of medical education terms. AMEE occasional
paper no. 3. Dundee: AMEE.
PBAs in T&O training – Trainees’ perspective
449
Med Teach Downloaded from informahealthcare.com by University College London on 05/18/15
For personal use only.