ArticlePDF Available


Traditionally, examiners’ reports on theses at the doctoral and Master’s level consist of two components: firstly, summative assessment where a judgement is made about whether the thesis has met the standards established by the discipline for the award of the degree, and, secondly, the developmental and formative component, where examiners provide feedback to assist the candidate to revise the thesis. Given this dual task of providing assessment and feedback, this paper presents the findings of a small-scale empirical study that aimed to gain insights into the connection or potential disjunction between feedback and assessment in six examiners’ reports. The main aim of this study was to identify the nature of examiners’ reports on Master’s and doctoral theses: is it primarily assessment or feedback? Our study suggests the crucial role of feedback in postgraduate thesis examination practice. Without feedback, there is little impetus for the candidate to progress, to close the gap between current and desired performance, and to attain the level needed to become a member of the scholarly community. The study concludes with the implications that a stronger focus on feedback might have for all stakeholders involved in the thesis examination process.
Examinersreports on theses: Feedback or assessment?
Vijay Kumar
, Elke Stracke
University of Otago, Higher Education Development Centre, PO Box 56, Dunedin 9054, New Zealand
University of Canberra, Faculty of Arts and Design, University Drive, Bruce ACT 2601, Australia
Examiner reports
Postgraduate students
Thesis examination
Traditionally, examinersreports on theses at the doctoral and Masters level consist of two
components: rstly, summative assessment where a judgement is made about whether
the thesis has met the standards established by the discipline for the award of the degree,
and, secondly, the developmental and formative component, where examiners provide
feedback to assist the candidate to revise the thesis. Given this dual task of providing
assessment and feedback, this paper presents the ndings of a small-scale empirical study
that aimed to gain insights into the connection or potential disjunction between feedback
and assessment in six examinersreports. The main aim of this study was to identify the
nature of examinersreports on Masters and doctoral theses: is it primarily assessment or
feedback? Our study suggests the crucial role of feedback in postgraduate thesis exami-
nation practice. Without feedback, there is little impetus for the candidate to progress, to
close the gap between current and desired performance, and to attain the level needed to
become a member of the scholarly community. The study concludes with the implications
that a stronger focus on feedback might have for all stakeholders involved in the thesis
examination process.
Ó2011 Elsevier Ltd. All rights reserved.
1. Introduction
Examiner reports play a crucial role in postgraduate examination both at the Masters and doctorallevel. At both levels, the
examiner report is the culmination of many years of supervised research.In Australia and New Zealand, for example, external
evaluation of a written thesis is essential if candidates are to be awarded the degrees for which they have entered. Two or
three examiners, who are not members of the supervisory committee, usually mark the thesis. Similarly, in the UK or
countries that follow the UK system (like Malaysia), examiners assess the thesis and prepare a written report. Even though
systems might vary with regard to any further examination requirements, for instance oral examinations, referred to as viva
or defence, all systems require written examiner reports. In this study we focus on two sets of examiner reports from New
Zealand and Malaysia respectively.
Examiners may consider the examination as a gate keepingtask, and/or as an opportunity to provide developmental
experiences (Joyner, 2003) to the candidate. The examiners usually make a summative judgement and also encourage
developmental experiences in the form of feedback (Kiley, 2009). In other words, a rst component of examiner reports is
a summative assessment where examiners make a judgement as to whether the thesis has met the standards established by
the discipline and the university for the award of the degree. The second component is developmental and formative, where
examiners provide feedback to assist the candidate to revise the thesis (Stracke & Kumar, 2010).
*Corresponding author. Tel.: þ61 2 6201 2492; fax: þ61 2 6201 2649.
E-mail addresses: (V. Kumar), (E. Stracke).
Contents lists available at ScienceDirect
Journal of English for Academic Purposes
journal homepage:
1475-1585/$ see front matter Ó2011 Elsevier Ltd. All rights reserved.
Journal of English for Academic Purposes 10 (2011) 211222
It should be noted that examinersassessments of theses are usually non-terminal in the sense that postgraduate
candidates are expected to take on board examinerscomments and revise their work. While an initial assessment is made,
which could range from accept with minor correctionsto resubmit, it is the norm to change this assessment to accepting the
thesis once revisions have been made and the goals met. Recent literature has highlighted the notion that doctoral examiners
assessments are in fact considered feedback on work in progress (Bourke, Hattie, & Anderson, 2004). Given this dual task of
examiners to provide summative assessment and feedback, this study aims to gain insights into the connection, or perhaps
potential disjunction, between feedback and assessment in examinersreports on theses. To understand this relationship, it is
essential to understand what assessment and feedback refer to in postgraduate thesis examination.
2. Theoretical background: assessment and feedback
2.1. Assessment
A conceptual denition of assessment refers to how much learning has taken place as a result of teaching (Gibbs &
Simpson, 200405). This denition emphasises, in the context of postgraduate supervision, the often-overlooked role of
supervisor and examiner as educators (Stracke, 2010). In this respect, assessment considers learning outcomes that is,
whether the outcomes meet the standards that have been established. In a sense, assessment provides information about
a performance. The performance standards are usually listed as assessment criteria or, in the case of a thesis, usually classied
as guidelines for examiners see Appendices A (Guidelines for Masters theses at University MAL (Malaysia) and B(Guidelines
for PhD theses at University NZ (New Zealand)) for examples. It should be noted that in postgraduate education at the
Masters and doctoral levels the institutions awarding the degree usually prepare assessment criteria. Examiners are asked to
decide if certain learning outcomes have been met. As an example, one of the learning outcomes of the PhD is whether the
thesis makes an original contribution to knowledge. If a candidate has met this criterion, the assumption is that the objectives
of this learning outcome have been met. But even though the examination criteria are made available to the examiners,
examiners may interpret the criteria based on their own scholarly understanding and interpretation. This notion is reported
in a study on doctoral examination (Mullins & Kiley, 2002) where empirical evidence suggests that many experienced
examiners do not make use of institutional criteria when assessing a thesis but rather use their own professional intuition to
assess learning outcomes. There may be the notion of the hidden curriculum (Snyder, 1971) by which examiners assess the
learning outcomes.
A second conceptual understanding of assessment emphasises the view of assessment as educational measurement, that
is, assessment is a measure of competence. Assessment refers to any appraisal (or judgement, or evaluation)(Sadler, 1989,
p. 120) and has been suggested to serve two purposes: summative and formative. Bloom, Hastings, and Madaus (1971)
dened summative assessment as those assessments given at the end of a semester/program or mid-semester with the
sole purpose of grading or evaluation of progress. Summative assessment indicates whether a learning goal has been ach-
ieved. Summative assessment is a passive measure of performance because it does not normally have immediate impact on
learning(Sadler, 1989, p. 120). In summative assessment, a nal grade is given in a thesis, it is usually a pass with several
degrees of acceptance criteria, which could range from the thesis being accepted, accepted with minor modications,
accepted with major modications, to a resubmission or even a fail.
In contrast, assessment that is given with an opportunity to improve the task is referred to as formative (Dunn, Morgan,
OReilly, & Parry, 2004, p. 18). Formative assessment does not carry a grade(Irons, 2008, p. 7) and concerns itself with
improvements to outputs that are developmental in nature. This seems to be pertinent at the postgraduate level, as writing
multiple drafts of chapters and engaging in formative assessment is a norm. Formative assessment is said to incorporate three
main components: diagnosing student difculties, measuring improvement over time, and, nally, providing information to
improve. Contrary to the passive nature of summative assessment, formative assessment is active in the sense that it triggers
and provides a sense of direction to achieve learning goals.
It has been suggested that when students receive summative assessment, theyhardly attend to formative feedback(Butler,
1988). However, Mastersand PhD candidates cannot ignore their examinersformative assessment/feedback. It is
a requirement of thesis examination procedures that the candidate attends to suggestions and comments (that are usually
moderated and prepared by the convener of the examination), and that revisions have to be made to the satisfaction of the
convenor or, possibly, an (internal) examiner.
2.2. Feedback
Arst conceptual understanding of feedback is that it closes a gap between current and desired performance (Parr &
Timperley, 2010). Feedback has been conceptualised as information about the gap between the actual level and the refer-
ence level of a system parameter which is used to alter thegap in some way(Ramaprasad, 1983, p. 4). Sadler (1989) adds that
what is essential in feedback is that it has to be active, in the sense that once the gap is identied, it has to be closed.
Traditionally feedback is conceptualised as information provided by an agent (e.g. teacher, peer, book, parent, self, experi-
ence) regarding aspects of ones performance or understanding(Hattie & Timperley, 2007, p. 81). Feedback is given to ensure
that learning goals are met. Sadler (1989) supports this view by noting that feedback is information given to the student
about the quality of performance(p. 142). In a model of feedback proposed by Hattie and Timperley (2007), effective
V. Kumar, E. Stracke / Journal of English for Academic Purposes 10 (2011) 211222212
feedback involves closing a gap in knowledge. Hattie and Timperley use the term feed upto refer to notions of where the
learner is going, feedbackto the notion of what progress is being made to achieve a goal, and nally feed forwardto refer to
the notion of where to next. In terms of postgraduate supervision, the supervisors usually play an active role in all three types
of feedback by providing input to ensure that specic learning goals are met, but the resulting thesis is subject to external
validation by the examiners. In an examination scenario, the examiners may have differences in the conceptualisations of
these goals that may differ from that of the supervisors. Kiley (2009) summarises supervisorschallenges when nominating
examiners in two categories: professional/academic issues and personality issues. An example of the rst category is
a different disciplinary perspective, in particular with cross-disciplinary and multidisciplinary dissertations. As for the second
category, supervisors and examiners might, for instance, have different understandings of intellectual courtesy and
A second conceptual understanding of feedback is that it provides developmental experiences and encourages self-
regulated learning (for elaborations, see Stracke & Kumar, 2010). In the postgraduate education process, feedback is given
during supervision, for instance after the candidate has completed a chapter. Feedback provides opportunities for post-
graduate students to practise skills and to consolidate the journey from a zone of current development to a zone of proximal
development (Vygotsky, 1978), that is, to move from being a novice to becoming an expert in a specialised eld of study, and
to achieve the tenacities of self-regulated learning. The main aim of feedback is to reduce discrepancies between current
understandings, performance and a goal(Hattie & Timperley, 2007, p. 86). In feedback, the focus is on specic aspects that
need improvement both supervisors and examiners may provide such feedback, with the examiners providing the last stage
of scaffolding the candidates learning experience.
Finally, feedback is often referred to as a form of communication. In a study by Kumar and Stracke (2007) on the doctoral
supervision process, it was reported that through feedback supervisors engage with supervisees. The term engagement refers
to strategies that writers use to to recognise the presence of their readers(Hyland, 2005, p. 365). This engagement could be
in the form of interactions that could range from referential utterances which provide information, directive utterances which
try to get the hearer (writer) to do something or, nally, expressive utterances which express the speakers (supervisors)
feelings (Kumar & Stracke, 2007). On a similar note, Higgins, Hartley, and Skelton (2001) highlighted the dialogical role of
feedback. Their argument was based on the notion that feedback should lead to [d]iscussion, clarication and negotiation
(p. 274).
The different conceptualisations of feedback that have emerged from Higher Education research focus on teacher/lecturer
as well supervisor feedback. However, examiners on theses also provide developmental experiences (Joyner, 2003) to the
supervisee. This is based on the theoretical underpinning that examiners view a thesis as work-in-progress (Bourke et al.,
2004; see also Stracke & Kumar, 2010). Therefore, examinersreports on research theses usually contain both assessment
as well as feedback.
2.3. Formative assessment and feedback
The distinction between summative and formative assessment may be clear cut: summative assessment makes a judge-
ment call on learning outcomes while formative assessment provides a sense of direction to achieve unattained goals.
Formative assessment seems to echo what feedback does, that is, closing a gap. If the information given to close a gap has an
immediate impact for learning, it can be considered as formative assessment or feedback. On the contrary, if information
given does not have an immediate impact on learning, it is summative assessment. In other words, while educators and
researchers may use different terms to refer to the process of closing a gap between achieved and desired goals, such as
formative assessment or assessment for learning (e.g. Parr & Timperley, 2010), formative feedback (e.g. Kiley, 2009), or
(bringing the two terms together) assessment feedback (e.g. Higgins et al., 2001), the terms refer to a common central goal
that is, a trajectory move towards attainment of a learning goal. In this paper, we use the term feedback to refer to this
trajectory in an effort to distinguish it clearly from summative assessment.
3. Methodology
Our small-scale study of six examiner reports aimed at identifying the nature of examinersreports on research theses, and
whether the reports provided primarily (summative) assessment or feedback.
3.1. Data sources and background information
Data for this study constituted six examiner reports that belonged totwo sets, one set of three reports on a Masters thesis
in Applied Linguistics (from University MAL, Malaysia), and one set of three reports on a PhD thesis in Applied Linguistics
(from University NZ, New Zealand). We chose to include both a PhD and a Masters thesis to gain an initial insight into the
under-researched processes involved in the assessment of postgraduate thesis examination practice (Tinkler & Jackson, 2000,
cited in Mullins & Kiley, 2002, p. 369). Both theses were assessed as a pass with major revisions. In each case, two examiners
had asked for major revisions, whereas one examiner recommended minor revisions.
The reports from Malaysia were written by two internal examiners (from the department) and one external examiner
(from outside the university). In the Malaysian postgraduate education system, candidates have to undertake 16 credits of
V. Kumar, E. Stracke / Journal of English for Academic Purposes 10 (2011) 211222 213
course work and spend two to three semesters to work on a thesis. Two academic staff supervised the thesis. Examiners are
given a set of guidelines to prepare the report (Appendix A). The candidate also has to undergo a compulsory viva as part of
the examination process.
For University NZ in New Zealand, the three examiner reports were for a PhD and the reports were written by two external
examiners (one international examiner, and one within New Zealand) and one internal examiner (same department). It needs
to be pointed out here that, in New Zealand, the thesis is the only compulsory item of examination for a PhD. A team of three
academic staff supervised the PhD. An oral examination is not compulsory but can be requested by examiners. Examiners are
given a set of guidelines to prepare their reports (Appendix B).
3.2. Data management and analysis
All examiner reports were available in electronic form. These reports were tabulated to enable coding of each sentence,our
unit of analysis. The descriptive analyses focused on the overall nature of each report, whether the assessor provided mainly
assessment or feedback. Our guiding questions were: Does the examiner provide clear judgement about whether the
candidate has achieved the learning outcomes to be awarded the degree? In these cases, we coded the comment as focussing
on assessment (A). With regard to feedback: Does the examiner identify goals that the candidate has yet to achieve? Most
importantly, does the examiner provide clues for the candidate to close the gap between achieved and desired goals to
become a member of the scholarly community of practice for which the degree will be awarded? Such instances were coded
as comments with a focus on feedback (F).
Both researchers read the six reports and identied and categorised all sentences into feedback or assessment based on
the theoretical underpinnings discussed above. At the outset of the research, both researchers drew up a list of the possible
coding categories, assessment(coded as A), feedback(coded as F), both(coded as B) or dont know(D) based on the data
collected. Once the researchers had completed the categorisation individually, they exchanged coding categories and dis-
cussed differences until there was consensus. At times, there was discussion about whether a comment was mainly
assessment or feedback, and sometimes comments showed overlaps. Such instances were double coded, as comments can fall
into both categories. For instance, in a comment like However, no reference has been made to alternative perspectives which
could illuminate the development of inner mental processes which Vygotsky referred to as cognition in ight’”, the rst half of
the sentence contains a clear judgement, i.e., alternative perspectives are lacking according to this examiner. However, the
subordinate clauses contain clues to the candidate about how to integrate such a perspective. Hence, this comment is an
example of double coding as it contains both assessment and feedback (B).
4. Findings and discussion
The rst part of this section presents and discusses the results of the analysis of the examinersreports. It describes the
overall nature of the set of three reports on a Masters thesis (from University MAL, Malaysia) (4.1) and, subsequently, of the
set of three reports on a PhD thesis (from University NZ, New Zealand) (4.2). In the second part of this section, we discuss the
dual task of examiners - assessment and feedback - and look at the connection or potential disjuncture between these two
essential components (4.3). It should be recalled that the focus of our descriptive analysis was to nd out whether examiners
provide mainly assessment or feedback.
4.1. University MAL, Malaysia
As has already been stated, the Masters level examination at University MAL included three examiner reports. The
examiners followed the guidelines quite closely by addressing all twelve points (Appendix A).
Examiner 1 at University MAL (E1-MAL) recommended accept with major revisionsbased on his summative judgement
that the thesis needed depth in the form of conceptual and theoretical discussions. Other summative assessments include
judgements about the formulation of the research questions: Further, the research questions are presented as two research
propositionsat the end of this section on study purpose(A).
Even though the examiner started the report with a negative tone by highlighting weaknesses in the choice of the title and
moving on with critical comments on what should be included in an abstract, he provided a good deal of feedback to enable
the candidate to move on with the task of meeting the required expected standard.
His critical assessments were always well substantiated with evidence from the thesis. When providing summative
assessment, E1-MAL also made suggestions to enable the candidate to close the gap. For example, when E1 found that the
scope of the study was inadequate, he suggested that the candidate should provide a richer and thickerexplanation of [.]
In another instance, even though he summatively concluded that the literature review has been fairly comprehensive and
critical(A), he took the initiative to suggest new references to assist the candidate to revise. For example, when he made an
assessment on the quality of the literature review, he suggested a current reference so that the candidate can locate her
literature review (and her study) in clearer perspective amid a comprehensive array of issues(F).
The report by E1-MAL contains both assessment and feedback. What seems unique about this report is that even though
the examiner was critical at the beginning, he provided useful comment in the form of feed forwardto enable the candidate
V. Kumar, E. Stracke / Journal of English for Academic Purposes 10 (2011) 211222214
to revise. Besides this, the examiner also played a collegial role by providing guidance to the candidate to ensure that she was
able to close the perceived gap between her actual and desired performance. Assessment that required revision was always
supported with clear and well-directed feedback. This seems to give aclear indication that E1-MAL viewed the thesis as work-
in-progress and thus focused on providing clear directions if certain criteria for the award of the degree were not met.
Examiner 2 at University MAL (E2-MAL) also recommended that the thesis be accepted with major modications. E2
report, however, started off with quite negative comments (A) such as grammatically incorrect,inaccurate,does not
mention,does not explain,lack of,too brief, which indicated that the candidate had not attained certain learning goals.
These types of comments were evaluative in nature. There were also instances of directing the candidate with sentences
starting with The Candidate needs to [.](F)and The Candidate should not [.](F), as well as It is the supervisorsduty to
check that [.](F). It should be noted that directives seldom allow for any forms of engagement. E2-MALs report was eight
pages in length, and the majority of the comments were assessment in nature. Unlike E1-MAL, who provided feed forward
after every assessment, E2-MAL often stopped at the evaluative judgement stage. An example of this is the following: There is
lack of recent literature related to [.] in the dissertation. (A) The studies cited are more than 10 years old(A). The candidate was
not given any further guidelines or a sense of direction on how this apparent gap could be closed.
However, E2-MAL provided well-directed feedback on surface features to guide the candidate to revise. For example,
comments such as Figure 4 should bear a specic heading(F) or The citation for any quotation should include a page number
(F) gave a clear indication to the candidate about what needed to be done.
In sum, E2-MALs report was more of an assessment where the examiner strongly embraced her role as gatekeeper.
The assessment focused more on what the candidate had not achieved than on what had been achieved. The report did not
engage the candidate to make new discoveries or provide indications on how she could close the perceived gap and move
forward from her zone of current development to the zone of proximal development needed for the Masters degree she was
E3-MALs report consisted mostly of assessment. Unlike E2-MAL, this examiner highlighted what had been achieved. This
is understandable as she recommended accept with minor changes. Her assessments were mostly positive and included
praise, such as [t]his timely work on .is thus of great relevance and value [.](A) or I am especially appreciative of the way in
which [the candidate] has managed to keep her writing concise yet comprehensive(A). The positive assessments highlighted the
candidates achievements. For example: The research issues that have been selected are current and very worthy of investi-
gation(A) or The main strength of the study lies in its scope and relevance(A). There was also negative assessment. However,
it was less directive, and often followed byan invitation to the candidate to consider her stand, for instance I think that [.]is
more a research than a teaching tool(F). Feedback such as this enables a candidate to self-reect and to make new discoveries
this is a tenet of self-regulated learning. It should also be noted that E3-MAL was collegial and expressive when making
requests to the candidate. A phrase such as It would have also been good if the candidate [.](F) or It would also have the
added benetof[.](F) seems to indicate the dialogical nature of this report. Overall, E3-MALs report provided positive
assessment as well as feedback that would have encouraged the candidate to undertake the recommended revisions.
4.2. University NZ, New Zealand
The PhD level examination at University NZ included three examiner reports. The three examiners followed the guidelines
(Appendix B) in varying degrees.
Examiner 1 at University NZ (E1-NZ) recommended accept with minor revisionsfor the PhD thesis. Hence, it is not
surprising that the assessment given is mainly very positive and encouraging. The report primarily offers summative
assessment. Positive comments dominate the report, such as:
The research methods, analyses, and interpretations of them are appropriate to the research questions posed, which in
turn are of signicance not only for educational practices but also for theories of literacy development and writing
The examiner clearly addresses the questions given by the University and makes clear judgements when praising the
candidate for his distinctive contribution,for offering intensive and thoroughanalyses, for writing clearlyand presenting
E1-NZs feedback can be interpreted as feed forward, as he suggests avenues to the candidate of where to next. For
instance, this examiner encourages the candidate to publish his research when he says that he expects an article-length
manuscript about the thesis [that] would be welcomed at such international journals as [.](F). He also points toward
future studies to be conducted following from the present research(F) and provides the candidate with a few suggestions about
how he could achieve this, for instance by expanding the task types used with the research participants.
To sum up, this report provides clear judgement (summative assessment) and also some clear feedback to the candidate.
The feedback is more precisely feed forwardand shows the candidate clear avenues for developing the research further, now
that he is at the threshold of becoming a full member of the community of practice of academics. It is noteworthy that the
overall positive and encouraging tone of this report can be traced back to the frequent use of comments that show the
examiners thoughts, feelings and opinions. Comments like I expector I was impressedpave the way for a dialogue, for
some interaction between the assessor and the candidate, which is an important part of the learning experience.
V. Kumar, E. Stracke / Journal of English for Academic Purposes 10 (2011) 211222 215
Examiner 2 at University NZ (E2-NZ) recommended accept with major revisions. He offers clear summative assessment as
well as feedback by offering many suggestions for the candidate to address the concerns that led to his judgement. His
assessment includes both positive and negative summative assessment, as these two examples illustrate:
The thesis clearly indicates that the candidate is well able to select, present and interpret the primary data that has
been collected(A).
There is a lack of coherence between the primary data that illuminate the mental processes of student writers and
the secondary data (collected apparently exclusively by email interviews) upon which the argument of the thesis is
While this examiners comments also fall mainly under summative assessment, he does offer clear guidance to the
candidate to address the concerns. His feedback is often directive and includes many suggestions, such as:
It is suggested that a stronger theoretical discussion should be included in the nal chapters of the thesis(F).
These secondary sources should be more fully explained and referenced, and relevant contextual information from
these (and other?) sources should be fully considered in Chapter 1, to be drawn upon in the discussion in the nal two
The candidate receives clear guidance about how to close the gap between what the examiner expects and the current
draft of the thesis. Use of the passive voice and modal conditionals allow for directness but focus on the written product, and
not on the candidate-writer. This allows the candidate to look at his writing in an objective way and start the revisions, as
requested by this examiner.
Like E2-NZ, Examiner 3 at University NZ (E3-NZ) also recommended accept with major revisions. His report offers many
instances of summative assessment and some feedback. In this report the candidate nds most of the (summative) assess-
ment in the rst half of the report. First, statements clearly address the university guidelines and answer the questions
provided (Appendix A). However, the examiner makes comments that go beyond the questions and expresses his concerns
with the thesis as it stands, for instance:
The positive comments made in [A] notwithstanding, the thesis falls short of dealing with the topic in terms of depth
and scope to fully meet the requirements of the PhD degree(A).
In this respect, the thesis does not seem to have much to contribute to the eld(A).
The candidate is confronted with a long list of such statements that fall under the negative summative assessment
category, before coming to the part where this examiner suggests how to improve the critical aspects. The examiner offers
many useful suggestions about how to go about the recommended revisions:
Granted that this is a qualitative study [.], some statistical data on the studentsrevision strategies will be very useful
and in fact necessary(F).
This is not just for a better understanding of each revision strategy, but such statistical data could also be subjected to
further in-depth analysis(F).
Often, this examiner offers feedback that opens up a dialogue with the candidate, for example when he asks a question, as
if the candidate was with him, face-to-face, such as In Table l, didnt Melinder also generate ideas?(F) or Quality by whose
Similarly, comments that reveal more of ones own opinion show the power of such feedback (Kumar & Stracke, 2007). It
can lead to a dialogue between the examiner and the candidate, despite the distance between the two interlocutors. In the
following, these expressive comments invite the candidate to look at things from a different perspective, namely the
From my personal experience, I can say that sometimes [I] do planning in the back of my mind when doing something
unrelated (e.g. taking a shower, cooking a meal, etc.)(F).
Passim: What is somewhat puzzling [.](F).
The examiners use of exclamation marks on a couple of occasions also underlines his involvement with the text he is
reading and his reaction to it:
Obviously it did, because they all received very good grades!(F)
The students have certainly produced quality texts by the teachers expectations, but not by the examiners!(F)
To conclude this rst part of this section, it is noteworthy to point out that, despite the variety in this small sample of
examination reports, all examiners provide both (summative) assessment and feedback, albeit with different weightings and
in different ways. As for the proportion of assessment and feedback in these six reports, we can observe some tendencies.
Despite the fact that all examiners provide both assessment and feedback, there are notable differences. E2-MAL provides the
least feedback. E1-MAL and E2-NZ provide assessment and feedback in a more balanced way than E3-MAL, E1-NZ and E3-NZ,
who tend to offer more assessment. Regardless of their nal recommendation, these examinersreports also show variety
with regard to the level of directness, the use of the expressive function, and/or the use of feedback as feed forward. Future
research, with a larger sample, might look at these factors in more depth.
V. Kumar, E. Stracke / Journal of English for Academic Purposes 10 (2011) 211222216
In the following, we focus on this dual role of the examiners as providers of assessment and feedback and look at the
connection or potential disjuncture between these two essential components of examinersreports.
4.3. The dual task of examiners: assessment and feedback in postgraduate thesis examination
Examination at the postgraduate level is very different from any other circumstance. In a public examination or a nal
assessment at the end of a semester, summative assessments are the norm. Often the candidates do not have anyopportunity
to see their work after the assessment, or to make amendments to improve. However, at the postgraduate level, the
examination is entirely different in the sense that a piece of work (thesis) is written under supervision and then sent out for
external assessment. Even though recommendations regarding the assessment of the thesis are made, the thesis can still be
re-worked to ensure that the learning goals are met to the satisfaction of the examiners. In other words, examiners
assessments of a thesis are not nal, and the candidate is given the opportunity to close any gap identied by the examiners.
For this, the candidate relies on the feedback offered by the examiners.
This particular examination situation is clearly evident from the data presented. All six examiners seem to play the role of
gatekeepers. All of them make assessments, some positive, some negative in tone. The positive assessments indicate that the
examiners are satised that the candidates have met the expected goals and attained the learning objectives stipulated for the
degree that they are seeking. The negative assessments are the ones that can create a disjuncture when examiners perceive
their roles purely as gatekeepers. While most of the examiners provide feedback to ensure that the potential disjuncture
between assessment and feedback does not occur, sometimes examiners stop with the provision of their (summative)
assessment. Stopping at this point of the assessment does not seem to provide any impetus for the candidates to progress, to
close the perceived gap in their performance, and to attain the level that will allow them to become members of the scholarly
community to which they wish to belong.
What seems to be the underlying concern potentially leading to disjuncture is the role of feedback in the assessment of
a thesis. Our data appear to indicate both a linear and recursive process of assessment and feedback. The process is linear
when the entire thesis, or some aspects of it, have met the standards as stipulated by the institution or decided by the
examiner. However, it becomes recursive if the standard has not been met. When a thesis is sent out for assessment,
examiners have to decide whether or not the thesis has met the standards for the award of the degree. If it has not, the
examiners are required to provide guidance to the candidate (as stipulated in the guide for examiners). This feedback is of
crucial importance for the student to be able to close the gap between current and desired performance (Parr & Timperley,
2010). Once feedback is provided, the candidate is expected to revise to the expected standards. A subsequent assessment is
made to ensure that the standard has been met. If it has, the thesis is accepted and the candidate has rightfully become
a member of the community of practice.
5. Conclusion and implications
With this study we aimed to gain insights into the connection and/or potential disjunction between assessment and
feedback in examinersreports through empirical study of examinersreports. The main aim of this study was to identify the
nature of examinersreports by nding out whether they primarily provide assessment or feedback, and to explore the link
between, or possibly the disjuncture between, assessment and feedback.
We acknowledge that the small sample size does not allow for any generalisation. We undertook this analysis to gain an
initial data-grounded insight into questions around assessment and feedback in thesis examination that we, in our role as
academics in Higher Education who supervise and examine postgraduate theses, had been reecting on in a more theoretical
way. A larger sample of postgraduate theses reports at both Masters and PhD level, from various disciplines and from other
countries, would allow for more insights and more robust ndings. Most importantly, this small-scale study does not show
any distinct difference in the way the examiners write their examination reports at Masters or PhD level. Further research
could examine differences between Masters and PhD theses at these two levels and (potentially) different examiners
feedback and develop recommendations for different approaches by supervisors, including their feedback.
Limitations notwithstanding, our study suggests the crucial role of feedback in postgraduate thesis examination practice.
Without feedback, there is little impetus for the candidate to progress, to close the perceived gap, and to attain the level
required to become a member of a scholarly community. Summative assessment alone cannot achieve this goal.
Given the crucial role of feedback in the assessment process and for the successful completion of the examination process
candidates will only be awarded the degree if they succeed in closing the gap between actual and desired performance,
between the submitted draft of the thesis and the nal one there appears to be a need to emphasise the role of feedback in
the postgraduate thesis examination process. Such an emphasis has implications for all parties involved - the examiners, the
university, supervisors and candidates alike.
Even though the majority of comments made by these examiners were essentially summative assessment, one can also see
that all examiners, regardless of their recommendation, looked at the thesis as work-in-progress and provided feedback.
Whereas E1-NZ considered the thesis as a useful basis for a journal article and tailored his feedback towards this goal, E2-NZ
and E3-NZ, who saw shortcomings in the thesis, clearly made suggestions tothe candidate as to how he could overcome these
and close the gap between the current draft and the expected revised version. Similarly, E1-MAL, E2-MAL and E3-MAL
provided differing degrees of guidance to the candidate.
V. Kumar, E. Stracke / Journal of English for Academic Purposes 10 (2011) 211222 217
However, the instances of disjuncture that we found in the data strongly suggest that examiner guidelines could spell out
more clearly that examiners need to offer such feedback, at least with regard to all aspects of the thesis that require changes.
Such an emphasis might require some policy changes at institutional level. For example, University NZ (New Zealand) could
make the instructions that allow for feedback (The reports should also contain specic comments on those parts of the thesis that
the examiners believe to require correction or amendment) more prominent (through formatting devices, for example) or
otherwise emphasise the importance of such comments. Likewise, even though University MAL (Malaysia) highlights at the
very beginning that [t]he objective is to help students to effectively incorporate all recommended amendments based on these
comments, the majority of the document asks the examiner to determinewhether a certain standard has been met, thus
focussing strongly on the assessment aspect of the examination report.
Another implication of a stronger emphasis of feedback in thesis assessment is the need for examiner training that would
allow examiners to reect and embrace more willingly their dual role as assessor and feedback provider. This would also
allow for a better understanding of the educational role of supervisors (Stracke, 2010) and examiners. Current training
materials (Evans & Tregenza, 2002) could also more strongly include this important aspect of the examination process.
Finally, we would suggest that if an examination report included more dialogical type of feedback, i.e., feedback that
encourages reection, it could lead more effectively to the desired learning outcomes at the postgraduate level. As this study
clearly shows, summative assessment alone does not augur well for the attainment of learning goals. Possibly, an emphasis on
feedback in assessment will also lead to a greater understanding of the cycle of feedback and revision on the candidates side.
The authors gratefully acknowledge the generosity of the examiners and the Malaysian student who contributed to this
study. The authors wish to record their appreciation to the anonymous reviewers and to thank the Universiti Putra Malaysia
and the University of Canberra for supporting this research. The authors have prepared this paper equally.
Appendix A. Guidelines for the preparation of a Masters thesis examination report from University MAL (Malaysia)
Guidelines for the preparation of a thesis examination report
Please submit a DETAILED REPORT using the following guidelines when examining the thesis. The objective is to help
students to effectively incorporate all recommended amendments based on these comments.
The examiner is expected to keep the thesis material condential until it is made public by the student through publication
or by deposition in the library.
1. Thesis topic (Title)
Determine whether the title is grammatically correct, contains important and pertinent keywords found in the abstract,
and reects the actual research issues addressed in the study. If the title requires improvement, do suggest a suitable title.
Document [.] (Appendix 1) provides additional guidelines for determining a suitable title for a thesis.
2. Abstract
Determine whether the abstract accurately reects the study that was conducted. The abstract should contain a (i) brief
statement of the problem or objectives, (ii) concise description of the research method and design, (iii) summary of the major
ndings and (iv) brief conclusion.
3. Research problems and objectives
Determine whether the background to the pertinent research issues is well discussed, the research problems well dened,
and the hypotheses address the dene research problem. Determine whether the objectives are clearly stated and met by the
research methodology design used and ndings. Suggest improvement, if necessary.
4. Scope and relevence
Determine whether the scope of the study is appropriate for the degree it is intended. The level of appropriateness is
a relative concept, and therefore, needs to be addressed by considering the following factors:
a. Field of study (example: pure sciences and social sciences have different perception of scope)
b. Research issues in a particular eld; and
c. Practicability of the addressed research problems (example: the scope could be limited by nancial, time and other
d. Research objectives
V. Kumar, E. Stracke / Journal of English for Academic Purposes 10 (2011) 211222218
5. Literature review
Determine whether the literature review:
a. is relevant to the research issues
b. is comprehensive and takes into consideration past and current literature
c. is well reviewed, summarised, organised and consistent with the sequence of the research issues addressed in the study
d. is proportionate relative to the rest of the thesis
e. contains too much text book materials (it should be kept to a minimum)
6. Methodology/Materials and methods
Determine whether the:
a. collection, strengths and weaknesses of the data used in the study are clearly specied
b. research design (e.g. sample size, choice of methods etc.) is suitable and appropriate to meet or address the specied
objectives or research issues of the study
c. use or choice of methods is well dened and justied
d. statistical analysis or package used is appropriate
e. methods used are properly and adequately referenced
7. Analysis and intepretation of results
Determine whether the
a. results obtained are in agreement with the stated objectives of study
b. interpretation of the ndings is logical or acceptable within the context of the issues of interest
c. analysis of the data using the chosen methodology has been properly specied
d. ndings are discussed with appropriate references
8. Presentation
Determine whether:
a. the sequence of chapters, and sections in each chapter are able to facilitate the understanding of the research issues
b. tables, pictures and any other form of summarized information properly labelled, numbered, and placed in the appro-
priate sequence and section of the thesis
c. the same research data is presented in more than one form (e.g. both table and gure)
d. gures especially photographs are clearly reproduced
9. References/bibliography
a. the extensiveness of the bibliography/reference list
b. whether current references are included
c. whether any reference cited in the text is missing or wrongly cited, and
d. whether the format used is consistent throughout the list
10. Accomplishments and/or merits
Indicate whether:
a. the author has clearly identied and discussed the contributions of the ndings to the knowledge in the area, and the
applicability of the ndings in addressing the research problems in the study
b. the stated objectives are achieved
c. there are any other accomplishment that merit a mention
V. Kumar, E. Stracke / Journal of English for Academic Purposes 10 (2011) 211222 219
11. Demerits
Indicate whether:
a. the main weaknesses of the research and their impacts on the ndings are properly addressed by the author
b. there are any other demerits (example: contents, language, relevance, etc.)
12. Recommendation
Conclude the evaluation of the thesis by stating your professional opinion on the overall acceptability of the thesis (after
taking into account all the above considerations), whether it is worthy of the degree pursued or otherwise. The outcome of the
examination should be reported as one of the following:
(a) Accepted
(i) with distinction when all or most of the research ndings have either been published or accepted for publication in
citation-indexed journals, and requires minimal improvement in spelling, grammar and syntax only;
(ii) accepted with some corrections in spelling, grammar and syntax.
(b) Accepted with minor modications
A thesis is accepted with minor modications if it requires any of the following: reformatting of chapters, improvement in
declaration of research objectives or statements, insertion of missing references, amendmentof inaccurately cited references,
require minimum improvement in spelling, grammar, syntax, and presentation.
(c) Accepted with major modications
A thesis is accepted with major modications if it requires any of the following but not additional experimental work or data
collection: major revision of the literature, major improvement in the description of the methodology, statistical analysis of the
research data, re-presentation of written data in the form of gures or tables, and improvement in the discussion of results.
The examiner may recommend the candidate seek the assistance of an editorial service if errors in grammar and syntax are
(d) Re-submission of thesis
The thesis should be recommended for resubmission if it does not meet the scope of the degree for which it is intended,
the objectives of the research are not met or when there are obvious aws in the research design or methodology, and
therefore, requires additional experiments or data collection.
(e) Fail
AFailstatus is given if the thesis does not achieve the level of the degree for which it is intended.
Appendix 1. Guidelines for determining the title of thesis by the examination committee
When preparing the report for a thesis being examined, the examiner is required to determine whether the title of the
thesis is grammatically correct and reective of the study undertaken (as stated in the Guidelines for Preparation of Thesis
Examination Reportand the Final Examination Report Form). In addition to the two, the examiners, being also members of
the Examination Committee, should consider the following guidelines when deciding on the most appropriate title for the
1. Ensure that important keywords are found in both the title and abstract of the thesis.
2. For titles in Bahasa Melayu, terms used are actually those found in the Dewan Kamusor Istilah Bahasa Melayufor the
relevant elds of study.
3. Do not allow the use of abbreviations (e.g. AMN etc.) and/or acronyms (e.g. UNITAR) unless they are universallyaccepted in
the eld of study e.g. DNA, ESL, PCR. Use, instead, the full terminology.
4. Do not allow the use of a colon (:) or dash () e.g. Bacillus subtilis amylase: Purication and Characterisationor Bacillus
subtilis amylase-Purication and Characterisation. The title may be replaced with Purication and Characterisation of Badllus
subtilis amylase.
5. Ensure that when both the common and scientic names of an organism (where applicable) are mentioned, the common
name is stated rst followed by the scientic name (including variety if known) in parenthesis.
6. Where possible avoid, do not allow the title to begin with The..... e.g. Use Effect of..instead of The Effects of...
7. Do not allow the use of phrases such as A study of...., Studies on..
V. Kumar, E. Stracke / Journal of English for Academic Purposes 10 (2011) 211222220
Appendix B. Excerpt from The PHD examination processfrom University NZ (New Zealand)
Written Reports from Examiners
Each of the examiners is requested to furnish a written report on the thesis together with an assessment of its ve-point
a) Accept, or accept with minor editorial corrections
(the corrections required are minor and can be completed in a short period of time, normally not longer than a few weeks.
The Convener of Examiners will check that the corrections have been made satisfactorily)
b) Accept after amendments have been made to the satisfaction of the Convener of Examiners in consultation with the internal
(the amendments required can be completed within a few months, normally not longer than two or three months. The
amendments will be made to the satisfaction of the Convener of Examiners in consultation with the internal examiner)
c) Revise and resubmit for examination
(the thesis is not of the required PhD standard and requires substantial revision involving up to six months of work or
possibly a little longer. The revised thesis will be resubmitted formally to all three examiners for a repeat examination)
d) Reject and refer to the appropriate authority within the University for consideration of the award of another degree
(the thesis is not of the required PhD standard and there is no likelihood that revisions will bring it up to that standard.
However, the thesis may meet the standards required of an alternative degree, possibly a Masters)
e) Reject with no right of resubmission
(the thesis is not of the required PhD standard and there is no likelihood that revisions will bring it up to that standard, nor
does the thesis meet the standards required of an alternative degree).
The examiners are asked to comment on the thesis with reference to the description of the degree (see Introduction
Examiners are requested to respond to the following questions:
Does the thesis comprise a coherent investigation of the chosen topic?
Does the thesis deal with a topic of sufcient range and depth to meet the requirements of the degree?
Does the thesis make an original contribution to knowledge in its eld and contain material suitable for publication in an
appropriate academic journal?
Does the thesis meet internationally recognised standards for the conduct and presentation of research in the eld?
Does the thesis demonstrate both a thorough knowledge of the literature relevant to its subject and general eld and the
candidates ability to exercise critical and analytical judgement of that literature?
Does the thesis display mastery of appropriate methodology and/or theoretical material?
The reports should also contain specic comments on those parts of the thesis that the examiners believe to require
correction or amendment.
The examiners form their own independent assessments of the thesis without discussion amongst themselves or with the
candidate. Should discussion be necessary amongst the examiners, it will be co-ordinated by the Convener.
The examiners send the reports directly to the Doctoral Ofce. From there, they are forwarded to the Convener of
Examiners. The examiners normally retain their copies of the thesis.
Bloom, B. S., Hastings, J. T., & Madaus, G. F. (Eds.). (1971). Handbook on the formative and summative evaluation of student learning. New York: McGraw Hill.
Bourke, S., Hattie, J., & Anderson, J. (2004). Predicting examiner recommendations on Ph.D theses. International Journal of Educational Research, 41,178194.
Butler, R. (1988). Enhancing and undermining intrinsic motivation: the effects of task-involving and ego-involving evaluation on interest and performance.
British Journal of Educational Psychology, 58(1), 114.
Dunn, L., Morgan, C., OReilly, M., & Parry, S. (2004). The student assessment handbook: New directions in traditional and online assessment. New York:
Evans, T. (concept), assisted by Treganza, K. (2002). Discussions on examining theses [CD]. Deakin University: Learning Services.
V. Kumar, E. Stracke / Journal of English for Academic Purposes 10 (2011) 211222 221
Gibbs, G., & Simpson, C. (200405). Conditions under which assessment supports studentslearning. Learning and Teaching in Higher Education, 1,329,
Retrieved on 12 June 2010 from.
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81112 .
Higgins, R., Hartley, P., & Skelton, A. (2001). Getting the message across: the problem of communicating assessment feedback. Teaching in Higher Education,
6(2), 269274.
Hyland, K. (2005). Representing readers in writing: student and expert practices. Linguistics and Education, 16, 363377.
Irons, A. (2008). Enhancing learning through formative assessment and feedback. London and New York: Routledge.
Joyner, R. W. (2003). The selection of external examiners for research degrees. Quality Assurance in Education, 11(2), 123127.
Kiley, M. (2009). You dont want a smart Alec: Selecting examiners to assess doctoral dissertations. Studies in Higher Education, 34(8), 889903.
Kumar, V., & Stracke, E. (2007). An analysis of written feedback on a PhD thesis. Teaching in Higher Education, 12(4), 461470.
Mullins, G., & Kiley, M. (2002). `Its a PhD, not a Nobel Prize: how experienced examiners assess research theses. Studies in Higher Education, 27(4), 369386.
Parr, J. M., & Timperley, H. S. (2010). Feedback to writing, assessment for teaching and learning and student progress. Assessing Writing, 15(2), 6885.
Ramaprasad, A. (1983). On the denition of feedback. Behavioural Science, 28(1), 413.
Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18,119144.
Snyder, B. R. (1971). The hidden curriculum. New York: Alfred A. Knopf.
Stracke, E. (2010). Undertaking the journey together: peer learning for a successful and enjoyable PhD experience. Journal of University Teaching & Learning
Practice, 7(1). Available at:
Stracke, E., & Kumar, V. (2010). Feedback and self-regulated learning: insights from supervisorsand PhD examinersreports. Reective Practice, 11(1),1932.
Tinkler, P., & Jackson, C. (2000). Examining the doctorate: institutional policy and the PhD examination process in Britain. Studies in Higher Education, 25(2),
Vygotsky, L. (1978). In Cole, M., John-Steiner, V., Scriber, S., & Souberman, E. (Eds.), Mind in society: The development of higher psychological processes.
Cambridge, MA: Harvard University Press.
V. Kumar, E. Stracke / Journal of English for Academic Purposes 10 (2011) 211222222
... They then make a recommendation, as to whether the Ph.D. degree can be awarded without changes or subject to minor or major modifications. This recommendation is presented in a written examiner's report that includes comments about the general quality of the work, and where modifications are required, feedback comments that indicate how the doctoral candidate can improve the work (Kumar & Stracke, 2011). This combination of a summative judgement and formative feedback makes the PhD examination a distinctive assessment process since it is "part grade and part gauge of what still needs to be done" (Holbrook, Bourke, Fairbairn & Lovat, 2014, p. 983). ...
... This feedback is considered to be formative since it provides the doctoral candidate with information that allows them to improve their work by reducing the gap between their current performance and a specific goal (Hattie & Timperley, 2007), allowing them "to attain the level required to become a member of a scholarly community" (Kumar & Stracke, 2011, p. 217). By providing formative feedback, examiners at doctoral level, also take on the role of mentors and teachers (Kumar & Stracke, 2011). ...
... We were also interested in analysing, the type of formative feedback that examiners were providing for the doctoral candidates. As reported in a number of studies regarding feedback at doctoral level (Holbrooke, Bourke, Love & Dally, 2004;Kumar & Stracke, 2011), the feedback is provided in various formats. The comments can include: editorial corrections of language mistakes; clear direct prescriptive instructions on what the doctoral candidates need to do to improve their work; and dialogic comments which usually are in the form of questions that are meant to help the doctoral candidates reflect on their work and explore alternative scenarios or explanations. ...
Full-text available
Achieving a PhD degree is viewed by academic institutions as a major landmark of success and achievement. It gives recognition to researchers and provides entry into academia. Considering its significance, a PhD degree is not awarded lightly and doctoral candidates undergo a rigorous examination process that involves the evaluation of a written thesis and the viva voce defence of this thesis. This study seeks to gain a better understanding of the way in which examiners go about assessing doctoral work with the aim of providing more transparent and clear guidelines for examiners. Data for the study included 50 written examiner reports for twelve doctoral candidates who submitted their thesis to the Faculty of Arts at the University of Malta, for the years 2017-2018. The findings suggest that examiners in their reports include summative comments about the quality of the work. They are impressed by work that makes a contribution to knowledge, is critical and analytical and is not marred by typographical or grammatical errors. At the same time, examiners provide extensive feedback to help students improve their work. This suggests a shift in the doctoral examination process from the traditional role of summative gate-keeping to a more formative learning experience.
... Thesis examiners' reports, undeniably, play a crucial role in postgraduate education and research as they constitute an important educational tool in maintaining standards, improving quality, and building research capacity (Bourke, Hattie, & Anderson, 2004;Kiley, 2009;Kumar & Stracke, 2011). They provide information on not only research but also on the student, supervisor, and the university (Starfield, Paltridge, McMurtrie, Holbrook, Lovat, Kiley, & Fairbairn, 2017). ...
... Drawing on similar studies such as Holbrook et al. (2007) and Kumar and Stracke (2011), we used a content analysis methodology. The choice of the qualitative content analysis enables attention to be paid to the naturally occurring data (here, the examiners' comments), which are not medicated through any means. ...
... In the second part of our discussion on form, we report and discuss the findings on sequencing of the presentation of summative and formative comments (see earlier discussion on theses). Examiners usually make judgements or evaluative comments (whether positive, negative, neutral or suggestion) regarding the theses they assess and give their assessment feedback on how students can make up for the weaknesses identified-be it examiners of MPhil or PhD theses (Golding, Sharmini, & Lazarovitch, 2014;Holbrook et al., 2004;Kumar & Stracke, 2011;Prieto, Holbrook & Bourke, 2016). What is, however, often unclear in the literature is the sequence in which such evaluative comments are usually presented by examiners, which is very important taking into the consideration the effects that examiners' comments are likely to have on recipients (Kumar & Stracke, 2011). ...
... [2,3] A number of criteria have been demonstrated in several reports; however, none provides a comprehensive and consolidated picture of its overall aspects. [4,5] The key issues such as the topic of PhD, academic status and research facility of the PhD supervisors, institutional policies on constitution of PhD committee and selection of examiners to establish a quality assessment system, and so on, contribute to producing good quality of PhD theses. [4][5][6][7] There has not been any established guideline to evaluate the legitimate contribution of each criterion involved in examination of the PhD theses, which ultimately ensures a PhD of good quality, meaning the purpose of the doctoral research is fulfilled. ...
... [4,5] The key issues such as the topic of PhD, academic status and research facility of the PhD supervisors, institutional policies on constitution of PhD committee and selection of examiners to establish a quality assessment system, and so on, contribute to producing good quality of PhD theses. [4][5][6][7] There has not been any established guideline to evaluate the legitimate contribution of each criterion involved in examination of the PhD theses, which ultimately ensures a PhD of good quality, meaning the purpose of the doctoral research is fulfilled. The length of a PhD thesis varies from subject to subject. ...
Full-text available
The number of PhD theses approved for the doctoral degree is on a steady rise in different disciplines across the globe. However, the quality may not meet the requirement of the academia and industries, especially in biomedical research and development (R&D), and that is not only contributing to the escalated problem of unemployment of such population having higher degrees but also diminishing the quality of education and research. Therefore, a harmonized approach shall be adopted by the universities for an examination of the quality of PhD theses and approval of the PhD degree. Quality of PhD theses of all universities, including central, state, private, and deemed to be (private or public), which are governed by the authorized regulators, such as University Grants Commission (UGC) in India, is important for the PhD students as well as mentors and granting institutions. On the basis of the current state of the art, concepts have been conceived to capture the merit of PhD degree or higher education. Evaluation of the quality of PhD can be achieved through the articulated examination system of the three key factors such as the merit of the PhD candidate, facility and position of the PhD mentor or guide, and the examination system of the granting universities. The discussion of this article has been concentrated in the quality of PhD in biomedical sciences to assess a comprehensive picture of PhD or higher education, which would not only be helpful for judging the merit of PhD but also redefine and implement policies by the regulators of higher education for continual improvement.
... In [8], Vijay Kumar and Elke Stracke mentioned that there are two components in examiners' reports on theses. One is summative assessment, where a judgement is made about whether the thesis has minimum standards required for the award of the degree. ...
Full-text available
Abstract: Research or PhD research is a systematic study and experimentation that can generate new knowledge or findings which can directly or indirectly be useful for the developing of society. Research may also involve developing new theories, or application of theories that are beneficial to many. Quality research involves performing the steps in research including problem definition, review of related research, design, experimentation, testing and verification of results and making conclusions following scientific methods and systematic procedures. Quality in research is essential as the findings may be used by many researchers to extend the research work resulting in new findings. In this paper some practical considerations in all the phases of research such as providing the basis for understanding the proposed work, highlighting the need of proposed research in view of existing related work, design of appropriate methodology for solving the proposed problem, experimentation with relevant data, systematic testing, verification and interpretation of results and making correct conclusions are presented. It is concluded that research performed with correct and required focus on each phase and following systematic procedures can lead to quality research. Ketwords - PhD, Quality research, Originality, Practical considerations
... The most recent large-scale study, Twumasi (2020), drawing on the assumption of evaluation as consisting of a triad (the evaluator, evaluation, and evaluate) makes far reaching and insightful comments on the comments across the various evaluative criteria identified by the UCC School of Graduate Studies. In addition to the negative and positive evaluative comments on the MPhil theses realized through epistemic adjectives, nouns, verbs, and adverbs, mitigating devices featured prominently as a way of demonstrating that the thesis was not only a work in progress but both a pedagogical and research document (Kumar & Stracke, 2011). ...
... Academic interest in the PhD program and scholarly narratives on doctoral research education have been predominantly recounted from the point of view of supervisors or supervision (Lee, 2008;Machin et al., 2019) and examiners or PhD assessment (Holbrook et al., 2007;Kumar & Stracke, 2011;Sharmini et al., 2015). Other studies have concentrated on the technical aspects and pragmatic areas of the doctorate such as choosing a supervisor, deciding on and narrowing down the topic, (alternative) methodological approaches to research, writing up the thesis and the viva voce (Dunleavy, 2003;Phillips & Pugh, 2000). ...
Full-text available
In the last two decades, academic reflections on the PhD experience and studies on various aspects of doctoral research education have attracted scholarly attention in the higher education and advanced academic literacy literature. In this article, I adopt a reflective-narrative framework to recount my engagement with the doctoral program at The Hong Kong Polytechnic University. I discuss how I tackled specific aspects of the PhD and thesis: choosing a topic/title for my project, writing the abstract, undertaking the literature review, conducting the analysis and my experience with the writing process. I also highlight the challenges I faced, how I overcame them and the insights I gained from them, thereby illustrating how doctoral narratives can be empowering and provide inspiration to current and future doctoral students. The experiences I recount reveal the complex nature of the doctorate as both an individual and a collective endeavor. Implications of the paper for the scholarship on narrative inquiry, doctoral thesis writing and academic identity are discussed.
The present case study explores the assessment practice of two examiners from different academic backgrounds on an undergraduate thesis work in mathematics education. Reflection notes and feedback from two instances of a thesis‐writing process are interrogated using a framework based on a semiotics perspective to meaning‐making. It is shown that the examiners utilize aspects that are ‘immediate and non‐contested’ to successively make accessible that which is ‘withdrawn yet to be revealed.’ In this process, varied interventional approaches are observed to support desirable aspects of teacher professional knowledge including critical‐analytical disposition. The study also highlights knowledge integration for teaching, scientific disposition, collaboration, and knowledge transformation as some of the desirable aspects of teacher professional knowledge. The findings indicate that knowledge integration and transformation remain an issue of concern for pre‐service teachers. This raises the question of how desirable traits for a teacher‐as‐a researcher can be promoted within the context of thesis work.
Full-text available
The thesis examiner’s report is an evaluation of a thesis, which includes dialogic and evaluative elements. The purpose of the study was to investigate the roles that examiners adopt for themselves and the language use in examiners’ reports on MPhil theses submitted to the School of Graduate Studies, University of Cape Coast. The study purposively selected 100 theses examiners reports from four disciplines. The study revealed that examiners adopted eight different roles in the reports. Another key finding of the study was that evaluator role was most frequent, and the least frequently occurring role was Institutional role. Again, examiners employed imperatives, personal pronouns, and adjectives in their adopted roles. The findings of the study serve to create an awareness for explicit guidelines for both fresh and experienced examiners in the task of postgraduate thesis examination.
Achieving a PhD degree is viewed by academic institutions as a major landmark of success and achievement. It gives recognition to researchers and provides entry into academia. A PhD degree is not awarded lightly and doctoral candidates undergo a rigorous examination process. This study seeks to gain a better understanding of the way in which examiners go about assessing doctoral work. Data for the study included written examiner reports presented to the University of Malta from four different faculties: Arts, Education, Science and Medicine & Surgery for the years 2017-2018. The findings suggest that the doctoral examination is a social practice with examiners using a combination of explicit criteria outlined in the University of Malta regulations, and implicit criteria based on their own personal expectations. Using a combination of these criteria, examiners look for doctoral work that makes a contribution to knowledge, is critical and analytical and is not marred by typographical or grammatical errors. The study also highlights a number of disciplinary differences in the expectations of examiners in different faculties and the need for more formal professional development for examiners to ensure a shared interpretation of criteria within a community of assessors.
Full-text available
This paper offers an interim analysis of written feedback on a first draft of a PhD thesis. It first looks at two sources of data: in-text feedback and overall feedback. Looking at how language is used in its situational context, we then coded the feedback and developed a model for analysis based on three fundamental functions of speech: referential, directive and expressive. It was found that expressive feedback benefited the supervisee the most. The interaction between the supervisor and the supervisee played an important role for the induction of the supervisee into the academic community, and suggests a peer-to-peer model in PhD education. Finally, this paper suggests the possibility of developing a taxonomy of good feedback practices in postgraduate supervision practice in Higher Education.
Full-text available
This paper provides insights into the doctoral journey of a supervisee by analysing written feedback provided by supervisors and thesis examiners. As one aim of doctoral education is to train scholars to become independent researchers, that is highly self‐regulated learners, this study paves the way for an understanding of the link between written feedback and the self‐regulated learning process. Based on an analysis of speech functions, written feedback provided by two supervisors and three examiners were classified into three main categories: referential, directive and expressive. The results indicate the value of expressive feedback for the development of self‐regulated learning in doctoral supervision.
This book is based on the argument that detailed and developmental formative feedback is the single most useful thing teachers can do for students. It helps to clarify the expectations of higher education and assist all students to achieve their potential. This book promotes student learning through formative assessment and feedback, which: enables self-assessment and reflection in learning; encourages teacher-student dialogue; helps clarify what is good performance; provides students with quality information to help improve their learning; encourages motivation and self-confidence in students; aids the teacher in shaping teaching. Underpinned by the relevant theory, the practical advice and examples in this book directly address the issues of how to motivate students to engage in formative assessment effectively and shows teachers how they can provide further useful formative feedback.
Feedback is one of the most powerful influences on learning and achievement, but this impact can be either positive or negative. Its power is frequently mentioned in articles about learning and teaching, but surprisingly few recent studies have systematically investigated its meaning. This article provides a conceptual analysis of feedback and reviews the evidence related to its impact on learning and achievement. This evidence shows that although feedback is among the major influences, the type of feedback and the way it is given can be differentially effective. A model of feedback is then proposed that identifies the particular properties and circumstances that make it effective, and some typically thorny issues are discussed, including the timing of feedback and the effects of positive and negative feedback. Finally, this analysis is used to suggest ways in which feedback can be used to enhance its effectiveness in classrooms.
In this article the authors attempt to shed some light on the PhD examination process, a process that is arguably far less transparent than those at other levels of British higher education. More specifically, the authors focus upon institutional policy from a sample of 20 British universities and attempt to document policy and interrogate the principles underpinning guidelines governing examination practices. They examine definitions of what constitutes a doctorate and policies governing a range of practices that are integral to the examination process. The research reveals that although there is a large degree of inter-institutional consistency regarding key criteria for the award of a PhD, close inspection of institutional policy suggests that the PhD examination is in fact conceptualised and operationalised in diverse ways.
Current literature provides useful insights into the role of assessment feedback in student learning, yet fails to recognise its complexity as a unique form of communication. This article outlines ideas emerging from ongoing research into the meaning and impact of assessment feedback for students in higher education. We argue that new models of communication are required to understand students' responses to the language of tutors' comments, and that issues of discourse, identity, power, control and social relationships should be central to any understanding of assessment feedback as a communication process. Implications of adopting an alternative perspective for research and practice are identified and discussed.
The external examiner (EE) is the most important arbiter of whether a student’s submission earns the research degree for which it is entered. Based largely on personal experience as a research degree examiner and administrator, suggests that there are two main characteristics that should be required of any potential external examiner: they should be sufficiently aware of the intellectual frontiers of their subject that they can judge whether the thesis makes a contribution to knowledge or scholarship sufficient to justify the award; and they should also be mature adults, of enough humanity to ensure that the examination process is a worthwhile and developmental experience for the candidate, irrespective of the outcome. Argues that there are thus two principles that underlie the successful selection of an external examiner (EE): institutions should have carefully constructed regulations defining the qualifications and experience expected of the EE, and a scrutiny framework sufficient to ensure that they are followed; and supervisory teams have a duty to prepare and inform themselves well in advance of the selection of external examiners. Ways in which these principles can be made effective will be discussed and current practice at Nottingham Trent University outlined.
Much evaluation of teaching focuses on what teachers do in class. This article focuses on the evaluation of assessment arrangements and the way they affect student learning out of class. It is assumed that assessment has an overwhelming influence on what, how and how much students study. The article proposes a set of 'conditions under which assessment supports learning' and justifies these with reference to theory, empirical evidence and practical experience. These conditions are offered as a framework for teachers to review the effectiveness of their own assessment practice.
The use of external examiners in the doctoral assessment process is seen as a quality assurance process in most higher education systems. This article suggests that the selection of examiners is a critical aspect of that process. Interview analysis highlights the professional/academic considerations involved in selecting suitable examiners, as well as the somewhat more difficult to determine personality issues. Most of the findings lead to an appreciation that experienced supervisors see one of their roles in selecting examiners as protecting their doctoral students from the ‘bad and mad’, and looking for those examiners who have empathy and understanding, while at the same time maintaining high standards and integrity. A particular concern raised in the article is that of inexperienced supervisors selecting examiners, given the finding that most experienced supervisors ensure that they know, or at the very least know of, the personality traits of potential examiners.