Graduate Centre Program September 2003

1
Graduate Centre ProgramSeptember 2003

• An Analysis of 81 Examiners Reports on
  University of Ballarat HDR students
• John McDonald

2
Aims of Workshop

• To examine patterns of classification
• To qualitatively analyse examiners comments
• To identify key characteristics of high quality
  and poor quality theses
• To make recommendations about revisions to
  supervision and examination

3
Analysis of Classifications

• Only 8/31 theses elicited examiners reports in
  complete agreement re. the classification
• 23/31 theses involved some level of disagreement
  over
• the classification
• 9/23 involved minor disagreement (between passed
  outright and passed subject to corrections)
• 10/23 involved major disagreement between passed
  outright or subject to corrections and
  deferral/resubmit or failure
• Only 1 of the 3 students passed without further
  examination was not required to make corrections

4
Conclusions about Classifications

• Students and supervisors must be prepared for
  disagreement between examiners classifications
• Students must be prepared to undertake
  corrections and probably some degree of rewriting
• There is a very good chance of being awarded the
  degree!

5
Qualitative Analysis of Examiners Comments

• There is no proforma for examiners comments
• Examiners comments ranged in length from 1/2
  page to 8 pages, with an average of about 3 - 4
  pages
• Commonly, about a third of the space involved
  pointing out corrections to grammar, punctuation,
  syntax, spelling etc.
• Not all the following comments apply to all the
  disciplines and theses

6
Characteristics of High Quality Theses (1)

• Title of thesis clearly reflects the focus and
  the argument
• A significant and substantial problem has been
  selected for investigation
• Early statement of the project aims
• Project presents a considerable advance on
  existing knowledge
• Key concepts or variables clearly defined and
  consistently used throughout the thesis
• Demonstrates systematic pursuit of consistent
  line of inquiry
• Thesis is well-planned and executed with each
  section clearly building on the last (coherent
  and unifying structure)
• Clear signposting and linking between paragraphs,
  sections and chapters
• Consistently (but not repetitively) remind the
  reader of the purpose, argument, or overall
  thrust of the thesis

7
Characteristics of High Quality Theses (2)

• Literature review is critical and evaluative, and
  sets forth an argument for why and how the study
  should be conducted
• Sophisticated theoretical ground-setting
  (including exposition of underlying assumptions,
  and relevance to the research aim)
• Balanced discussion of the rationale for
  selecting a methodology and method (including
  up-to-date methodological literature)
• Appropriate research design which allows the
  research questions to be answered
• Meticulous account of the procedure
• Rich variety of evidence employed to develop a
  balanced argument
• Advanced analytical skills to demonstrate a deep
  understanding of the problem

8
Characteristics of High Quality Theses (3)

• Clear chain of evidence laid down
• Discussion is disciplined and not excessively
  speculative
• Conclusions are well drawn and convincing (relate
  the outcomes back to the research aims)
• Clear and strong knowledge claims made about the
  exact contribution/s of the thesis
• Elegant, precise and economical expression
• Evidence of systematic proof-reading and error
  correction

9
Characteristics of Poor Quality Theses (1)

• Objectives and protocol of the study not stated
• Research questions either not significant or were
  self-evident (no risk of outcome)
• Principle purpose/s or arguments of the thesis
  are difficult to discern (reader is hard-pressed
  to extract the original contribution)
• No clear delimitations to the study
• Aimless contextualising of a problem
• Overly-simplistic comments and generalisations
• Scope of the thesis was overly-ambitious
• Definitions of key term either omitted or
  imprecise

10
Characteristics of Poor Quality Theses (2)

• Grasp of the literature has serious limitations
  (unaware of major, relevant works, or uses older
  works that are no longer authoritative or never
  were authoritative)
• Serial description of the literature (with scant
  critical analysis or argument emerging)
• Dated review of the literature (must be revisited
  or continually visited)
• No clear connections between the focus of the
  study and the logic or foundations of the
  research on which it is based.

11
Characteristics of Poor Quality Theses (3)

• Theoretical perspectives or conceptual frameworks
  that are left implicit throughout a thesis
• Unaware of the alignment or compatibilities of
  particular theoretical and methodological
  approaches
• Broad overview of theory that lacks depth or
  persuasiveness (especially noted by a reliance on
  undergraduate texts without reference to primary
  authors)
• Missing or undeveloped rationale for a particular
  theoretical approach
• Inadequate description of the sample selection
  strategy (inclusion and exclusion criteria not
  stated)
• Poorly reproduced photos, confusing diagrams,
  inadequately labelled tables
• Arguments that are intrinsically weak

12
Characteristics of Poor Quality Theses (4)

• Large slabs of (qualitative) data used to present
  a point when smaller excerpts with richer/deeper
  analyses are needed
• Lack of understanding of appropriate statistical
  analyses and interpretation, or insufficient
  detail on how data analysis was undertaken
• Triangulation often claimed but rarely delivered
• Sweeping, unfounded conclusions that have little
  or no basis in evidence

13
Characteristics of Poor Quality Theses (5)

• Poor expression is an irritant which detracts
  from the candidates argument
• Text is unnecessarily long and wordy. Material is
  repeated.
• Littered with spelling/typographical errors (not
  ready for examination)
• Incorrect or inconsistent referencing
• References omitted
• Lack of critical self-evaluation of the research
• Supervisors that do not have depth of knowledge
  in the field to advise the student. Raising false
  expectations and providing inadequate oversight.

14
Some Recommendations

• Recruit associate supervisors with requisite
  skills and knowledge to plug the gaps (esp.
  research design and data analysis)
• Prepare students for the reality of re-writing
  and re-submission
• Clearer and more expansive criteria against which
  to assess the thesis
• More prescriptive re. formatting of comments
• Meticulous proof-reading (use a range of readers)