Promoting constructive alignment through programme specification and subject benchmarks

1
Promoting constructive alignment through
programme specification and subject benchmarks

• Warren Houghton

School of Engineering and Computer Science,
University of Exeter
2
Plan.

• Programme Specification
• How we went about it in Engineering at Exeter
• Discussion of process
• Threshold standards
• Defining differentiated assessment criteria at
  module level
• Example and discussion
• Helping students to manage their own learning
• Levels of thinking about learning processes
• A reflective framework for thinking about
  learning teaching

3
Themes

• Responsibility
• Alignment
• Reflection

4
Context of examples

• University of Exeter
• mid ranking old university
• research lead
• Department of Engineering
• in School of Engineering and Computer Science
• small general engineering department
• 26 full time academic staff
• approx 400 U/G students
• Heavily constrained by PEI accreditation

5
Context of examples
Common first year
6
How we wrote Programme Specifications

• Put Subject Benchmark Statement to one side !
• Wrote aims and ILOs for existing programmes
• many iterations - emergent outcomes
• Wrote aims and ILOs for existing modules
• many iterations
• drawing out what staff were already doing
• Then, checked against Benchmark Statements etc.
• Did not try to achieve one-to-one mapping

7
Why not use Benchmark Statements as blueprints?

• What authority should we give the Benchmark
  Statements ?
• What does the QAA say ?
• Is there a correct answer ?
• How can we obtain a set of required ILOs ?
• From industry?
• Do we have to take responsibility, with our own
  ideas?

8
William Perrys positions

• 1. Absolute right answers are provided by
  Authority.
• 2. Authority may make us find his absolute right
  answers ourselves.
• 3. Authority may not have found all the absolute
  right answers yet . . .
• 4. Anyone has a right to his own opinion, but
  better keep Authority happy.
• 5. Everything is relative.
• 6. I may have to make some decisions for myself.
• 7. I must commit myself to a viewpoint.
• 8. I must take responsibility for what I commit
  to.
• 9. I am confident in my personal commitment, but
  I will keep an open mind.

9
What position do we take ?

• 1. Absolute right answers are provided by
  Authority.
• 2. Authority may make us find his absolute right
  answers ourselves.
• 3. Authority may not have found all the absolute
  right answers yet . . .
• 4. Anyone has a right to his own opinion, but
  better keep Authority happy.
• 5. Everything is relative.
• 6. I may have to make some decisions for myself.
• 7. I must commit myself to a viewpoint.
• 8. I must take responsibility for what I commit
  to.
• 9. I am confident in my personal commitment, but
  I will keep an open mind.

10
What position are we encouraged to take ?

• 1. Absolute right answers are provided by
  Authority.
• 2. Authority may make us find his absolute right
  answers ourselves.
• 3. Authority may not have found all the absolute
  right answers yet . . .
• 4. Anyone has a right to his own opinion, but
  better keep Authority happy.
• 5. Everything is relative.
• 6. I may have to make some decisions for myself.
• 7. I must commit myself to a viewpoint.
• 8. I must take responsibility for what I commit
  to.
• 9. I am confident in my personal commitment, but
  I will keep an open mind.

11
How should we use benchmarks?

• NOT as some definitive right answer
• We have to take responsibility for
  creating/recreating the curriculum
• drawing on
• our own experience
• others experience - set out in benchmarks etc.
• an iterative, reflective, process

12
Curriculum experienced by teachers and students
Curriculum development
Emergent outcomes
13
A limitation of benchmarks?

• They are not explicitly multidimensional

14
A two-dimensional table for each assessment
criteria heading
MEng
Assessment criteria
Breadth and depth of programme
BEng
BSc
3rd
1st
2.2
2.1
Degree classification (performance)
15
MEng and BEng programmes volumes in a space
that is at least three-dimensional.
16
Dimensions of learning outcomes
Order of thinking (e.g. Blooms hierarchy)
difficulty of concepts
Range of concepts
17
Simplistic levels view of a degree
Order of thinking
e.g. Blooms hierarchy
Range of concepts
18
Allowing for other dimensions
Order of thinking
e.g. Blooms hierarchy
Range or difficulty of concepts
19
and allowing for life before university !
Order of thinking
e.g. Blooms hierarchy
Range or difficulty of concepts
20
Plan .

• Programme Specification
• How we went about it in Engineering at Exeter
• Discussion of process
• Threshold standards
• Defining differentiated assessment criteria at
  module level
• Example and discussion
• Helping students to manage their own learning
• Levels of thinking about learning processes
• A reflective framework for thinking about
  learning teaching

21
Threshold standards

• benchmarking implies . . .
• all graduates will meet all threshold standards
• we need to show how
• we may have to change our assessment
• QAA(2000) Engineering Benchmark Statement.

22
Threshold standards

• Effective in (e.g.) the Royal Navy
• Different in HE why?
• Education and training are different
• Is certification realistic / useable ?
• Too much / different for employers to read
• PDP offers a solution to both problems

23
Examplesetting and assessing threshold
standards in core academic modulesin
Engineering at Exeter
24
Traditional examination

• 3 hr paper
• Choose 5 out of 8 questions
• Pass mark 40
• Pass provides evidence of 25 of ILOs tested
• But which 25 ?
• What can we build further learning on ?

25

• For all 1st and 2nd year engineering modules

• define detailed ILOs/assessment criteria . . .

26
Module specification A/B structure
27
Assessment - examinations

• Paper A
• Covering list A ILOs only
• Typically, short straightforward questions
• No choice
• Expected mark gt80
• Criteria referenced
• Paper B
• All ILOs, and some choice
• Longer, more challenging questions with no easy
  parts
• Expected average lt 40

28
Doesnt this approach mean that we are blatantly
teaching to the examination?
Yes !
29
Development

• Accepted because of PEI accreditation
• evidence that students with different marks had
  achieved identifiably different learning outcomes
• Originally developed as part of a scheme to give
  better guidance to students

30
Impact on staff

• Staff find it hard to split ILOs this way
• BUT
• asking for differentiated ILOs seems to work
  better than just asking for single level ILOs

31
Realism

• Any test is demanding when the pass mark is 80
• If we are genuinely going to test all ILOs they
  must be achievable.
• We have to be honest.

32
Deep vs. surface learning

• Are we encouraging surface learning?
• Other factors enable deep learning . . .
• Consider structure of the learning (noun)
• Hierarchy of concepts
• What happens if students try to understand
  complex concepts when they havent grasped the
  components?
• A/B approach makes deep learning possible

33
a problem

• of success ?

34
Supporting students

• A/B split originally introduced for student
  guidance
• students asked to identify progress against ILOs
  on weekly basis
• now whole of 1st year
• ILOs are a prerequisite to PDP

35
Why is PDP important ?

• It is about students becoming autonomous,
  independent, thinking for themselves
• the real purpose of HE
• It enables students to articulate what they can do

36
PDP

• is not a bolt on extra
• it is an integral part of learning in HE
• it must be addressed by all academic teaching
  staff

37
Academic staff

• view PDP in qualitatively different ways
• and
• many have difficulty with valuing PDP
• Why?

38
Levels of thinking about teaching

• Biggs (1 to 3)
• Focus on 1. what the student is
• 2. what the teacher does
• 3. what the student does
• plus?
• 4. how the student can manage what
  the student does (PDP)

39
Ways of thinking about Generic Graduate Attributes

• Barrie (1 to 4)
• 1. Necessary basic PRECURSOR skills but
  irrelevant as they are a prerequisite for
  university entry
• 2. Useful skills that COMPLEMENT or round out
  disciplinary learning
• 3. These are the abilities that let students
  TRANSLATE, make use of or apply disciplinary
  knowledge in the world
• 4. They are the abilities that infuse and ENABLE
  university learning and knowledge

40
Biggs - teaching(plus)
Barrie - attributes (skills)

• 1. What student is
• 2. What teacher does
• 3. What student does
• 4. How student manages learning

• 1. PRECURSOR, irrelevant
• 2. Useful COMPLEMENT
• 3. TRANSLATE learning
• 4. ENABLE university learning

41
Paradigm shifts
Reflect
Theorise
Kolb
Experience
Plan
42
Paradigm shifts - double loop learning
Paradigm shift
43
Biggs - teaching(plus)
Barrie - attributes (skills)

• 1. What student is
• 2. What teacher does
• 3. What student does
• 4. How student manages learning

• 1. PRECURSOR, irrelevant
• 2. Useful COMPLEMENT
• 3. TRANSLATE learning
• 4. ENABLE university learning

44
Learning to learn

• Teachers must
• have theories of learning
• not just bags of skills (see Ramsden)
• STUDENTS need
• study skills
• AND
• learning about learning
• theories of learning
• tools for metacognition / reflection / self
  management

45
A Reflective Framework
46
The created learning environment must be
designed so that students can manage their own
learning within it.

• It must offer
• real choices for students to make
• resources to support different choices
• information required to make choices (ILOs etc.)
• assessment outcomes clearly linked to choices
  (i.e. aligned assessment)

Student engaged in PDP
47
P.B.L. ?
48
A dialogue about learning is essential
PDP as a bolt on extra
49
PDP

• should be an integral part of the academic
  experience
• requires teachers who are reflective
  practitioners
• should have a profound impact on learning
• can be used as a tool for curriculum development

50
Plan.

• Programme Specification
• How we did it in Engineering at Exeter
• Discussion of process
• Threshold standards
• Defining differentiated assessment criteria at
  module level
• Example and discussion
• Helping students to manage their own learning
• Levels of thinking about learning processes
• A reflective framework for thinking about
  learning teaching

51
Summary
Alignment
Responsibility
Reflection
52
Student taking responsibility
53
How do we achieve alignment of