1
Department Seminar on
Teaching Learning
IT University of Copenhagen ((( May 28, 2008 )))
2
Intro to Constructive Alignment
The SOLO Taxonomy
Theory Practice
IT University of Copenhagen ((( May 28, 2008 )))
3
Agenda (Morning)

• Welcome Presentation of Programme
• by Mads Tofte Claus Brabrand

0900

• Introduction to Teaching Learning
• Constructive Alignment The SOLO Taxonomy

0915
--- 15 break ---

• From Theory to Practice
• From Content to Competence

1010
--- 15 break ---

• Group Exercise (heterogeneous groups)
• Discussion of Theory in Practice

1105
1200

• Lunch
• One hour lunch break

4
Agenda (Afternoon)

• Course Descriptions
• Good advice on how to write course descriptions

1300

• Main Group Exercise (homogeneous groups)
• Application of Theory in Practice

1330
short breaks
1610
Short presentations of Good Examples
--- short (10) break ---
1640
Poster Session (Tour I Tour II)
short break
--- short (10) break ---

• End Note
• by Mads Tofte Claus Brabrand

1820
1830

• Dinner Bar
• Dinner Bar

5
Agenda (Morning)

• Welcome Presentation of Programme
• by Mads Tofte Claus Brabrand

0900

• Introduction to Teaching Learning
• Constructive Alignment The SOLO Taxonomy

0915
--- 15 break ---

• From Theory to Practice
• From Content to Competence

1010
--- 15 break ---
1105

• Group Exercise (heterogeneous groups)
• Discussion of Theory in Practice

1200

• Lunch
• One hour lunch break

6
First Exercise
T

• Before we start Focus exercise

• Write down answer to
• "what is good teaching?"
• 2)

Swap Post-Its...
7
Teaching Learning (theory)

• Constructive Alignment SOLO Taxonomy

John Biggs world famous Bible of Teaching
Learning
Teaching for Quality Learning at
University - What the student does
(see compendium in your registration bag
paper)
Teaching Teaching
Understanding Understanding
19 minute short-film on Teaching
Learning (available on DVD in 7 languages,
epilogue by John Biggs)
8
Exercise
T

• Buzz Session

1) Discuss w/ neighbour "which of the
messages of the film did you find
particularly relevant ?" 2) Write it on a
Post-It 3) Swap Post-Its
9
Alignment

• Make explicit ILOs
• (Intended Learning Outcomes)
• (and tell this to students)

Exam ILOs Teaching
10
Student Motivation

• Susan (intrinsic motivation)
• - wants to learn !
• Robert (extrinsic motivation)
• - to pass exams !

11
Constructivism

• Transmission is Dead
• (lectures )
• Knowledge is Actively Constructed !

!
active teacher passive students
risk
12
SOLO Taxonomy

• Hierarchy for Competences
• Deep learning (not surface) !

5 generalize, theorize, predict, 4 explain,
analyze, compare, 3 describe, combine,
classify, 2 recite, identify, calculate,
13
Stud Learning Focus

• Focus on Student Learning !
• (instead of what teacher does
• labelling students good/bad)
• Student activitation ? learning

14
Break (until 1010)

• Please put the Post-Its on the wall

"What is good teaching?"
Film message(s)?"
15
Agenda (Morning)

• Welcome Presentation of Programme
• by Mads Tofte Claus Brabrand

0900

• Introduction to Teaching Learning
• Constructive Alignment The SOLO Taxonomy

0915
--- 15 break ---

• From Theory to Practice
• From Content to Competence

1010
--- 15 break ---
1105

• Group Exercise (heterogeneous groups)
• Discussion of Theory in Practice

1200

• Lunch
• One hour lunch break

16
From Theory to Practice
From Content to Competence ? advocate a shift
in perspective
Elaborate on SOLO (for Competences) ?
advocate SOLO for competences
New Danish Grade Scale ? show relation to new
grade scale
--- After lunch ---
How to write Course Descriptions ? concrete
advice
AFTER LUNCH
17
From Content to Competence

• Old way my 2004 Concurrency course descr
• Given in terms of a 'content description'
• Essentially

• Goal is...
• To understand
• deadlock
• interference
• synchronization
• ...

This is a bad idea for 2 reasons...!
18
1) Problem with 'content' as aim

• Problem with 'content as learning goals ?!?

analyze ... theorize ...
analyze systems explain causes
define deadlock describe solutions
Stud. C

• Goal is
• To understand
• deadlock
• interference
• synchronization
• ...

agreement
tacit knowledge from research-based
tradition (not known by stud.)
?
name solutions recite conditons
Teacher
analyze systems explain causes
Stud. B
BUT, even if it were possible to agree, we know
that the exam will dictate the learning anyways.
Stud. A
Censor
19
2) Problem with 'understanding'

• Problem with 'understanding'as learning goal ?!?

• Goal is
• To understand
• deadlock
• interference
• synchronization
• ...

concept of deadlock ?!
?
The answer is simple
It cannot be measured (!)
20
'Competence' as objectives !
Competence knowledge capacity to act upon it

• 'Competence' as learning objectives !
• Evaluation

Have the student do something, and then measure
product and/or process

• Objective !
• To learn to
• analyze systems for...
• explain cause/effects...
• prove properties of...
• compare methods of...
• ...

Note'understanding' is (of course)
pre-requisitional (!)
?
Note' inherently operational ( verbs)
'SOLO' Structure of the Observed Learning
Outcome
21
Neighbour Discussion
T
Discuss with neighbour "does this make sense
?!?" (content ? competence)
22
From Theory to Practice
From Content to Competence ? advocate a shift
in perspective
Elaborate on SOLO (for Competences) ?
advocate SOLO for competences
New Danish Grade Scale ? relation to new
grade scale
--- After lunch ---
How to write Course Descriptions ? concrete
advice
AFTER LUNCH
23
Advantages of 'SOLO'
Production of new knowledge

• Advantages of 'SOLO'
• Constructed for research-based university
  teaching
• Converges on research (at SOLO 5)

24
SOLO (elaborated)
Note the list is non-exhaustive
QUALITATIVE
QUANTITATIVE
SOLO 2 uni-structural
SOLO 3 multi-structural
SOLO 4 relational
SOLO 5 extended abstract

• theorize
• generalize
• hypothesize
• predict
• judge
• reflect
• transfer theory (to new domain)

• analyze
• compare
• contrast
• integrate
• relate
• explain causes
• apply theory (to its domain)

• combine
• structure
• describe
• classify
• enumerate
• list
• do algorithm
• apply method

• define
• identify
• count
• name
• recite
• paraphrase
• follow (simple) instructions

25
Post-It exercise
T
Write down 1-2 key competences (i.e.,
verbs) (for your course)
26
From Theory to Practice
From Content to Competence ? advocate a shift
in perspective
Elaborate on SOLO (for Competences) ?
advocate SOLO for competences
New Danish Grade Scale ? relation to new
grade scale
--- After lunch ---
How to write Course Descriptions ? concrete
advice
AFTER LUNCH
27
The Danish 7 Step Scale
For an excellent performance which completely
meets the course objectives, with no or only a
few insignificant weaknesses .
12
A
Excellent
For a very good performance which meets the
course objectives, with only minor weaknesses
10
B
Very good
Grade Degree of realization of
course objectives!
For a good performance which meets the course
objectives but also displays some weaknesses
7
C
Good
For a fair performance which adequately meets the
course objectives but also displays several major
weaknesses
4
D
Fair
02
E
Adequate
For a sufficient performance which barely meets
the course objectives
00
Fx
Inadequate
For an insufficient performance which does not
meet the course objectives
-3
F
For a performance which is unacceptable in all
respects
Unacceptable
28
Intended Learning Outcomes !

• Consequence
• Every course has to explicitly define
• Intended Learning Outcomes (!)

29
Break (until 1105)

• Please put the Post-Its on the wall

After break Heterogeneous group exercise (meet
here for instructions)
Key competences for your course?
30
Agenda (Morning)

• Welcome Presentation of Programme
• by Mads Tofte Claus Brabrand

0900

• Introduction to Teaching Learning
• Constructive Alignment The SOLO Taxonomy

0915
--- 15 break ---

• From Theory to Practice
• From Content to Competence

1010
--- 15 break ---
1105

• Group Exercise (heterogeneous groups)
• Discussion of Theory in Practice

1200

• Lunch
• One hour lunch break

31
Group Exercise
T
Note Groups are in the programme

• Group Exercise (in heterogeneous groups)

Discuss how main messagesof the film relate to
your ownteaching (either given/received)
32
LUNCH!

• We resume here at 1300

33
Agenda (Afternoon)

• Course Descriptions
• Good advice How to

1300

• Main Group Exercise (homogeneous groups)
• Application of Theory in Practice

1330
short breaks
1610
Short presentations of Good Examples
--- short (10) break ---
1640
Poster Session (Tour I Tour II)
short break
--- short (10) break ---

• End Note
• by Mads Tofte Claus Brabrand

1820
1830

• Dinner Bar
• Dinner Bar

34
Example Genetics 101 (at AU)

• Old course description
• New course description

Purpose To give the students a basic
understanding of the fundamental laws of genetics
and populational genetics and elementary
knowledge of Content - evolution, selection,
mutation, variation, in-breeding, equibrilia,
recombinations,

• After the course, the students are expected to be
  able to
• locate genes on chromosomes
• do simple calculations (eg, recombination
  frequencies, evolutionary equilibria, ..)
• describe and perform connexion-analysis
• describe fundamental genetic concepts (e.g.,
  mutation, variation, selection, )
• describe and analyze simple inheritancies
• analyze inheritance of multiple genes
  simultaneously

35
Concrete Recommendations (4x)
1) Use 'standard formulation' a) puts
learning focus on the student b) competence
formulation "to be able to"

• Intended Learning Outcomes Genetics
  101
• After the course, the students are expected to be
  able to
• locate genes on chromosomes
• do simple calculations (e.g., recombination
  frequencies, in-breeding
  coefficients, Hardy-Weinberg,
  evolutionary equilibria).
• describe and perform connexion-analysis
• describe fundamental genetic concepts
  (e.g., mutation variation,
  in-breeding, natural selection).
• describe and analyze simple inheritancies
• analyze inheritance of multiple genes
  simultaneously

4) Avoid 'understanding-goals' "To
understand X", "Be familiar with Y",
"Have a notion of Z", ...!
V
N
N
V
N
V
V
N
V
N
V
V
N
V
3) Use 'Verb Noun' formulation What the
student is expected to do with a given
matter .
2) List sub-goals as 'bullets' Clearer than
text
N
V
36
Another Example Semantics

• Applicable to all courses
• Quantum Mechanics 101
• Aesthetics History of Dancing
• Migration Patterns of Flock Birds

• Old course description
• New course description

Purpose To give the students an understanding of
- transition systems, big-step vs. small-step
semantics, side-effects, laziness,
termination, types, static semantics, dynamic
semantics, equivalence, bisimulation,
environment-store-model, and structural induction.

• After the course, the students are expected to be
  able to
• describe the meaning of a wide range of
  programming constructs
• explain fundamental concepts, techniques, and
  results within semantics ()
• analyze the meaning of a wide range of
  programming constructs
• compare semantic descriptions
• reason about semantic descriptions
• prove consequences of semantic descriptions
• implement semantic descriptions (in familiar
  programming languages)

37
Alignment Implementation Process

• Process (course specific)

1) Think carefully about overall goal of
course (what are the stud. to learn?)
2) Operationalize these goals and express
them as intended learning outcomes
alignment
learning incentive
learning support
3) Choose carefully the form(s) of
examination ( intended learning outcomes)
4) Choose carefully the form(s) of teaching
( intended learning outcomes)
38
Starting Point

• Content description (Concurrency '04'05)

What is the overall goal of the course...? (what
are the students to learn) think big picture
39
Overall Course Philosophy

• Example Concurrency
• Philosophy Model-Based Design for Concurrency

40
Implementation Process

• Process (course specific)

1) Think carefully about overall goal of
course (what are the stud. to learn?)
2) Operationalize these goals and express
them as intended learning outcomes
alignment
3) Choose carefully the form(s) of
examination ( intended learning outcomes)
4) Choose carefully the form(s) of teaching
( intended learning outcomes)
41
Learning Goals

• Intended Learning Outcomes(based on The SOLO
  Taxonomy)

Noteexplicitly included as a non-goal
?
42
Alignment Impl.

• ILOs
• construct models
• apply methods
• relate specs?models
• test models
• define properties
• verify modelsprops
• analyze models
• compare models
• implement models
• relate modelsimpl

• Process (course specific)

1) Think carefully about overall goal of
course (what are the stud. to learn?)
2) Operationalize these goals and express
them as intended learning outcomes
alignment
learning incentive
3) Choose carefully the form(s) of
examination ( intended learning outcomes)
4) Choose carefully the form(s) of teaching
( intended learning outcomes)
43
On the Role of the Exam

"To the teacher, assessment is at the end of
the teaching-learning sequence of events, but
to the student it is at the beginning" -- John
Biggs (2003)
Exam is a "necessary evil"
From
Conceptual change
Exam is a powerful motivational
learning- guiding pedagogical tool (for the
teacher) !!!
To
44
Alignment Impl.

• ILOs
• construct models
• apply methods
• relate specs?models
• test models
• define properties
• verify modelsprops
• analyze models
• compare models
• implement models
• relate modelsimpl

• Process (course specific)

1) Think carefully about overall goal of
course (what are the stud. to learn?)
2) Operationalize these goals and express
them as intended learning outcomes
alignment
learning support
3) Choose carefully the form(s) of
examination ( intended learning outcomes)
4) Choose carefully the form(s) of teaching
( intended learning outcomes)
45
Questions?
My research and teaching
Cognition structures
SOLO Analyses
Intended learning outcomes (ILO)
Association new old
"understanding" content ? competence
The SOLO Taxonomy
'TLA' Teaching/Learning Activities
Teacher models levels 1 - 2 - 3
Student models Susan Robert
M
S
The Short-Film
I
'The Book'
Model-based design for Concurrency
Experiences Pre vs. Post
Student activation
Satisfaction
analyze explain
Students at Uni
?
"What is good teaching?"
Exam
Tips'n'Tricks ???
Constructive Alignment
John Biggs
46
Tips'n'Tricks (activation)

• Neighbour discussions

• Post-It exercise

• focus zoom in
• anonymous (!)
• swap'able
• everyone will engage
• empathetic control
• shared knowledge pool

• more questions (students dare ask them)
• better questions (students had a chance
  to discuss)

Phil Race
1-2 min timeout

• Frequent breaks

• Form variation

pulse reader measurements
lecturing blended with in-class activation
exercises
47
Tips'n'Tricks (cont'd)

• Use many examples(build on student
  pre-knowledge)

• Explicit structure

1. xxxxxxxxxx 2. yyyyyyyyyy 3. zzzzzzzzzz 4.
wwwwwww
1. xxxxxxxxxx 2. yyyyyyyyyy 3. zzzzzzzzzz 4.
wwwwwww
1. xxxxxxxxxx 2. yyyyyyyyyy 3. zzzzzzzzzz 4.
wwwwwww
1. xxxxxxxxxx 2. yyyyyyyyyy 3. zzzzzzzzzz 4.
wwwwwww
?

• self evident to you teacher
• not to a learner student
• (esp. during learning process)

• "Less-is-more"

• Student 'recap' at end

• analyze
• compare
• relate

common deadlock, uncommon deadlock,
A-synchronization, B-synchronization, hand-shake,
multi-party synchronization, multi-party
hand-shake, binary semaphores, generalized
semaphores, blocking semaphores, recursive locks,
...
vs.
now
after 1 day
after 1 week
after 2 weeks
after 3 weeks
Emphasize depth over breadth (coverage)
48
Now, please "3-minute recap"

• Please spend 3' on thinking about and writing
  down the most important points from the talk
  now!

Immediately
After 1 day
After 1 week
After 2 weeks
After 3 weeks
49
Agenda (Afternoon)

• Course Descriptions
• Good advice How to

1300

• Main Group Exercise (homogeneous groups)
• Application of Theory in Practice

1330
short breaks
1610
Short presentations of Good Examples
--- short (10) break ---
1640
Poster Session (Tour I Tour II)
short break
--- short (10) break ---

• End Note
• by Mads Tofte Claus Brabrand

1820
1830

• Dinner Bar
• Dinner Bar

50
Common SOLO Competences
From a study of 632x courses at NAT/(AUSDU)
51
Group Exercise
T
Note Groups are in the programme

• Group Exercise (in homogeneous groups)

Rewrite course description (for your groups
course) incorporate recommendations
(Anna will distribute old course descriptions
for your convenience)

• Do NOT (under ANY circumstances) in fact ever
  use the verbs
• to understand
• to know
• to be familiar with
• to have a notion of

• internal cognitive structure !
• non-operational !
• cannot be measured !

52
Object-Oriented Prog. Intro

• After the course, the student is expected to be
  able to
• relate a given Java model to the real world
  phenomena that it models
• explain, use, and combine in programming
  variables, types, expressions, loops,
• design and implement class hierarchies using
  single inheritance comprising 3-5 classes.
• design and implement programs on the order of 500
  lines of Java code.
• use and design generic classes and generic
  methods.
• explain and use recursive methods.
• construct classes that implement a given Java
  interface.
• identify and use packages.
• construct classes that adhere to given class
  invariants.
• describe and reason about simple class
  invariants.
• explain which methods are being called during
  execution of a given program.
• explain the memory evolution during execution of
  programs.
• compare expected execution times (of simple
  algorithms).

53
Digital Culture

• After the course, the student is expected to be
  able to
• explain the use of the concept of culture in a
  digital context
• relate new media culture to a broader historical
  context
• apply the concepts introduced on the course in a
  critical analysis of cultural og social phenomena
  related to the use of the internet and
  information technology in everyday life
• reflect on different methodological perspectives
  on the study of digital culture and be able to
  select and then use them in their own practice
• use and compare relevant sociological and
  cultural theories on a broad selection of topics
  within the areas of online media, digital culture
  and digital communication
• independently identify and formulate a current
  and original problem/question related to the use
  of IT and digital communication, choose a
  relevant method to apply in the study,
  hypothesise the expected results, and finally
  discuss and perspectivate the actual results of
  the study in the final assignment

54
Agenda (Afternoon)

• Course Descriptions
• Good advice How to

1300

• Main Group Exercise (homogeneous groups)
• Application of Theory in Practice

1330
short breaks
1610
Short presentations of Good Examples
--- short (10) break ---
1640
Poster Session (Tour I Tour II)
short break
--- short (10) break ---

• End Note
• by Mads Tofte Claus Brabrand

1820
1830

• Dinner Bar
• Dinner Bar

55
Key References

• Teaching for Quality Learning at University
  (what the student does)John BiggsThe Society
  for Higher Education and Open University Press,
  2003(Note 3rd edition available)
• Teaching Teaching Understanding
  UnderstandingClaus Brabrand Jacob Andersen19
  minute award-winning short-film (DVD)Aarhus
  University Press, Faculty of Science, University
  of Aarhus, Denmark
• Constructive Alignment and the SOLO Taxonomy A
  Comparative Study of University Competences in
  Computer Science vs. MathematicsClaus Brabrand
  Bettina DahlKeynote Paper for Koli 2007 Conf.
  on Computing Education ResearchKoli National
  Park, Finland
• Evaluating the Quality of Learning The SOLO
  TaxonomyJohn B. Biggs Kevin F. CollisNew
  York Academic Press, 1982

56
Thank You!
Film's homepage
((( http//www.daimi.au.dk/brabrand/short-film/
)))
57
BONUS SLIDES
58
The New Danish Grade Scale
Conversion (between EU countries)
7 steps
4 steps
4 steps
8 steps
8 steps
ECTS
10 steps
10 steps
SCALE
...
...
...
...
...
21 steps
21 steps
A, B, C, D, E, Fx, F

• Problems (comparability EU nations)
• Information loss (10 steps ? 7 steps)
• (13,11) ? A (9,8) ? C
• The 13 (exception grade) doesnt exist in
  other scales!
• Some places only access if you have top grade (
  13)
• and a number of other motivations

pigeon hole principle
59
The New Danish Grade Scale
--

Degree of realization of course objectives
A 10 B 25 C 30 D 25 E 10
Not passed
passed
ECTS Scale
10
25
30
25
10
0 (minimum)
22,5 (interval-mid)
50 (interval-mid)
100 (maximum)
77,5 (interval-mid)
Divide by 10 round off
0
2
5
10
8
Symmetric transplacement
-2
-5
Add 2
02
4
7
12
10
Avoid using negative grades
00
-3
60
Grading ?
10 Grade degree of realization of course
objectives (overall evaluation) 9 Absolute
grading only (i.e., no relative grading)
12

• Learning Goals
• After the course, the students are expected to be
  able to
• locate genes on chromosomes
• do simple calculations (e.g., ).
• describe and perform -analysis
• describe fundamental genetic concepts
• describe and analyze inheritancies
• analyze inheritance of multiple genes

A
For an excellent performance which completely
meets the courseobjectives, with no or only a
few insignificant weaknesses.
Excellent
10
B
For a very good performance which meets the
course objectives, with only minor weaknesses
Very good
7
C
For a good performance which meets the course
objectives but also displays some weaknesses
Good
4
D
For a fair performance which adequately meets the
course objectives but also displays several
major weaknesses
Fair
02
E
For a sufficient performance which barely meets
the course objectives
Adequate
00
Fx
For an insufficient performance which does not
meet the course objectives
Inadequate
-3
F
For a performance which is unacceptable in all
respects
Unacceptable
Karaktergivning 12 'Udtømmende' 10
'Omfattende' (med nogle mindre væsentlige
mangler) 7 'Omfattende' (med en del
mangler) osv...
...set gennem en forskers øjne (forsknings-baseret
undervisning)
61
Top 10 Competences

• Top 10 Competences for
• Computer Science vs.
• Natural Science ( PHYSCHEMBIOMOL.BIO) vs.
• Mathematics

62
Notes on Skill Acquisition

• From the world of psychoanalysis
• Skill acquisition progresses according to the
  following stages of learning
• 1. Unconscious incompetence
• 2. Conscious incompetence
• 3. Conscious competence
• 4. Unconscious competence
• 5. Capacity for moving consciously between
  stages 3. and 4. required by a teacher

63
Impersonalization

• A taxonomy / language for teaching impersonalizes
  teaching
• Emotional detachment (aka. dissociation)
• The teacher is good/bad
• identity good/bad teacher
• The methods are good/bad
• behavior good/bad method
• knowledge good/bad method
• With dissociation
• more capable of dealing with critique ? better to
  listen to constructive advice (just like with
  our research)

Neutological levels model of the mind, NLP
behavior
reactions
knowledge
experience
moral
ethics
identity
convictions
capabilities
interaction
64
UNALIGNED COURSE
?
Teachers intention
Students activity

• e.g.
• explain
• relate
• prove
• apply

"Dealing with the test"
Exams assessment

• e.g.
• memorize
• describe

• e.g.
• memorize
• describe

65
ALIGNED COURSE
?
Teachers intention
Students activity

• e.g.
• explain
• relate
• prove
• apply

• e.g.
• explain
• relate
• prove
• apply

• e.g.
• explain
• relate
• prove
• apply

Exams assessment

• e.g.
• explain
• relate
• prove
• apply

• e.g.
• explain
• relate
• prove
• apply

66
Definition Good Teaching

• Definition
• Good news
• We now know how to do this
• Alignment!!!
• Explicitly defined course objectives (as verbs)!
• Discourage surface-learning!
• Encourage depth-learning!
• Less-is-more depth rather than breadth of
  coverage!

Good teaching is getting most students to use
the higher cognitive level processes that the
more academic students use spontaneously --
Teaching for Quality Learning at University,
John Biggs, 2003
67
The BLOOM Taxonomy (1956)

• The BLOOM Taxonomy

Analysis
Evaluation
Synthesis
SOLO 45
Qualitative
Application
Comprehension
Quantitative
SOLO 23
Knowledge

really intended to guide the selection of
items for a test rather than to evaluate the
quality of a students response to a particular
item -- (Biggs Collis, 1982)