Conceptualising and measuring the quality of WIL curricula

1
Conceptualising and measuring the quality of WIL
curricula
learning ? teaching ? higher education research

• Calvin Smith (PhD)

2
Evaluation - scope

• Educational (any) evaluation is defined by its
  scope and purpose
• E.g. experience, outcomes, ROI, serendipitous
  outcomes, costs, flexibility, validity, currency
• Measure and articulate the relationships between
• key aspects of the learning design and
  environment, and its coherence, and
• students satisfaction and learning outcomes

3
Many different manifestations of WIL-related
curricula

• Go by different names
• Work experience
• Work-based learning, work-place based learning
• Work placement
• Internship
• Work-integrated learning
• Are underpinned by different philosophies of the
  learning-work nexus
• Have different espoused learning objectives
• Have different practical qualities / properties
• Details vary enormously
• But can we ask What are the dimensions that they
  have in common?

4
Whats the dimension?
Whats the dimension?
5
What do WIL curricula share?

• A concern with authenticity
• Real not realistic
• Consequential and meaningful work
• A concern with action in the workplace
• Engagement in, not passive experience of, work
• A concern with integrative learning
• Not just application but reflection upon learning
• Mutually informing dialectic of learning

6
What (else) do WIL curricula share?

• Dispersion of students
• Dependency on partnerships/relationships with
  workplaces/industries
• Increased risks
• Students in workplaces (OHS-type risks)
• Reputational risks for university if things go
  wrong

7
Administrative overheads

• Establishment and maintenance of industry partner
  (IP) relationships / and an IP pool (including
  troubleshooting bad IP relationships)
• Dispersed students (academic contact)
• High risk unpredictable WP environments (OHS)
• Preparation of students (induction)
• Preparation of IPs (negotiation / training)
• Integrating uni support services and IP support
  services

8
Framework for good practice in WIL

• Authenticity
• Integration
• Alignment
• Administration

AIAA
9
Framework for good practice in WIL

• Authenticity
• having a real work/workplace context/environment
  where disciplinary KS will be applied in real
  life situations by implementing real work TLAs
  (Herrington Herrington, 2006)
• Authentic learning is
• Experience-based (students and others
  experiences of the real work/workplace are
  shared) (Kolb 1984 in Cowan 2006, Herrington
  Herrington, 2006)
• and situated (contextual, embedded in a
  real-world context or application environment)
  (Lave Wenger 1991, Herrington, Oliver Reeds
  2003)
• Involves real high-stakes applications of
  knowledge and skills in that environment to
  produce meaningful outcomes for the organisation
  and meaningful learning for the student

AIAA
10
Framework for good practice in WIL
AIAA

• Integration
• pedagogical
• Linking disciplinary/theoretical KSA and
  practical/applied KSA
• Linking university and work/workplace experience
• (Abeysekera 2006 OConnor 2004, Orrell 2004,
  Cooper et al. 2003, Billett 2001)
• Recognition of and reflection on the application
  of disc KSA in real-world workplace (Billett
  200120-21)
• non-pedagogical
• Linking of inter- and intra- institutional
  resources and services for WIL practice
• e.g. coordination of industry partners and
  universitys staff support and counselling
  resources for high-stress placements

11
Framework for good practice in WIL
AIAA

• Alignment
• Between the learning objectives, the activities
  done in situ and the assessments (Biggs 1999)
• academic and workplace goals must be
  complementary / mutually supporting
• curriculum needs to be founded on the
  contributions and circumstances of the workplace
  environments (Billett 20016).

12
Framework for good practice in WIL
AIAA

• Administration/management
• Pedagogical
• Preparation of students for WIL (e.g. inductions,
  roles and responsibilities, troubleshooting)
• Supervising, monitoring of TLAs, support and
  feedback for student in situ
• Monitoring the curriculum to ensure that it
  follows the principles of authenticity, alignment
  and integration(p), as well as the other elements
  of the model
• non-pedagogical
• paperwork (insurance, learning contracts,
  induction sessions and guidance manuals etc.)
• relationship w/ industry/organisation partners
• Creation and maintenance

13
Evaluation project

• Measurement study
• Samples
• 23 from a Griffith WIL course (criminology
  students)
• 78 from a UK collaborator (business students)
• Method
• Lit review
• Conceptual development
• Item development
• PCA then PAF
• SEM (tentative and preliminary!)

14
A I A A
GSs GAs
15
Measurement study

• Typical questions

16
Authenticity

• The work I did
• was important and consequential,
• resulted in a worthwhile outcome,
• was relevant to the organisation
• afforded the application of theory to work
• developed me as a beginning professional.

17
Integration (p)

• The activities set by/agreed with my academic
  supervisor required me to
• critically appraise the theories I had learned
  in class to the work I was doing in the workplace
• apply the theories I had learned in class to the
  work I was doing in the workplace
• reflect on the application of disciplinary
  knowledge and theories to the work I was doing
• critically appraise the practices I engaged in
  or witnessed in the workplace

18
Alignment

• The assessments set by/agreed with my academic
  supervisor focused on
• the critical appraisal of the applicability of
  theory to the workplace
• the application of theory in the workplace
• the application or development of the
  professional skills I had learned in the
  workplace
• The workplace that hosted me gave me plenty of
  opportunities to do work that was relevant to
  what I was supposed to be learning

19
Direct effects (SEM) RMSEA0.085 CFI0.953
The work
The activities
Assessment
20
Administration (p)

• There was a simple system for getting students
  connected with relevant industry partners /
  workplaces
• The system for getting students connected with
  relevant industry partners / workplaces ensured a
  good fit between me and the workplace
• There was comprehensive, clear and useful
  documentation about the roles, rights and
  responsibilities of all the parties (myself, the
  workplace supervisor and the academic supervisor)

21
Integration (p)

• The universitys student support services (e.g.
  counselling, learning support, career
  development, health service, library) were an
  integral part of the work-integrated-learning
  course I did
• The workplace's support services were an integral
  part of my workplace experience

22
Administration (p)

• My academic supervisor got back to me quickly
  when I wanted to discuss something
• It was easy for me to contact my academic
  supervisor during my placement
• My interactions with my academic supervisor
  during my placement were useful to me

23
Direct effects (SEM) CHI48.8 (df32 P.114)
CFI0.925
24
Some issues for future

• Small N (? study continues but building up data
  base ARC discovery application for 2009)
• OK so there are some good measures here, but do
  they make a difference to the quality of student
  learning or experience? (? exploring
  relationships with DVs)
• Do they stand up across different disciplines (?
  group comparative study when database is larger
  already have film, engineering, criminology,
  business)
• Watch this space

25
Thank you ?

• Calvin Smith
• Griffith Institute for Higher Education
• Griffith University
• Gold Coast
• calvin.smith_at_griffith.edu.au

26
Refs

• Billett, S. (2001). Learning in the workplace
  Strategies for effective practice. Crow's Nest
  Allen and Unwin.
• Abeysekera, I. (2006). Issues Relating to
  Designing a Work-integrated Learning Program in
  an Undergraduate Accounting Degree Program and
  its Implications for the Curriculum. Asia-Pacific
  Journal of Cooperative Education, 7(1), 7-15.
• Herrington, J., O l i v e r, R. (2000). An
  Instructional Design Framework for Authentic
  Learning Environments. Educational Technology
  Research and Development, 48( 3), 26.
• Lave, J., Wenger, E. (1991). Situated learning
  Legitimate peripheral participation. Cambridge
  Cambridge University Press.
• Orrell, J. (2004). Work-integrated Learning
  Programmes Management and Educational Quality.
  Paper presented at the Australian Universities
  Quality Forum.

27
Dialectic of learning
Codified knowledge Discipline skills Generalisable
, transcendent A priori
Applied knowledge Workplace skills Generic
Skills Specific, immanent Experiential
Reflection
Application
Explicit
Tacit
28
Evaluation project outcomes

• Scales that can be used by others
• Influence on the design of SEC evaluation
  instrument/s
• Further research
• Outcomes measures in the present instrument also