IEN Inaugural Annual Conference September 24th 2004

1
IEN Inaugural Annual ConferenceSeptember 24th
2004

• Evaluation A tool for meaning-making in a Higher
  Education Quality Assurance culture

2
Content

• Conceptions about how mind works
• The OTC culture as a context for the Quality
  Assurance framework
• The OTC Quality Assurance framework
• Evaluation as a tool for meaning making within
  the OTC culture
• How meanings are communicated as catalysts for
  both individual and collective development and
  progression

3
Conceptions about how mind works Bruner
(1995)

• Computational View
• Information processing
• Culturalism
• Meaning-making
• Situated learning and thinking

4

• Education is not simply a technical business
  of well managed information processing..It is a
  complex pursuit of fitting a culture to the needs
  of its members and of fitting its members and
  their ways of knowing to the needs of the
  culture
• Bruner 198643

5
Guiding Principles
Vision and Mission
Ethos
Pedagogical Practice

• O.T.C. QA Framework
• Quality Control
• Quality Assurance
• Continuous Quality Improvement

6
Guiding principles

• Inclusiveness
• Openness
• Relevance
• Student centeredness
• Accountability
• Accessibility

7
College Vision and Mission

• The Open Training College will be recognized for
  the major role it plays in the advancement of the
  inclusion of people with disabilities in Irish
  Life the excellence of our training our
  commitment to our students the quality of our
  relationships with agencies.

8
Mission/vision statement cont.

• our advancement of adult lifelong-learning
  opportunities and our contribution to change and
  best practice at the level of the service-user,
  the staff member and the agency in the disability
  sector.

9
College ethos and values

• Enhancing the status of people with a disability
• Fundamental principles
• Valuing students as colleagues
• Close working relationships
• Cost effectiveness and value for money
• Strategic planning

10
Pedagogical practice

• Shared understanding of the adult learning
  process
• Courses designed to provide and stimulate diverse
  learning experiences, building of the skills and
  knowledge that our learners bring to their
  learning
• Clearly articulated learning outcomes and related
  assessment procedures

11
The Open Training College Quality Assurance
framework

• Please see handout

12
Evaluation process model for course evaluation

• Please see handout

13
How do we evaluate?

• Responsive Evaluation
• To covey a message rather than specific
• data
• 
  Stake (1975)
• Evaluation is an OBSERVED VALUE compared to
  some STANDARD
• Scriven (1967)

14
OTC standards

• Represented in the
• Guiding principles
• Vision and Mission statement
• Ethos
• Pedagogical Practice

15
Interpreting the evaluation summaries

• The basic task of an evaluator is made barely
  tolerable by the fact that he/she does not have
  to solve this equation in some numerical way nor
  to obtain a descriptive summary grade but needs
  merely to make a comprehensive statement of what
  the programme is observed to be, with useful
  references to the satisfaction and
  dissatisfaction that appropriately selected
  people feel toward it
• Stake (1975)

16
Making meanings

• Meaning making involves situating encounters with
  the world in their appropriate cultural contexts
  in order to know what they are about. Although
  meanings are in the mind they have their
  origins and their significance in the culture in
  which they are created. It is this cultural
  situatedness of meanings that assures their
  negotiability and ultimately, their
  communicability.
• 
  Bruner (19863)

17
How meanings are communicated

1. Evaluation summaries (QA coordinator)
2. Meaning-making within and as a result of the
   shared micro- culture of the OTC (tutors)
3. Feedback to Programme Boards through Annual
   Course Review Reports (tutors)
4. Recommendations made to Academic Council
   (Programme Boards)

18

1. Ratification of improvements and developments
   (Academic Council)
2. Implementation of the same (OTC team)
3. Dissemination to all stakeholders (QA
   coordinator)

19
Summary

• We make meanings of evaluation data by filtering
  our interpretations through our minds in order to
  measure findings against specified standards
• Macro and micro cultures shape mind and provide
  us with a toolkit by which we construct and
  communicate meanings

20

• If meanings from evaluations procedures are to be
  catalysts for development and progression
  evaluation procedures must be embedded within a
  clearly defined shared organizational culture to
  which its members are committed.