DATA COLLECTION AND DATA ANALYSIS

1
Perbincangan REKABENTUK KURIKULUM PENYELIAAN
PENGUJIAN, PENGUKURAN DAN PENILAIAN PEMBELAJARAN
Uthm 11 DEC 2007 Prof. Madya Dr. Wahid Razzaly
Dekan Pusat Pembangunan Akademik
2
At our doorstep.

• The stakeholders are evaluating our product and
  system
• How do we react?
• None of our business
• Lets change
• Doctor, Engineer, and Teacher.

3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
Our Minister Statement in 2003
9
Workshop Outcomes

• Participant will be able to
• Describe the implication of MQF
• Identify OBE processes
• Design a program
• Implement assessment

10
MALAYSIAN QUALIFICATION FRAMEWORK
11
Approach to Quality Assurance in HE
Promote public confidence that quality of
provision delivery of HE is always maintained
enhanced by HEIs
Nationally agreed Criteria Standards
Objective, independent reporting
Transparent procedures
12
Expected Outcome of MQF

• Increased public confidence in the quality and
  standards of qualifications
• Clarity of typical learning pathways for
  progression through and across sectors
• Recognition of life long learning through
  Accreditation of Prior Experience Learning
  (APEL)
• Establishment of a common currency for credit
  accumulation transfer based on learning
  outcomes
• Increased public-private partnerships
• Parity of esteem among academic, professional and
  vocational qualifications.
• Transparent readable qualifications -
  presentation of qualifications in forms that
  enable stakeholders to gauge their intended
  outcomes as well as the system of education the
  holders have undergone.
• Better articulation of equivalence of
  qualifications from other countries

13
STRUKTUR MQF
14
TAHAP LALUAN PENDIDIKAN MQF
Pengiktirafan Pembelajaran Terdahulu PPPT / APEL

Sijil Diploma Pasca Siswazah Postgraduate
Professional Awards Fellow Master Craftsmanship
Ph.D Kedoktoran
8
Sarjana Profesional (4 Thn) Sarjana Penyelidikan
, Kursus, Gabungan
7
Sarjana Muda (kepujian) (3-5 Thn)
Sijil Diploma Siswazah
6
Diploma Lanjutan
Diploma Lanjutan
5
Diploma Kemahiran
Diploma Teknikal Vokasional
4
STPM/ STAM Matrikulasi Asas
3
Sijil 3 Sijil 2 Sijil 1
Technical Vocational Certificate
2
1
SPM Lain-lain kelayakan diiktiraf
15
Hasil Pembelajaran Bidang Program
KEMAHIRAN PRAKTIKAL/ TEKNIKAL
Pengurusan maklumat pembelajaran sepanjang hayat
kemahiran Komunikasi berpasukan
PENGETAHUAN ILMU BIDANG
KEBERTANGGUNGJAWABAN SOSIAL
Kemahiran mengurus keusahawanan
Penyelesaian masalah penyelesaian saintifik
NILAI, SIKAP, PROFESIONALISME
16
8 General Outcomes Domains

• MASTERY OF BODY OF KNOWLEDGE
• PRACTICAL SKILLS
• SOCIAL ACCOUNTABILITY
• ETHICS, SHARED VALUES PROFESSIONALISM
• SCIENTIFIC METHOD, CRITICAL THINKING PROBLEM
  SOLVING
• COMMUNICATION SKILLS TEAM WORK
• INFORMATION MANAGEMENT LIFE LONG LEARNING
• ENTREPREUNERSHIP

17
StandardGuidelines on Standard of Specific
Disciplines at Bachelor Degree Level, Vol 1,
Ministry of Education, Malaysia, 2003.

• Standard that is explicit but not rigid
  covering.
• Educational Vision
• General Educaional goals
• Qualifications
• Learnng outcomes
• Programme design
• Assessment
• Entry criteria
• Academic staff
• Educational resources

18
OBE at GLANCE
19
OBE Any Issues?

• Points
• Well structured Cur
• Measurable lo
• Career Oriented
• Jaminan overseas job?
• TnL updated
• Generate rnd

• - - Points
• Budaya
• Tak nampak
• Beza?
• OBE ISO?
• Banyak kerja
• Idealistic
• Q
• Generalisation of students lc
• Success measurement
• Apa beza?
• More SCL?

20
OBE Any Issues?

• Competent output
• Transparent LO
• Global product
• Meeting Industry Req

• ----
• Sukar difahami
• Beza?
• Delivery?
• Hasilnya mana?
• How to measure?
• Why bother?
• Penerimaan Pelajar?
• Kenapa sekarang?

21
OBE Malaysian Experience

• 2002
• Quality Assurance Division, Ministry of Education
  established.
• MQF was initiated by BJK and LAN

• 2003
• MQF Consultative Programmes initiated that
  includes all.
• Malaysia becomes a Provisional Member of WA

22
OBE Malaysian Experience

• 2004
• Code of Practice Manual was published by The
  Ministry of Education
• Each discipline was supposed to have its own
  Guidelines for Standard for Bachelor Programme.
• All accreditation process must follow the Code of
  Practice, except for the professional bodies.

23
OBE Malaysian Experience

• 2006
• BJK and LAN merged to form Malaysian
  Qualification Agency (MQA)
• July Dato Dr. Sharifah Hapsah Syed Hasan
  Shahabudin became the VC of UKM and stepped down
  as MQA Director.

• 2005
• Dec MQF was presented and accepted by the
  Cabinet
• Effective application of New EAC Manual OBE
  Based
• WA official monitoring the Accreditation Process
  in Malaysia UKM and UTM were among the first to
  experience.
• All EAC accreditation will be based on the new
  Manual.
• The basis of evaluation is whether the academic
  system embraced OBE. (Appendix C

24
OBE Malaysia 2010

• What do you think the scenario would be in your
  institution?
• Top 18 WCU (Prof. Hamid)

25
Background
Quality Assurance Division (QAD),MoHE

• OBE has its origin in the USA in the 1980s.
• Washington Accord Signed 1989
• Malaysia became Provisional Member in 2003
• substantial equivalency of accreditation systems.
  
• graduates prepared to practice engineering at the
  entry level

26
Visi

• Berazam menerajui aplikasi sains dan teknologi
  untuk kemakmuran sejagat

27
Misi

• Untuk melahir dan melatih ahli profesional dan
  teknologi bersahsiah mulia dan berdaya saing di
  arena global melalui program akademik yang
  holistik dan pembudayaan ilmu serta penyelidikan
  yang berteraskan konsep tauhid.

Frazer Hill, 20062006
28
FALSAFAH PENDIDIKAN UNIVERSITI

• Pendidikan di Universiti ini adalah usaha
  berterusan untuk menerajui program akademik yang
  berorientasikan pasaran dan berfokuskan pelajar
  menerusi pembelajaran-melalui-pengalaman
  (experiential learning) untuk menghasilkan sumber
  manusia terlatih dan profesional sebagai
  pemangkin ke arah kemajuan negara yang mampan

29
Hasil Program (KUiTTHO)

• Practice and application-oriented graduates who
  have
• 1. have acquired and able to apply knowledge of
  science, and engineering fundamental
  (fundamental knowledge)
• 2. in-depth technical competence in a specific
  engineering discipline (technical competence)
• 3. ability to communicate effectively/use ICT
  effectively(communication)
• 4. ability to use techniques, skills, and
  modern engineering tools necessary for
  engineering practice and easily adaptable to
  industrial needs(adaptability)
• 5. ability to identify problems, create
  solutions, innovate and improve current
  practices (problem solving)
• 6. understanding of professional and ethical
  responsibilities and commitment to the
  community(ethics)
• 7. a recognition of the need for, and an
  ability to engage in, life-long learning
  (adaptability to new situations and demands by
  applying and/ or updating knowledge and
  skills)(lifelong learning)
• 8. ability to function effectively in groups in
  ways that contribute to effective working
  relationships and the achievement of goal both as
  a leader as well as and effective team
  player(teamworking)
• 9. ability to have an international
  perspective on social, cultural, global and
  international responsibilities of a professional
  engineer and the need for sustainable
  development(social awareness)
• 10. ability to appreciate aesthetic values
  through development and applications of personal
  judgement (appreciation of aesthetic values)

30
Sistem Kualiti di KUiTTHO.

• 2 sistem utama
• Sistem QAA
• Unit Jaminan Kualiti Akademik
• Pejabat pengurusan Akademik
• Fakulti
• Sistem Pengurusan Kualiti ISO90012000
• dan disokong bersama oleh sub-sistem yang
  pelbagai.
• Rekabentuk
• Penyampaian
• Penilaian
• Sokongan (HROD,Hartabina, Perpustakaan, PDP,
  SACAD, ..etc)

31
INSTITUTIONAL QUALITY SYSTEM
National Global Aspirations
Institutional Vision, Mission Goals
Learning Outcomes
Program Design
STUDENTS
EFFECTIVE

GRADUATES
DELIVERY
STAFF ACADEMIC FACULTY, PHYSICAL FACILITIES,
FINANCE, MATERIALS, TECHNOLOGY, SUPPORT SERVICES,
ETC
GOVERNANCE ADMINISTRATION/MANAGEMENT
QUALITY MANAGEMENT
32
Quality Management System Process Model
ISO QUALITY MANAGEMENT SYSTEM CONTINUAL
IMPROVEMENT
I n t e r e s t e d p a r t y
I n t e r e s t e d p a r t y
Management Responsibility
S a t I s f a c t i o n
R e q u i r e m e n t s
Resource management
Measurement, analysis, improvement
Product realization
Output
Product/ services
Input
(and/or service)
Quality Management System
33
Problem .nurturing Quality
Assessment
Delivery
Design
Therefore, Lets discuss the issue today.
34
Persoalan..?

• Mampukah kita mencapai VISI kita atau PRODUK
  yang mempunyai kriteria (outcomes) yang
  ditetapkan melalui sistem pengurusan pembelajaran
  sedia ada di KUiTTHO?

35
OUTCOME BASED Academic Quality Assurance
Conceptual Understanding
outcomes
Do our activities contribute towards the
development of effective Graduates (outcomes)?
Eg. LI, PSM which outcome?
36
What is Outcome-Based Education ?
Outcomes Based Education focuses on student
learning by 1. Using learning outcome statements
to make explicit what the student is expected to
be able to know, understand or do 2. Providing
learning activities which will help the student
to reach these outcomes 3. Assessing the extent
to which the student meets these outcomes through
the use of explicit assessment criteria.
37
OBE
destination orend of journey
OBE Focusing on the outcomes of
programme implementation
Always begin with the end in mind - Steven
Coveys 7 Habits of Highly Successful People
38
OBE Philosophy
OUTCOMES
39
Culture Shift towards OBE

• Regulatory
• QA Code of Practice / MQF (QAD, Ministry of
  Higher Education ) also requires OBE
• EAC Manual being revised to incorporate OBE
• Anticipation of real benefits
• More directed coherent curriculum
• - An antidote to curriculum chaos. Felder
  2003
• Graduates will be more relevant to industry
  other stakeholders
• Continual Quality Improvement (CQI) is an
  inevitable consequence

40
Some immediate Advantages

• Always alert on quality of graduates
• More effective innovative teaching
• - PBL, SCL, etc
• More industry input
• (KUiTTHO BSC target 10 of the Curriculum)

41
Quality Assurance Division (QAD), Ministry of
Higher Education

• Entrusted with committed to ensuring quality in
  Higher Education
• OBE elements in - QA Code of Practice
• OBE elements in MQF

42
Outcome-Based Education - A Shift in Focus

• The greater focus on outcomes and objectives
  means that all stakeholders must have a say in
  establishing them, and measuring them.
• (Stakeholders industry, faculty, students,
  government, alumni, etc)

43

• Jenkins, M. G., "Standards and Codes in
  Mechanical Engineering Education Confounding
  Constraints or Helpful Hindrances?,"
  Standardization News, Vol 27, No 9, pp 20-25, 1999

44
OBE PLANNING STRATEGIES
45
KICK START _at_ KUiTTHO
REVIEW4/06
S1,S2/0506
Implement OBE curriculumImprove teaching
techniquesOutcomes assessment
3
T r a n s l a t i o n a l
5/05
Design OBE curriculum - Design courses -
Course outcomes
2
Going on !
OBE awarenessSetting programme
objectivesSetting programme outcomes
1
46
P 1 PEO LO
PEO Later / on the job
LO at the end of Programme
47
(No Transcript)
48
Different Levels of Learning Outcomes
EAC Manual Programme Educational Objectives
PEO are broad goals describing expected
achievements of graduates in their career and
professional life after graduation. Programme
Outcomes - Statements that describe what students
are expected to know, think, and able to do by
the time of graduation.
49
Relationships between subject LOs, Course LOs
POs and compliance to the stakeholders
Institutional requirements eq UTM Graduate
Attributes
Professional Bodies eg /EACBEM
Quality Assurance JPT
Industrires
CUSTOMERS, STAKEHOLDERS, CONSTITUENTS
Parents
.. etc
PO-1
.. etc
PRORGRAME EDUCATIONAL OBJECTIVES
PO-2
PO-4
PO-5
TDAs, KJs, KPs, KKs, JKA and JKKS
PO-3
Programme Specs
LO-2
LO-4
LO-5
LO-3
.. etc
LO-1
LEARNING OUTCOMES
TDAs, KJs, KPs, KKs, JKA and JKKS
Programme Specs
Subject LO-1
Subject LO-2
Subject LO-3
Subject LO-5
Subject LO-4
SUBJECT LEARNING OUTCOMES
.. etc
Subject coordinator, Academic Staff
Course Outlines
Management support and commitment
50
OBE _at_ UTM
Students-Staff meeting/dialogue
Employability survey
Traditional methods
Industrial/employers survey
Entry/Exit Survey
Customer survey
Exams, projects, quizzes,test etc
graduate survey
Learning Style Survey
Assessment, review and evaluation stage
Check
External advisor
Academic auditing
Info from professional bodies
Peer evaluations Observations Learning Portfolios
Visiting Profs report
External examiner
Sijil Pengajaran Pengajian Tinggi
HRD Programmes
Problem based Learning
Traditional Approach
Actions
E-Learning
Active/corporative Learning
Teaching with Tech. Academic Advising Teaching
Approach Teaching Portfolio Problem based learning
Implementation Stage
Do
Incorporating Generic Skills in T L
PTK
Teaching Portfolio
Setting POs
At UTM / faculty level
Dept/Staff level
Course Outlines
Program Specification
Plan
Setting LOs
E-Learning
Planning Stage
Mapping Template
Weekly course notes
Documented Manuals
UTM Graduate Attributes
Management support and commitment
51
OBE Model Hierarchy

• 1. Programme educational objectives (PEO) are
  developed from a number of sources including
  professional accreditation bodies, employer
  groups, the university educational principles and
  the professional experience of staff teaching in
  the discipline.
• 2. The programme outcomes(PO) for a degree are
  clearly written statements about the knowledge,
  skills and attitudes of its graduates. It should
  link to the PEO
• 3. From these POs the curriculum of the course
  is constructed, the subdivision of structure into
  units is made, and the outcomes specific to each
  of the units are derived.

52
(No Transcript)
53
(No Transcript)
54
Section 3.0(i) Programme Objectives

• An engineering programme seeking accreditation
  shall have published Programme Objectives that
  are consistent with the mission and vision of the
  IHL, and are responsive to the expressed interest
  of various groups of programme stakeholders.
• The following are examples of performance
  indicators expected for Programme Objectives-
• defined, measurable and achievable
• linked to Programme Outcomeshave own niche
• detailed out and documented
• Published
• consistent and linked to mission vision of IHL
  and stakeholder needs
• linked to curriculum design
• reviewed and updated
• The process of establishing the educational
  objectives should be evaluated by the panel by
  examining the evidence provided by the programme.
  

55
Section 3.0(ii) Processes Results

• The programme shall have a clear linkage between
  Programme Objectives and Programme Outcomes
  having the process of formulating Programme
  Objectives and the process of assessing and
  evaluating the achievement of Programme
  Objectives with documented results and the
  evaluation results are used in the continual
  improvement of the programme.
• The following are examples of performance
  indicators expected for Processes Results-
• Established processes can be shown available in
• formulating Programme Objectives
• assessing achievement of Programme Objectivese
• valuating achievement of Programme Objectives
• the performance target of the Programme
  Objectives is achieved
• evaluation results are used in the CQI of the
  programme.

56
Section 3.0(iii) Stakeholders Involvement

• The programme shall produce the evidence of real
  involvement of programme stakeholders with regard
  to Sections 3.0(i) and (ii).
• The following are examples of performance
  indicators expected for Stakeholders
  Involvement-
• high degree of involvement in defining Programme
  Objective statements
• high degree of involvement in assessing the
  achievement Programme Objectives
• high degree of involvement in assessing
  improvement cycles (CQI)
• involvement in strategic partnership
• The involvement of stakeholders should be of
  prime importance for the the programme.

57
Section 4.0(i) Programme Outcomes

• An engineering programme seeking accreditation
  must have published Programme Outcomes that have
  been formulated considering items (a) to (j)
  given in Section 4.0 of the manual, and/or any
  added outcomes by the programme that can
  contribute to the achievement of its stated
  Programme Objectives. The Programme Outcomes must
  be shown linked to the Programme Objectives.
• The following are examples of performance
  indicators expected for Programme Outcomes-
• covers (a) to (j)
• linked to Programme Objectives
• have own wordings
• have own niche
• defined, measurable and achievabledetailed out
  and documented
• Published
• consistent and linked to Programme Educational
  Objectives.
• reviewed and updated

58
Section 4.0(ii) Processes and Results

• The programme shall also establish a process of
  measuring, assessing and evaluating the degree of
  achievement of Programme Outcomes by the
  students. The results of this assessment process
  shall be applied for continual improvement of the
  programme.
• The following are examples of performance
  indicators expected for Processes and Results-
• processes for all elements of criteria are
  quantitatively understood and controlled
• clearly linked to mission, programme objectives,
  and stakeholders needs
• seen as benchmarks by other institutionssystematic
  evaluation and process improvement in place
• support areas involved
• common sources of problems understood and
  eliminated
• outcomes in line with national needs
• sustained resultsresults clearly caused by
  systematic approach
• sound, highly integrated system
• deployed throughout the programme, faculty, and
  IHL

59
The Right Perspectives

• Professionalism
• Knowledge management
• Understanding of National Aspiration
• Understanding Vision, Mission, Educational
  Philosophy
• Strategy
• Understanding Quality
• Then.how to do.

60
Falsafah Pendidikan Negara

• Pendidikan di Malaysia adalah suatu usaha
  berterusan ke arah memperkembangkan lagi potensi
  individu secara menyeluruh dan bersepadu untuk
  mewujudkan insan yang seimbang dari segi intelek,
  rohani, emosi dan jasmani berdasarkan kepercayaan
  dan kepatuhan kepada Tuhan. Usaha ini adalah bagi
  melahirkan rakyat Malaysia yang berilmu
  pengetahuan berketrampilan, berakhlak mulia,
  bertanggungjawab dan berkeupayaan mencapai
  kesejateraan diri serta memberi sumbangan
  terhadap keharmonian dan kemakmuran masyarakat
  dan negara.

61
KUKUM Mission Vision

• Vision
• To become a world-class academic and research
  institution in order to produce prominent leaders
  of the country
• Mission
• To realize the country's aspirations and to
  contribute towards global civilisation

62
KUKUM Curriculum Structure

• Curriculum Structure
• The curriculum structure utilized in KUKUM is
  based on the 'lab-intensive' approach, whereby
  60 of the teaching is done in laboratory
  settings, whereas 40 is done via formal lecture
  sessions.
• All students need to collect 135 credits before
  being eligible for graduation. Of this, 120
  credits comprise of core engineering
  mathematics courses, whereas 15 credits comprise
  of university-required courses.
• Laboratory Work
• The 'practical-oriented' / 'lab-intensive'
  approach used in KUKUM requires 60 of the
  teaching/learning process be done in a laboratory
  setting. Students are required to submit lab
  reports of the work assigned during lab sessions.
  In addition, lab examinations are also conducted
  at the end of the semester. Lab-based evaluation
  makes up to 40 of the total marks in any given
  core course.

63
Quality Teaching from Students
Perspective.from survey

• Relates to real world applications
• Teach at students level
• Make learning fun
• Concern for students
• Enthusiastic

64
Quality Teaching

• Promote Student Centered Learning
• Current Knowledge
• Writing and Publication of Journals
• Leadership Role
• Research Based Teaching and Consultancy
• Professional Responsibility

65
Current Knowledge

• Continuous Learning put theory into practice
• Advancement of knowledge
• What knowledge is worth knowing
• Global thinker
• Multi Disciplinary

66
Knowledge Management Access to an Untapped
Resource
Knowledge
At the first American Conference of Artificial
Intelligence, 1980, Edward Feignebaum coined a
phrase. It was KNOWLEDGE IS POWER
67
(No Transcript)
68
Curriculum Design

• Professionalism
• Knowledge management
• Understanding of National Aspiration
• Understanding Vision, Mission, Educational
  Philosophy
• Strategy
• Understanding Quality
• Then.how to do.

69
Characteristics of Quality Teaching

• Research indicates that students, faculty and
  administrators alike agree that quality teaching
  
• establishes a positive learning environment
• motivates student engagement
• provides appropriate challenges
• is responsive to students learning needs, and
• is fair in evaluating students learning.

70
DESIGNING
71
OBE at GLANCE
72
Curriculum Development
73
(No Transcript)
74
(No Transcript)
75
(No Transcript)
76
(No Transcript)
77
(No Transcript)
78
(No Transcript)
79
INSTITUTIONAL QUALITY SYSTEM
National Global Aspirations
Institutional Vision, Mission Goals
Learning Outcomes
Program Design
STUDENTS
EFFECTIVE

GRADUATES
DELIVERY
STAFF ACADEMIC FACULTY, PHYSICAL FACILITIES,
FINANCE, MATERIALS, TECHNOLOGY, SUPPORT SERVICES,
ETC
GOVERNANCE ADMINISTRATION/MANAGEMENT
QUALITY MANAGEMENT
80
ASSESSMENT

• Definition of Test, Measurement, Assessment and
  Evaluation.
• To understand the various types and approaches to
  assessment
• To understand the application of Blooms Taxonomy
  and Ebels Relevance Guidance in constructing
  exams
• To understand the good practices in grading

81
Understanding the concepts

• Assessment
• Is the process of gathering information
  about how learners are progressing in their
  learning. It gathers information about what
  learners know and can demonstrate as a result of
  their learning processes
• Nitko, A.J. (1996) Educational assessment of
  students, Englewood Cliffs Prentice-Hall.
• Examine the above definition and discuss the
  following
• 1. List the various ways in which lecturers
  gather information about learners progress in
  learning. Which of these ways are more useful
  than others and why?
• 2. What does the phrase progressing in their
  learning mean to you?
• 3. Is it sufficient to assess what pupils know
  and can demonstrate? (knowledge and skills only)
• 4. Propose your own definition of assessment

82
Measurement

• Measurement refers to the process by which
  attributes or dimensions of some physical object,
  process or opinion are determined. The process
  depends on the use of standard instruments such
  as rulers, questionnaires, standardized tests
  etc.
• In measurement we are not assessing anything.
  We are simply collecting information relative to
  some established rule or principle.
• To measure is to apply a standard scale or
  measuring device to an object, events, or
  conditions, according to practices accepted by
  those who are skilled in the use of the device or
  scale.
• Kizlik B. (2003) measurement assessment and
  evaluation in education at www.adprima.com/measure
  ment.htm

83
Evaluation

• Evaluation is a process that enables us to
  determine the value of something. It allows us to
  make judgments about a given situation. When we
  evaluate, we yield information regarding the
  worthiness, appropriateness, goodness, validity,
  legality of something for which a reliable
  measurement or assessment has been made.
• It is the process of making judgment about
  the quality of a learners performance using the
  information gathered during assessment
• Ogunniyi, M.B. (1991) Educational measurement
  and Evaluation, Singapore Longman.

84
Testing

• Testing is just one of a number of strategies
  for measurement. It is a process by which we can
  formally gather valid information about the
  performance of pupils in given subjects. It comes
  in many forms such as multiple choice testing,
  essay testing, completion items testing, true
  false testing, etc.
• Activity Draw a concept map based on the need to
  distinguish the concepts assessment, testing,
  measurement and evaluation

85
Measurement
Testing
Evaluation
Testing, Measurement, Evaluation ASSESSMENT
86
Testing, Measurement Evaluation Some
Differences
87
Purposes of assessment

• Diagnose learners strengths and needs
• Provide feedback on teaching and learning
• Provide basis for instructional placement
• Inform and guide instruction
• Communicate learning expectations
• Motivate and focus learner attention and effort
• Provide practice applying knowledge and skills
• Provide a basis for learner evaluation
• Provide basis for evaluating programme
  effectiveness
• McTighe, J. and Ferrarra, S. (1994)
  Performance based assessment in the classroom,
  Pennsylvania, Educational leadership, 4-16.

88
Forms of assessment

• Formative
• any assessment that is ongoing meant to improve
  learning and help direct the teaching learning
  process. It is sometimes called continuous
  assessment
• Discuss how teachers formatively assess their
  pupils indicating the potential barriers they
  often encounter. Suggest ways of overcoming these
  barriers
• Summative
• happens at the end of a course or programme
  aimed at determining the effectiveness of the
  whole learning episode at its completion. The
  continuous assessment mark along with the end of
  year, programme or course marks are often
  aggregated in some way to arrive at a decision
  about the effectiveness of the entire learning
  episode
• In what way is your teaching subject summatively
  assessed? What issues about summative assessment
  in your subject are currently topical?

89
Assessment is effective when it

• is student-centred
• is congruent with instructional objectives
• is relevant
• is comprehensive
• is clear in purpose, directions and expectations
• is objective and fair
• simulates behaviour/product/performance
• incites active responses
• shows progress/development over time

90
Types Approaches to Assessment

• There numerous terms used to describe different
  types and approaches to learner assessment.
  Although somewhat arbitrary, it is useful to
  these terms as representing dichotomous poles.
  (McAlpine, 2002)

Formative
Summative
Formal
Informal
Final
Continuous
Product
Process
Convergent
Divergent
McAlpine, M. Principles of assessment, Glasgow
University of Glasgow, (2002)
91
Formative vs Summative Assessment

• Formative assessment is designed to assist the
  learning process by providing feedback to the
  learner, which can be used to identify strengths
  and weakness and hence improve future
  performance. Formative assessment is most
  appropriate where the results are to be used
  internally by those involved in the learning
  process (students, teachers, curriculum
  developers).
• Summative assessment is used primarily to make
  decisions for grading or determine readiness for
  progression. Often done at the end of an
  educational activity and is designed to judge the
  learners overall performance. In addition to
  providing the basis for grade assignment,
  summative assessment is used to communicate
  students abilities to external stakeholders,
  e.g., administrators and employers.

92
Informal vs Formal Assessment

• With informal assessment, the judgements are
  integrated with other tasks, e.g., lecturer
  feedback on the answer to a question or preceptor
  feedback provided while performing a bedside
  procedure. Informal assessment is most often used
  to provide formative feedback. As such, it tends
  to be less threatening and thus less stressful to
  the student. However, informal feedback is prone
  to high subjectivity or bias. 
•  
• Formal assessment occurs when students are aware
  that the task that they are doing is for
  assessment purposes, e.g., a written examination.
  Most formal assessments also are summative in
  nature and thus tend to have greater motivation
  impact and are associated with increased stress.
  Given their role in decision-making, formal
  assessments should be held to higher standards of
  reliability and validity than informal
  assessments.

93
Continuous vs Final Assessment

• Continuous assessment occurs throughout a
  learning experience and is most appropriate when
  student and/or instructor knowledge of progress
  or achievement is needed to determine the
  subsequent progression or sequence of activities.
  Continuous assessment provides both students and
  teachers with the information needed to improve
  teaching and learning in process. Obviously,
  continuous assessment involves increased effort
  for both teacher and student.
•  
• Final assessment is that which takes place only
  at the end of a learning activity. It is most
  appropriate when learning can only be assessed as
  a complete whole rather than as constituent
  parts. Typically, final assessment is used for
  summative decision-making. Obviously, due to its
  timing, final assessment cannot be used for
  formative purposes.

94
Process vs Product Assessment

• Process assessment focuses on the steps or
  procedures underlying a particular ability or
  task, e.g. the cognitive steps in performing a
  mathematical operation. Because it provides more
  detailed information, process assessment is most
  useful when a student is learning a new skill and
  for providing formative feedback to assist in
  improving performance.
•  
• Product assessment focuses on evaluating the
  result or outcome of a process. Using the above
  example, we would focus on the answer to the math
  computation. Product assessment is most
  appropriate for documenting proficiency or
  competency in a given skill, i.e., for summative
  purposes. In general, product assessments are
  easier to create than product assessments,
  requiring only a specification of the attributes
  of the final product.

95
Divergent vs Convergent Assessment

• Divergent assessments are those for which a range
  of answers or solutions might be considered
  correct. Examples include essay tests, and
  solutions to the typical types of indeterminate
  problems posed in PBL. Divergent assessments tend
  to be more authentic and most appropriate in
  evaluating higher cognitive skills. However,
  these types of assessment are often time
  consuming to evaluate and the resulting judgments
  often exhibit poor reliability.
• A convergent assessment has only one correct
  response (per item). Objective test items are the
  best example and demonstrate the value of this
  approach in assessing knowledge. Obviously,
  convergent assessments are easier to evaluate or
  score than divergent assessments. Unfortunately,
  this ease of use often leads to their
  widespread application of this approach even when
  contrary to good assessment practices.

96
Assessment vs Evaluation

• Depending on the authority or dictionary
  consulted, assessment and evaluation may be
  treated as synonyms or as distinctly different
  concepts. If a distinction exists, it probably
  involves what is being measured and why and how
  the measurements are made.
• In terms of what, it is often said that we assess
  students and we evaluate instruction. This
  distinction derives from the use of evaluation
  research methods to make judgements about the
  worth of educational activities. Moreover, it
  emphasizes an individual focus of assessment,
  i.e., using information to help identify a
  learner's needs and document his or her progress
  toward meeting goals.
• In terms of why and how the measurements are
  made, the table by Apple Krumsieg, (1998)
  compares and contrasts assessment and evaluation
  on several important dimensions.

97
Assessment Evaluation Compared
Apple, D.K. Krumsieg, K. Process education
teaching institute handbook, Pacific Crest (1998)
98
Quotes on Tests Exams

• Why do we do it? Tests arent fun to take and
  certainly not fun to grade, and sometimes it
  seems life would be a lot simpler if learning
  and not tests and grades were more important
  (Speaking of Teaching, Stanford University
  Newsletter on Teaching, Fall 1992, Vol.4, No.1)
• one of the greatest problems in
  institutional forms of learning is that students
  study for the tests and exams, instead of
  studying to grasp the object of learning and
  instead of studying for life (Bowden Marton,
  The University of learning Beyond qualityand
  competence in higher education, London, Kogan
  Page, 1998)

99
Blooms Taxonomy Descriptions(useful in writing
objectives)

• Knowledge recall of previously learned material
  such as specific facts, terms, methods, concepts,
  principles or complete theories. Knowledge
  questions can be identified by key verbs such as
  define, identify, list, match, select, state and
  label.
•  
• Comprehension ability to grasp the meaning of
  facts, principles, charts, graphs, etc.
  (understand). Comprehension questions can be
  identified with key verbs such as convert,
  explain, distinguish, paraphrase, interpret,
  summarize, predict.
•  
• Application ability to use learned information
  or ideas (concepts, principles, laws, theories)
  in novel and often practical situations.
  Application questions can be identified with key
  verbs such as compute, determine, demonstrate,
  modify, operate, prepare, produce, show, use and
  solve.

100
Blooms Taxonomy Descriptions(useful in writing
objectives) contd.

• Analysis ability to break down large pieces of
  information in order to examine the structure and
  interrelationships among its component parts,
  recognizing unstated assumptions, logical
  fallacies in reasoning, irrelevant data,
  organizational structure, facts vs inferences.
  Analysis questions can be identified by key verbs
  such as analyze, diagram, differentiate,
  illustrate, infer, outline, points out, relate,
  subdivide, and relate.
• Synthesis ability to combine various elements or
  parts into a new structural whole such as a new
  communication, plan of operations, classification
  systemin other words, the formulation of a new
  pattern. Synthesis questions can be identified by
  key verbs such as categorize, design, devise,
  formulate, reorganize, write, create, and plan.
• Evaluation ability to make a judgment about the
  value of something for a given purpose, based on
  definite criteria such as judging conclusions
  based on data, judging the value of art works
  using internal criteria or external standards of
  excellence. The students may be given the
  criteria or develop it himself. Evaluation
  questions can be identified by key verbs such as
  compare, critique, evaluate, appraise, contrast,
  criticize, and judge.

101
Blooms Taxonomy Ebels Relevance Guide Compared
Bloom, B.S. and others (1956) Taxonomy of
educational objectives. Handbook I The cognitive
domain. New Your David McKay Co. Ebel, R.L.
(1965) Measuring Educational Achievement.
Englewood Cliffs, NJ Prentice Hall Inc.
102
Grades and Grading

• There are varieties of ways of grading, from
  setting up an absolute standard to using a curve.
  Whatever model adopted, make sure the students
  know in advance how they will be evaluated.
  Grading policies should be spelt out on the
  syllabus.
• Academic performance mastery of knowledge and
  skills should be the focus of grading.
• Encouraging an orientation towards learning
  rather than towards grades will doubly assist the
  students they will comprehend and retain
  information better and continue to learn how to
  learn more efficiently and effectively.

103
Assessment Methods

• Direct Assessment Methods
• Directly determine whether students have mastered
  the content of their academic programs.
• Require students to display their knowledge and
  skills as they respond to the instrument itself
  (i.e. objective tests, essays, presentation, and
  classroom assignments).

104

• Direct assessment methods include
• Standardized exams
• Locally developed exams
• Oral exams
• Portfolios (work collected over time)
• Performance appraisal
• Oral presentations
• Projects, demonstrations, case studies,
  simulations
• Capstone experience (embodied in capstone
  courses)
• Juried activities
• Evaluation of field work.
• Behavioral observations

105

• Assessment Methods
• II. Indirect Assessment Methods
• Ask students to reflect on their learning, what
  they have learned and experienced, rather than to
  demonstrate it (i.e. surveys and interviews).
• Details about instructional or curricular
  strengths that can not be provided by direct
  methods alone.

106
Indirect assessment methods include

• Written surveys and questionnaires
• Entering students
• Current students
• Graduating seniors
• Faculty
• Alumni
• Employers
• Parents
• Exit interviews
• Focus groups
• External examiner
• Archival records

107
Planning and Managing test

• Determine the test outcomes
• Determine the Learning Content to test
• Develop Test Discriminatory Table
• Develop Item / Question
• Review of Item / Question
• Analysis of Item
• Difficulty Index
• Discrimination Index
• Learning Taxonomy
• Administration of Testing
• Preparatory Stage
• Implementation Stage
• Coordination Stage

108
Test development principles

• Validity
• Generally content validity and the responsibility
  of subject matter expert
• Reliability
• The extent to which scores are consistent across
  different scorers.
• Fairness
• The extent to which score interpretations are
  valid and reliable regardless of race, origin,
  gender, disability etc.

109
Assessment of Projects
Definition Objective

• Implementation
• Coordination Committee
• Supervisor Supervisory Guidelines
• Supervision (Log book, Seminar, Report)

Mgt. Implementation Of Project

• Assessment
• Seminar presentation
• Report
• Grading Passing Criteria

110
Statistical application

• Assessment is the integration of both
  quantitative and qualitative data to provide
  information on he nature of the learner, what is
  learned and how it is learned.
• Statistics Analysis
• Mean
• Median
• Mod
• Standard Deviation
• Variance
• Presentation
• Cummulative Frequency
• Histogram / Bar Chart
• Normalisation
• Making Decision

111
Grading and Reporting

• Why Grading?
• Grading Principles
• Easily understood by learners
• Well informed to learners
• Fair to all
• Support and strengten the learning proses
• As widely acceptable as possible
• Norm Reference or Criterion Based Grading?

112
Summary

• Academically Speaking,
• The Vision of University depends on the Mission
  depends on the Strategy depends on the Objectives
  depends on Measures depends on Initiatives
• University Excellence
• Graduate Excellence
• Programme Excelence
• Programme Design Excellence
• Subject Design Excellence
• Delivery Excellence
• Assessment Excellence
• Therefore everybody needs to be excellent in
  practising assessment

113
Linking to

• LO writing
• TOS
• Grading

114
Thank you