Approaches to Assessing Critical Thinking

1
Approaches to Assessing Critical Thinking
EDU Lunchtime Seminar
12 May 2000

• Kam-Por Kwan x6287 etkpkwan_at_
• Patrick Lai x6294 etktlai_at_

2
Seminar series

• 12 May 2000 Approaches to Assessing Critical
  Thinking
• 18 May 2000 Overcoming the Challenges in
  Developing Students Critical Thinking
• 9 June 2000 Measuring Students Critical
  Thinking Problems and Possible Solutions

3
Outline of todays session

• Overview of approaches to assessing critical
  thinking (CT)
• Examples
• Relative advantages and disadvantages
• Appropriateness for different purposes of CT
  assessment
• Applicability to your context
• PolyU Special Interest Group for Teaching and
  Assessing CT

4
Purposes of CT assessments

• Programme/subject evaluation evidence of success
  in promoting CT
• Authentic assessment
• evidence of effectiveness of CT instruction
• motivating students to be better at CT
• Diagnostic feedback on students CT level
• for designing instruction
• for faciliating students growth development
• Research into CT instructions
• Selecting students for admission

5
Definition of CT

• Ennis (1993 180) Critical thinking is
  reasonable reflective thinking focused on
  deciding what to believe or do
• APA Delpi Report (1990) Critical thinking is
  the process of purposeful, self-regulatory
  judgement. This process gives reasoned
  consideration to evidence, contexts,
  conceptualisations, methods and criteria.

6
Deciding what to believe or do

• Judge the credibility of sources
• Identify conclusions, reasons, and assumptions
• Judge the quality of an argument acceptability
  of its reasons, assumptions, and evidence
• Develop and defend an issue
• Ask appropriate clarification questions
• Plan experiments amd judge experimental designs
• Define terms in a way appropriate for the context
• Be open-minded
• Try to be well-informed
• Draw conclusions when warranted, but with caution

7
CT Dispositions and skills

• Two components of critical thinking (Facione ,
  1996)
• Affective dispositions the necessary attitudes
  and habits in approaching problems, questions,
  and issues for CT to take place
• Cognitive skills the mental ability needed for
  critical thinking

8
Critical thinking dispositions
Open-mindedness
Inquisitiveness
Systematicity
Dispositions towards CT
Analyticity
Cognitive maturity
Truth-seeking
CT Self confidence
9
Critical thinking skills
Inference
Analysis
Explanation
CT Skills
Interpretation
Evaluation
Self-regulation
Source Facione (1996)
10
Approaches to CT assessments

• Students self-reported skills / gains in CT
• Students performance on standardised published
  CT tests
• Instructor-constructed CT tests
• Student interviews / protocol analysis
• Students performance on tasks requiring CT
  skills
• performance assessment
• rubric for assessing CT in students work

11
Self-reported skills /gains in CT

• Students self reports on
• skills in CT
• gains or progress in CT skills
• Examples
• Items from CSEQ
• Items from IDEA
• Relatively easy and inexpensive to use
• Less reliable and valid

12
Published CT tests

• Standardised tests with well-established
  psychometric properties
• Examples
• The California Critical Thinking Test College
  level (CCTST)
• Cornell Critical Thinking Test, Level Z (1985)
• Watson-Glaser Critical Thinking Appraisal (1980)
• Reliability and validity established , often with
  norms for comparison
• Expensive to use, non subject-specific, implicit
  cultural and contextual assumptions

13
Instructor-constructed CT tests

• Developed by instructor
• Can be tailored for specific subject, context, or
  purpose
• Relatively costly to design (both in time and
  effort), reliability and validity need to be
  determined, difficult to compare across groups or
  contexts

14
Interview or protocol analysis

• Students given a problem to solve in an interview
• to find out ways person understands a concept or
  solves a problem
• Researcher analyses students explanation, and
  infers from that to form the researchers
  description
• Result consists of categories which describe
  different ways of understanding a concept or
  solving a problem
• These categories stand in a logical hierarchical
  relationship with one another and have
  implications for teaching

15
Interview method

• Students given a problem to solve in an interview
• They are asked to think aloud and tell the
  interviewer their thoughts as they are going
  through
• Probing/Prompted questions
• You mentioned about X. How does this occur?
• Do these symbols have any meaning to you?
• Why did you give up your original hypothesis?

16
Protocol / data analysis

• Verbatim transcription of audio-taped interviews
• Transcription printed and read through with a
  particular research question in mind
• Descriptive words or phrases attached to short
  sections of the transcript as a potential
  category
• Sections of the transcript that could fit into
  this potential category labelled by
  sub-categories
• Content of one excerpt compared and contrasted
  with the content of another similarly labelled
  excerpt
• Labelled excerpt organised into flow charts.

17
Performance assessments

• Students written and / oral descriptions
  analysed by the Holistic Critical Thinking
  Scoring Rubric (HCTSR) (Facione and Facione,
  1994)
• HCTSR measures
• critical thinking skills to reach a judicious
  judgment which include analysis, interpretation,
  evaluation, inference, explanation
• assesses dispositions to pursue reasons and
  evidence fair mindedly and open-mindedly to reach
  sound, objective decision
• HCTSR consists of four levels - level 4 signifies
  a critical thinker

18
Assessing CT under the 3-P model
19
CT Assessment for different purposes
20
SIG for teaching assessing CT

• Sharing of resources and experiences in teaching
  and assessing CT through
• e-newsletter
• regular meetings
• Collaboration in designing and evaluating CT
  instructions
• trying out innovative methods for CT instruction
• small-scale classroom research
• Contact persons
• Patrick Lai etktlai_at_polyu.edu.hk x6294
• K. P. Kwan etkpkwan_at_polyu.edu.hk x6287