Learning Objectives and Classroom Assessment

1
Learning Objectives and Classroom Assessment

• Jeff Froyd, Texas AM University

2
Workshop Presenter

• Jeff Froyd, Director of Academic Development
• Educational Achievement Division, College of
  Engineering, Texas AM University
• Project Director, Foundation Coalition

3
Acknowledgement

• Russ Pimmel, NSF Program Officer
• Former Professor of Electrical Engineering at the
  University of Alabama
• Assembled much of the material for similar
  workshops

4
Objectives Participants will gain experience in

• Describing the rationale for preparing and using
  learning objectives in an individual course
• Preparing specifications for high quality
  learning objectives
• Writing learning objectives for a single course
• Preparing specifications for assessment
  processes/tools
• Generating alternative assessment processes/tools
  for a single course
• Selecting assessment processes/tools for a single
  course

5
Agenda

• Background for learning objectives
• Specifications for learning objectives
• Writing learning objectives
• Background for classroom assessment
• Specifications for assessment processes/tools
• Background on alternatives for assessment
  processes/tools
• Generating alternatives for assessment
  processes/tools
• Selecting assessment processes/tools
• Review workshop activities

6
Agenda

• Background for learning objectives
• Specifications for learning objectives
• Requirements for specifications
• Team exercise develop specifications for
  learning objectives
• Workshop exercise improve specifications for
  learning objectives
• Writing learning objectives
• Individual exercise write objectives for a
  course or a portion of a course
• Team exercise review individual objectives
• Workshop exercise develop a list of suggestions
  for writing learning objectives
• Individual exercises revise objectives Team
  exercise review individual objectives
• Workshop exercise reflection

7
Agenda

• Background for classroom assessment
• Specifications for assessment processes/tools
• Team exercise develop a set of specifications
  for assessment processes/tools
• Workshop exercise improve sets of specifications
  for assessment processes/tools
• Generating alternatives for assessment
  processes/tools
• Team exercise select some learning objectives
  and generate alternative assessment
  processes/tools
• Background on alternatives for assessment
  processes/tools
• Selecting assessment processes/tools
• Individual exercise select a set of learning
  objectives and generate alternative assessment
  processes/tools
• Individual exercise select one or more
  assessment processes/tools that you would use in
  your course
• Team exercise share and review choices of
  assessment processes/tools
• Review workshop activities

8
Learning Objectives

• Background for Learning Objectives

Session Objective At the end of the session,
participants will describe themselves as more
confident in their ability to hold productive
conversations with their colleagues regarding the
place and importance of learning objectives in
the teaching-learning process
9
Stakeholders for Learning Objectives

• Who are the stakeholders in conversations about
  preparing and applying learning objectives?
• Faculty
• Students
• Employers
• Accreditation organizations

10
EC Program Outcomes

• (a) an ability to apply knowledge of mathematics,
  science, and engineering
• (b) an ability to design and conduct experiments,
  as well as to analyze and interpret data
• (c) an ability to design a system, component, or
  process to meet desired needs
• (d) an ability to function on multi-disciplinary
  teams
• (e) an ability to identify, formulate, and solve
  engineering problems
• (f) an understanding of professional and ethical
  responsibility
• (g) an ability to communicate effectively
• (h) the broad education necessary to understand
  the impact of engineering solutions in a global
  and societal context
• (i) a recognition of the need for, and an ability
  to engage in life-long learning
• (j) a knowledge of contemporary issues
• (k) an ability to use the techniques, skills, and
  modern engineering tools necessary for
  engineering practice.

11
Blooms Taxonomy of Cognitive Learning

• Knowledge defines, recalls, matches, reproduces
• Comprehension explains, gives examples
• Application discovering, assessing, computing
• Analysis breaking down, organizing, inferring
• Synthesis creating, putting together
• Evaluation appraising, judging, selecting

Western Michigan University, 25 October 2002,
Kalamazoo, Michigan
12
Purpose of Learning Objectives

• Communicate expectations for a course
• Provide a context for what will be learned

http//ublib.buffalo.edu/libraries/projects/tlr/im
portance.html
13
Objectives and Students

• Objectives help students
• Clarify their personal goals
• Provide framework for measuring their success.
• Reduce their anxiety
• Improve their studying effectiveness
• Objectives help instructors
• Guide preparation of classroom material
• Make homework assignments
• Aid in test design

Source - http//ublib.buffalo.edu/libraries/projec
ts/tlr/importance.html
14
Strategies For Workshop Teams

• Be positive, supportive, and cooperative
• Limit critical or negative comments
• At least 5 positive comments for every negative
  comment
• Be brief and concise in discussions
• Avoid lengthy comments, stories, or arguments
• Stay focused

15
Team Roles

• Assign team roles follow through on
  responsibilities
• Coordinator -- Coordinates discussion develops
  consensus
• Recorder -- Writes down the ideas reports them
• Gatekeeper -- Keeps the team on the subject
• Timer -- Makes sure the team stays on schedule
• With smaller teams combine gatekeeper timer

16
Workshop Team Roles

• For first exercise
• Coordinator Individual with largest class last
  semester
• Recorder/Reporter Individual on left of
  coordinator
• Gatekeeper/timer -- Individual on left of
  recorder
• Timer -- Individual on left of gatekeeper
• Roles rotate clockwise on subsequent exercise

17
Team Exercise

• Form teams of four people
• Time 5 minutes
• Develop at least four (4) advantages and four (4)
  disadvantages of preparing learning objectives
  for a course.

18
Team Exercise

• Advantages
• Communication with students expectations and what
  they need to learn
• Offers focus in preparing material to accomplish
  goals
• Evaluation tool
• Tie a course to program or curriculum
• Clarifying goals to students seeks buy-in and
  self-assessment
• ABET Assessment tool
• Stating goals facilitates communicates with
  colleagues for better coordination
• Written goals facilitates communications with
  employers and other external groups
• Makes course easier to transfer among faculty
  members because much work is already done

• Disadvantages
• Could be too narrow or focused
• Explicitly stating learning objectives takes
  (too?) much time
• Busy work for faculty
• Could turn into curriculum nightmare
• Poorly written objectives dont help
• Could be cumbersome, ambiguous
• Could be more difficult to measure

19
Learning Objectives

• Preparing Specifications for Learning Objectives

• Session Objectives
• At the end of the session, participants will
• Write specifications for learning objectives
• Describe themselves as more confident in their
  ability to describe quality learning objectives.

20
Form of Learning Objectives

• Write objectives as student outcome statements
• Objectives should answer the questions
• "What must students do to prove that they have
  succeeded?"
• "What should students be able to do as a
  consequence of instruction?"

Source - http//ublib.buffalo.edu/libraries/projec
ts/tlr/importance.html
21
Elements of an Objective

• Objective must contain three basic elements
• Verb describing an observable action
• Conditions of this action
• When given x you will be able to..."
• Level of acceptable performance

Source - http//ublib.buffalo.edu/libraries/projec
ts/tlr/importance.html
22
Verbs for Objectives

• Verbs for constructing concrete objectives
• analyze compute classify collaborate
• compare appreciate contrast define
• demonstrate direct derive designate
• discuss display evaluate know
• identify infer integrate interpret
• justify list understand organize
• grasp report respond solicit
• state synthesize name explain

Modified from http//www.brown.edu/Administration/
Sheridan_Center/pubs/syllabus.htmlcoura
23
Learning Objectives Verbs for Categories in
Blooms Taxonomy

• Knowledge
• Define, describe, list, reproduce, enumerate
• Comprehension
• Classify, explain, discuss, give example,
  summarize
• Application
• Determine, develop, compute, chart, utilize
• Analysis
• Correlate, diagram, distinguish, outline, infer
• Evaluation
• Compare contrast, critique, justify, conclude
• Synthesis
• Adapt, combine, compare, contrast, design,
  generate

24
Complex Versus Simple Objectives

• One complex objective versus several simple ones?
• High-level versus low-level objectives
• Example
• One complex objective (4 or 5 weeks of classes)
• Given a verbal description of a digital module,
  develop an implementation using any of 7
  different logic devices
• 15 to 20 simpler objectives (1 or 2 per class)
• Given a verbal description, draw the truth
  table
• Given a truth table, obtain a minimum-cost
  equation
• Draw the the NAND-gate implementation for an
  equation

25
Comparison Of Complex Simple Objectives

• Multiple simple objectives
• More manageable chunks for students
• Explicit objective(s) for each class
• Simple (more manageable) homework problems and
  test questions
• Single complex objective
• Student's attention directed to the overall
  process
• May lead to higher level learning
• Students must deal with complexity
• Students must subdivide problem on their own

26
Advantages Of Simple Objectives

• Advantages of simple objectives are more
  important in
• Large classes rather than small classes
• Introductory courses rather than advanced courses

27
How To Deal With Understand In Objectives

• How do you write objectives when you want
  students to understand a complex concept,
  system, or process
• Identify specific tasks that indicate
  understanding
• Specify objectives for each task
• Similar comments apply to know, appreciate,
  value

28
How To Deal With Understand In Objectives --
Example

• In our computer architecture course we want
  students to understand a sample architecture
  made up of several modules
• What would students be able to do if they
  understood
• Objectives
• Students should be able to identify
• All the modules and interconnecting signals
• Modules involved in a given system-level
  operation
• Output values for a given input values for each
  module
• Sub-module changes given a system level change

29
Writing Objectives Piecemeal Approach

• Writing low-level objectives for a whole course
  may be overwhelming
• Use a piecemeal approach
• Write your lectures and define the homework as
  usual
• After each class -- write down what you expect
  the students to be able to do
• These become a list of objectives
• Give them to the students before each exam
• Use them to write the exam
• As semester progresses -- may become comfortable
  writing the objectives before you prepare your
  lecture

30
Evolving Objectives

• In a 3-credit semester course
• Russ Pimmel (UA, now NSF) started with over 100
  objectives
• Four offerings later -- down to about 50
• Eliminated peripheral stuff that was not
  central
• Broadening, informational, perspective material

31
Objectives and Homework Assignments

• Homework assignments should match objectives
• Students need to practice and explore the skills,
  knowledge, and attitudes defined in objectives
• Frequently require supplementary homework
  problems
• In some of my courses
• 1/3 of homework is from textbook
• Rest are supplementary problems
• With well defined objectives
• Writing homework problems is straightforward
• Same is true for exam questions

32
Students Use Of Objectives

• Survey in 400-level required course
• Did you find the objectives helpful?
• Yes --- 52
• No ---- 48
• Did you read the objectives?
• Frequently ------ 22
• Occasionally -- 37
• Never ------------- 41

33
Team Exercise -- Guidelines For Learning
Objectives

• Task
• Write 3 or 5 guidelines for good learning
  objectives
• What are the common features?
• What should objectives look like?
• Think of guidelines as specifications
• Methodology
• Brain storm individually -- 2 minutes
• Establish consensus as a team -- 5 minutes
• Report team results -- 3 minutes
• Revise guidelines as a team-- 2 minutes

34
Team Exercise

• Must be testable and measurable
• Achievable
• Clearly and precisely articulated
• Appropriate to course and audience
• Relate to program objectives
• Linked to course outcomes to allow for assessment
• Simple, one sentence, common format
• Simple better than complex
• Should be specific and unambiguous
• Relate to topic coverage

35
Learning Objectives

• Preparing Learning Objectives

• Session Objectives
• At the end of the session, participants will
• Write learning objectives for one or more courses
  that they teach
• Describe themselves as more confident in their
  ability to describe quality learning objectives.

36
EC Program Outcomes

• (a) an ability to apply knowledge of mathematics,
  science, and engineering
• (b) an ability to design and conduct experiments,
  as well as to analyze and interpret data
• (c) an ability to design a system, component, or
  process to meet desired needs
• (d) an ability to function on multi-disciplinary
  teams
• (e) an ability to identify, formulate, and solve
  engineering problems
• (f) an understanding of professional and ethical
  responsibility
• (g) an ability to communicate effectively
• (h) the broad education necessary to understand
  the impact of engineering solutions in a global
  and societal context
• (i) a recognition of the need for, and an ability
  to engage in life-long learning
• (j) a knowledge of contemporary issues
• (k) an ability to use the techniques, skills, and
  modern engineering tools necessary for
  engineering practice.

37
Blooms Taxonomy of Cognitive Learning

• Knowledge defines, recalls, matches, reproduces
• Comprehension explains, gives examples
• Application discovering, assessing, computing
• Analysis breaking down, organizing, inferring
• Synthesis creating, putting together
• Evaluation appraising, judging, selecting

Western Michigan University, 25 October 2002,
Kalamazoo, Michigan
38
Challenges in Engineering Education

• Challenges
• Challenge of lifelong learning
• Challenge of problem solving
• Challenge of engineering design
• Challenge of transfer

39
LASSI SCALE ENGR111 (Mean) CVEN349 (Mean) Significance
Skill Component
Information Processing 60.68 60.29 0.930
Test Strategies 64.33 63.27 0.794
Selecting Main Ideas 55.18 59.29 0.342
Will Component
Anxiety 60.52 67.12 0.147
Attitude 42.47 34.56 0.080
Motivation 63.30 59.29 0.397
Self-regulation Component
Concentration 61.31 54.56 0.144
Self-testing 52.47 37.92 0.006
Study Aids 60.24 45.04 0.005
Time Management 55.23 47.65 0.134
40
Lifelong Learning at Penn State
Self-Directed Learning Readiness Survey (SDLRS),
Guglielmino Associates, http//www.guglielmino73
4.com/prod01.htm, March 2003. 27
Although the data suggest a slight upward upward
trend, the trend proved not to be statistically
significant based upon an analysis of variance
(ANOVA). Thus the cross-sectional study did not
find evidence of an increase in readiness for
self-directed learning, even for students in the
later semesters who are taking elective courses
and their capstone courses. Litzinger, T., Wise,
J., Lee, S., and Bjorklund, S. (2003) Assessing
Readiness for Self-directed Learning,
Proceedings, ASEE Annual Conference
41
Challenge of Problem Solving

• Despite individual professors dedication and
  efforts to develop problem solving skill,
  general problem solving skill was not developed
  in the four years in our undergraduate program.
  Students graduated showing the same inability
  that they had when they started the program. Some
  could not create hypotheses some misread problem
  statements. During the four-year undergraduate
  engineering program studied, 1974-1978, the
  students had worked over 3000 homework problems,
  they had observed about 1000 sample solutions
  being worked on the board by either the teacher
  or by peers, and they had worked many open-ended
  problems. In other words, they showed no
  improvement in problem solving skills despite the
  best intentions of their instructors.

Woods, D. et al (1997) Developing Problem
Solving Skills The McMaster Problem Solving
Program, Journal of Engineering Education,
42
Challenge of Problem Solving

• Ineffective approach 1. give the students
  open-ended problems to solve This, we now see,
  is ineffective because the students get little
  feedback about the process steps, they tend to
  reinforce bad habits, they do not know what
  processes they should be using and they resort to
  trying to collect sample solutions and match past
  memorized sample solutions to new problem
  situations.

43
Challenge of Problem Solving

• Ineffective approach 2 Show them how you solve
  problems by working many problems on the board
  and handing out many sample solutions
• This, we now see, is ineffective because teachers
  know too much. Teachers demonstrate "exercise
  solving". Teachers do not make mistakes they do
  not struggle to figure out what the problem
  really is. They work forwards not backwards from
  the goal. They do not demonstrate the "problem
  solving" process they demonstrate the "exercise
  solving" process. If they did demonstrate
  "problem solving" with all its mistakes and
  trials, the students would brand the teacher as
  incompetent. We know we tried!

44
Challenge of Problem Solving

• Ineffective approach 3 Have students solve
  problems on the board
• Different students use different approaches to
  solving problems what works for one won't work
  for others. When we used this method as a
  research tool, the students reported "we learned
  nothing to help us solve problems by watching
  Jim, Sue and Brad solve those problems!"

45
Challenge of Problem Solving

• Through four research projects we identified why
  and how these and other teaching methods failed
  to develop process skills and which methods were
  successful in developing the skills
• Woods, D.R., J.D. Wright, T.W. Hoffman, R.K.
  Swartman and I.D. Doig (1975) "Teaching Problem
  Solving Skills," Annals of Engineering Education,
  1, 1, 238-243.
• Woods, D.R. et al. (1979) "Major Challenges to
  Teaching Problem Solving" Annals of Engineering
  Education, 70, No. 3 p. 277 to 284, 1979 and "56
  Challenges to Teaching Problem Solving" CHEM 13
  News no. 155 (1985).
• Woods, D.R. (1993a) "Problem solving - where are
  we now?" J. College Science Teaching, 22,
  312-314.
• Woods, D.R. (1993b) "Problem solving - what
  doesn't seem to work," J. College Science
  Teaching, 23, 57-58.
• Woods, D.R. (1993c) "New Approaches for
  developing problem solving skills," J. College
  Science Teaching, 23, 157-158.

46
Challenge of Engineering Design

• The literature is filled with positive comments
  from students, instructors, and industrial
  sponsors who have participated in capstone design
  courses. The vast majority of participants feel
  that the course benefited all involved.
• The nature of capstone design courses, however,
  often leads to a purely subjective evaluation
  with little or no hard evidence of actual
  benefits. Born, for example, does not attempt to
  prove the value of senior level design courses.
  He simply states that he is convinced from his
  experiences that such courses are valuable. Other
  educators have similar feelings as to the
  relative costs and benefits of capstone design
  courses.

Dutson, A.J., Todd, R.H., Magleby, S.P.,
Sorensen, C.D., (1997) A Review of Literature on
Teaching Engineering Design Through
Project-Oriented Capstone Courses. Journal of
Engineering Education
47
Challenge of Transfer

• Researches posed this problem to people.
• "Suppose you are a doctor faced with a patient
  who has a malignant tumor in his stomach. It is
  impossible to operate on the patient, but unless
  the tumor is destroyed the patient will die.
  There is a kind of ray that can be used to
  destroy the tumor. If the rays reach the tumor
  all at once at a sufficiently high intensity, the
  tumor will be destroyed. Unfortunately, at this
  intensity the healthy tissue that the rays pass
  through on the way to the tumor will also be
  destroyed. At lower intensities the rays are
  harmless to healthy tissue, but they will not
  affect the tumor either. What type of procedure
  might be used to destroy the tumor with the rays,
  and at the same time avoid destroying the health
  tissue?"

48
Challenge of Transfer

• Consider the following story
• "A small country was ruled from a strong fortress
  by a dictator. The fortress was situated in the
  middle of the country, surrounded by farms and
  villages. Many roads led to the fortress through
  the countryside. A rebel general vowed to
  capture the fortress. The general knew that an
  attack by his entire army would capture the
  fortress. He gathered his army at the head of
  one of the roads, ready to launch a full-scale
  direct attack. However, the general then learned
  that the dictator had planted mines on each of
  the roads. The mines were set so that small
  bodies of men could pass over them safely, since
  the dictator need to move his troops and workers
  to and from the fortress. However, any large
  force would detonate the mines. Not only would
  this blow up the road, but it would also destroy
  many neighboring villages. It therefore seemed
  impossible to capture the fortress. However, the
  general devised a simple plan. He divided his
  army into small groups and dispatched each group
  to the head of a different road. When all was
  ready he gave the signal and each group marched
  down a different road. Each group continued down
  it road to the fortress at the same time. In
  this way, the general captured the fortress and
  overthrew the dictator."

49
Challenge of Transfer

• After the subjects read and summarized this
  story, they were asked to solve the tumor problem
  under the guise of a separate experiment.
• Given the clear analogy, you might think that
  performance would be near ceiling. Surprisingly,
  only 30 of the subjects offered a convergence
  solution.
• Moreover, when these same subjects were given the
  suggestion that they should use the General
  story, 80 provided a convergence solution.
• This finding demonstrates that half the subjects
  could apply the General story to the tumor
  problem when they were instructed to but did not
  do so on their own.

50
Focusing Activity (8 minutes)

• INDIVIDUALLY use 3 minutes to write your
  description of learning, what it is, what it
  looks like, how you might recognize when it has
  occurred, etc.
• AS A PAIR use 5 minutes to discuss descriptions
  with someone sitting next to you. If you have
  additional time, develop a consensus description
  of learning.

51
Focusing Activity

• ??

52
Individual Exercise -- Writing Learning Objectives

• Individually write a set of objectives for a
  topic representing a few classes
• Something that you recently did in class
• Follow your teams guidelines
• Questions to consider about your objectives
• Do they define student behavior?
• Are they observable, measurable?
• Can you write homework exam problems?
• Are they consistent with the instructors intent?
• Time 15 minutes

53
Team Exercise -- Reviewing Learning Objectives

• Review each others objectives
• Questions to consider in reviewing objectives
• Do they follow your teams guidelines
• Do they define student behavior?
• Are they observable, measurable?
• Can you write homework problems exam questions?
• Are they consistent with the instructors intent?
• 15 minutes

54
Individual Exercise -- Revising Your Learning
Objectives

• Rewrite your learning objectives based on your
  teams review
• Report on biggest improvement
• 10 minutes

55
Biggest Improvement

• Put quantifiable goals that students would
  achieve
• Clarify my objectives
• Classify learning objectives to go from basic to
  specific
• Make a different level of objectives from whole
  course to chapter to each lecture
• Make few higher level objective to organize
  multiple lower level objectives to support
• Identify more complex goals and separate into
  simpler components for teaching communications
• Improve clarity
• Make more understandable to the student
• Use proper verbiage to clarify

56
Minute Paper

• Write a one-sentence answer to the following
  question
• What is the muddiest point about learning
  objectives?
• (What is the most confusing point?)

57
Muddiest Point

• How to incorporate those that relate to attitude
  and enthusiasm for lifelong learning
• Why do we spend time writing things that we do
  and we do best?
• Answer to my question about the level and number
  of learning objectives that are appropriate for
  publication in my syllabus. Are there
  intermediate not published objectives?
• How to relate a given objective to Blooms
  Taxonomy level, and how important is it to do so?
• Ways of measuring each learning objective
• How many objectives should be used level of
  detail then how do we practically measure them?
• Being brief vs. Being precise and accurate
• Being able to link objectives to larger program
  outcomes or goals
• Organizing subject matter to achieve learning
  objectives

58
Team Exercise -- Reflection on Learning Objectives

• Assume that you are a debate team
• Write the single best pro and con arguments for
  the statement
• Using learning objectives improves student
  leaning.

59

• Classroom Assessment

60
Feedback on Learning Objectives

• ??

61
Agenda

• Background for classroom assessment
• Specifications for assessment processes/tools
• Team exercise develop a set of specifications
  for assessment processes/tools
• Workshop exercise improve sets of specifications
  for assessment processes/tools
• Generating alternatives for assessment
  processes/tools
• Team exercises select some learning objectives
  and generate alternative assessment
  processes/tools
• Background on alternatives for assessment
  processes/tools
• Selecting assessment processes/tools
• Individual exercise select a set of learning
  objectives and generate alternative assessment
  processes/tools
• Individual exercise select one or more
  assessment processes/tools that you would use in
  your course
• Team exercise share and review choices of
  assessment processes/tools
• Review workshop activities

62
Classroom Assessment

• Background for Classroom Assessment

Session Objective At the end of the session,
participants will describe themselves as more
confident in their ability to hold productive
conversations with their colleagues regarding the
place and importance of assessment instruments
and processes in the teaching-learning process
63
Pre-Assessment

• Exercise
• Classroom Assessment
• Individual
• Write a one-sentence answer to the following
  question
• What can you do in the last few minutes of class
  to determine how well your class learned what you
  taught that day?

64
Classroom Assessment

• At end of session, participants will be able to
• Define several classroom assessment tools
• Discuss the importance of using classroom tools
  in the teaching-learning process
• Write assessment tools for their classes

65
Resources on the Web

• www.hcc.hawaii.edu/intranet/committees/FacDevCom/g
  uidebk/teachtip/assess-1.htm
• www.hcc.hawaii.edu/intranet/committees/FacDevCom/g
  uidebk/teachtip/assess-2.htm
• www.siue.edu/deder/assess/catmain.html

66
Types Of Assessment

• Assessment
• Used in many contexts
• Used for several different foci
• Classroom
• Teacher focus concerns your performance
• Student focus concerns their individual
  performances
• Program
• Alumni (as a group) focus ABET Criterion 2
• Graduates (as a group) focus ABET Criterion 3
• Individual graduate focus Comprehensive exams

67
Types Of Assessment

• Classroom, teacher focus
• Question -- How effective was a lecture,
  assignment, lab?
• Tools -- One-minute paper, student survey
• Classroom, student focus
• Question Did a specific student achieve the
  learning objectives?
• Tools -- Exams, reports, presentation
• Program, graduates (as a group) focus
• Question -- How well did a group of students
  achieve a set of objectives (outcomes) in a
  program or course?
• Tools -- Standardized tests, design project
  report analysis

68
Classroom Assessment

• Two fundamental questions
• How well are learners learning?
• How effectively are teachers teaching?
• Deals with better learning and more effective
  teaching
• Provides feedback about effectiveness as teachers
• How students learn
• How they respond to particular teaching
  approaches.
• Gives students a measure of their progress as
  learners

Modified from www.hcc.hawaii.edu/intranet/committe
es/FacDevCom/guidebk/teachtip/assess-1.htm
69
Incorrect Assumptions About Teaching Learning

• Instructors assume students learn what they teach
  
• Tests, concept inventories, and term papers
  provide disappointing evidence to the contrary
• Students have not learned as much or as well as
  expected
• Gaps between what was taught and what was learned
• Sometimes considerable gaps
• Instructors notice gaps too late to remedy the
  problems
• Classroom assessment can uncover gaps earlier

www.hcc.hawaii.edu/intranet/committees/FacDevCom/g
uidebk/teachtip/assess-1.htm
70
Classroom AssessmentGetting Started

• Planning
• Select one, and only one, of your classes
• Choose a simple and quick technique
• Implementing
• Make sure the students understand the procedure
• Analyze students responses as soon as possible
• Responding -- Close the feedback loop
• Tell students what you learned and what you will
  do about it - motivates students to become
  actively involved

www.hcc.hawaii.edu/intranet/committees/FacDevCom/g
uidebk/teachtip/assess-1.htm
71
Classroom Assessment Five Suggestions

• Dont use any technique that does not appeal to
  you
• Don't make it into a self-inflicted chore or
  burden.
• Try it yourself before you use it with students
• Allow more time than you think you will need
• To carry out the assessment
• To respond to it
• Make sure to "close the loop"
• Let students know
• What you learned from their feedback
• How you and they can use that information to
  improve learning

www.hcc.hawaii.edu/intranet/committees/FacDevCom/g
uidebk/teachtip/assess-1.htm
72
Team Exercise

• Process
• Brain storm individually -- 1 minutes
• Establish consensus as a team -- 4 minutes
• Task Write 2 to 4 guidelines for good classroom
  assessment tools (Think of guidelines as
  specifications)
• What are the common features?
• What should they look like?
• Do all guidelines have to apply to all tools?

73
Team Exercise

• Tool should be efficient use of time for
  instructor and class
• Easy for students to use
• Can be used across different classes and
  situations
• Built into teaching methodology and doesnt look
  like an add-on
• Tied to learning objectives or limited number of
  topics
• Flexible, not dependent on particular technology
• Done repeatedly throughout semester
• Easy for faculty to administer and evaluate
• Self-documenting, easy to show results to others
• Clear and precise instructions

74
Team Exercise

• Process
• Brain storm individually -- 2 minutes
• Generate different ideas as a team -- 3 minutes
• Task Generate as many possible good classroom
  assessment tools that might satisfy your team
  guidelines
• Generate different ideas

75
Team Exercise

• Muddiest point
• Quizzes on specific objectives
• What is clear?/What is unclear?
• What did you learn today?
• Strengths/Improvements/Insights (SII)
• Self or peer assessment
• Who did you share X with today?
• Weekly feedback on homework instruction what
  was clear? What was unclear?
• Continuous Improvement What was beneficial?
  What would you change?

76
Team Exercise

• Process
• Brain storm individually -- 2 minutes
• Generate different ideas as a team -- 3 minutes
• Task Generate as many possible good classroom
  assessment tools where the student work product
  is primarily graphical and the tools might
  satisfy your team guidelines
• Generate different ideas

77
Team Exercise

• List topics with smiling/frowning faces
• Flow charts/flow sheets
• Process flow of your problem solving process
• Road map to get from A to B
• Bar graph of confidence related to specific
  objective
• Free body diagram
• Web of association to a particular term
• Story board of set up or explanation of
  particular concept
• Fit concept into network or tree of concepts
• Draw a concept

78
Examples of Assessment Techniques

• Background Knowledge Probe
• Students respond to short-answers or
  multiple-choice questions
• General information on their level of preparation
• Minute Paper (most widely used)
• Students write brief response to
• "What was the most important thing you learned
  during this class?"
• What important question remains unanswered?

www.hcc.hawaii.edu/intranet/committees/FacDevCom/g
uidebk/teachtip/assess-2.htm
79
Examples of Assessment Techniques (Cont.)

• Muddiest Point (simplest technique, remarkably
  efficient)
• Students jot down a quick response to one
• "What was the muddiest point in ....... ?
• A lecture, a discussion, a homework assignment
• One-Sentence Summary
• Students answer the questions "Who does what to
  whom, when, where, how, and why?" (WDWWWWHW)
• Synthesize answers into a simple, informative,
  grammatical sentence.

www.hcc.hawaii.edu/intranet/committees/FacDevCom/g
uidebk/teachtip/assess-2.htm
80
Examples of Assessment Techniques (Cont.)

• What's the Principle?
• Students state the principle that best applies to
  a few problems

www.hcc.hawaii.edu/intranet/committees/FacDevCom/g
uidebk/teachtip/assess-2.htm
81
Concept Map

• A concept map is a set of nodes that represent
  concepts connected by a labeled links that
  describe a link between concepts.

Concept A
Describe how concept A and concept B are related?
Concept B
82
Team ExerciseBuilding a Concept Map

• Start with a subset of the concepts on the
  following page and construct a concept map that
  shows the concepts you have selected and how they
  are related.
• Exchange concept maps and share insights

83
FeedbackDerivativeFinite Element
AnalysisIntegralLinear MomentumAngular
MomentumEnergyInterestMassIdeal Gas
LawFicks First LawFicks Second LawVectors
Dot ProductVectors Cross ProductOrdinary
Differential EquationsKirchoffs Voltage
LawSecond Law of ThermodynamicsKirchoffs
Current LawModelingProblem-SolvingForceOhms
LawResistanceComplex NumbersLogarithmic
FunctionElectric FluxDecision
TheoryDivergenceIndirect CostCapacitanceBendin
g MomentFeedback
First Law of ThermodynamicsEntropyHeatElectric
FieldMagnetic FieldPartial Differential
EquationsDeterminantsReturn on
InvestmentPhasorsBrainstormingExponential
FunctionConductivityChemical KineticsSpecific
HeatElasticityMalleabilityPlasticityResiliency
PermittivityCurrentElectric PotentialCurlPres
entation SkillsDemocracyProfitDensityMolecule
PhaseShearRheology Frequency
ResponseEigenvalue, Eigenvector
Sinusoidal FunctionsWorkDisplacementVelocityAc
celerationResistivityLeadershipHess
LawZeroth Law of ThermodynamicsElectric
PotentialMagnetic FluxDesignMaxwells
EquationsPowerDuctilitySpring
ConstantStressStrainPartial DerivativePermeabi
lityChargeMagnetic PotentialGradientParagraph
Rate of ReturnFrequencyAtomRoot
LocusTorqueInductanceTorsionPolymerKinetic
Theory of Gases
84
Cowans Teaching Examples

• Bridge design
• Design and build two different bridges and grade
  on the lower performance design
• Problem-solving script
• Illustrate script for one type of problem, ask
  students to develop a script for another type of
  problem

Cowan, J. (1998) On Becoming an Innovative
University Teacher Reflection in Action.
Buckingham SRHE and Open University Press.
85

• Some Personal Observations

86
Relating Student Performance On Exams To
Objectives

• Write exam using objectives
• Select objectives for exam questions from list
• Many objectives -- test questions represent a
  sample
• Exam question may involve more than one objective
• Use some hard and some easy questions
• Identify questions ( objectives) a high
  percentage missed
• Review idea in class -- give additional work
• Modify lecture, reading, or homework for future
• Change the objective

87
Using Cooperative Learning In-class Exercises

• Examine students work during the in-class
  exercise
• If all have a good approach -- may be wasting
  time
• If all are lost -- may need more explanation
• If one-half to two-thirds have a good approach --
  level and pace are right
• Collect and show a few solutions to in-class
  exercises
• If all have correct approach -- may be wasting
  time
• If all are wrong -- may need more explanation
• If one or two are correct -- level and pace are
  right

88
One-Minute Papers

• Common questions
• What one thing should be changed about ____?
• What one thing should not be changed about ____?
• What do you think about ____?
• What is the muddiest point about ____ ?
• Ask about
• Course or lecture
• Text or chapter
• Assignment or test
• Teaching style or class activity

89
Some Colleagues Observations
90
Informal Assessment Techniques

• General Guidelines
• Keep them anonymous
• Use then frequently better feedback
• Close the loop
• Let students know results of the process
• David Cordes

91
Informal Assessment Techniques - Daily Activities

• One-minute paper
• At the end of the lecture, ask students for
• The most important topic that we covered today
• The one topic you are still confused about
• Single sheet of paper, no names
• Can read on the way back to the office
• Look for common problems
• Look for did they understand my focus?
• David Cordes

92
ME 360 - Plus / Delta Assessment 1

• On one side of sticky pad
• Put a in upper left hand corner
• What is something that worked well or made more
  sense in lab this week?
• On other side of sticky pad
• Put a D in upper left hand corner
• What is something that could have been done
  better in lab this week?
• Stick on the door on your way out
• Joey Parker

93
ME 372 - 1st Day of Class

• What are a valid set of units for a mass moment
  of inertia? (Dynamics concept)
• What is the difference between a capacitor and a
  resistor? (Circuits concept)
• What is the equation of the straight line that
  passes through the points X2, Y7 and X7, Y2?
  (Math concept)
• Joey Parker

94
Team Exercise

• Process
• Brain storm individually -- 2 minutes
• Establish consensus as a team -- 5 minutes
• Report team results -- 3 minutes
• Revise products as a team -- 2 minutes
• Task Write 3 to 5 guidelines for good classroom
  assessment tools (Think of guidelines as
  specifications)
• What are the common features?
• What should they look like?
• Do all guidelines have to apply to all tools?

95
Team Exercise

• Simple, quick, easy to administer and evaluate
• Provide immediate feedback to students they
  need to know youre using the results
• Tied to a specific topic or objective
• Gives usable, valuable information
• Identification of strengths and areas of
  improvement
• Give clear and precise instruction to the student
• Using graphical and textual feedback
• Result in action

96
Team Exercise

• Process
• Brain storm individually -- 2 minutes
• Establish consensus as a team -- 5 minutes
• Report team results -- 3 minutes
• Revise products as a team -- 2 minutes
• Task Generate as many possible good classroom
  assessment tools that might satisfy your team
  guidelines
• Generate different ideas that you have generated
  or have seen

97
Team Exercise

• Ask students to identify an application based on
  concept presented in class
• Person A teaches Person B, and then they write
  what was hard to convey and what was hard to
  learn
• Identify an objective for class and rate their
  comfort level with number or bar graph
• Ask student to give example to transfer example
  in class to another context
• Students examine 2-3 methods/opinions/processes
  for similar purposes, ask groups to analyze
  similarities and differences, solicit results,
  and then invent their own that combines
  methods/opinions/processes
• Ask students to identify concept based on a
  diagram.
• Ask students to write problem that could be
  solved by concept presented in class

98
Individual Exercise

• Individually write a set of assessment tools for
  a class or a topic representing a few classes
• Follow your guidelines
• Consider the following questions about your tool
• Can your students understand the task?
• Can your students do the task quickly?
• Can you analyze the results quickly?
• Can you summarize and report the results easily?
• Does it assess student learning?
• 5 minutes

99
Team Exercise

• Review each others objectives and assessment
  tools
• Consider the following questions
• Does the tool follow your guidelines?
• Can your students understand the task?
• Can your students do the task quickly?
• Can you analyze the results quickly?
• Can you summarize and report the results easily?
• Does it assess student learning?
• 15 minutes

100
Individual Exercise

• Rewrite your assessment tools based on your
  teams review
• Identify the major improvement
• 5 minutes

101
Minute Paper

• Write a one-sentence answer to the following
  question
• What is the muddiest point about classroom
  assessment?

102
Individual Exercise

• How can students be more involved in assessing
  each other, as well as the self-assessment we
  have largely concentrated on, and do this in
  reasonable time.
• How often is a reasonable amount of assessment?
  How much is too much?
• Going from creating an assessment tool to
  actually implementing the assessment.
• Incorporation of feedback will not be every time
  easy and straight forward
• Will students see the value of classroom
  assessment? Yes, with good approaches. Yes, with
  good feedback.
• Being able to design a clear, concise assessment
  tool that is both beneficial to students and
  professor

103
Team Exercise

• Assume that you are a debate team
• Write the single best pro and con arguments for
  the statement
• Using classroom assessment tools improves
  student learning.

104
Workshop Objectives Action Items

• Recall objectives
• At end of session, participants will be able to
  define, discuss, write
• Learning objectives for their courses
• Assessment tools for their classes
• Workshop provided a structure for experience in
  writing
• Learning objectives
• Assessment tools
• Your charge In one of your courses next
  semester use
• Learning objectives
• Classroom assessment tools

105

• Questions?

106

• Some Personal Observations