Assessment Made Easy Maris Lown, EdD

1
Assessment Made EasyMaris Lown, EdD
2
Assessment answers this question How do we know
students are learning what we think we are
teaching?
3
Assessment belongs to the scholarship of
teaching. It is a study of situated teaching
practices . . . using particular forms of
research and knowledge. Litterest Tompkins,
(p. 10)
4
Types of Outcomes

• Learning

• Success

• Describes what we want students to do with the
  content they are learning.

• Identifies whether or not the course or program
  is effective.

5
A learning outcome reflects what students can do
with the content.
6
A Success Outcome reflectsa standard of judgment
that a discipline, a department or an institution
chooses to apply to students work.
7
Learning Outcomes

• Are measurable
• Begin with an action verb
• Stress higher-order thinking skills
• Are a learner oriented essential ability or skill
• Lead to the identification of assessment tools
• Identify what a student is able to do with the
  content
• Provide the foundation for success outcomes

8
Success Outcomes

• Are measurable
• Begin with a metric
• Stress a standard of judgment
• Are learner oriented
• Reported in terms of numerical scores that place
  them in an achievement level based on developed
  and tested criteria
• Build on learning outcomes

9
Outcomes

• Learning
• Describes what we want students to do wth the
  content.
• Includes a performance indicator
• Give a speech that is designed to convince the
  audience of a specified point of view.

• Success
• Identifies whether or not the course is
  effective.
• Includes a metric
• 85 of students will achieve a grade of C or
  better on the course final exam.

10
Comparison Objectives Outcomes

• Yields behaviors which are predictable.
• Primarily concerned with analysis.
• Usually evaluates discrete specific behaviors
• Evaluates behaviors that are most readily
  assessed and therefore may ignore behaviors that
  are more important.

• Yields behaviors which are unpredictable.
• Primarily concerned with synthesis.
• Usually evaluates abilities or skills.
• Evaluates outcomes of education which are more
  difficult to assess and are often ignored.

11
Assessment Direct Methods

• Integrated Learning

• Exams
• Collaborative projects
• Research papers
• Videotaped interactions
• Journaling
• Case studies
• Performances
• Standardized exams

• Portfolios
• Creations
• Projects
• Internships
• Capstone projects
• Oral examinations
• Concept Maps
• Service learning

12
Assessment Indirect Methods

• Attitudes, Perceptions, Experiences

• Student surveys
• Focus groups
• Course evaluations
• Faculty surveys
• CCSSE

• Interviews
• Graduate surveys
• Employer surveys
• Alumni surveys
• Student Satisfaction Inventory

13
Culture of Assessment
Hierarchy of Assessment of Student Learning
What evidence have you gathered documenting that
these changes have Improved Student Learning?
LEVEL 5
How have you Changed instruction/the curriculum
to Improve Teaching and Learning based on these
Data?
LEVEL 4
LEVEL 3
What Data have you Gathered Regarding Achievement
of PLOs/CLOs?
LEVEL 2
What Tools are you using to Assess Achievement of
these PLOs?
LEVEL 1
What are the Program/Course Learning Outcomes?
14
CURRICULUM DEVELOPMENT Institutional Learning
OutcomesProgram Learning Outcomes
Course Learning OutcomesUnit Learning
OutcomesLearning Activities
Evaluation Tools
15
(No Transcript)
16
Middle States

• Level 5 requires evidence of changes made due to
  assessment departmental minutes.
• We will be required to show a portfolio of
  assessment projects/artifacts.
• Must be faculty driven.
• How can we embed program assessment into Five
  Year Program Assessments?
• Will we be required to connect learning outcomes
  to success outcomes?

17
Program Review Process

• Connect to PLOs
• Can include direct assessment
• Papers,
• Exams
• Can include indirect assessment
• Student surveys
• Exit interviews
• Job placement
• Provide samples of program reviews

18
Institutional Learning Outcomes

• Skills/abilities with which every student
  graduates.
• Critical thinking
• Reading
• Writing

19
Program Learning Outcomes

• Connect to institutional learning outcomes.
• Includes three or more general education
  skills/abilities.
• Includes additional discipline skills/abilities.
• General education skills/abilities can be
  customized to discipline competencies.
• Students in programs take ENG 101 and 102.
• ETS Proficiency Profile.

20
Course Learning Outcomes

• Connect to program learning outcomes.
• Includes general education skills/abilities
  tailored to discipline.
• Focuses on discipline skills/abilities.
• How many?

21
Mapping Institutional Learning Outcomes to
Program Courses
Program College Wide Learning Outcomes College Wide Learning Outcomes College Wide Learning Outcomes College Wide Learning Outcomes
Program Name Critical Thinking Communication Teamwork Information Literacy
Biology AS Biology 101 Biology 102 English 101 Speech 115 English 102 Biology 102 History 101 English 101
Nursing AAS Nursing 101 Nursing 102 Nursing 203 Nursing 204 English 101 English 102 or Speech 115 English 102 Nursing 203 Nursing 204 English 101
Liberal Studies AA History 105 English 101 Speech 115 English 102 English 101
22
Mapping Program Learning Outcomes to
Program Courses
Name of Program _______BIOLOGY___________________
_____________________ Degree ____AA_______
Biology Program Courses Program Level Student Learning Outcomes Program Level Student Learning Outcomes Program Level Student Learning Outcomes Program Level Student Learning Outcomes Program Level Student Learning Outcomes Program Level Student Learning Outcomes
Biology Program Courses Explain basic biological principles and defend scientific perspectives Apply the scientific method to hypothetical or laboratory problems Evaluate the relationship between mankind and the natural world Utilize critical thinking skills to solve biologic problems. Communicate scientific material effectively in oral and written formats. Demonstrate information literacy skills and technology skills when gathering scientific information
BIO 111 X X X X X X
BIO 112 X X X X X X
BIO/CHE/PHY electives (2) X X X X X X
CHE-111 X X X X
CHE-112 X X X X
ENG-101 X X X
ENG-102 X X X
MAT 143 or 171 X X X
MAT 144 or 172 X X X
23
Mapping PLOs to CLOs, Political Science
PLOs CLOs CLOs CLOs
POLI 101 POLI 102 POLI 220 POLI 221
Explore political science methodology Explain basic political science concepts Examine the three branches of government and how they check and balance each other Analyze how states gain and lose power Investigate the origins of environmental problems
Compare and contrast political ideologies and theories of governance Interpret political ideologies including conservatism, liberalism, socialism, communism, and fascism Evaluate the electoral process and how candidates are selected to represent us at the federal level Compare and contrast the state system and the international system Analyze the goals of environmental interest groups
Analyze political concepts, theories, and ideologies in spoken and written form Discuss the strengths and weaknesses of the political process Compare and contrast the ideological differences of the two major parties Analyze the contributions of international institutions Evaluate environmental legislation including the EPS and the CWA
24
Learning outcomes drive the selection of learning
activities and assessment tools. They lay the
foundation for success outcomes.
25
Mapping CLOs to Learning Activities Assessment
Tools
Course CLO Learning Activity Assessment
NURS I Implement care using the Critical Thinking Model and the Nursing Process Scaffolded patient case study Nursing Care Plan
NURS II Assess human needs across the life span Simulation of a newborn in distress Newborn Assessment
NURS III Analyze therapeutic communication when interacting with patients YouTube examples of communication approaches Video taping patient interaction
NURS IV Implement care focusing on complex pathophysiology Simulation of a patient in multisystem failure and illness trajectory Concept Map
NURS IV Collaborate with members of the health care team when delivering care Management Case Studies Journal
26
Collecting Evidence

• Record data
• Identify changes made
• Collect additional data
• Discuss whether changes made a difference
• Can be memorialized in department minutes, annual
  meeting, or a faculty in-service

27
Institutional Effectiveness Model

• Strategic Plans
• AMP
• Technology Master Plan
• Facilities Master Plan
• Matrix
• Non-credit

28
Institutional Effectiveness Model

• Surveys
• Community Needs Assessments
• Community College Survey of Student Engagement
• Student Satisfaction Inventory

29
Institutional Effectiveness Model

• Academic Assessment
• Proficiency Profile (e.g.)
• Program Assessment
• Program Review
• Departmental Plans
• Data Books
• Graduate Surveys
• Employer Surveys
• Graduation Rates
• Transfer Rates

30
Institutional Effectiveness Model

• Non Credit Assessment
• Targeted outcomes assessment
• Administrative Services Assessment
• Targeted outcomes assessment
• Planning, assessment and budget
• Student Services Assessment
• Targeted outcomes assessment

31
No assertions without evidence
Middle States
32
And now the easy part!
33
Learning Management Systems

• Applications that ease assessment
• Use technology where applicable
• Grade books that provide assessment information

34
(No Transcript)
35
Portfolio Assessment

• Samples of students work over time
• Uses a rubric
• Offers a deep but narrower approach
• Gives a richer texture
• Can be resource intensive
• Can use a sample rather than the entire
  population
• Can be used for general education assessment
• Can be used for prospective employers

36
Capstone Assessment

• Paralegal Program Trial Notebook
• Incorporates all Program Learning Outcomes
• Used at the end of Litigation II
• Juried Review
• Uses a rubric related to outcomes
• Report that includes level 5 and trended data

37
Assessment of Majors Courses

• Select 2 to 4 courses in the discipline (majors
  courses, popular courses)
• Select assessment method
• Biology
• Assess BIO 111
• Assess BIO 112
• Assess BIO 105
• Assess Bio 106

38
Embedded Exam Questions

• Department develops questions that relate to
  student learning outcomes.
• Questions are different for each course but
  similar in that they assess the same behaviors,
  e.g.
• Analyze an organic compound, analyze an inorganic
  compound
• Identify normal lung sounds, identify abnormal
  lung sounds

39
Blue Print Assessment Tools
CLO Remembering Understanding Applying Analyzing Evaluating Creating
1 3 23 14 7, 30
2 15 6 20 21
3 7, 27 8, 24
4 13, 25 1 22
5 9 2 26 19
6 10 17, 28 4
7 15 11 18
8 16 5, 29 12
40

• Everything that counts is not countable and
  everything that is countable does not count.
• A. Einstein

41
What is not countable

• Athletics
• Work study
• Service learning

• FYE
• Trips
• Clubs
• Speakers
• Study abroad

42
Middle States, 2017

• Preparation, 2011-2014
• Assessment process
• Budget, planning, assessment
• Department plans
• Catalog
• Syllabi
• Preparation of SSR, 2014-2017

43
What gets measured get improved!
44
The central value of learning outcomes assessment
is using evidence and data to make teaching more
effective.
45
Whenever things sound easy,it turns out theres
one part you didnt hear.
D. Westlake