Scholarship of Teaching and Learning

1
(No Transcript)
2
Task
1. How would you describe your experience with
Scholarship of Teaching and Learning? Circle
one Extensive Moderate Hardly-Any None 2.
OUTCOMES List 3 topics or issues you hope to
learn more about at this workshop.
3
Continuum of Experience with SOTL A
Metaperspective
?
O
Origin of SOTL What Why of SOTL
Initiating a SOTL Program Faculty SOTL Projects
Bridges to Productivity Questions, Designs
Resources
4
A Conceptualization of Teaching Related
Activities
Teaching
Scholarly Teaching
Scholarship of Teaching Learning

• Scholarly Teaching
• Teaching that entails certain practices of
  classroom assessment and evidence gathering
  teaching that is informed not only by the latest
  ideas in the field but by current ideas about
  teaching generally and specifically in the field
  and teaching that invites peer collaboration or
  review.
• Scholarship of Teaching
• An act of intelligence or artistic creation
  becomes scholarship when it possesses at least
  three attributes it becomes public, it becomes
  an object of critical review and evaluation by
  members of ones community, and members of ones
  community begin to use, build upon, and develop
  those acts of mind and creation.
• (Definitions after Shulman)

5
An ineffective deed of teaching
Plane of the Venn Diagram
Teaching
A superior deed of scholarly teaching
Scholarly teaching
An excellent deed of teaching
Scholarship
Quality (or excellence) of any teaching-related
activity is an independent dimension not
represented in the plane of the Venn diagram.
6
Contemporary forms of research and creative
activity
Teaching
Scholarly teaching
Scholarship

• Distinction between scholarship of teaching
  and learning as a domain of academic achievement
  and as a campus program
• SOTL as academic achievement (yellow ellipse)
• SOTL as a campus program can span all
  teaching related activities (leftmost three
  ellipses).
• A SOTL program encourages movement in
  teaching-related activities toward the right in
  the diagram.

7
Why SOTL?

• Synthesis
• Building community
• Wisdom of practice
• Generativity
• Reserve capacity

8
Task

• What reasons, these or others, are most important
  on your campus?
• What reasons might be irrelevant to faculty on
  your campus?

9
Origin and Evolution of SOTL (In Progress)
Not a new idea (Hutchins 1923) Scholarship of
teaching coined (Boyer 1990)
Reform Concepts CA/CR (Angelo Cross 1993) New
epistemology (Schön 1995) New American scholar
(Rice 1996) Scholarship Assessed (Glassick et al
1997)
Implementing Strategies AAHE FFRR CASTL -
Carnegie Scholars - Teaching Academy -
Professional Societies - Knowledge Media Lab
Peer Review of Teaching
Recent Articulations Shulman, Change 31
(4) 1999 Hutchings Shulman Change 31 (5)
1999 Kreber Cranton JHE 71 (4) 2000 Richlin
in Kreber (Ed.) New Directions 86 2001
10

• What is certain is that most Ph.D.s become
  teachers and not productive scholars as well. A
  Ph.D. candidate who plans to be a teacher must
  know his field and its relation to the whole body
  of knowledge. It means too that he must be in
  touch with the most recent and most successful
  movements in undergraduate education, of which he
  now learns officially little or nothing. How
  should he learn about them? Not in my opinion by
  doing practice teaching upon the helpless
  undergraduate. Rather he should learn about them
  through seeing experiments carried on in
  undergraduate work by the members of the
  department in which he is studying for the
  degree
• Robert Maynard Hutchins

11
Origin and Evolution of SOTL (In Progress)
Not a new idea (Hutchins 1923) Scholarship of
teaching coined (Boyer 1990)
Reform Concepts CA/CR (Angelo Cross 1993) New
epistemology (Schön 1995) New American scholar
(Rice 1996) Scholarship Assessed (Glassick et al
1997)
Implementing Strategies AAHE FFRR CASTL -
Carnegie Scholars - Teaching Academy -
Professional Societies - Knowledge Media
Lab Peer Review of Teaching
Recent Articulations Shulman, Change 31
(4) 1999 Hutchings Shulman Change 31 (5)
1999 Kreber Cranton JHE 71 (4) 2000 Richlin
in Kreber (Ed.) New Directions 86 2001
12

• On the high ground, manageable problems lend
  themselves to solution through the use of
  research-based theory and technique. In the
  swampy lowlands, problems are messy and confusing
  and incapable of technical solution. ...the
  problems of the high ground tend to be relatively
  unimportant to individuals or to the society at
  large, however great their technical interest may
  be, while in the swamp lie the greatest problems
  of human concern. ...Shall the practitioner
  remain on the high ground where he can solve
  relatively unimportant problems according to his
  standards of rigor, or shall he descend to the
  swamp of important problems where he cannot be
  rigorous in any way he knows how to describe?
• Schon, D. The New Scholarship Requires a New
  Epistemology, Change, Nov./Dec. 1995 p. 28

13
R. Eugene Rice. 1996. The New American Scholar.
AAHE. p. 14
14
Origin and Evolution of SOTL (In Progress)
Not a new idea (Hutchins 1923) Scholarship of
teaching coined (Boyer 1990)
Reform Concepts CA/CR (Angelo Cross 1993) New
epistemology (Schön 1995) New American scholar
(Rice 1996) Scholarship Assessed (Glassick et al
1997)
Implementing Strategies AAHE FFRR CASTL -
Carnegie Scholars - Teaching Academy -
Professional Societies - Knowledge Media Lab
Peer Review of Teaching
Recent Articulations Shulman, Change 31
(4) 1999 Hutchings Shulman Change 31 (5)
1999 Kreber Cranton JHE 71 (4) 2000 Richlin
in Kreber (Ed.) New Directions 86 2001
15
Origin and Evolution of SOTL (In Progress)
Not a new idea (Hutchins 1923) Scholarship of
teaching coined (Boyer 1990)
Reform Concepts CA/CR (Angelo Cross 1993) New
epistemology (Schön 1995) New American scholar
(Rice 1996) Scholarship Assessed (Glassick et al
1997)
Implementing Strategies AAHE FFRR CASTL -
Carnegie Scholars - Teaching Academy -
Professional Societies - Knowledge Media
Lab Peer Review of Teaching
Recent Articulations Shulman, Change 31
(4) 1999 Hutchings Shulman Change 31 (5)
1999 Kreber Cranton JHE 71 (4) 2000 Richlin
in Kreber (Ed.) New Directions 86 2001
16
CASTL Carnegie Academy for the Scholarship of
Teaching learning

• Higher Education
• K-12

17
CASTL Carnegie Academy for the Scholarship of
Teaching learning

• Carnegie Scholars
• Teaching Academy
• Professional Societies
• Related Carnegie Initiatives
• Knowledge Media Lab
• Resources http//www.carnegiefoundation.org/
  CASTL/highered/index.htm

18
CASTL Carnegie Academy for the Scholarship of
Teaching learning

• Pew National Program for Carnegie Scholars
• http//www.carnegiefoundation.org/CASTL
• /highered/Pewscholars.htm

19
CASTL Carnegie Academy for the Scholarship of
Teaching learning

• Teaching Academy Campus Program AAHE
  Carnegie
• Level I Campus Conversations
• Level II Going Public
• Level III National Teaching Academy
• http//www.carnegiefoundation.org/CASTL/highered/t
  eachingacademy.htm

20
CASTL Carnegie Academy for the Scholarship of
Teaching learning

• AAHE CASTL Campus Program WebCenter
• Campus Reports
• List of "Going Public Grants"
• Resources
• http//aahe.ital.utexas.edu

21
CASTL Carnegie Academy for the Scholarship of
Teaching learning

• Professional Societies
• Networking
• Invitational Small-Grants
• Register Reports
• http//www.carnegiefoundation.org/CASTL/highered
  
• /collaborations.htm

22
CASTL Carnegie Academy for the Scholarship of
Teaching learning

• Knowledge Media Lab
• Tour
• Exhibition Scholarship Under Construction
• Gallery
• Multimedia Tutorial for SOTL
• http//www.carnegiefoundation.org/KML/index.htm

23
CASTL Carnegie Academy for the Scholarship of
Teaching learning

• Other Resources
• Bibliography of SOTL
• Text of Key Papers

24
Origin and Evolution of SOTL (In Progress)
Not a new idea (Hutchins 1923) Scholarship of
teaching coined (Boyer 1990)
Reform Concepts CA/CR (Angelo Cross 1993) New
epistemology (Schön 1995) New American scholar
(Rice 1996) Scholarship Assessed (Glassick et al
1997)
Implementing Strategies AAHE FFRR CASTL -
Carnegie Scholars - Teaching Academy -
Professional Societies - Knowledge Media
Lab Peer Review of Teaching
Recent Articulations Shulman, Change 31
(4) 1999 Hutchings Shulman Change 31 (5)
1999 Kreber Cranton JHE 71 (4) 2000 Richlin
in Kreber (Ed.) New Directions 86 2001
25
An Exemplary Course Portfolio And A Superb Model
of SOTL An Alternative Approach to General
Chemistry Assessing the Needs of At-Risk
Students with Cooperative Learning
Strategies
Dennis Jacobs Professor of Chemistry University
of Notre Dame
Rita Naremore, Simon Brassell, Shanker Krishnan,
David Parkhurst Constructing and Evaluating a
Course Portfolio Making Good Teaching Apparent
October 26 2000
26
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies Dennis Jacobs
(University of Notre Dame)

• Contents of Course Portfolio
• Overview
• Rationale
• Why develop an alternative approach?
• Implementation
• Changes introduced to foster learning.
• Targeting at-risk students.
• Impact
• Documentation and assessment of immediate and
  longer-term effects.
• Library
• Examples of videos of group discussions, tests,
  on-line quizzes, questionnaires.
• http//kml.carnegiefoundation.org/gallery/djacobs/

Constructing and Evaluating a Course
Portfolio Making Good Teaching Apparent
October 26 2000
- 2 -
27
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies Dennis Jacobs
(University of Notre Dame)

• Course Portfolio Design

Constructing and Evaluating a Course
Portfolio Making Good Teaching Apparent
October 26 2000
28
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies Dennis Jacobs
(University of Notre Dame)

• Rationale
• Recognized Problems
• At-risk students (Math SAT 630)
• dropped out of General Chemistry.
• didnt take any advanced science.
• frustrated by large lecture format.
• Alternate Course Design
• Similar requirements and lectures.
• Comparable exams.
• Various activities involving structured
  cooperative learning.
• Initial Comments on Proposal
• Only delaying inevitable failure.
• Efforts should be focused on the best not the
  at-risk students.

Constructing and Evaluating a Course
Portfolio Making Good Teaching Apparent
October 26 2000
- 4 -
29
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies Dennis Jacobs
(University of Notre Dame)

• Documentation
• At-risk students
• markedly more likely to
• drop or fail the course
• in the traditional class
• format.

Constructing and Evaluating a Course
Portfolio Making Good Teaching Apparent
October 26 2000
- 5 -
30
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies Dennis Jacobs
(University of Notre Dame)

• Implementation
• Aim
• Provide improved learning opportunities for
  at-risk students.
• Develop more effective teaching in large lecture
  format.
• Alternate Course Design
• Introduced opportunities for structured
  cooperative learning including
• discussion of concepts in pairs.
• small group in recitation sections.
• work as pairs in laboratory.
• Mandatory recitation sections
• more time committed to class.
• direct contact with instructors.

Constructing and Evaluating a Course
Portfolio Making Good Teaching Apparent
October 26 2000
- 6 -
31
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies Dennis Jacobs
(University of Notre Dame)

• Traditional vs. Alternative Classes
• Similarities
• Size (250 students), text, chapters.
• Lecture time (3 hr), lab time (2.5 hr).
• Lecture format (Powerpoint slides and
  demonstrations).
• Exam format and many exam questions.
• Differences in Alternative Section
• Mandatory recitations (1 hr/wk, 20 students)
  attendance 95 vs. 10.
• Weekly homework (10 vs. 30, graded).
• On-line quizzes (www chapter reviews).
• Weekly feedback from homework, group problems,
  on-line quiz, in-class questions.
• Personal contact with instructor and follow-up if
  performance declined.

Constructing and Evaluating a Course
Portfolio Making Good Teaching Apparent
October 26 2000
- 7 -
32
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies Dennis Jacobs
(University of Notre Dame)

• Impact of Alternative Section
• Assessment Strategies
• Effects on conceptual understanding,
  problem-solving and self-confidence
• feedback from students.
• evaluation of individual elements of the
  cooperative learning activities.
• Immediate and long-term benefits
• retention of at-risk students.
• success in advanced science classes.
• Data Collection
• Recording in-class learning activities.
• Tracking individual grades and progress.
• Longitudinal study of at-risk students
• progress in subsequent classes.

Constructing and Evaluating a Course
Portfolio Making Good Teaching Apparent
October 26 2000
- 8 -
33
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies Dennis Jacobs
(University of Notre Dame)

• Measurement of Impact
• Success of At-risk Students
• Better grades in General Chemistry.
• Improved retention in class.
• Higher success rate in subsequent classes.

Constructing and Evaluating a Course
Portfolio Making Good Teaching Apparent
October 26 2000
- 9 -
34
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies Dennis Jacobs
(University of Notre Dame)

• Measurement of Impact II

Constructing and Evaluating a Course
Portfolio Making Good Teaching Apparent
October 26 2000
- 10 -
35
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies Dennis Jacobs
(University of Notre Dame)

• Measurement of Impact III

Constructing and Evaluating a Course
Portfolio Making Good Teaching Apparent
October 26 2000
- 11 -
36
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies Dennis Jacobs
(University of Notre Dame)

• Conclusions
• Dennis Jacobs Course Portfolio
• Succinct and reflective.
• Focused on specific learning objectives.
• Uses readily available information
• grades, course materials, etc.
• Documents the positive effects of changes in
  instructional methodology.
• Benefit for Dennis Jacobs
• Ready justification for resources to support
  recitation sections.
• Value of Course Portfolios
• Demonstrable outcomes of teaching practices that
  can be peer-reviewed.

Constructing and Evaluating a Course
Portfolio Making Good Teaching Apparent
October 26 2000
- 12 -
37
Origin and Evolution of SOTL (In Progress)
Not a new idea (Hutchins 1923) Scholarship of
teaching coined (Boyer 1990)
Reform Concepts CA/CR (Angelo Cross 1993) New
epistemology (Schön 1995) New American scholar
(Rice 1996) Scholarship Assessed (Glassick et al
1997)
Implementing Strategies AAHE FFRR CASTL -
Carnegie Scholars - Teaching Academy -
Professional Societies - Knowledge Media
Lab Peer Review of Teaching
Recent Articulations Shulman, Change 31
(4) 1999 Hutchings Shulman Change 31 (5)
1999 Kreber Cranton JHE 71 (4) 2000 Richlin
in Kreber (Ed.) New Directions 86 2001
38
Task
Think about your own background in SOTL. What
additions would you make to the map Origins and
Evolution of SOTL in each category? Reform
Concepts Implementing
Strategies Recent Articulations
39
OutlineThe IUB Experience

• The Administrative Perspective
• Design Principles
• Kinds of Resources Needed
• Changes to the Faculty Culture
• The Departmental Perspective
• Going from Zero to Sixty in 10.2 Seconds
• The Individual Faculty Members Perspective

40
How Did We Set Up a Campus SOTL Program?

• The Ingredients
• A dedicated director
• A faculty advisory committee
• A core of committed researchers willing to share
  their work
• Massive Administrative Support
• The Key Faculty Input

41
Design Principles

• Get out the crowd to initial events
• Serve food
• Connect with national
• Programs
• Resources
• Be inclusive

42
Design Principles (Cont.)

• Involve librarians
• Partner with stakeholders
• Identify needs
• Campus conversations
• Faculty committees
• Exhibit administrative support

43
What Resources Were Needed?

• FinancialMoney is a great motivator!
• HumanPeople will go where the money is!
• StructuralOther rewards are important in the
  academic environment. Integrate SOTL into this
  reward structure.

44
Deploying Available Resources Some Examples

• Expand rewards
• Travel grants
• Presentation/research grants
• Visibility for contributions

45
ONE EXAMPLE OF A STRUCTURAL CHANGE

• IU FACULTY SUMMARY REPORT PRIOR TO ACADEMIC
  YEAR 1999-2000
• TEACHING ACTIVITIES
• Courses taught (weekly contact hours reported by
  course number in tabular form).
• Development or major revision of course(s) during
  the year.
• Dissertation, Research and Field Work Committees
• Teaching awards and honors, including those of
  your students.

46
FACULTY SUMMARY REPORT Revised in Academic
Year 1999-2000

• TEACHING ACTIVITIES
• Courses taught (weekly contact hours reported by
  course number in tabular form).
• Activities directed at improving instruction,
  learning, or course administration. (Please
  describe rationale for/description of
  innovations, methods/measures for assessing
  outcomes, and results.)
• Please note Scholarly activity related to
  teaching and learning (e.g. investigation/research
  , dissemination/publication of results) should be
  reported under the section on Research/Creative
  Activities.

47
FACULTY SUMMARY REPORT, CONTINUED

• Development or major revision of course(s) during
  the year.
• Dissertation, Research and Field Work Committees
• Teaching awards and honors, including those of
  your students.

48
Effects to Date

• 1999-2000 Campus SOTL Program of Events
• 15 events - 930 total registrants
• 53 participants per event
• 515 individuals registers for one or more events
• 43 requests for videotapes
• SOTL community formed and functioning
• Developmental process for scholars in formation
• Several SOTL-related research projects identified

49
The Program in Year 2

• 10 Events Scheduled
• - 75 Participants per Event (Average over 4
  events)
• More Efforts to Link SOTL With Other Initiatives
• Preparing Future Faculty
• The Concern with Research Ethics
• The Course Portfolio Project

50
Working At the Departmental Level

• In the beginning
• A concern for and commitment to excellent
  teaching
• A key group of faculty members willing to begin
  the conversation about teaching in the unit

• Next?
• Collecting data
• Sharing the results
• Designing follow-up to the initial research
• Involving others
• Extending the questions

51
An Example of Department-Based Research

• The problem a group of faculty members
  concerned that teaching excellence was getting
  short shrift in the faculty evaluation process
• The solution appoint a Committee on Teaching to
  make recommendations

52
Working Toward a Solution

• Issue 1 a lack of confidence in student
  evaluations
• Survey the facultyWhat can/should students tell
  us about our teaching?
• Develop and test a new evaluation instrument

• Issue 2 a lack of good procedures for peer
  evaluations
• Gain a consensus for how peer evaluation might be
  made better
• Develop and test a peer evaluation procedure

53
Issue 1 Student Evaluation

• A departmental form was developed and used along
  with the traditional university form in all
  classes for 4 semesters.
• Results using the two forms were compared, and
  statistical analyses were conduced to determine
  reliability and validity.

54
A Brief Summary of Results

• Faculty members who taught 10 required
  undergraduate courses were ranked in terms of
  their overall student evaluation scores.

• The differences among the top 5 faculty were very
  small, but the difference between the top 5 and
  the bottom 5 was sizeable.
• 86 of variance in the rankings was predicted by
  responses on one item I learned a lot in this
  course.

55
Issue 2 Peer Evaluation

• The faculty consensus traditional classroom
  visitation is not very revealing of anything
  useful. What we really need is evaluation of
  course structure and materials used in the class
  for teaching and evaluation. Wed like to have
  periodic external peer review of these materials.

56
Ongoing Concerns with Peer Evaluation

• What are we doing? Were working toward the
  development of course portfolios, to be done over
  time and turned in by faculty members every
  second or third year, not every year.

• The biggest concern IT TAKES TOO LONG TO PUT
  ALL THIS STUFF TOGETHER!

57
The Campus Tie

• The departmental course portfolio effort should
  eventually tie in with a campus-wide effort now
  underway, conducted by an interdisciplinary team
  in conjunction with a Pew Foundation funded
  project directed by Dan Bernstein at the
  University of Nebraska. For more information
  about this project, see http//www.unl.edu/peerrev
  /

58
TASK Refining Your Own Campus SOTL Program or
Plan
Think for a minute about your own campus. What
administrative priorities or structures might
help in setting up a campus level SOTL program?
What hurdles might need to be overcome on your
campus?
59
Other Campus Models

• Elon College
• The Citadel
• Rockhurst University
• Abilene Christian University
• Notre Dame University
• Middlesex Community College

60
Elon College

• Multidisciplinary, Multiyear, 72,000 Investment
• 6000 projects in each of 3 years
• Projects directed by faculty-student research
  teams
• Learning for BOTH student and teacher
• Eight projects selected in years 1 and 2
• Create intellectual engagement
• New thinking in diverse fields
• Application of learning to life
• Opening spaces for reflective integration

61
The Citadel

• Mission
• Increased campus awareness of and participation
  in SOTL
• Focus
• Communication, Resources and Continuing Education
  
• Self-selected research projects

• Highlights
• Biweekly, participatory meetings with assignments
• http//www.citadel.edu/carnegie
• Effectiveness
• 15 of full-time, tenure-track faculty at
  bi-weekly meetings
• 12 of full-time, tenure track faculty in
  classroom research
• Administrative Support
• Attendance at functions
• Financial support

62
Rockhurst University

• Beginnings
• (Fall 1998) All University Symposium
• (Spring 1999) Follow-up Symposium
• Year-long Carnegie faculty seminar
• (2000-2001) Carnegie faculty seminar continued
• The Rockhurst Carnegie Seminar
• Central Questions
• Seminar Members
• Discussions
• Methods

• Products
• Formal Letter on SOTL
• Faculty SOTL Projects
• Selected Key Issues and Observations
• Obstacles to Discussion
• Interdisciplinary/Collaborative Approaches
• Where's the beef?
• Scholarly Teaching as best first path
• To be a good consumer of the SOTL

63
Abilene Christian University

• 19 Faculty doing SOTL Projects
• Strong Institutional Support
• Stipends for materials resources
• Travel to teaching-related conferences
• Ongoing peer meetings,videoconferences

64
University of Notre Dame

• Initial campus conversations with 90 leaders
• SOTL needed support
• RFP resulted in 9 funded SOTL projects
• Sample research question Do new teaching
  methods in intro engineering affect students
  learning?
• Support for SOTL teams includes these elements
• 5,000 per team for student time, equipment,
  supplies, faculty time
• Consulting with methodology experts
• Group meetings 2x/semester for mutual support
• Help in dissemination of results
• http//www.nd.edu/kaneb/Carnegie.html

65
Task Campus Models for SOTL

• List 2 features of the models you have just
  seen that might be most applicable to your
  campus.

66
Working at the Level of the Individual Researcher
67
Deploying Available Resources Some Examples

• Expand rewards
• Travel grants
• Presentation/research grants
• Visibility for contributions

68
Task
"MINUTE PAPER" 1. What were the 2 most
important points presented in the first half of
this workshop? 2. What 2 things would you most
like to learn tomorrow? .
69
(No Transcript)
70
How Could I do Scholarship of Teaching and
Learning?

• Different Genres of SOTL
• Reports on Particular Classes
• Reflections on Years of Teaching
• Larger ContextsComparisons
• Formal Research
• Meta-Analyses

71
Task
Which 3 of these genres would be easiest for
faculty on your campus to do?
72
Approaches to Research
73
Classroom vs. Traditional Research

• Classroom
• Professors practice
• Obtain knowledge applicable in limited
  circumstances
• Specialized training not essential
• Students and Professor

Traditional State-of present research Build or
verify theory Specialized training usually
essential Field and Researcher
Origin Purpose Requirements Benefits
74
Action Research

• Research carried out by practitioners with a
  view to improving their professional practice and
  understanding it better.
• Quoted in Borg, Gall Gall Applying Educational
  Research 3rd Ed. Longman, 1993 p. 390

75
Classroom Research

• Classroom Research is not traditional research
  conducted in or on classrooms. It is a specific
  methodology designed for discipline oriented
  teachers without training or experience in the
  methods of educational research. Classroom
  Research is ongoing and cumulative intellectual
  inquiry by classroom teachers into the nature of
  teaching and learning in their own classrooms.
  Inquiry into a question about how students learn
  typically leads to new questions and thus to
  continual investigations through classroom
  research.
• Cross, K.P. Steadman, M. Classroom Research,
  Jossey-Bass, 1996 p. xviii

76
IMPROVING TEACHING AND LEARNING VIA CLASSROOM
RESEARCH
INFORMATION GATHERING
IMPETUS
Formative Assessment
EFFECT
Goals
Summative Assessment
Analysis
SCHOLARLY PRODUCTIVITY
Questions
Recorded Observation
Reflection
Existing Scholarship
Synthesis
Institutional Database
APPLICATION
Improved Teaching and Learning
77
Classroom Assessment

• Classroom assessment is systematic and formative
• Class is the unit of measurement rather than the
  individual
• Conditions of learning may be assessed rather
  than student performance.
• Right and wrong are not the emphasis.
• Unexpected rather than expected responses are
  often most useful.

INFORMATION GATHERING
IMPETUS
Formative Assessment
EFFECT
Goals
Summative Assessment
Analysis
SCHOLARLY PRODUCTIVITY
Questions
Recorded Observation
Reflection
Existing Scholarship
Synthesis
Institutional Database
APPLICATION
Improved Teaching and Learning
Angelo, T. Cross, K.P., Classroom Assessment
Techniques A Handbook for College Teachers (2nd
Ed) Jossey-Bass (1993)
78
Effective Grading

• Primary Trait Analysis (PTA) building scales
  that make performance criteria explicit in order
  to categorize/classify student work.
• Provides a documentary source for changes in
  student learning.
• Improves grading.

INFORMATION GATHERING
IMPETUS
Formative Assessment
EFFECT
Goals
Summative Assessment
Analysis
SCHOLARLY PRODUCTIVITY
Questions
Recorded Observation
Reflection
Existing Scholarship
Synthesis
Institutional Database
APPLICATION
Improved Teaching and Learning
Walvoord, B. and Anderson, V., Effective Grading
A Tool for Learning and Assessment, Jossey-Bass
1998.
79
The Course Portfolio (I)

• I was familiar with teaching portfolios but
  thinking about teaching as scholarly inquiry
  began to lead me in the direction of something I
  had not seen anyone else doing a portfolio that
  focused on the course rather than on all of ones
  teaching. Being a social scientist, I began to
  think of each course as a kind of laboratory -
  not a truly controlled experiment of course but
  as a setting in which you start out with goals
  for student learning, then you adopt teaching
  practices that you think will accomplish these
  and along the way you can watch and see if your
  practices are helping to accomplish your goals,
  collecting evidence about effects and impact.

INFORMATION GATHERING
IMPETUS
Formative Assessment
EFFECT
Goals
Summative Assessment
Analysis
SCHOLARLY PRODUCTIVITY
Questions
Recorded Observation
Reflection
Existing Scholarship
Synthesis
Institutional Database
APPLICATION
Improved Teaching and Learning
W. Cerbin quoted in Hutchings, P. (Ed.) The
Course Portfolio, AAHE 1998
80
CLASSROOM RESEARCH
INFORMATION GATHERING
IMPETUS
Formative Assessment
EFFECT
Goals
Summative Assessment
Analysis
SCHOLARLY PRODUCTIVITY
Questions
Recorded Observation
Reflection
Existing Scholarship
Synthesis
Institutional Database
APPLICATION
Improved Teaching and Learning
Cross, K.P. and Steadman, M., Classroom Research,
Jossey-Bass 1996
81
Task
Which kind of information gathering (previous
slide) is most common on your campus? Which kind
is most needed? Why?
82
Asking the Question
83
Framing Research Questions(Goal Approach)

• Define a goal
• Answer questions about the goal
• Create a one (or two) sentence summary of the
  specific goal
• Ask what evidence would reveal
• -the present state?
• -that the goal is achieved?
• Write possible research questions

84
Framing Research Questions(Issue Approach)

• Criteria
• Investigable (not necessarily empirical)
• Bounded and well-defined
• Significant (not necessarily statistically)
• Considerations
• Length of time needed
• Complexity of procedures
• Availability of subjects
• Availability of support (resources, personnel,
  funds)

85
Example

• Less framed
• Do students who help others learn an academic
  discipline learn it better themselves?

• More framed
• Do students in CMSC 250 who tutor students in
  CMSC 150 perform better on the CMSC 250 final
  exam than students who do not tutor but have
  similar grades in CMSC 150?

86
Making Vague Questions Answerable

• 1. Do students learn more in small classes?

• 1. Do students in sections of M118 enrolling
  fewer than 50 students perform better on the
  departmental final exam than students from
  sections enrolling more than 75 students?

87
Making Vague Questions Answerable

• What is the optimum number of homework
  assignments to give in a beginning math class?

• Do students enrolled in M036 who are given a
  homework assignment every week perform
  differently on the departmental final exam than
  students enrolled in M036 who are given homework
  every day?

88
Making Vague Questions Answerable

• Do students enrolled in S333 who are given 6
  exams per semester evaluate the overall course
  effectiveness on BEST item 1 differently from
  students in S333 who have only 1 exam per
  semester?

• What is the effect of the number of exams in a
  course on students opinions about the course?

89
Task 1

• Write the tentative issue or question to be
  addressed in your project.
• Form a group with two colleagues (groups of
  three)
• Discuss your tentative project with your
  colleagues for the purpose of framing your goal,
  issue or question in the clearest and most
  assessable way. Encourage your colleagues to
  questions you and comment.
• Write your (clarified and assessable)
  goal/issue/question again.
• Assist each of your colleagues in completing
  steps 3 4.

90
Design Frameworks
91
What is a research design?

• A plan or protocol for carrying out a research
  project
• An underlying scheme that governs functioning,
  developing, or unfolding
• Regardless of your definition, a good design
  promotes efficient and successful data gathering
  and analysis.

92
Quantitative or Qualitative Research?

• Quantitative
• Empirical, statistical
• Goal hypothesis testing or confirmation
• Design predetermined, structured

• Qualitative
• Fieldwork, constructivist
• Goal hypothesis generating, making meaning
• Design flexible, evolving

93
Comparisons, continued

• Quantitative
• Sample large, representative
• Measures scales, tests, surveys
• Researcher is outsider
• Findings precise, reliable

• Qualitative
• Sample small, purposeful
• Measures interviews, observations
• Researcher is insider
• Findings rich, deep

94
Guiding Questions in Choosing Methodology

• What approach fits your research problem?
• Do you have the skills/resources to carry out the
  methods?
• Will your audience find these approaches
  acceptable?
• Provided by Samuel Guskin, Professor Emeritus,
  School of Education, Indiana University

95
Choosing the Measures to Answer the Question
96
Examples of Quantitative Measures

• course exam, project, paper scores
• survey scores (Likert)
• scores on standardized scales and tests
• counts (participation, visits)
• measures of time use
• institutional research data (GPAs, grades,
  admissions scores, demographics)

97
Examples of Qualitative Measures

• observations
• interviews
• focus groups
• student projects, essay exams (summative)
• reflective statements (formative)
• reports of others (counselors, etc.)

98
Illustrations of Qualitative and Mixed Methods

• Qualitative case study
• Quantitative study enhanced by qualitative data
• Qualitative study enhanced by quantitative data
• Provided by Samuel Guskin, Professor Emeritus,
  School of Education, Indiana University

99
Task
Design an investigation to address the research
question you framed earlier.
100
Summary of Standards

• Clear Goals
• Does the scholar state the basic purpose of his
  or her work clearly? Does the scholar define
  objectives that are realistic and achievable?
  Does the scholar identify important questions in
  the field?

Glassick, C.E., Huber, M.T., Maeroff, G.I.
Scholarship Assessed Jossey-Bass (1997) pp. 22-36
101
Summary of Standards

• Adequate Preparation
• Does the scholar show an understanding of
  existing scholarship in the field? Does the
  scholar bring the necessary skills to his or her
  work? Does the scholar bring together the
  resources necessary to move the project forward?

Glassick, C.E., Huber, M.T., Maeroff, G.I.
Scholarship Assessed Jossey-Bass (1997) pp. 22-36
102
Summary of Standards

• Appropriate Methods
• Does the scholar use methods appropriate to the
  goals? Does the scholar apply effectively the
  methods selected? Does the scholar modify
  procedures in response to changing circumstances?

Glassick, C.E., Huber, M.T., Maeroff, G.I.
Scholarship Assessed Jossey-Bass (1997) pp. 22-36
103
Summary of Standards

• Significant Results
• Does the scholar achieve the goals? Does the
  scholars work add consequentially to the field?
  Does the scholars work open additional areas for
  further exploration?

Glassick, C.E., Huber, M.T., Maeroff, G.I.
Scholarship Assessed Jossey-Bass (1997) pp. 22-36
104
Summary of Standards

• Effective Presentation
• Does the scholar use a suitable style and
  effective organization to present his or her
  work? Does the scholar use appropriate forums
  for communicating work to its intended audiences?
  Does the scholar present his or her message with
  clarity and integrity?

Glassick, C.E., Huber, M.T., Maeroff, G.I.
Scholarship Assessed Jossey-Bass (1997) pp. 22-36
105
Summary of Standards

• Reflective Critique
• Does the scholar critically evaluate his or her
  own work? Does the scholar bring an appropriate
  breadth of evidence to his or her critique? Does
  the scholar use evaluation to improve the quality
  of future work?

Glassick, C.E., Huber, M.T., Maeroff, G.I.
Scholarship Assessed Jossey-Bass (1997) pp. 22-36
106
Where to Publish and Present
107
How to Find Potential Sources of External Funding
108
Closing Evaluation of Workshop