Spanning the Divide Between Institutional Research and Classroom Assessment

1
Spanning the Divide BetweenInstitutional
Research andClassroom Assessment

• San Diego Community College District
• San Diego Mesa College
• Bill Grimes, M.A., Director of District
  Institutional Research and Planning
• Yvonne Bergland, Ph.D., Dean of Instructional
  Services, Resource Development, and Research
• Gail Conrad, M.A., Mesa Interim VPSS, Former
  SLOAC Coordinator
• Susan Mun, M.A., Research Analyst (CBR)
• Strengthening Student Success, San Jose, CA
• October 3 October 6, 2007

2
Presentation Learning Outcomes

• Explain the key elements of a Campus-Based
  research model
• Identify the steps that colleges need to take for
  implementing this model
• Evaluate the advantages and challenges of this
  model

3
Recommendation from WASC (Accreditation Process)

• The college should develop and implement a plan
  to meet current and future needs for
  institutional research that is accurate, timely
  and actionable

4
Recommendation from WASC(continued)

• Toward this end, the college should carefully
  consider how instructional research is positioned
  in the college so that it may support the entire
  institution from a fair, unbiased and informed
  stance, thereby strengthening various planning
  and institutional improvement efforts. (I.B).

5
Recommendation from WASC(continued)

• In order to build upon their efforts to
  strengthen institutional effectiveness and to
  foster a culture of evidence throughout the
  district, the district office and the colleges
  should cooperate in the development of an
  enhanced research function with both strong
  district and strong college components. (I.B.3,
  I.B.6, IV.B.2.b.)

6
Preparation Checklist

• Central contact person(s) and location
• Mechanism to inform the college of research
  activities
• Mechanism for identifying research needed for
  making effective data-driven decisions.
• A year-long research agenda that 1) integrates
  campus-wide information, research activity and
  needs, 2) takes advantage of existing district
  research (avoids duplication of effort), 3)
  identifies and addresses information gaps as
  expressed by the campus community.
• A College Research Committee to support and guide
  the functions listed above.
• (See CD document Preparation Checklist)

7
Setting the Stage

• San Diego Community College District (SDCCD)
• City, Mesa, and Miramar College and Continuing
  Education
• Second largest community college district in
  California
• Total enrollment exceeds 140,000 students
• Mesa College
• Largest of the three colleges in the SDCCD
• Functioning Research Committee
• Well-established Program Review process and
  Student Learning Outcomes Assessment Cycle
  (SLOAC)
• Integration of Academic and Student Services
  Program Reviews

8
District / CBR Model
9
SAN DIEGO MESA COLLEGE OFFICE OF INSTRUCTIONAL
SERVICES, RESOURCE DEVELOPMENT AND
RESEARCH RESEARCH FUNCTION LINES OF
RESPONSIBILITY
DISTRICT COLLEGE
Presidents Cabinet
Assistant Chancellor, Student Services
President
Vice President, Student Services
Vice President, Admin. Services
Vice President, Instruction
Dean, Instructional Services, Research and
Resource Development
Director, Institutional Research and Planning
Campus-Based Researcher
District-wide Research Committee
Mesas Research Committee
10
(No Transcript)
11
District/Campus Collaboration
Basic Skills Program Review SLO
Support Accreditation support Surveys District-wid
e studies Chancellors Cabinet reports State
reports District IR web site
Basic Skills Program Review SLO
Support Accreditation support CCSSE/CCFSSE Researc
h Agenda Faculty/Staff requests Educational
Master Plan Marketing promotion
Campus-based Researcher
Supports data-driven decisions that increase
student access, retention, and success
12
Campus-Based Research
Research Requests
Surveys
College-wide Strategic Planning
13
Examples ofRequests for Research Reports

• Student Athletes Study
• Student success (ESOL, Math, Fast Track HS, PERG,
  Learning Communities, short-term courses)
• Marketing, outreach recruitment (GIS maps,
  BLAS, CHIC, transfer, Bridges program, students
  on probation/DQ, applicants who did not enroll)
• Graduate follow-up surveys (Chemistry, PTA)
• Transfer
• (see CD handout CBR Project Log)

14
Requests for Research to Support Assessment of
Student Learning

• Support direct assessment of SLOs
• Spanish 101 Multiple Independent Scoring of
  Essays
• Indirect Assessment of SLOs
• CCSSE/CCFSSE/Basic Skills Oversample
• Silver Process
• Examination of Learning Indicators
• Instruction Successful course completion rates,
  retention, persistence, etc.
• Student Services Programs Retention,
  persistence, etc.

15
Educating the Campus

• Flex Day Presentations
• District Research - Ways It Can Assist
• Developing Effective Research Questions
• Using Rubrics in Assessment (Part I)
• Using Rubrics in Assessment (Part II)
• Pivot Tables An Interactive Tool to Support
  Decisions
• FTES Basics (Part I)
• CBR and the Research Agenda
• How to Develop a Survey
• Follow up to CBR and Mesa College Research
  Agenda Retrospective
• Student Services SLOs College Planning
• (See CD document Flex presentations)

16
Educating the Campus

• Research Planning Agenda, GIRPA, CCSSE
• Presidents Cabinet
• Academic Senate
• Classified Senate
• Associated Students
• Presidents Executive Staff
• Academic Affairs Committee
• School meetings (7)
• Chairs Committee
• Deans Council
• Research Committee
• Student Services Administrators
• Student Services Council
• See CD document Research Planning Agenda

17
District / Campus Communication

• Bi-monthly joint meetings
• Weekly Research Planning meetings including all
  District analysts and CBR(s)
• District-Wide Research Committee meetings
• District representation at campus committee
  meetings
• District / campus liaisons
• Project Tracker / Research Log

18
Benefits

• Collaboration between campus and district
  research office has become substantially stronger
  and richer.
• Sophistication of research initiatives has
  increased.
• Research has been a pathway to integration
• Interactions between student services programs
  and academic success are becoming more clearly in
  focus.

19
Challenges

• Growing pains
• Need to strengthen connection to SLO development,
  measurement, and linkages to traditional
  institutional research
• Challenge of new or redefined policies based on
  new requests, context and campus framework

20
Serving Two Masters (or more?)

• Director of District IRP Supervises
  policy-related issues, project plans, and weekly
  meetings amongst the analysts in the SDCCD
  research community
• Dean of Instructional Services, Resource
  Development, and Research Supervises day-to-day
  operation, fulfillment of campus research
  requests, and committee work (with exception of
  District-Wide Research Committee)

21
Measures of Productivity

• Research activity and strategic involvement in
  SLO development has skyrocketed
• Sophistication of the requests and the
  stakeholders has been increasing
• Rubric design for rating sophistication of
  research requests
• Measures of increased faculty campus
  participation
• Measures of district-wide benefits

22
Surprises

• Research-based momentum has sustained itself
  through changes in leadership, circumstances, and
  policies
• Similarities between District and Campus Research
  Offices
• Exponential increases in workload
• Urgency
• Prioritization
• Sharing / exchange
• Requests for research

23
Guidelines for Implementingthe Research Planning
Agenda

• FERPA ? GIRPA
• Data sensitivity Access, security, use,
  dissemination
• Level I public information which is highly
  aggregated, or broadly categorized, such as
  enrollment figures, transfer rates, or any other
  institution-wide data
• Level II General Requests for Research Reports,
  survey data, and data that are disaggregated, or
  broken out by categories, to some extent, such as
  success rates or student progress at the program
  level
• Level III Special Requests for Research Reports
  and sensitive information that is highly
  disaggregated, such as student contact
  information, data at the Course Reference Number
  (CRN) level, student records

(See CD documents GIRPA and GIRPA FAQs)
24
Questions???
bgrimes_at_sdccd.edu yberglan_at_sdccd.edu gconrad_at_sdccd
.edu smun_at_sdccd.edu
Email

Phone [phone] (District) (619)
388-2509 (Mesa)
District Web Site www.sdccd.net/research/ Mesa
Web Site www.sdmesa.edu
25
Operational Logistics

• The Research Office continues to support
  District-wide projects where it is easier to
  complete two or more colleges work.
• Projects are shared between the CBR and Research
  Office to maximize efficiency of time, data
  collection, analysis, or reporting.
• CBR is the lead on a project self-contained to
  the campus and its unique research agenda.
  (Campus Liaison would support or provide
  temporary backup)
• Essential role of documentation to 1) enhance
  learning, 2) allow for constructive review, 3)
  provide historical perspective for future
  researchers, and 4) maximize efficiency.
  Reinforces principle of sharing and teaching
  among research community.
• Secure and shared network space located at each
  campus for downloading and uploading files in a
  safe and secure manner.
• Continue process of maintaining individual daily
  To Do list

26
Research at Mesa

• Support the Mesa College mission
• Create a culture of evidence
• Assist faculty, staff, administration with
  research
• Reinforce data-driven decision-making
• Educate the campus community
• ?

27
Research at Mesa

• Create a culture of evidence
• Assist faculty, staff, administration with
  research
• Reinforce data-driven decision-making
• Educate the campus community
• Support the Mesa College mission

28
Request for Research Report Form

Research question
Research report requested
Reason and rationale for request
Date research report is needed
Parameters of research report
Signatures from requestor, Department Chair, and
Dean or Manager
See CD document Request for Research Report form
29
Web Links

• Request for Research Report Form
• http//www.sdmesa.edu/instruction/research/pdf/RES
  EARCHREQUESTFORM.pdf
• Research Planning Agenda
• http//www.sdmesa.edu/instruction/research/pdf/Res
  earchPlanningAgendaApproved2-27-07.pdf
• GIRPA
• http//www.sdmesa.edu/instruction/research/pdf/Dat
  aSensitivityGIRPA.pdf
• GIRPA FAQs
• http//www.sdmesa.edu/instruction/research/pdf/Dat
  aSensitivityGIRPAFAQs.pdf