Assessing International Education at our Institutions: An Overview

1
Assessing International Education at our
Institutions An Overview

• Darla Deardorff, Duke University
• David Comp, University of Chicago
• Elaine Meyer-Lee, Saint Marys College

2
Goals

• Gain overview of assessment in international
  education including brief history
• Explore ways of thinking about internationalizatio
  n outcomes assessment
• Discuss some examples of current assessment
• Important Note This presentation is not a
  methods, how-to, or findings, but overview and
  examples to inspire

3
Why assess?

• Wed love to hear from you!

4
Why assess international education?

• Inform continuous quality improvement
• Advocate for international education
• Satisfy regional accrediting bodies
• Add to basic understanding of student growth in
  this area

5
Why Assess? Measure success

• Moving beyond
• OUTPUTS to OUTCOMES
• What are meaningful outcomes (results) of
  internationalization efforts?

6
__________________________________________________
________________________   INTERNATIONALIZATION at
institutions of higher education   Inputs/Resourc
es needed for implementation of components of
internationalization Activities/Components of
Internationalization (college leadership, faculty
involvement, curriculum, study abroad,
international students/scholars/faculty,
international co-curricular units) Outputs of
Internationalization (i.e., number of
international students, number of study abroad
programs, number of students studying foreign
languages, etc.) Outcomes of Internationalizatio
n Intercultural competence what is it? How do
higher education administrators define it?
intercultural experts? How can it be
assessed?   Long-Term Impact of
Internationalization _____________________________
__________________________________ Figure 2.
General program logic model applied to
internationalization. (Deardorff, 2004, Page 58)
7
Brief Historical Overview

• Where has the international ed field been in
  regard to assessment?

8
Recent History of Assessment in International
Education

• NAFSA Association of International Educators
• New Teaching, Learning and Scholarship -
  Knowledge Community.
• NAFSA Task Force on Research, 2004.
• SECUSSA Research Committee, 2003-2005.
• Forum on Education Abroad
• Focus on organizational goals.
• Outcomes Assessment Committee.
• Guide to Assessment in International Education.
• Partnership with Frontiers Journal.
• Standards of Good Practice.

9
Recent History of Assessment in International
Education

• Association of International Educators (AIEA)
• Burn, B., Smuckler, R.H. A Research Agenda for
  the Internationalization of Higher Education in
  the United States Recommendations and Report
  Based on August 10-11, 1995 Meeting.
• International Education Forum, Association
  Journal during the 1990s
• 2004 conference session with significant
  discussion on research in international education
  (transcripts distributed).
• American Council on Education (ACE)
• 2004 Roundtable on Research in International
  Education, Baltimore, Maryland

10
Recent History of Assessment in International
Education

• Association for Studies in International
  Education (ASIE)
• The Association is an inter-organizational body
  whose mission is to encourage serious research
  and publications dealing with international
  education and academic mobility, to stimulate
  interest in such work (both in the international
  education community and in academic circles in
  general), and to develop and promote ways to
  disseminate this work in cost-effective and
  accessible formats. The initial objective and
  primary focus of the Association in fulfillment
  of this mission is the publication and
  distribution of the Journal of Studies in
  International Education. (ASIE)
• ASIE organizations include AIEA Association of
  International Education Administrators CBIE
  Canadian Bureau for International Education The
  CIEE Council on International Educational
  Exchange EAIE European Association for
  International Education IEAA International
  Association of Australia IEASA International
  Education Association of South Africa NAFSA
  Association of International Educators NUFFIC
  Netherlands Organization for International
  Cooperation in Higher Education UKCOSA The
  Council for International Education and, WES
  World Education Services.

11
Ways of approaching assessment

• Some key points

12
LEVELS OF INTERNATIONALIZATION OUTCOME ASSESSMENT

• Institutional
• Program (inc. faculty)
• Student (inc. alumni)
• Student outcome assessment is not a measurement
  of learning but an integral part of learning. Can
  include language/disciplinary assessment,
  personal growth, intercultural competence, etc.
• Involve students in negotiating outcomes

13
ASSESSING OUTCOMES

• Starting point
• Mission Statement
• into
• Goals
• into
• Measurable Objectives

14
Assessment Methods?

• Methods determined by clearly articulated
  assessment questions what do we want to
  measure?
• Include
• DIRECT METHODS authentic assessment,
  portfolios, observation
• INDIRECT METHODS surveys, focus groups,
  interviews
• COMBINATION OF QUANTITATIVE AND QUALITATIVE

15
Internationalization Assessment-Contexts

• At Home curriculum, extra-curricular
  activities, community involvement/impact,
  domestic/internatl student interaction, policy,
  financial, campus climate
• Abroad (Cross-border) involves cross-border
  delivery of education through exchange, distance
  and e-learning, branch campuses, partnerships,
  host community impact

16
Assessment Cycle

• Define outcomes (based on mission/goals) and
  establish measurable criteria
• Identify appropriate assessment methods
• Collect data
• Analyze data
• Reflect on needed changes
• Design and apply changes

17
INTERNATIONALIZATION ASSESSMENT

• Multi-method, multi-perspective
• Ongoing
• Integrated

18
Assessment Examples

• Saint MarysChicago and beyondICC Study

19
Saint Marys College Context

• Catholic Womens Liberal Art
• (about 1600 students)
• 20 of students study abroad
• 8 semester options
• 9 short-term options

20

• Encompasses much more than study abroad (see
  brochures), and assessing as much of it as
  possible
• Is adopting Byrams definition of intercultural
  competence
• Is launching a portfolio-based honors program in
  Intercultural Leadership

21
Intercultural Leadership Honors Program

• Capture Saint Marys strengths and CWILs unique
  location at the intersection of intercultural
  (local and global) and leadership education in
  the context of a womens college
• Guide students through a developmental process
  into a deeper capacity for intercultural
  leadership
• Bring coherence and provide an organizing
  framework for every effort (curricular and
  co-curricular) being undertaken at Saint Marys
  College to encourage intercultural leadership
• Portfolio assessment of students proficiency as
  an intercultural leader in
• 6 proficiency areas
• Recognize the Leader Within
• Articulate Your Ethical/Spiritual Center
• Engage With Value Diversity
• Dialogue on Power Privilege
• Create Inclusive Equitable Community
• Make Your Difference in the World

22
Education Abroad Evaluation Assessment Goals

• Evaluate programs effectiveness
• Not satisfaction, measure Students
• Changes in sensitivity to cultural differences
• American identity development
• Own sense of growth toward goals
• Explore correlations of change with
• Intercultural engagement while abroad
• (interaction and reflection)
• Program or demographic characteristics

23
Education Abroad Assessment Methods/Design

• Longitudinal Research Design
• Pre-test/Post-test (some grad follow-up)
• Procedures 70 response rate so far
• Control Group
• 30 First-year students
• Focus Group
• Study Abroad Re-entry Course
• Written reflection and Kolbs LSI

24
Education Abroad Assessment Instruments

• Study Abroad Participants
• Intercultural Development Inventory (IDI)
• American Identity Questionnaire
• Pre Goals/motivations from SMC Study Abroad
  Survey
• Post Program Impact Measure of Intercultural
  Engagement from SMC Study Abroad Survey

• Control Group
• Intercultural Development Inventory (IDI)
• American Identity Questionnaire

25
Data Collection Timeline
26
On-campus International and Intercultural
Education Assessment

• Qualitative Methods
• Student Intercultural Learning Showcase
• Community Education Program during International
  Education Week
• Reports up on CWIL web-site
• Student faculty intercultural learning grants
  reports
• International Fellows 6-mo. reports exit
  interviews
• Quantitative Methods
• Institutional Research College Student Survey,
  HERI Faculty Survey, etc.

27
On-campus International and Intercultural
Education Assessment

• Mixed Methods
• International Student Services Survey
• Comprehensive survey to assess students ability
  to fully participate in the college
• Comparable with Study Abroad Survey s Measure of
  Intercultural Engagement
• Individual programs for faculty development,
  study abroad pre-departure re-entry, or
  international education week surveys
• Intercultural Learning/Living Floor survey
  Intercultural Development Inventory
• Catalyst Trip survey interviews

28
Assessing Internationalization at the University
of Chicago

• Ad Hoc Committee on International and Regional
  Programs at the University of Chicago (April 15,
  1995)
• Ad Hoc Committee on International Programs at The
  University of Chicago (June 20, 2001)

29
Assessment Highlights in the Field

• Assessment Tools
• The IES MAP (Model Assessment Practice) for Study
  Abroad Charting a Course for Quality.
  (Institute for the International Education of
  Students (IES), 2003).
• The IES MAP focuses on four academic areas
• Student learning environment
• Student learning assessment and intercultural
  development
• Resources for academic and student support
• Program administration and development

30
Assessment Highlights in the Field

• Assessment Tools
• Standards of Good Practice for Education Abroad.
  (The Forum on Education Abroad, 2005).
• The Standards of Good Practice for Education
  Abroad use a queries based approach under the
  following eight standards
• Ethics and integrity
• Mission, policies and procedures
• Academic framework
• Organizational and program resources
• Student selection and code of conduct
• Learning environment
• Student learning and development
• Health, safety and security

31
Institutions currently assessing students ICC?
32
Frequency of students ICC assessment (ideal)
33
ICC assessment methods currently being used
34
SUMMARY

• Young, wide-open field
• Great need for better methods and more data on
  many different dimensions of international
  education
• Assessment data need to be utilized to provide
  feedback to students, improve the program and so
  on
• Assessment needs to be multi-method,
  multi-perspective, ongoing, intentional,
  integrated, part of strategic plan

35

• CONTACT INFORMATION
• Darla Deardorff d.deardorff_at_duke.edu
• David Comp dcomp_at_uchicago.edu
• Elaine Meyer-Lee meyerlee_at_saintmarys.edu
• LINKS For more handouts or bibliography on
  assessing intercultural outcomes of study abroad
  programs, go to
• http//www.saintmarys.edu/cwil/php/intercultural.
  learning/IILOutcomes.php
• http//www.forumea.org/related_links.html