Baiting the Hook: Creating a Culture of Assessment

1
Texas Association for Institutional Research
February 6, 2008

• Baiting the Hook Creating a Culture of
  Assessment
• Evilu Pridgeon Lisa Casto

2
Overview of Presentation

• AiDallas
• History
• SACS
• How it happened
• What we did
• The Results

3
The Art Institute of Dallas

• Private College
• SACS accredited since 1997
• Program Accreditation in Interior Design and
  Culinary Arts
• 1700 students
• 100 full- and part-time faculty (60/40)

4
Degrees Offered

• Culinary Arts Associate of Applied Science (AAS)
• Restaurant Catering Management, AAS
• Art of Cooking Certificate
• Design
• Advertising, Bachelor of Fine Arts (BFA)
• Graphic Design, Associate of Applied Arts (AAA),
  BFA
• Web Design and Interactive Media, BFA

5
Degrees (Cont.)

• Fashion Design, AAA, BFA
• Fashion Retail Management, BFA
• Interior Design BFA
• Media Arts Animation
• Video Production AAA
• Digital Filmmaking Video Production BFA

6
History at AiDallas

• Accredited in 1997
• Re-accreditation visit by the Southern
  Association of Colleges and Schools (SACS),
  Commission on Colleges (COC) in 2003
• After two monitoring reports, placed on probation
  Dec. 2006 because of lack of adequate
  documentation of ongoing assessment

7
How It Happened

• Confusing directions
• Poor communication
• Lack of support from leadership
• Perpetual crisis mode

8
How It Happened

• Models were too complicated
• Each department was insulated
• IE Committee was a rubber stamp
• Only two people reviewed plans

9
What We Did

• Created an action plan
• Made major staff changes (president, DIER)
• Hired Institutional Effectiveness Associates (IEA
  -- Jim and Karen Nichols) for training of entire
  faculty/staff and IE Committee
• Developed timeline and shared with Ai community
• Hand-picked committee

10
Training

• Nichols
• All Faculty and Staff
• Grouped by Administrative/Support and Academics
• Rubrics
• Five column model
• Constructive feedback
• Objective writing (SMART)
• Committee
• Selected members attended conference

11
Best Interest of the Student (BIOS)

• All outcomes and objectives were student-focused
• Academics improve the programs to benefit
  students
• Administrative/Educational Support customer
  service focused to improve services to students

12
The Institutional Effectiveness CommitteeKey to
Our Success

• Hand-picked from recommendations
• Primarily faculty should always be
  faculty-driven
• Committed to the success of the school
• Motivated to assist
• Possessed leadership skills with opportunity for
  further development
• Nearly all academic departments represented
• Included a few curmudgeons to keep us honest

13
The Institutional Effectiveness Committee -- Key
to Our Success

• Varied Skill Sets (writing, math, reasoning)
• Continued training in assessment
• Invested in the schools success
• Provide communication and leadership
• Continue to reward and recognize committee every
  quarter

14
Outcomes/Objectives Development

• Faculty/staff brainstormed as a group in each
  department
• From the long list, a short list was chosen
• Turn into IE Committee for review each year
• Outcomes/Objectives change yearly

15
Rubrics Development

• Increased resources in library
• All IEC members received reading material
• Faculty in each department developed rough drafts
  
• Refined by small group which included some IE
  members, faculty from within the department as
  well as wordsmiths from other departments
• Rubrics are continuously refined based on faculty
  and Professional Advisory Committee (PAC)
  members.

16
The Total Process

• Plans are turned in quarterly with previous
  quarters assessment data
• Each plan reviewed by two or more members
• Constructive feedback
• Focus on meaning and logical processes, not
  grammar
• Chair and IE Director review and give feedback,
  sometimes in person
• Departments make corrections and resubmit

17
Finalization

• Departments submit narrative describing
  department changes and explaining significant
  data
• Subset of committee edits narratives and plans
  into one report
• Report is reviewed by President and VPAA
• Final report submitted to SACS
• Significant changes are reported at all faculty
  and staff meeting

18
Communication

• Updates provided at school-wide meetings
• Improvements in programs/services announced
  yearly
• Annual Presentations to
• Executive Committee
• Board of Trustees

19
The Results

• Mock Visiting Committee said
• Real sense of shared commitment
• If you involve people in the process the
  decision- making -- they own it.
• IE process was embraced
• Faculty/staff felt they were being heard and
  validated.

20
Mock Visit Results (Cont.)

• Employees expressed feeling empowered
• Feel involved, believe it has been a worthwhile
  process and hope it will continue
• It has become part of the culture
• Lots of positive comments about the IEC
• It (IEC) has worked very well has been very
  helpful

21
Comments from Special Visiting Committee

• No basis for making recommendations in any area
• The Committee perceived the results of those
  changes (action plan) to be very positive in
  establishing an evaluation system that apparently
  has significant support from the faculty and
  administration as well
• . . . the Committee perceived a strong
  commitment from those parties faculty and
  academic administrators to continue with the
  efforts underway to enhance the quality of
  learning among their students

22
December, 2006

• The Art Institute of Dallas was removed from
  probation with no further sanctions

23
TIMELINE FOR FY 2008
DATE EVENT RESPONSIBLE PARTY
August 9, 2007 All FY 08 plans due All
August 11, 2007 Institutional Effectiveness Meeting Lisa, Pridge
November 1, 2007 Data Collection Point for Summer 07 EC and Directors
November 5, 2007 Institutional Effectiveness Meeting Lisa, Pridge
January 28, 2008 Data Collection Point for Fall 07 EC and Directors
Week of January 28 Institutional Effectiveness Meeting Lisa, Pridge
May 5, 2008 Data Collection Point for Winter 08 EC and Directors
Week of May 5 Institutional Effectiveness Meeting Lisa, Pridge
August 14, 2008 All FY 08 IE plans due EC and Directors
Week of August 14 Institutional Effectiveness Meeting Lisa, Pridge


24
Ongoing

• Institutional Effectiveness is a part of our
  culture
• Ongoing training for new faculty and committee
  members
• The road goes on forever
• and the party never ends

25
Contact Information

• Lisa Casto, Director of Library Services
• Chair, Institutional Effectiveness Committee
• lcasto_at_aii.edu
• 214-692-8080, ext. 1246
• Evilu Pridgeon, Director of Institutional
  Effectiveness Research
• epridgeon_at_aii.edu
• 214-692-8080, ext. 1177