Survey Says? Assessing Library Instruction Through Blackboard

1
Survey Says?Assessing Library Instruction
Through Blackboard

• Lynda Kellam
• Amy Harris
• Jackson Library, UNCG
• November 18, 2008

2
In the Beginning

• Campus-wide New focus on assessment
• In the Library Objectives for First-Year and
  Upper-Level Instruction
• What do students know before the library
  instruction session?
• What are students learning in library instruction
  sessions?
• How well are they retaining concepts over time?
• Lack of time for students to do pre- and
  post-tests in class

3
Getting Started

• The survey tool in Blackboard was the ideal place
  for library instruction assessment
• Determining audience
• Designing the survey
• Obtaining Institutional Review Board Exemption
  from full review

4
Designing the survey

• Same questions for pre- and post-test
• Included consent questions
• Used mainly open-ended questions
• Topics chosen were
• Finding books and databases
• Constructing a basic search
• Characteristics of a scholarly article
• Website evaluation
• Citation styles
• Asked if students had previous LI experience

5
Blackboard implementation

• Course Builder or Instructor access to the
  Blackboard course space
• Survey Tool within Blackboard
• Creation and importation of survey
• Launch of survey into course space
• Data extraction

6
The Institutional Review Board

• Protects human subjects involved in research
• Necessary when you plan to publish results
• Exempt v. non-exempt procedure
• Preparation for Institutional Review Board
  exemption from full review
• Approved for exemption!

7
Pilot Project

• Spring 2008
• One 200-level Political Science class
• One 300-level Political Science class
• Five English 101 classes
• Participation Rates
• Pre-test (88 to 23)
• Post-test (38 to 14)
• Our Blog for tracking issues
• Issues
• Question evaluation
• Post-test participation rates
• Changes made (new questions and new sites)

8
Second Stage

• Fall 2008
• One 200-level Political Science class
• Four English 101 classes
• Two English 102 classes
• Participation Rates
• Pre-test (95 to 25)
• Post-test (75 to 14)
• Same number of classes, but doubled
  participation. How?

9
Data analysis

• Cleaning out extraneous code
• Visual analysis pre-session (who are these
  students?)
• Creation of codebook
• Coding data
• Importing data into SPSS
• Descriptive statistics

10
Participation Comparison
11
Crosstabulation for fall 2008
LI Class Crosstabulation LI Class Crosstabulation LI Class Crosstabulation
Freshman Sophomore Junior Senior NA Total
No Instruction 42 15 4 0 4 65
Previous Instruction 3 13 5 3 0 24
UN S 15 1 1 0 0 17
No Answer 0 0 0 0 1 1
Total 60 29 10 3 5 107
12
Keyword Searching Question (fall 2008)

• Pre-test Keyword Question
• Type below exactly what you would type in the
  search box of a database to find information on
  the study habits of college students.
• Post-test Keyword Question
• Type below exactly what you would type in the
  search box of a database to find information on
  drug testing in athletics.

Pre-Test Post-Test
Correct Answer 6.0 62.0
Wrong Answer 91.0 37.0
13
Lessons Learned

• Instructors are partners in this project
• Departmental Institutional Review Board
  representative was an excellent resource
• Rethinking questions/survey writing
• Survey fatigue
• Scalability concerns
• Limits of approach
• Use of data