ICT Assessment, the Librarian

1
ICT Assessment, the Librarians View

• Eileen Stec, Instruction Outreach Librarian,
  Rutgers University
• April 18, 2007

2
What are our choices?

• Multiple choice or similar questions
• ICT online assessment or print assignments/in
  class presentation using authentic tasks
• Electronic or analog portfolio

3
Answer this question

• 1. What is a list of books, journal articles, or
  other materials about a certain topic?
• CHOOSE ONLY ONE ANSWER.
• Bibliography
• Keyword
• Library catalog
• Research database
• Subject heading
• Question from Project SAILS Validation Phase III

4
Answer to question 1

• What is a list of books, journal articles, or
  other materials about a certain topic?
• CHOOSE ONLY ONE ANSWER.
• Bibliography (correct)
• Keyword
• Library catalog
• Research database
• Subject heading

5
SAILS multiple choice assessment

• Pros
• Simple to score
• Easy to compare individuals or cohorts
• Lower cost to build administer than ICT
• Designed using ACRL Information Literacy Standards

• Cons
• Proof of memorization skills
• Cant be generalized to real-world tasks
• Disagree with many answers

6
Authentic Tasks allow assessment of
Lower Order thinking skill Outcome 2.2.a.
Identifies keywords, synonyms, and related terms
for the information needed. Higher Order
thinking skill Outcome 3.3.b. Extends initial
synthesis, when possible, to a higher level of
abstraction to construct new hypotheses that may
require additional information. The
Information Literacy Competency Standards for
Higher Education. American Library Association,
2000.
7
Authentic tasks
Authentic tasks can be created electronically, as
they are in the ICT assessment http//www.ets.org
/ictliteracy/ OR
8
Authentic Tasks
..in print IRIS Online
Catalog Assignment.   1. Briefly describe your
topic ____________________________________________
______ 2. List below, keywords you would use to
locate a subject heading. (Method
1.) ____________________ _____________________
____________________________  3. Use any of
the three subject heading search methods
demonstrated in the Handbook to locate a useful
book you can check out on your topic. As you do
so, place an X in front of all the methods
listed below that were helpful ___ Method 1.
Keyword to find a useful subject heading. ___
Method 2. Browsing alphabetically for a subject
heading. ___ Method 3. Finding hidden treasure
Using a book to help you find other books.   4.
Provide information about the book you located
and checked out Title ___________________________
________________________________ Call Number
__________________________________________________
___ Library that holds the book
________________________________________ A
helpful subject listed in the books record
____________________________  
9
Authentic Tasks
With a rubric Very Good. All of the conditions
in Acceptable are met, however, this student
has provided additional information for the
assignment, e.g. located more than one book,
shown evidence of thoughtful concepts culled from
reading the book, provided additional subject
headings or narrower/broader keywords than
required. Acceptable. Each question (1-6)
contains at least one correct answer and the book
the student reports has the status Checked
Out Unacceptable. Not all of questions 1-6 are
answered or the information seems phony OR The
book the student was supposed to check out of the
library is on the shelf or is a non-circulating
book (Reference, etc.)
10
Portfolios Authentic tasks

• Work samples anchored to performance objectives
• Work samples collected during the process of
  instruction (no special tests created)
• Regular assessment using a rubric, student
  responses evaluated and scored
• The Systematic Design of Instruction. 2001.
  Walter Dick, Lou Carey James O. Carey.

11
ICT

• Pros
• -Can access partial understanding.
• -Real world scenarios.
• -Assessment validated.
• -Designed using ACRL Information Literacy
  Standards
• Cons
• -Expensive to construct.
• -Core level low on library-like tasks.

12
Authentic home grown

• Pros
• -Inexpensive to develop.
• -Real world scenarios individualized.
• Cons
• -Very time-consuming to score.
• -Assessment not validated.

IRIS Online Catalog Assignment.   1. Briefly
describe your topic ______________________________
____________________ 2. List below, keywords you
would use to locate a subject heading. (Method
1.) ____________________ _____________________
____________________________  3. Use any of
the three subject heading search methods
demonstrated in the Handbook to locate a useful
book you can check out on your topic. As you do
so, place an X in front of all the methods
listed below that were helpful ___ Method 1.
Keyword to find a useful subject heading. ___
Method 2. Browsing alphabetically for a subject
heading. ___ Method 3. Finding hidden treasure
Using a book to help you find other books.   4.
Provide information about the book you located
and checked out Title ___________________________
________________________________ Call Number
__________________________________________________
___ Library that holds the book
________________________________________ A
helpful subject listed in the books record
____________________________
13
Portfolios - Authentic

• Pros
• -Individualized and real world assessment.
• -Rubrics, properly tested and applied can compare
  one students work to another.
• Cons
• -The most time-consuming of all assessments.
• -requires work over time, not a one-shot
  assessment

14
Resources

• ACRL Information Literacy Standards
• http//www.ala.org/ala/acrl/acrlstandards/standard
  s.pdf
• Project S.A.I.L.S.
• https//www.projectsails.org/
• ETS ICT Assessment
• http//www.ets.org/ictliteracy/