Assessment

1
Assessment

• Developing an Assessment

2
Assessment Planning Process

• Analyze the environment
• Agency, clients, TR program, staff resources
• Define parameters
• Function, content, implementation strategy
• Select or develop assessment
• Establish assessment protocols
• Train staff interns on protocols
• Peterson Stumbo, 2000

3
Assessment Implementation Process

• Review assessment protocol
• Prepare for assessment
• Administer assessment to client
• Analyze or score assessment results
• Interpret results for placement into programs
• Document results of assessment
• Reassess client as necessary/monitor progress
• Peterson Stumbo, 2000

4
Selection of Assessment Procedure

• Determine the purpose
• Specify the exact content to be covered
• Identify other selection criteria
• Search assessment resources
• Compare possible procedures to selection criteria
• Select assessment

5
Dont Create an Agency Assessment When

• A commercial assessment is readily available that
  fits purpose clients
• The staff has limited expertise or interest in
  test development or reliability validity
• Programs are not based on a comprehensive program
  design
• Stumbo Peterson, 2009

6
Reality

• In actual practice, more therapeutic recreation
  specialists rely on agency-specific assessments
  than on nationally available tests.
• Peterson Stumbo, 2000, p. 221
• Why?
• Content of commercial assessments rarely matches
  content of programs or agencies

7
Rationale for Teaching Development to
Undergraduate Students

• Many individuals feel that assessment
  development and validation is a complex enough
  process that it should not be advocated as a task
  for the typical TRS. Individuals with bachelors
  degrees may not have the conceptual and
  statistical expertise for development and
  validation. Another view states that TRSs are
  more likely to develop their own assessments
  (good or bad) than buy commercial assessments so
  instruction in correct methodology is important.
  (Stumbo, 2002, p. 163)

8
Developing an Assessment

• Select the assessment content
• Comes from content of program
• Based on current literature, needs of clients,
  and TR models or theories
• Develop list of programs designed for a specific
  group
• Define client goals

9
Developing an Assessment

• Determine the assessment purpose
• Placement into programs
• Basic screening, identifying problems, narrow
  problems, reassess problems
• Understand client group
• Disabilities, education, backgrounds, etc.

10
Developing an Assessment

• Determine type of assessment
• Interviews, observations, self-administered
  surveys, record reviews
• How many clients assessed?
• How much staff time is available for assessment?
• What are the skills of staff?
• What will yield the best information to place in
  programs?

11
Developing an Assessment

• Select the client outcomes to be assessed
• Outline the assessment content
• Make a two-way chart
• Make sure assessment is representative of content
  of program
• See p. 170

12
Developing an Assessment

• Select the type of items to use
• Closed-ended or open-ended
• Yes/no, true/false, matching, multiple choice,
  rating, ranking closed
• Fill-in-the-blank, short answer, extended
  answer open
• Match the items to specific client outcomes

13
Developing an Assessment

• Make the final selection of items
• See Stumbo, 2002, p. 181-182
• Norm group
• Validity reliability

14
Developing an Assessment

• Make manual/protocol
• Directions for administration, scoring,
  interpretation, and reporting
• Reliability validity
• See Stumbo, 2002, p. 182-183
• Pilot test make revisions

15
Whats Needed in a Protocol

• Preparation
• Environmental conditions
• Resources
• Length of time
• Administration
• Instructions for interview, observation, test
• What to do in exceptional circumstances

16
Whats Needed in a Protocol

• Scoring
• How answers to each question s/b scored
• How the total assessment is to be scored (e.g.
  subscores, total)
• Analysis and Interpretation
• How items or test is to be interpreted for
  program placement

17
Whats Needed in a Protocol

• Reporting/documentation
• How results are to be recorded
• How results are to be reported
• Stumbo Peterson, 2009

18
Do Create an Agency-Specific Assessment When

• An appropriate commercial assessment cant be
  located
• TR programs or clients have unique
  characteristics
• A comprehensive program is designed and results
  in a clear view of program content to be used in
  an assessment

19
Do Create an Agency-Specific Assessment When

• TR staff have time to make an assessment
• TR staff know how to design an assessment
• TR staff know how to design an assessment
  protocol
• Stumbo Peterson, 2009