PRAXIS

1

• PRAXIS
• Speech-Language Pathology
• and
• Audiology
• Tests
• Barbara Parr
• Assessment Specialist, Licensure Development
  Group
• Assessment Development Division
• November 18, 2006

2
Overview

• The Praxis Series
• Licensure Tests
• Test Development Process
• Test Preparation Products and Services

3
The Praxis Series
Professional Assessment for Beginning Teachers

• Licensure
• State regulated process
• Grants permission/legal authority to practice
• Indicates the professional has met the standards
  for training and practice experience
• Indicates readiness for independent professional
  practice
• Indicates minimal competence

4
The Praxis Series

• Licensure Testing
• High stakes
• Required, not voluntary
• One of multiple requirements
• Measure knowledge, skills, and abilities deemed
  important for professional practice
• Does not predict performance
• Must be legally defensible and professionally
  credible

5
The Praxis Series

• Age of the program over 55 years old
• User states jurisdictions
• Each state or agency adopts the most appropriate
  tests for its licensure titles
• Each state or agency sets its own cut score for
  each test
• Praxis staff work with states and agencies to set
  cut scores (standard setting studies)
• Number of test titles 140

6
The Praxis Series

• Number of administrations per year 8
• Number of test centers across the country 650
• Number of test takers for selected tests
• PPSTgt150,000
• Elementary Education Curriculum, Instruction,
  and Assessmentgt 35,000
• Speech-Language Pathology gt5,000
• Audiology gt 1,500

7
The Praxis Series

• Test Offerings
• Praxis I PPST (Pre-Professional Skills Tests)
• Measure basic proficiency in reading, writing,
  and mathematics
• Computer and paper/pencil delivery
• Praxis II Subject Assessments
• Specialty Area tests (140 titles)
• Principles of Learning and Teaching (PLT) tests
  (4 titles)
• Multiple Subjects Assessment for Teachers (MSAT)
  (2 titles)

8
The Praxis Series

• Praxis III Classroom Performance Assessments
• Direct observation in the classroom
• Related Programs
• School Leaders Series
• School Leaders Licensure Assessment (SLLA)
• School Superintendent Assessment (SSA)
• ParaPro Assessment

9
Praxis Speech-Language PathologyandPraxis
Audiology

• Intended for examinees who have completed a
  degree program.
• Recognized as the national examination in
  speech-language pathology and audiology
• The test is one of several requirements for the
  Certificate of Clinical Competence issued by the
  American Speech-Language-Hearing Association
  (ASHA).
• Some states use the examination as part of the
  licensure procedure.
• Complete information may be about
  certification/licensure from the authority (ASHA
  or state agency) from which certification or
  licensure is sought.
• .

10
TEST DEVELOPMENT COMMITTEE
ETS RESEARCH
ASHA

• Format outcomes of the domain analysis into a
  survey
• Conduct a large-scale confirmation survey

• Analyze Standards
• Analyze Domain
• Define Subset of Domain
• Draft Test Specifications
• Recommend Test Design

• Test Development Committee consists of ASHA
  representatives, content experts, and higher
  education faculty.
• The committee is diverse in terms of race,
  gender, and educational setting (elementary,
  secondary, college, etc.)

11
(No Transcript)
12
Item Development

• Item writers are referred to ETS by ASHA.
• Beginning an online initiative to access writers
  throughout the country
• Each item that appears on the test undergoes
  external and internal reviews.

13
Item Types

• Discrete multiple choice
• Case studies
• Assess knowledge of possible applications to
  clinical situations and issues.
• Research articles
• Assess ability to synthesize information and to
  apply it to specific examples

14
Praxis Speech-Language Pathology and Audiology

• 2 hours
• 120 Multiple Choice Questions
• Cost 75

15
Registration

• All information is available online www.ets.org

16
Test Takers with Disabilities

• All requests for testing accommodations are
  subject to approval by ETS and must meet ETS
  documentation criteria.
• Reasonable testing accommodations are provided to
  allow candidates with documented disabilities
  (recognized under the ADA) an opportunity to
  demonstrate their skills and knowledge.
• http//www.ets.org/disability/index.html

17
Test Preparation Products and Services

• Test at a Glance available free online
• Test specifications
• Sample questions, answers with explanations
• SLP eStudy Guides will be available in January,
  2007
• Workshops available for faculty of institutions
  of higher education

18
Questions and Comments
19
Praxis Speech Pathology and AudiologyStatistics

• Kathryn L. Ricker
• Associate Psychometrician
• Center for Statistical Analysis
• ETS, Princeton, NJ

20
Purpose of the Praxis Tests

• The Audiology and Speech Language Pathology tests
  measure examinees academic preparation in and
  knowledge of the fieldit is one of several
  requirements for the Certification of Clinical
  Competencealso used by some states for licensure
• These tests are to aid in determining
  preparedness to enter the professionthey are NOT
  diagnostic, and scores beyond the cut score are
  NOT meant to be predictive of success in the
  profession

21
Whats a psychometrician?

• Oversees all statistical processes related to the
  test title
• Equating
• Item Analysis
• Differential Item Functioning
• Test Analysis
• Consults on measurement issues
• Scoring, test delivery
• Ensures that statistics are used appropriately to
  ensure fairness and validity (tools, not weapons,
  or magic)

22
What is Item Analysis?

• A statistical description of how examinees
  responded to each item in a test
• Reveals the difficulty of the items
• Shows the relative attractiveness of each option
  in a multiple-choice item
• Indicates how well the item discriminates among
  the examinees

23
What Can IA Help Detect?

• Wrong key
• Typographical errors which make the key incorrect
• Key has become incorrect because it is obsolete

24
Test Form Equating

• Test forms are not exactly equal in
  difficultyscores must be aligned statistically
  to be FAIR
• Scaled scores mean the same thing across forms,
  raw scores do not!
• Use a set of items common to an old
  form-performance on these items helps establish
  equivalence of the forms

25
Differential Item Functioning

• Statistical means of examining tests for bias for
  one group over another
• Males vs. Females, Ethnicity (if sample size
  allows)
• Conditions on total test score (i.e., compares
  item performance relative to like test scores)
• Used as a guide for careful examination by
  committees of item reviewers

26
Some Notes on the Test Stats
27
Notes from the Test Stats

• Repeaters-tend to have lower mean test scores by
  about 1-1.5 standard deviationsrepeaters need to
  try to use the diagnostic information and test
  prep materials that are providedthis requires
  time!
• Subscores (category scores) are less reliable
  than total test scoresbe cautious when
  interpreting raw scores at the level of the
  content categories!

28
What can these stats tell me to help my students
prepare?

• Individual level feedback
• Institutional level feedback

29
Understanding Individual Scores
30
Understanding Individual Scores
Raw category score
31
Understanding Individual Scores
Raw category score
Best possible score
32
Understanding Individual Scores
Rough estimate of typical performance
Raw category score
Best possible score
33
Understanding IHE Statistics

• Sample size is important!
• Be careful when interpreting statistics with
  small samples (especially less than 30-50
  candidates)
• Score distributions
• By gender, ethnicity
• Detailed Category Score Information
• Detailed Score Information by Quartile
• Why are National and State-wide data both
  included for comparisons?

34
(No Transcript)
35
(No Transcript)
36
FAQs

• I took the test twice and got too different raw
  scores (when I add up my number correct in each
  category) but the same scaled score. Whats up
  with that?
• I got a scaled score of 590, but the conditional
  standard error of measurement (CSEM) is 10.
  Doesnt that really mean I could have gotten a
  610?
• My friend and I took the same test at the same
  time and our number correct are the same, but our
  scaled scores are different. Doesnt that mean
  there was some kind of mistake made?

37
It Sure Isnt the National Exam That I Took

• Beth Mineo Mollica, Ph.D.
• University of Delaware
• Praxis SLP National Advisory Committee

38
Striving for

• Balance across content areas
• Keeping reading burden reasonable
• Currency
• Logical association with practice demands

39
Covering the Breadth of the Field
40
Reflects Practice Demands

• Application of knowledge to clinical situation
• Question posed in context of clinical situation
• Series of questions tied to clinical scenario
• Extracting information from research
• Integration of information
• Working knowledge of law/regulations and their
  application

41
Logistics

• 120 questions
• 2 hours in duration
• Multiple-choice format
• Item clusters tie multiple questions to a
  single reading passage

42
Establishing Test Items

• Every item reviewed for currency and accuracy
• Large inventory of new items generated and
  extensively vetted

43
This investigation was motivated by observations
that when persons with dysarthria increase
loudness, their speech improves. Some studies
have indicated that this improvement may be
related to an increase of prosodic variation.
Studies have reported an increase of fundamental
frequency (F0) variation with increased loudness,
but there has been no examination of the relation
of loudness manipulation to specific prosodic
variables that are known to aid a listener in
parsing out meaningful information. This study
examined the relation of vocal loudness
production to selected acoustic variables known
to inform listeners of phrase and sentence
boundaries specifically, F0 declination and
final-word lengthening. Ten young, health women
were audio-recorded while they read aloud a
paragraph at what each considered normal
loudness, twice-normal loudness, and half-normal
loudness. Results showed that there was a
statistically significant increase of F0
declination, brought about by a higher resetting
of F0 at the beginning of a sentence and an
increase of final-word lengthening from the
half-normal loudness condition to the
twice-normal loudness condition. These results
suggest that when some persons with dysarthria
increase loudness, variables related to prosody
may change, which in turn contributes to
improvement in communicative effectiveness.
However, until this procedure is tested with
individuals who have dysarthria, it is uncertain
whether a similar effect would be
observed. Watson Hughes (2006) The
relationship of vocal loudness manipulation to
prosodic F0 and durational variables in healthy
adults. JSLHR, 49, 636-644.
44
Question 1
Which of the following represent the independent
variable(s) used in the Watson and Hughes
study? (A) Prosody of dysarthric speech (B)
Gender of the speaker (C) F0 declination and
final-word lengthening (D) Vocal loudness (E)
Speech intelligibility and communicative
effectiveness
45
Question 1
Which of the following represent the independent
variable(s) used in the Watson and Hughes
study? (A) Prosody of dysarthric speech (B)
Gender of the speaker (C) F0 declination and
final-word lengthening (D) Vocal loudness (E)
Speech intelligibility and communicative
effectiveness
46
Answer to Question 1
An independent variable is manipulated or
selected by the researcher to determine its
effect on the dependent variable(s). Watson and
Hughes manipulate vocal loudness to determine its
effect on prosodic F0 and durational variables.
Therefore, (D) is the correct answer. (A) and (E)
are incorrect, as those variables were not part
of the Watson and Hughes study. As only women
were studies, the effect of speaker gender was
not addressed, so (B) is incorrect. (C) is
incorrect, as both F0 declination and final-word
lengthening represent dependent variables that
change as a consequence of vocal loudness.
47
Question 2
Watson and Hughes are cautious when suggesting
that the speech of some persons with dysarthria
improves due to the prosodic changes that result
from increasing vocal loudness. Of the following,
which is the most likely reason for this
caution? (A) Only women were studied. (B)
Several acoustic variables related to speech
prosody were not included. (C) The prosody of
persons with dysarthria may not show
similar loudness effects. (D) The results were
not statistically significant. (E) Reciting a
paragraph aloud is unlike spontaneous
speech.
48
Question 2
Watson and Hughes are cautious when suggesting
that the speech of some persons with dysarthria
improves due to the prosodic changes that result
from increasing vocal loudness. Of the following,
which is the most likely reason for this
caution? (A) Only women were studied. (B)
Several acoustic variables related to speech
prosody were not included. (C) The prosody of
persons with dysarthria may not show
similar loudness effects. (D) The results were
not statistically significant. (E) Reciting a
paragraph aloud is unlike spontaneous
speech.
49
Answer to Question 2
At the end of the abstract, Watson and Hughes
suggest a relationship between increased vocal
loudness and improvement in communicative
effectiveness in some persons with dysarthria.
This relationship is not directly supported by
their study, as individuals with dysarthria were
not tested, so (C) is the correct answer. (A),
(B), and (E) are incorrect as they do not address
the relationship of ddysarthria, vocal loudness,
and improved communicative effectiveness. (D) is
incorrect for the same reason and, furthermore,
the results were statistically significant.
50
Question 3
Which of the following best describes the
experimental design of the Watson and Hughes
study? (A) A longitudinal design (B) A
multiple-baseline design (C) A between-subjects
design (D) A within-subjects design (E) A mixed
between- and within-subjects design
51
Question 3
Which of the following best describes the
experimental design of the Watson and Hughes
study? (A) A longitudinal design (B) A
multiple-baseline design (C) A between-subjects
design (D) A within-subjects design (E) A mixed
between- and within-subjects design
52
Answer to Question 3
Watson and Hughes use the same group of subjects,
who recite a paragraph under 3 conditions at
normal, twice-normal, and half-normal
loudness. This is an example of a within-subjects
design, in which the dependent variables (in this
case, prosodic F0 and durational variables) are
measured repeatedly in the same subjects under
different task conditions (in this case, vocal
loudness). Therefore, (D) is the correct answer.
(A) is incorrect because a longitudinal design
would require tracking change in the dependent
variables over time. (B) is incorrect, as the
study included no baseline measures. (C) and (E)
are incorrect because there was only a single
group of subjects.
53
Interpretation of PRAXIS Results in the
Preparation of Students The Audiology PRAXIS

• Janet Koehnke, Ph.D.
• ETS Consultant Member of ASHA National Advisory
  Committee on the PRAXIS

54
Agenda

• The New Face of the Audiology PRAXIS
• Number of questions
• Types of questions
• Preparing Audiology students for the PRAXIS
• Common areas of difficulty
• When to take the PRAXIS

55
The New Audiology PRAXIS

• In 2004 we began the process of reviewing the
  test
• Determined areas needing updating and revising
• Decided to make some changes to the format of the
  questions
• New questions and new question types have been
  developed

56
The New Audiology PRAXIS

• Note The Foundation remains the same!!
• 2 hour test
• Multiple-choice format
• Breakdown of question content areas
• From http//www.ets.org/Media/Tests/PRAXIS/pdf/034
  0.pdf

57
The New Audiology PRAXIS

• The Changes
• More applied questions
• Less rote memorization
• From http//www.ets.org/Media/Tests/PRAXIS/pdf/034
  0.pdf

58
The New Audiology PRAXIS

• Other changes
• New audiogram form used throughout test
• Some questions use the entire form
• Other questions may use only the audiogram or
  only the tympanograms, etc.

59
The New Audiogram Form
60
The New Audiology PRAXIS

• A sample question using the audiogram form

61
Case Description
62
Audiogram Question 1
63
Audiogram Question 2
64
For More Examples

• Go to http//www.ets.org/Media/Tests/PRAXIS/pdf/03
  40.pdf
• OR

65
Preparing students for the PRAXIS

• Question content areas have not changed
• Question formats have, in some cases, changed
• Test results from the New PRAXIS over the past
  2 years indicate certain areas where students
  seem to be having difficulty

66
Preparing students for the PRAXIS

• Standards, Laws, Regulations
• HIPAA
• ANSI
• IDEA
• WHO
• FDA
• FERPA

67
Preparing students for the PRAXIS

• Screening
• ASHA versus state guidelines
• Medical audiology
• Research statistics
• Correlation
• Interpretation of acoustic reflexes
• Industrial audiology
• Hearing protection
• Psychoacoustics
• Temporal integration

68
Time to take the PRAXIS?

• In light of
• Changes in degree requirements
• Changes in test format
• It is recommended that students take the
  Audiology PRAXIS in the Fall of their last year
  in the program
• WHY?
• Majority of coursework completed
• Considerable clinical experience completed

69
Time to take the PRAXIS?

• Why not take it earlier to get it out of the
  way?
• Applied nature of the test
• Test is designed to be taken near the end of the
  doctoral program
• Test is presently offered 7 times/year
• We should encourage students to take the PRAXIS
  in November or January of their last year
• If for some reason they do not pass, they can
  take it again in March, April or June

70
Thanks for your attention!QUESTIONS?
71
Answer to sample question 1

• From http//www.ets.org/Media/Tests/PRAXIS/pdf/034
  0.pdf

72
Answer to Audiogram Q 1
From http//www.ets.org/Media/Tests/PRAXIS/pdf/034
0.pdf
73
Answer to Audiogram Q 2

• From http//www.ets.org/Media/Tests/PRAXIS/pdf/034
  0.pdf

74
Audiogram Questions 3 4
75
Answers to Audiogram Q. 3 4

• From http//www.ets.org/Media/Tests/PRAXIS/pdf/034
  0.pdf