An Adaptive Approach to Psychological Test Battery Selection

1
An Adaptive Approach to Psychological Test
Battery Selection
Tammie Olm B. Sc. (Hons.), and Graeme Senior
Ph.D., University of Southern Queensland,
Toowoomba, Queensland, Australia
Introduction
Composite Calculation Compute deviation
quotient for the composite (DQ) as a Standard
Score using DQ ((Xc-Mc)/SDc) x 15 100
Standard Error of estimate (SEe) can be
calculated calculated for confidence intervals
Which are placed symmetrically around the
predicted true score (DQPT) DQPT rcc (DQ-100)
100
Advantages The alternate verbal composite
(VCC) is more clearly related to word knowledge,
including a measure of word recognition (STW),
word meaning (Vocabulary), and word
pronunciation (WRAT-3 Reading). The combination
of these measures is supported by factor analytic
studies. Additionally, WRAT-3 Reading serves a
dual purpose of providing an estimate of formal
educational achievement and can be employed in
discrepancy analysis with naming and verbal
fluency measures, thus increasing the efficiency
of testing time. The alternate verbal
composite may be calculated with minimal loss in
composite reliability (I.e. rvcc 0.95, rvci
0.96).
One of the greatest challenges facing clinicians
is how to process and interpret the large amount
of test data obtained from a psychological
assessment. Fixed test batteries have the
advantage of a specified number of tests for
which the relationships and psychometric
properties are generally well known and which are
combined into composite scores for ease of
interpretation and analysis (i.e. WAIS/WMS). This
often comes, however, at the cost of long
administration times or the administration of
tests which are poorly suited to the needs of the
client. Conversely, flexible batteries, in
which test composition varies, have the advantage
of assessing behaviour within practical
constraints imposed by specific client abilities,
time limitations, and test availability.
However, the relationships amongst measures are
either unknown, or more commonly not incorporated
into the system of analysis. Clearly both
approaches have strengths and weaknesses. The
purpose of the current study is to bridge the gap
between these two approaches utilising the
psychometric properties of flexible measures to
generate analysis and interpretative guidelines
consistent with fixed batteries. This poster
demonstrates the utility of mathematical
procedures to achieve composites suitable for
analysis and interpretation from measures chosen
to address specific clinical issues.
Illustrative Examples
The following three examples illustrate the use
of these computational procedures to permit
substitution of other measures into a composite.
3) Visual Memory Composite A combined
WAIS-III/WMS-III factor analysis conducted by
Tulsky and Price (2003) found the Faces subtest
to have markedly low loadings on visual memory
(Faces I 0.32 Faces II 0.37). This and other
research suggests facial recognition is a unique
cognitive ability which may be unsuitable for
inclusion into composites of visual memory
performance. Alternate visual memory composites
for both immediate and delayed visual memory were
calculated substituting Visual Reproduction (I
and II) for Faces (I and II). The resulting
composite scores increase the reliability
measuring visual memory performance. Table 2.
Composite Reliability Comparison
1) Working Memory Specific tests may be
unsuitable for the individual client (e.g.
including Arithmetic (AR) in WAIS-III Working
Memory (WMI) in a client with specific numerical
deficits). Alternatively, the clinician may wish
to combine WMI subtests from the WAIS-III and
WMS-III as suggested by factor analytic research
(Tulsky Price, 2003). Comparisons of the WMI
reliabilities with AR, with AR removed and with
Spatial Span substituted for AR are shown in
Table 1. Table 1. Composite Reliability
The third condition produces a single composite
for working memory which includes WAIS-III and
WMS-III measures which does result in a
substantial reduction in reliability and permits
the exclusion of the Arithmetic subtest.




Mathematical Procedures
In order to generate composites that reflect
different combinations of tests, the following
must be available or computed 1) means and
standard deviations for an appropriate normative
sample 2) measures of internal consistency 3)
correlations among all variables combined in the
composite. For ease of computation, all
measures should be converted into standardised
scores using the same scale. We recommend for
ease and consistency across measures using Scaled
Scores (Mn10, SD3) for tests, and Standard
Scores (Mn 100, SD 15) for composites. Composi
te Computations For each measure, raw scores
must be converted to Scaled Scores. Compute
the composite mean which is the sum of the
individual test means (Mc 10 x number of
tests). Compute the composite standard
deviation calculated from test variances.
Compute composite reliability from test
reliabilities and correlations between all tests.
rcc (?rxx2?rxy)/(n2?rxy)
Discussion
These computational procedures allow clinicians
to flexibly meet diverse clinical requirements
without compromising the advantages of a fixed
battery approach. As demonstrated, combining
tests chosen for appropriate clinical or
theoretical reasons can result in equally or more
reliable composites than those currently
available. The true advantage, however, lies in
the ability to combine any tests for which
sufficient psychometric properties are known to
compute more stable and reliable composites. The
advantage of the fixed battery approach is that
the degree of error is known. Application of
these algebraic methods to combine tests chosen
for clinically relevant reasons also permits the
degree of error to be known. In this way some of
the advantages of fixed batteries can be applied
to their flexible counterparts without
compromising clinical utility.

• 2) Verbal Comprehension
• Time constraints provide a substantial
  limitation to real-world clinical practice.
  Assessment may be detrimentally influenced by
  testing fatigue caused by use of a lengthy
  testing battery. In many instances it would be
  optimal for the clinician to use efficient tests
  which serve a dual purpose without compromising
  reliability. Using the proposed method, the
  clinician could satisfy these practical
  constraints, combine tests to measure the
  underlying cognitive construct and evaluate the
  reliability of this composite score.
• WAIS-III VCI
• Vocabulary - word meaning
• Similarities - abstract verbal relationships
• Information - general knowledge
• Alternate VC Composite
• Spot The Word (STW) - word discrimination
• Wide Range Achievement Test 3 (WRAT 3)
• Reading - word pronunciation
• WAIS-III Vocabulary - word meaning

References
Tulsky, D. S. Price, L. R. (2003). The Joint
WAIS-III and WMS-III Factor Structure
Development and Cross-Validation of a Six-Factor
Model of Cognitive Functioning, Psychological
Assessment, 15, 149-162.