Broad-Based Office Skills Testing

1
Broad-Based Office Skills Testing
Los Angeles County Office of Education PERSONNEL
COMMISSION
2
The Problem

• Many clerical/secretarial classifications
• Differ on specific job content and levels of
  responsibility
• Many obstacles to class consolidation -
  logistical, political, fiscal
• Each class has a unique examination
• Variably content and quality
• Not consistently updated
• 2-3 month recruitment examination cycle time
• No overall competency architecture across the
  classes
• Core competencies and competency progression
• Pass point paradoxes (lower pass points for
  higher level jobs based on selection ratios)

3
Purpose and Objectives

• Improve testing quality, reliability, and
  efficiency
• Improve coherence of clerical examination plans
• Reduce redundant testing
• Reduce time to place candidates on eligibility
  lists
• Ensure eligibility lists are always available

4
Strategy

• Core test battery for as many classifications as
  possible
• Annual calendar of assessment opportunities
  (e.g., quarterly)
• Computer-based testing
• Score banking and certification to different exam
  plans
• Different weights and cut-off scores for
  different classifications based upon relevancy
  and level needed.

5
Strategy Similar to SAT ACT for applying to
college

• Sign up and take the test
• Apply to the colleges of choice
• Send scores
• Colleges set their own pass-points and weights
  for final selection decisions

1. Sign up and take the test
2. Apply to the jobs of choice
3. Scores populated
4. Each job has its own pass-points and weights for
   final selection decisions

6
Score Data Flow
7
Development of the Test Battery

• Reviewed job analyses from Agency-Wide
  Classification Study - 2004-2007
• Re-stated duties into generalized work activity
  (GWA) statements with no reference to specific
  job content.
• Map GWAs to each classification
• Identify relevant assessment tools for GWAs
• Via GWAs, map assessment tools to Classifications
• Derive relevancy and weight of assessment tools
  to classifications.

8
Example GWAs

• Greet, interact, and provide information to
  others in person and on the telephone
• Verify, input, and correct data and information
• Receive, sort, amd categorize information and
  materials
• Understand, interpret, explain, and follow
  directions, rules, and regulations
• Operate a personal computer and other common
  electronic office equipment
• Format, edit, keyboard and type documents.
• Gather compile, collate, and assemble information
  and materials
• Maintain and coordinate calendars and schedule
  meetings and events
• Receive, track, and record monetary/numerical
  information and data

9
Matrix of GWA with Classifications
Class GWA Typist Clerk Secretary Accounting Clerk Receptionist
Greet, interact, and provide information to others in person and on the telephone 1 2 1 2
Verify, input, and correct data and information 2 1 2 0
Receive, sort, amd categorize information and materials 2 2 2 1
Understand, interpret, explain, and follow directions, rules, and regulations 2 1 2 1
10
Competencies Assessed - Tests

1. Following Instructions (MC)
2. English Usage Grammar (MC)
3. Microsoft Outlook (MC)
4. Microsoft Excel (Performance, OPAC)
5. Microsoft Word Skills (Performance, OPAC)
6. Telephone - Customer Service (SJ, OPAC)
7. Sequencing and ordering speed and accuracy
   (timed)
8. Computational speed and accuracy (timed)
9. Checking and comparing speed and accuracy (timed)
10. Data Entry speed and accuracy (OPAC)

11
Matrix of Tests with GWAs
Test GWA Telephone Customer Service Data Entry speed and accuracy Sequencing and ordering speed and accuracy Following Written Instructions
Greet, interact, and provide information to others in person and on the telephone x
Verify, input, and correct data and information x x
Receive, sort, and categorize information and materials x
Understand, interpret, explain, and follow directions, rules, and regulations x
12
Matrix of Tests with Classifications
Class Test Typist Clerk Secretary Accounting Clerk Receptionist
Telephone Customer Service 1 2 1 2
Data Entry speed and accuracy 2 1 2 0
Sequencing and ordering speed and accuracy 2 2 2 1
Following Written Instructions 2 1 2 1
13
Result

• Linkage of tests-to-tasks-to-classifications to
  establish job relatedness
• Derived weighting scheme for each classification
• Relevant tests
• Relevant weights
• Job specific competencies identified in the job
  analyses are assessed in specialized examination
  parts
• Min Quals
• Interview
• Performance

14
Applicable Classifications

• Clerk
• Intermediate Clerk
• Senior Clerk
• School Clerk
• Senior School Clerk
• Data Control Clerk
• Senior Data Control Clerk
• Typist Clerk
• Intermediate Typist Clerk
• Temporary Office Worker
• Senior Typist Clerk
• Department Assistant, Dance
• Department Assistant, Music

• Department Assistant, Theater
• Department Assistant, Visual Arts
• Receptionist
• Reader
• Information Resources Specialist
• Media Dispatching Clerk
• Secretary
• Division Secretary
• Legal Secretary
• School Administrative Secretary
• Senior Division Secretary
• Executive Legal Secretary
• Executive Assistant

15
Affected Positions
Classification LACOE Positions of Clerical Positions
Senior School Clerk 110 27
Secretary 78 19
Division Secretary 46 11
Senior Typist Clerk 35 9
School Administrative Secretary 30 7
Intermediate Typist Clerk 28 7
Senior Division Secretary 24 6
TOTAL 351 87
16
Administrative Practices

• Score Life
• 2 years
• Test Administration
• Every 3 months
• Test Retake
• Once every 6 months
• Retake Scope
• Whole Battery not individual tests for
  logistical reasons
• Weighted or Qualifying-Only
• Qualifying Pass/Fail-only for Promotional Exams
• Weighted for Open recruitment

17
Administrative Practices

• Effective Scores
• Only the candidates most recent score, not the
  highest score, can be certified to a list.
• For candidates whose scores enable them to be
  placed on an eligibility list (pass the exam),
  the certified scores stands for the duration of
  eligibility on the respective list.
• Scores already certified are not updated by
  retakes
• Ones score (and relative ranking) on a list can
  not be improved until eligibility expires (one
  year).
• Multiple (10) Pass Points
• Low individual test pass points
• Reduced Adverse Impact
• 3 apparent factors
• Computer skills, speed and accuracy, verbal

18
Recruitment

• Continuous offering of the BBC
• Continuous recruitment for all affected
  classifications
• Establishes a pre-tested pool of potential
  candidates for the affected classes
• For most classifications, limited or no public
  posting for job specific recruitments
• Notifications sent to pre-tested pool, based on
  test cut-off score
• Invitations to apply to take the specialized exam
  parts for the classification

19
No need to go to the general population for each
recruitment
20
Status

• Officially launched on February 9th, 2009
• Advertised on our PC website Weekly Job
  Opportunities emails.
• Two exam administrations in March April phased
  in the BBC in the context of specific
  classification recruitments.
• June and October 2009 BBC administrations as part
  of annual calendar

21
Process Improvements Since Start

• Candidates are automatically notified via email
  when particular job classifications are open for
  recruitment.
• Candidates receive their score report along with
  an Examination Information Sheet to address
  frequently asked questions.
• Scores can (soon) be viewed on-line with ID and
  Password
• Candidates who miss the Scantron exam portion can
  re-register to take the next exam administration.
  
• Candidates who are unable to attend the first
  Computer Exam can attend the Computer Make-up
  exam typically scheduled the following week.

22
Benefits Received and Expected

• Test quality improvement through
• Data monitoring and psychometric analysis
• Empirical validation research
• Development of alternate forms
• Administration efficiencies through
• Managed efficiencies in test administration and
  score managements
• Minimizing retesting of the same candidates for
  different classifications
• Improved communications with candidates
• Recruitment enhancement through
• Maintaining a pre-tested pool of applicants
• Having something to offer at job fares
• Reduced eligibility list production cycle time

23
Next Project BBA

• Broad-Based Administrative Classifications
• Administrative, Analyst, and Professional Series
• Focus on college grads
• Same general model
• Launch in Q1, 2010

24
Thank you!

• Questions?
• Comments?