Lessons from the evaluation of learndirect telephone guidance trial

1
Lessons from the evaluation of learndirect
telephone guidance trial

• Becci Newton (IES) Rosie Gloster (CESI)

2
Background context

• Skills White Paper Getting on in business,
  Getting on at work Leitch Review of Skills
  called for universal, high quality IAG service
• Decision to develop an advancement and careers
  service
• Existing adult guidance delivered through
  nextstep
• learndirect well-placed to deliver distance
  guidance trial supported by e-learning

3
Multi-method evaluation approach
4
Demand, no displacement

• Between Jan-July 2006, 46,000 calls and 14,000
  action plans
• Seasonal demand, but overall upward trend
• Callers not nextstep clients

5
Satisfaction high, good outcomes

• 86 per cent agreement that the overall quality of
  the service was good
• Soft outcomes more likely than hard outcomes

6
High quality, cost-effective

• Scored quality analysis of calls
• 52 per cent of calls graded good or excellent
• Scores did not differ between client groups
• Working with one advisor beneficial
• Limitations to cost-effectiveness analysis
• different groups, services, includes learndirect
  set-up costs
• Marketing affects costs
• Cost per session much the same for learndirect as
  nextstep
• Supply only learndirect more cost-effective
• Overall learndirect cost-effective may become
  more so

7
Lessons demand-side

• Diversity and equal opportunities pros and cons
• reduced risk of stereotyping as no visual clues
• but no information eg if presentation is a
  barrier
• gender mix of adviser and user can affect quality
• older adults under-represented
• can does not mean will
• cant does not mean wont
• users may want mixed media
• Different users may need different services
• preferences for f2f, telephone etc.

8
Lessons supply side

• Quality separate from medium
• impacts outcomes and satisfaction
• enhanced by co-location of advisers
• training development, formal informal
• Make the medium work for the user
• dont generalise needs
• language and translation services
• Continuity with same adviser important but good
  records facilitate client sharing

9
Lessons supply side

• Similar cost to face-to-face service and no
  displacement effect
• cost-effective due to call centre system?
• f2f appointments cancel (pushes cost up)
• Targeted or open services / Voluntary or
  mandatory
• learndirect targeted but open to all
• outcomes for mandated customers different?
• implications for relationships and impartiality?

10
Lessons understanding impact

• Satisfaction is high, but
• users wanted and expected - more local labour
  market information
• integration with local services important
• learndirect voluntary, unclear how mandation
  would affect satisfaction outcomes
• Outcomes only partially determined by guidance
  and satisfaction
• not all users will achieve hard outcomes
• some may revise decisions/plans (still positive)

11
Contacts
Ruth Hawthorn NICEC 0207 267 3572 ruthhawthorn_at_bto
penworld.com
12
Publication details

• An evaluation of the UfI/learndirect telephone
  guidance trial (DfES) Research Report 833, Page
  R, Newton B, Hawthorn R, Hunt W, Hillage J,
  (2007)
• ICT and the over 50s do telephone advice lines
  and computers make it harder for older people to
  use guidance services? NICEC briefing, Hawthorn
  R, (2008)

13
Discussion ideas

• What distance remote services already exist
  around the EU?
• How to generate demand for new service/medium?
• What possibilities exist to integrate distance
  remote services with local agencies for local
  labour market information?
• Implications of electronic services on take-up by
  diversity factors?

14
thank you
www.employment-studies.co.uk