New Ideas

1
New Ideas Training Methods to Dramatically
Improve Productivity

• Ron Clark
• I/H/R Research Group
• Las Vegas, NV

2
Many of you in this room are in the call center
business, or buy interviewing from telephone
banks. This has been a controversial subject
area for the last few years. ? Lots of
work has gone to off-shore locations. ?
There are behind-the-scenes debates over
loss of telephone work to the internet. ?
Then, there is the growth of the cell phone
culture, a whole host of privacy issues, and
TCPA regulations.
3
From an academic standpoint, there is talk that
telephone work is no longer representative
? Too many refusals. ? Call-blocking
devices. ? Cost pressure to avoid
cell-phone samples. ? Deadlines which do
not allow enough time to perform
callbacks. ? A great variation in work
quality, partially caused by tremendous
cost pressure and the use of temp
workers to fill seats.Because of issues such as
these, some say that telephone data collection is
dead.I disagree.
4

• However, if you want to compete cost effectively
  
• It is time to act .
• Call centers can do things to make interviewing
  better.
• Things which are inexpensive, but which can
  significantly improve the quality of the work.
• Things which also will save lots of money at the
  same time.

5

• Refusal rates are at the center of the problem.
• ? The higher the refusal rate, the less
• representative the ending sample of
• completed interviews.
• ? Especially, if no attempt is made to
  convert
• refusals to help decrease non-response bias.
• ? The higher the refusals, the more expensive
• the work.
• ? High refusal rates also make it difficult
  to
• meet deadlines.
• ? And, refusals negatively impact interviewer
  
• morale.

6

• Going to focus on several interrelated issues ...
• ? The hiring process for call center workers.
• ? The wording and content of the
• questionnaire introduction.
• ? Training issues regarding the way the
• "intro" is thought of and read by the
• interviewer.
• ? And, methods to evaluate workers from the
• standpoints of both quality and profitability.

7

• But first, lets set a background for both the
  problem and opportunity.
• Years ago, very few people refused to be
  interviewed by telephone.
• Im going to guess 10 to 15 max in the
  mid-1960s. In fact, most people apologized for
  not doing surveys.
• But now, refusals in general are very high.

8

• The MRIA (in Canada) tracked refusal percentages
  as ...
• 1995 1999 2002
• 66 68 78
• (RDD -- 50 incidence moderate
• Q length/public policy subject.)

9

• In the USA, both CMOR and CASRO data show similar
  general magnitudes for refusal rates ...
• 2003 CMOR estimate was
• 75.4
• After that 2003 measurement, the
• yardstick was changed to voluntary
• self-reporting.

10

• What is going on?
• In part, refusals should be viewed as
  situational.
• Obviously, there is considerable mythology about
  situations to avoid. And, simple common sense
  can develop a fairly good list of times when
  refusals will be high
• Periods of intense media viewing, either
• among the general population or for
• specific segments.
• Super Bowl? World Series NBA playoffs?
  NCAA?
• American Idol? Academy Awards?
• Political Debates? International
  Events?

11
? Holiday periods both National
Religious. Mothers/Fathers Day?
Thanksgiving weekend? ? When the target
respondent will likely not be home
? Natural Disasters, Weather, Etc. What
is not so obvious is that almost everyone
responds to a telephone survey occasionally. It
all depends on the subject and timing of the
call. Since most refusals happen within a few
seconds of the interviewer initiating the
conversation, lets think specifically about the
introduction and whatever else is said
up-front.
12

• MAJOR PREMISE 1
• The introduction may be the most important part
  of the questionnaire.
• Dealing with the intro in the right conceptual
  manner is also the most important part of
  interviewer training.
• o Controls non-response bias
• o Major influence on C/P/I

13

• Unfortunately, there appears to be considerable
  conflict and confusion regarding intros -- both
  in practice and in theory.
• Looking at actual questionnaires, listening to
  interviewers call our homes, and reviewing much
  of the professional literature, there are
  obviously a variety of styles
• - Some intros are very short. Others are
• painfully long and tedious.
• - Some explain the purpose of the study, or
• attempt to motivate the potential respondent
• as to why they should participate.
• - Some include the interviewer's name.

14

• - Some identify the sponsor of the research
• and/or the name of the call center.
• - Many quote a questionnaire length --
• perhaps favorably shaded.
• - Some combine elements of both the intro and
• screening by asking for a specific type of
• person with certain qualifications right up
• front. (Note Self-selection is fast, but
  not
• technically acceptable.)
• - Others attempt to scientifically select from
• all household members, using either a "last
• birthday" or complex enumeration method.
• - And, if there is an incentive, should that be
• in the intro?

15

• - Should there be a mention of how the
• telephone number was pulled, or how this
• person's name got on the list?
• - Do we need to remind them the call might be
• monitored? Remember, this is a legal
• requirement in many states. Do we always
• follow legal guidelines?
• - And, then there is the famous we are not
• selling anything.
• - Etc. Etc. Etc.
• Later on, we are going to propose a different way
  to deal with the various elements often put into
  the introduction.

16

• For now, let's say many of these aspects do not
  fit well with what the introduction needs to
  accomplish.
• To add to the confusion of what might be an
  optimal introduction, some industry associations
  are publishing semi-official guidelines" for
  research practices which impact introductions.
• MRA CMOR
• CASRO

17

• Think of this ...
• o These guidelines are not very consistent.
• o Will we see more of this in the future?
• AMA? AAPOR?
• o If you belong to MRA or CASRO, do you
• have to follow their rules? What if the
• guidelines conflict?
• o Will the government intercede with their
• own set of rules?
• Should guidelines be the same for all studies?

18

• MAJOR PREMISE 2
• Should introductions be different for certain
  classes or categories of research?
• Yes.

19

• Hi. I'm with the FBI and would like to ask you
  a few questions.
• That intro will work every time -- and who cares
  what else is included.
• Same thing for the IRS, Centers for Disease
  Control (CDC) or various agencies involved in law
  enforcement.
• These organizations have a certain amount of
  "moral authority. And, their call to you is of
  obvious importance.
• Typical research surveys have neither.

20

• Some degree of moral authority, importance or
  interest can be added
• (or created) by ...
• o An easy-to-recognize worthy and important
• purpose. (terrorism?)
• o An existing and important business
• relationship with customers.
• o A well-known name. (JD Power?)
• o An important product in use by the
• respondent.
• o Ties with affinity groups. (cat lovers or
  PETA)
• o Etc.

21

• MAJOR PREMISE 3
• Whenever we can truly add moral authority, we
  should try to do it.
• However, be realistic.
• There is no such thing as an important" study
  about ...

22

• - Fast Food
• - Grocery products
• - Home improvement
• - Financial services or credit cards
• - Newspaper readership
• - Telephone service options
• - Radio listening patterns
• - Advertising recall
• - Aluminum siding or stucco
• - Automobile dealers
• - Etc.
• Instead, the interviewer needs to have the use of
  clever words and flawless technique.

23

• It is useful to look at a couple of introductions
  to see contrasting styles
• "Hi. We're doing a short opinion
• survey and would like to ask you
• a few brief questions."
• (6 seconds)
• or ...

24

• Hi. My name is ____ and I'm calling from ____
  on behalf of _____, a national research firm. We
  are conducting a state-wide survey to ask
  people's opinions on current and future issues
  facing our communities. Your opinions can
  directly influence changes that may be made to
  current policy with regard to local issues.
• We are not selling anything, and everything you
  tell me will remain completely confidential. The
  survey will take about 15 minutes depending on
  your responses. This call may be monitored for
  quality control purposes.
• I was hoping to speak with the adult in your
  household who had the most recent birthday.
  Would that be you?
  
• 
  (44 seconds)

25

• MAJOR PREMISE 4
• Regardless of what goes into an introduction,
  almost everyone agrees that the purpose of the
  intro is to ...
• Gain cooperation
• Following that thought, one might assume that an
  emphasis on really, really good intros might
  yield a fairly good cooperation rate.
• But, why are refusals so high? Some hypotheses
  include ...

26

• - A strong Privacy trend in recent years
• - A massive level of telemarketing in
• general.
• - Disappointment with sugging
• in specific.
• - Because of outsourcing a less tolerant
• attitude toward call center workers who
• don't sound American.
• - Perhaps, an interviewer labor pool less
• skilled than in earlier times. (Could this
• be tied to use of daily temps?)

27

• - A churn-and-burn emphasis on
• minimizing costs at the expense of
• recruiting and training a quality staff?
• - Could it be that our industry can no
• longer pay a living wage to attract a
• large enough intelligent, literate, and
• conscientious labor force?
• - Is it possible that questionnaire writers
• are not as skilled as the old-timers
• from the past?
• You can add or subtract to this list at will.

28

• MAJOR PREMISE 5
• Refusals are controlled almost entirely by the
  introduction and how it is administered.
• If we can learn how to control refusals, we can
  learn a lot about interviewer training in general.

29

• MAJOR PREMISE 6
• Refusal conversions are not as expensive as one
  might think. But they can significantly improve
  statistical accuracy via a lowered Non-Response
  Bias.
• Put simply, Non-Response Bias is a potential
  error factor introduced into the data set because
  some eligible (and qualified) respondents were
  not interviewed for whatever the reason.

30

• An example ...
• Completes
  Refused
• (N 300)
  (N 900)
• Buy product "A" 60
  ????
• So what is the real percentage? Is 60 actually a
  good estimate of the market for this product?
• In reality, we don't know -- because we have no
  idea if the people who refused are the same as,
  or different, than people who were willing to
  respond.

31

• Let's assume that via a miracle, we called back
  the refusal group and were able to complete the
  interview with each.
• Furthermore, we found the level of Brand A
  purchase for this refusal group to be 40 .
• Here are the restated figures ...
• Initial
• Completes Refused
  Total
• ( 300 )
  (900) (1200)
• Buy Product A 60 40
  45
• For this case, notice the refusal group is
  actually a better measure of the total market
  than the original group who completed the survey.

32

• One obvious answer to this situation is to use
  "refusal conversion" as a standard part of your
  operating procedures ...
• o Wait a few days
• o Then, call back with a different
• "intro" and person.
• However, it would be nice not to have the refusal
  problem in the 1st place.
• And, that brings us back to ...

33

• What can we do with interviewer training and the
  questionnaire introduction? First, a few
  comments about the intro, and then we'll move to
  the specifics of training.
• o Spend time thinking about what is really
• needed in the intro. If it seems crucial, do
  it.
• If not, don't.
• o Beware of the cut-and-paste syndrome. Most
• intros come from an earlier questionnaire.
• Some companies use the same intro for every
• job they do.
• o Make sure you view the intro as a living,
• changeable thing.

34

• ? Listen to how it works. And, ask the
• interviewers.
• ? Pretest intros and run experiments.
• ? Allow flexibility -- even encourage it.
• Consider multiple intros and put them
• on the screen.
• ? Sell this philosophy to your clients. Quite
• frankly, most clients are not good at
• devising introductions.
• - Offer a price reduction?
• - Include control of the intro as part of
  your proposal.
• - Lobby for changes during client
  monitoring
• and the program approval process.

35

• Enforce strict compliance of what you decide. Do
  not let the interviewers do whatever they want to
  do.
• Don't use big words remember that a significant
  proportion of potential respondents have just a
  6th grade education.
• And, unless there is moral authority,
  introductions should be as short as possible.

36

• Now for the interviewer hiring and training parts
  of the equation.
• Screening candidates
• o Measure typing speed too slow and theyll
• never do open-ends.
• o Test vocabulary suggest at least 80
• percentile vs. 8th and 9th grade levels.
• o Be able to read complex sentences at first
• glance. Make sure to include actual question-
• naire sequences.
• o Must be smooth, natural readers with good
• inflection. If they sound as if they are
• reading, refusal rates will be higher.
• o Use a spy for each hiring session.

37

• What can we learn from the LA Times and ancient
  myths? About 15 years ago, I saw a one page
  article in the LA Times that asked ...
• How quickly did you fall in love
• with your mate?
• (Asked of those who actually got married.)
• The answer might surprise you.
• Social researchers found a large number of people
  who actually end up getting married fall in love
  within ...

38

• 7 seconds
• Is it Love at first sight?
• Or, Struck by Cupid's arrow?
• Maybe the age-old stories have a bit of truth
• As researchers, how can we learn from this?

39

• MAJOR PREMISE 7
• In concept, the interviewer must make the
  respondent "fall in love" within ...
• 3 to 4 seconds

40

• If the 3 to 4 second rule actually applies, the
  first thing to remember is that elements read
  after the first few seconds do nothing to help
  lower the refusal rate.
• Unless, of course, the study and sponsor have
  moral authority and/or importance.
• If you want to explain the study, mention
  monitoring or recording, talk about how the
  findings will be used, identify the subject
  matter of the questionnaire
• - Do it after cooperation is accomplished.
• - Besides, the more you identify the project
• the first time you call, the more difficult
  it
• will be to perform refusal conversions
  later.

41

• There are plenty of sociological studies which
  find first-time impressions of people are formed
  almost immediately.
• In person, judgmental criteria include
• o Appearance
• o Body type and dimensions
• o Clothing
• o Bearing
• o Voice
• o And so forth
• There is no reason this principle does not apply
  over the telephone.
• However, only the voice would count.

42

• Learnings from the British Census study
• - British, American and Canadian workers
• - Experiments, digitized voice analysis
• FINDINGS
• Voice inflection should always go up at the end
  of a question.
• ... and I'd like to ask you a few questions.
• Upward is friendly, interested
• and concerned.
• Downward is passive, resigned, bored, ready for a
  refusal.

43

• The pitch and clarity of voice is important ...
• - You can teach people to raise or lower the
• pitch of their voice.
• - Force them to practice it.
• - The greeting Hi! will help those with low
• voices.
• - Conversely, Hello will tone-down shrill
• or squeaky voices.
• - Good posture also helps.
• - Avoid gum, tongue studs, etc.
• Accent-neutral counts.

44

• The pace of speaking is a factor, as well.
• - Too fast creates tension and is
• aggressive, pushy.
• - Too slow is frustrating to listen to, and
• suggests a long survey.
• - The pace of the introduction can be slightly
• faster than for the body of the
  questionnaire.
• - Voice inflection is crucial. If you listen
  to
• people talk amongst themselves or with
• friends, they communicate using multiple
• dimensions . . .
• ? With facial expressions
• ? Using their hands
• ? Other forms of body language
• ? And, in a relaxed casual manner

45

• Obviously, it is far more difficult to express
  yourself over the telephone.
• Thus, when it comes time for reading survey
  questions from a computer screen, many clearly
  sound as if they are reading in a flat, boring
  manner.

46

• MAJOR PREMISE 8
• Interviewers need to learn how to speak as if
  they were having a face-to-face conversation,
  even though the interviewer's side of the talk is
  totally scripted.
• People want to give their ideas and opinions.
  They do not want to answer questions.

47

• Ways to sound more like a two-way conversation
  ...
• - Use your hands while asking the
• question.
• - Focus on the respondent as a person,
• not just a source of answers.
• - Never refer to the questionnaire as a
• physical thing, e.g., "The next
• question is ...
• - Don't pound the keyboard.

48

• MAJOR PREMISE 9
• Practice the intro every day before dialing.

49

• MAJOR PREMISE 10
• Measuring refusal rates
• - Construct BELL CURVE-type ratings for each
• job, using Z-scores from the average refusal
• rate to calculate statistical measures.
• - High correlation between refusals and poor
• CPH production rates.
• - Weight ratings according to the number of
• contacts.
• - Routinely retrain (or fire) lowest outliers.

50

• But, be careful. Too much focus on lowering
  refusal rates may cause some workers to
  disposition refusals as something else.
• USEFUL FINDING
• Compared to all workers, refusal rates will be
  about 1½ times higher for those in the lower 1/3
  of your staff.

51

• Putting it into Practice.
• NOW LETS HAVE A RING-RING MEETING.
• Use this as your introduction
• Hi. Were doing a short survey and would like
  to ask you a few questions.