Improving Response Rates Lessons from Physician Surveys

1
Improving Response RatesLessons from Physician
Surveys

• PMRS Ottawa Chapter
• February 26, 2004

2
Presentation Overview

• Survey Response Rates The state of the art
• Particularities of Physician Surveys
• Response rate boosting tactics What works and
  what does not.
• On the use of Monetary Incentives in Physician
  and Consumer Surveys
• Q A period

3
Current response rates

• Academic Surveys published between 1961 and 1977
  71
• Academic surveys published in 1991 54
• Academic surveys published between 1986 and 1995,
  sample size over 1,000 respondents 52
• Commercial/marketing physician surveys (2002)
  20
• RETICULUM surveys 12 to 66

4
Current response rates

• Surveys of executives, published in 1991 21
• PMRS Members surveys 15.7(1997)11.3 (2000)
• One-time telephone surveys 16 (1997) 13 (2002)

5
Physician Surveys Particularities

• More homogenous populations
• Highly-regulated professionals
• Better sampling frames
• Better record-keeping

6
Physician Surveys Particularities

• Highly-solicited respondents
• Highly-educated respondents
• Well-connected respondents

7
Physician Surveys Particularities

• Surrounded by tough gatekeepers
• Addicted to monetary incentives

8
Tactics that boost response by 50 or more

• Monetary Incentives
• Multiple contacts multiple contact modes

9
Monetary Incentives

• (Gallagher, 2001)
• 1st contact by mail, no incentive 11
• 2nd contact by phone, no incentive 22 (cumul)
• 3rd contact by courier, 20 incentive 57
  (cumul)
• (Malin, 2000)
• 1st mailing, no incentive 17
• 2nd mailing, no incentive 13
• 3rd mailing, 50 incentive 66
• Cumulative response rate 76

10
Multiple Contacts Contact Modes

• Typical response rates after multiple mailings
• 1st mailing X
• 2nd mailing X/2
• 3rd mailing X/4
• (CDC, 1997)
• 1st contact by First Class mail 60
• 2nd contact by Fedex 72 (cumul)
• 3rd contact by phone 96 (cumul)

11
Tactics that boost response by a few points

• Pre-notification by phone
• Personalization
• Advertising the survey
• Choice of sponsors
• Shortening the questionnaire
• Instituting a draw

12
Pre-notification by phone

• (Osborn, 1996)
• No Pre-notification 64
• Pre-notification 77
• (Ward, 1994)
• No Pre-notification 69
• Pre-notification 84

13
Personalization

• First Class mail
• Commemorative stamps
• Stamped return envelope
• Name address printed on the envelope
• Personalized salutation
• Full date on Cover Letter
• Handwritten signature
• Handwritten note

14
Personalization

• (Maheux, 1989)
• Handwritten thank you note 30
• No thank you note 22
• (Streiff, 1999)
• Stamped return envelope 38
• Business-reply envelope 32

15
Choice of sponsor

• (Asch, 1994)
• Veteran Affairs return address pulled 20 more
  than a Hospital Department of Medicine
• (RETICULUM, 2000)
• A joint study with IMS Health, Royal College,
  College of Family Physicians 22
• IMS Health alone 12

16
Tactics that dont boost response

• Pre-notification by mail
• Offering non-monetary incentives (pens, mouse
  pads, candy, booklets, software..)
• Mailing surveys on a specific day of the week
• Promising anonymity
• Gimmicks

17
On the use of Monetary Incentives

• Even symbolic sums will boost response
• (Everett,1997)
• 0 45
• 1 63 (one-dollar bill included in mailing)
• (Donaldson, 1999)
• 0 46
• 5 58 (five-dollar cheque included)

18
On the use of Monetary Incentives

• Larger incentives, Higher response rates
• (Asch, 1998)
• 2 incentive 46
• 5 incentive 63
• (Gunn, 1981)
• 0 incentive 58
• 25 incentive 69
• 50 incentive 77

19
On the use of Monetary Incentives

• Larger incentives, Higher response rates
• UP TO A POINT
• (VanGeest, 2001)
• 5 incentive 60
• 10 incentive 68
• 20 incentive 67
• (RETICULUM/ IMS Health, 2000)
• 25 incentive 22
• 50 incentive 34
• 75 incentive 36

20
On the use of Monetary Incentives

• Pre-paid incentives outperform
• Post-paid incentives
• (Berry, 1987)
• 20 incentive, pre-paid 78
• 20 incentive, post-paid 66

21
Monetary Incentives in Lay Surveys

• Sparse data
• Controversial practice
• Banned in certain jurisdictions

22
Monetary Incentives in Lay Surveys

• (James Bolstein, 1992)
• 0 52
• 2 64
• (Dillman and al., 1999) in 5 different studies
• 2 incentive boosted response by 19 to 31

23
Monetary Incentives in Lay Surveys

• Pre-paid incentives will outperform post-paid
  incentives
• (Johnson McLaughlin, 1990)
• 5 pre-paid 83
• 10 post-paid 72
• (James Bolstein, 1992) survey of small
  contractors
• 1 pre-paid 64
• 5 pre-paid 72
• 50 post-paid 57

24
On the use of Monetary Incentives

• Why and how do they work???

25
On the use of Monetary Incentives

• Respondent appreciated, not taken for granted
• Value-creating
• Attention grabbing Secretary
• Attention grabbing Physician
• Pre-paid incentives create trust

26
On the use of Monetary Incentives

• The Pitfalls
• Point of no-return
• Cost
• Fraud
• Ethical Issues

27
On the use of Monetary Incentives

• Pre-paid incentives
• (Gallagher, 2001)
• 46 replied
• 3 declined and returned the 20-dollar pre-paid
  incentive
• 51 declined, but pocketed the 20-dollar
  pre-paid incentive

28
PMRS Ottawa Chapter

• Thank you very much
• Q A (in both official languages!)