Minimizing Non-response in a Survey of New Businesses

1
Minimizing Non-response in a Survey of New
Businesses

• Tom Barton
• David DesRoches
• Third International Conference on Establishment
  Surveys
• June 19, 2007
• Montréal
• Québec, Canada

2
Kauffman Firm Survey (KFS) Background

• Sponsored by the Ewing Marion Kauffman
  Foundation- www.kauffman.org/
• Longitudinal survey of businesses that began
  operations in 2004
• Baseline Survey completed in 2006, annual
  follow-up surveys planned for three years

3
Survey Design

• Web and CATI data collection
• Advance letter and a reminder postcard
• Web-First approach No CATI follow-up
  mentioned in advance materials
• Web non-responders were called

4
Barriers to Survey Completion

• Surveys used as marketing tools
• Lack of time for business owners
• Gatekeepers
• No contacts

5
Overcoming Barriers to Completion

• Use a multi-mode approach, emphasizing Web option
• Streamline survey instrument
• 50 post-pay incentive

6
How Interviewers Address Barriers

• Stress research
• Not a sales pitch
• Not soliciting money Kauffman Foundation
• Acknowledge how busy they are

7
Telephone Interviewer Training

• Be honest about the difficulty of the study
• Use what was learned from Pilot Study
• Practice a smooth and short introduction
• Practice quick response to common objections,
  such as lack of time and Im not interested.
• Role plays with scenarios, including those with
  gatekeepers.

8
Managing Gatekeepers

• Ask for the business owner first
• Stress 5 minutes of the owners time
• Push the incentive
• Dont just leave the toll-free number
• If you talked to the owner on a previous call,
  tell the gatekeeper.

9
Data Collection Results

• 12-month data collection period
• Total sample size 32,469
• 4,930 interviews completed - CATI (77), Web
  (23)
• A low eligibility rate (35)
• 49 weighted response rate

10
Data Collection Results- Continued

• 375,000 calls made to complete 3,783 CATI
  interviews
• Converted 538 refusals, or 11 of the total
  completes and 8 of total refusals - screened out
  another 1,062
• Of all refusal conversions, 478 (89) completed
  by phone, and 60 (11) by Web

11
Research Question

• When we compare business owners who initially
  refused and then completed with business owners
  who completed without an initial refusal, do they
  differ on
• Measures of size
• Type of industry
• Form and location of business
• Demographics

12
Size Comparison
No initial refusal (N 4,392) Initial refusal- (N 538)
Total Assets Median 19,500 Median 18,000
Total Revenue Median 7,000 Median 3,500
Number of employees Mean 1.87 Mean 1.90

13
Technology Status
No initial refusal (N 4,392) Initial refusal- (N 538)
High tech 2 2
Medium Tech 13 13
Low Tech 85 85
Intellectual Property 1.30 1.35

14
Legal Status Comparison
No initial refusal (N 4,392) Initial refusal- (N 538)
Sole Proprietorship 36 35
Partnership 5 6
LLC 31 29
Sub-S Corp 20 20
C-Corp 8 9

15
Business Location Comparison
No initial refusal (N 4,392) Initial refusal- (N 538)
Home 49 48
Rented Space 41 42
Purchased Space 5 6
Client Location or other 5 4
16
Demographics of Business Principal
No initial refusal (N 4,392) Initial refusal (N 538)
Age 44 46
Gender Female Male 32 68 29 71
Education Associates or lower Bachelors or higher 54 46 52 48
17
Summary

• No major differences between refusals and
  non-refusals
• Surveys of new businesses offer significant
  challenges
• Prepare interviewers for the challenges
• Ask only what you need
• Consider multiple modes and incentives

18
Questions, Comments and Paper Requests

• Please contact Tom Barton at
• Tbarton_at_mathematica-mpr.com
• Mathematicas Web site is
• www.mathematica-mpr.com/