Uplift Analysis with the Quadstone System

1
Uplift Analysiswith the Quadstone System

• Monday, 7th January 2005
• 7.30am PST / 10.30am EST / 3.30pm GMT / 16.30 CET
• Any trouble getting into the conference call
• contact support_at_quadstone.com.

2
How to ask questions

• Return to Meeting Manager
• Use Chat

3
Uplift Analysis

• Nicholas J. Radcliffe
• Chief Technology Officer

Agenda MOTIVATION Demo 1 Up-sell example
(binary outcome) When is Uplift Modelling
important? Demo 2 Deep-sell example (continuous
outcome) TECHNICAL CONSIDERATIONS Practical
considerations and guidelines Small population
issues and extensions The quality measure
Qini TRIAL How to get a trial copy datasets
4
We have to find a way of making the important
measurable, instead of making the measurable
important

• Robert McNamara

I know half the money I spend on advertising is
wasted,but I can never find out which half
John Wanamaker
5
Demo 1 Up-sell example

• Binary outcome
• SCENARIO
• Mobile phone company
• 3G MMS Video phone promotion
• Some mass advertising
• - non-targeted customers can purchase
• Direct calling campaign to drive further sales
• Random 250k chosen from 10m base for trial
• c. 75k actually targeted c. 175k as control

6
When is Uplift Modelling Important?
7
Two Separate Benefits

• Not targeting people who are little affected
• Reponse Dont spend money targeting or offer
  discounts to people who will buy anyway
• Attrition Dont spend money trying to save
  people who will go anyway
• Targeting people with low probability but high
  responsiveness
• Response Do spend money on people who arent
  very likely to buy if you do, but are very
  responsive to offers/contact
• Attrition Do spend money trying to save people
  who arent at huge risk of attrition, but can be
  made much more likely to stay

8
When would a conventional model be misled?
High pre-existing purchase rate
9
When are negative effects likely?

• Sometimes, our actions actually drive customers
  away, especially when

attrition risk
dissatisfied / angry customers
risqué / offensive communications
intrusive contact mechanisms
forgotten standing charges
10
Demo 2 Deep-sell example

• Continuous outcome
• SCENARIO
• Grocery retailer
• Direct mail campaign to increase spend
• Weekly Spend measured in 12-week pre-period
  (AWS)
• Also in 6-week post period (AWSPostCampaign)
• Objective is difference (PostMinusPreAWS)
• Random 250k chosen from 10m base for trial
• c. 75k actually targeted c. 175k as control

11
Control Group Structure

• Control group
• Must be representative technique will give
  misleading results otherwise
• In practice, this means randomly select controls
  from target group
• There must be enough of them

All possible recipients
Targets
12
Population Size

• Population size
• Rule of 500 to detect a x difference
  (uplift), x of the smaller population (usually
  controls) should ideally be at least 500 people
• So if looking for 1 difference, control group
  needs to have at least 50,000 people
• So consider longitudinal controls contact half
  now, half later

13
Pruning and Validation

• Pruning
• Autopruning is implemented, based on qini
  variance
• In practice, fairly unaggressive, so recommend
  manual pruning
• Validation
• Ordinary test-training fine if there is enough
  data
• If not, consider k-way crossvalidation

14
Small Population Extensions

• Bagging (oversampling method) and k-way
  cross-validation
• Analysis candidate selection
• useful if there are too many analysis
  candidates
• Stronger pruning (variance-based)
• Stratification
• Not part of product, but potentially available as
  an extension if purchased

15
Return on Investment

• Key thing is that Campaign ROI depends on the net
  effect (i.e. uplift) of action, not apparent
  response
• (reduction in churn) (value of saved people)
  (cost of action)
• (increase in purchase rate) (value of purchase)
  cost
• (increase in spend) (cost of action)
• etc.
• Quadstone System has many suitable ROI FDL
  functions ( fx) built in (even without uplift
  license)

16
So how do youmeasure whats important?
17
Quality Measure Considerations

• Can only estimate uplift by segment
• This is what we are used to with control groups
• One person does not have a (knowable, measurable)
  uplift
• Generalizing measures like classification
  error/accuracy or R2 doesnt look promising
• Rank statistics do seem more promising because
  they can sometimes be computed on a segmented
  basis

18
Can we use/modify the Gini for Uplift?
Overall uplift x
x
Possibility of negative effects
uplift
0
100
x
of customers targeted
19
Summary When to use Uplift

• Uplift modelling is just a better way of
  modelling the true effect of an action
• Particularly relevant to
• Retention (where its the number/value of people
  you save thats important
• Up-sell, cross-sell, deep-sell (where its the
  incremental revenue or profit thats important)
• Risk management actions (where its the reduction
  in risk achieved thats important)

20
Where to find out more

• www.quadstone.com/system/uplift/
• For more in-depth training our Uplift Analysis
  course. Contact support_at_quadstone.com

21
Questions and answers
22
After the webinar

• These slides, the data and a four-week trial
  license are available via www.quadstone.com/traini
  ng/webinars/
• Any problems or questions, contact
  support_at_quadstone.com

23
Uplift Quick Reference

• Building uplift models
• Ensure random control group exists
• Set partition field with P interpretation (1 for
  treated, 0 for control)
• Set objective (binary, continuous/discrete)
• Hit go
• Pruning
• Switch to test dataset
• Hit Autoprune
• Creating results field
• Use Uplift as difference

• Using difference viewers
• Crossdistribution Viewer places partition field
  on ? axis automatically
• For view shown, drag count to depth, duplicate
  mean (ObjectiveField) and drag on to height
• Can configure which population is viewed by
  right-clicking on functions
• Using ROI Functions
• These are available under fx in Table Viewer when
  deriving new field.

24
Upcoming webinars
Thursday, 17th February 2005 Data Preparation in
the Quadstone System Version 5 7.30am PST /
10.30am EST / 3.30pm GMT / 16.30 CET

• If theres a webinar topic youd like to see,
  please let us know via support_at_quadstone.com.
• www.quadstone.com/training/webinars/

25
Your feedback
Suggestions or feedback? Please enter them in the
feedback form or send them to support_at_quadstone.co
m
26
Modifying the Gini for Uplift?
Unaffected by action
Negatively affected by action
x
Positively affected by action
uplift
0
100
x
of customers targeted
27
The Shape of the Qini Curve
?
Why is this flat?
neutral
ve
x
ve
uplift
0
100
x
of customers targeted