Planning for Marketing Campaigns

1
Planning for Marketing Campaigns

• Qiang Yang and Hong Cheng
• Hong Kong University of Science and Technology
• (qyang, csch)_at_cs.ust.hk

2
Marketing Planning definition

• In business marketing literature (Dibb et al.
  1996), planning for marketing campaigns refers to
  developing action plans by taking into account
  customer segmentation, marketing objectives and
  budgetary constraints.
• Corporations and institutions are interested in
  executing marketing plans to affect their
  customers
• turning reluctant customers into active ones
• prevent customers from churning (leaving for
  competitors)

3
A Marketing Example

• Suppose a company is interested in marketing to a
  group of customers in the Customer table
• In addition, we have a database of past plans

A candidate plan is Step 1 Send mails
Step 2 Call home Step 3 Offer low
interest rate
4
Marketing Planning modeling

• Two types of marketing planning
• Direct marketing for individual customers
• Segmentation marketing for group of customers ?
  Our Focus
• We formulate Marketing Planning as
• A variation of MDP, and
• Search for a plan as AND-OR search

5
Related Work (I)

• Classical Planning does not apply
• We have uncertainty with the result of actions
• Difficult to formulate as relations and logic
  formulas
• Probabilistic Planning (Draper et al. 1994) also
  requires logical formula
• Again, in our domain there is no logical
  representations of actions

6
Related Work (II)

• MDP (Bellman 1957, Howard 1960, Sutton 1988)
• The aim of MDP is to find a policy in which to
  direct an agents action no matter where the
  agent is observed to land.
• An optimal action is chosen based on the agents
  observed resulting state.
• It is more suitable for direct marketing
  (Pednault et al. 2002).
• Our planning problem
• Can be formulated as a variation of MDP planning
• However, our plan is more like a classical plan
  (sequence of actions), rather than a reactive
  policy

7
Can we formulate as an MDP?

• An MDP Approach (e.g., Sun and Sessions 01)
• First, optimally solve the MDP for all states
• Then, extract a marketing plan from the state
  space
• Problem this approach will not provide optimal
  solution
• We show a counter example next

8
Counter Example
Actions a1, a2, utilities marked at
leaves Assume all transition probabilities
p(s,a,s) 0.5 Cost(a1)2, Cost(a2)1.
Best plan a2a1, Utility2
MDP Plan a1a2, Utility1.5
9
Explanation on the Example

• Optimal MDP is a conditional plan
• At S0, select a1
• At S1, select a1
• At S2, select a2
• Marketing Plan is a sequence
• At S0 select a2
• At S3, S4, select a1

10
Intuition of our Approach

• We formulate the planning problem as a search in
  an AND-OR graph.
• Each state is an OR node, with the actions
  forming the OR-branches.
• Each outcome node is an AND node, the arcs
  connecting the outcome node to the state nodes
  are the AND edges.
• Actions have costs
• State has utilities (equal to of converted
  customers)

11
Objective of MPlan Algorithm

• Objective convert customers from negative (-)
  class to positive () class with lowest cost
• Plan sequence lta1, a2, angt
• Plan Cost
• Success Threshold E(Sn)gts, where
• E(Sn) is the expected value of the state
  classification probability p(s) of all terminal
  states s the plan leads to
• s is a user-defined probability threshold
• Length constraint
• the number of actions must be at most Max_Step

S0
a1
a2

an
Sn
12
Reducing the State Space Size

• Major difficulty
• potentially many states and sequences of actions
• Observation
• Significant sequences are frequently traveled
  by past marketing campaigns
• Thus, we can abstract out these trails using
  abstraction
• We preprocess the problem using Frequent String
  Mining (Find frequent paths from A to B)

13
Marketing Planning Algorithm 3 Steps

• Step1 Frequent string mining
• A string mining algorithm will mine the
  Marketing-log table to discover frequent
  sequences whose support is at least a user
  specified min-support value.
• Support is the number of occurrences of a
  state-action sequence in a database.
• For connection, we are only interested in
  frequent sequences both beginning and ending with
  states, in the form

14
Marketing Planning Algorithm

• Step 2 Construct an AND-OR space.
• We piece together the segments of paths from
  frequent sequences mined in Step. If
• and are two
  frequent paths, then is a
  connected path in the AND-OR space.
• The state space is greatly reduced by ignoring
  statistically trivial paths.

15
MPlan Search Algorithm Step 3

• We used a function to
  estimate a plan.
• s is the initial state, p is a plan.
• g(p) is a function estimating the cost of plan p.
• h(s,p) is a heuristic function estimating how
  good the plan is for transferring customers.
• Based on the function f, we perform a best-first
  search for plans until the termination condition
  is met.

16
MPlan Search Algorithm Step 3 cont.

• In each iteration, we select the plan with the
  minimum value of f(s,p) from the queue.
• We compute E(s) to estimate how good the plan
  is at transferring customers to a desirable
  class.
• E(s) is the expected value of the
  state-classification probability of all terminal
  states the plan leads to. Its defined as
• , if is a
  non-terminal state
• , if is a terminal state.

17
MPlan Search Algorithm (See our paper for details)

• We design a MPlan search algorithm for searching
  high-utility plans.

• Insert all one-action plans into Q.
• While (Q not empty)
• Get a plan with minimum value of f(s,p)
  from Q.
• Calculate E(s) of this plan.
• If ( E(s) gt Success_Threshold)
• Return Plan
• If (length(Plan) gt Max_Step)
• Discard Plan
• Else
• 7.1 Expand plan by appending an action.
• 7.2 Calculate f(s,p) for the new plans
  and insert into Q.
• end while
• Return plan not found

18
Example MPlan in action

• Suppose we have an AND-OR graph on the right.
• Suppose all transition probabilities in the
  figure p(s,a,s) 0.5
• Cost(a1)1, Cost(a2)2.
• Success_Threshold 60
• Max_Step 2.
• p(s1)0.4, p(s2)0.4
• p(s3)0.6, p(s4)0.8
• p(s5)0.4, p(s6)0.6
• p(s7)0.3, p(s8)0.5

19
Example Step 1

• Queuelta1, a2gt
• f(s0,a1)1.6
• f(s0,a2)2.6
• Delete a1 from Queue and calculate its expected
  probability of belonging to the positive class.
• E(s0) 40 lt Success_Threshold.
• Append a2 to a1 and insert a1a2 into Queue.

20
Example Step 2

• Queuelta2, a1a2gt
• f(s0,a2)2.6
• f(s0,a1a2)3.4
• Delete a2 from Queue and calculate its expected
  probability of belonging to the positive class.
• E(s0)40 lt Success_Threshold.

21
Example Step 3

• Queuelta1a2gt.
• f(s0,a1a2)3.4
• Delete a1a2 from Queue and calculate its expected
  probability of belonging to the positive class.
• E(s0)60 gt Success_Threshold.
• Output plan a1a2.

22
Experimental Result

• We would like to test
• Customer conversion rate Transition Rate
• We define a transition rate to denote the of
  customers converted from negative (-) class to
  positive () class
• Planning efficiency cost is low, CPU time is low
• Data Generation
• We used the IBM Synthetic Generator Quest to
  generate a Customer table.
• It has two classes ( and -) and nine attributes.
  
• We will design plans based on the Customer table
  and Marketing-log database to convert the 100K
  customers.
• The positive class has 30K records and the
  negative has 70K.
• A classifier is trained using the C4.5 decision
  tree algorithm. The classifier will give p(s).

23
Experimental Result

• This figure shows the transition rate as a
  function of Success Threshold s.
• When s is low, the plans found dont guarantee
  high probability of success, so transition rate
  is low at first.
• As s increases, so does transition rate because
  plans found have higher probability of success.
• When s is too high, no plans can be found for
  some states because their probability of success
  cannot exceed s. So transition rate decreases.

Transition Rate () vs. Success Threshold s.
24
Experimental Result (II)

• This figure shows the CPU time as a function of
  Success Threshold s.
• When s is low, plans can be easily found for most
  initial states. So CPU time is low at first.
• When s increase, CPU time increases quickly
  because the searching process takes longer time.
• If the case is that there is no plan satisfying
  the Success Threshold, the searching process
  doesnt terminate until all plans are expanded
  longer than Max Step.

• CPU Time vs. Success Threshold s.

25
Conclusion

• We formulated Marketing Planning problem as a
  variation of MDP
• The problem is different from traditional MDP in
  that we require plan be non-conditional (sequence
  of actions)
• We also developed a planning algorithm MPlan
  based on And-Or tree search

26
Future Work

• Include plan utilities in preprocessing step
  (frequent sequence mining)
• Consider missing data problem
• Formulation as Partially Observable MDP problem
• Scale up with even more data