Simple UCB for Parallel Tracking

1
Simple UCB for Parallel Tracking

• Craig Saunders and John Shawe-Taylor
• University of Southampton

2
Motivation

• Response rates moving in parallel implies that
  the difference between individual responses and
  mean should be stable

3
Upper Confidence Bounds

• Applying the upper confidence strategy means
  keeping estimates of means and variances of the
  
• where

4
Variances

• Can be estimate for the binomial distribution
  first for
• where is the number of times arm has
  been played
• Giving variance for of

5
UCB Strategy

• The UCB strategy plays the arm that maximises the
  upper confidence estimate
• where is the trial number.

6
Some comments

• Strategy breaks down when k 2.
• Can increase the coefficient of the variance of
  to 1 to attempt to compensate
• We omitted the factor by mistake from
  our submission
• It became clear that the trials were not
  sufficiently long or repeated sufficiently to
  fully elucidate the deficiencies of the submission

7
Results I

• Official results
• Baseline results
• Additional results
• Increase no of seeds from 10 to 100
• Increase length of runs from 100,000 to 1,000,000
  trials

Min Max Mean Std
Task 1 36 103 65.6 23.04
Task 2 8069 20063 13270 3837.8
Task 3 67 108 80.7 11.2
8
Results II
9
Conclusions

• Promising strategy for parallel moving response
  rates
• New ideas required for other two tasks
• Hopefully inspired others to become involved with
  the challenge
• Clearly exposed some weaknesses in the challenge
  evaluation