Weak Learning DNF under uniform distribution

1
Weak Learning DNF under uniform distribution

• A parity function weakly approximates f
• Find this function
• KM algorithm
• Form a tree, pruning a node if there are no large
  coefficients starting with that substring

2
Strong learning Boosting

• Recall Freunds algo
• Construct weak hypotheses h1,h2,h3
• At step i
• Distribution Di more weight to x on which
  hi,,hi-1 were wrong
• Form hypo hi on Di
• Combine hypotheses using some rule
• Does parity approximate f on Di?
• Yes.. Answer on next slide
• Needs a distribution-independent weak learner
• We dont have one for DNF

3
Strong learning DNF

• Let f be a DNF having s-terms
• Lemma f has a fourier coefficient of value at
  least 1/(2s1)

4
Can we tweak KM?

• Cool fact KM works for real-valued functions as
  well
• Idea Construct function g such that
• Depends on L?(g) in running time

5
Converting (f,D) to (g,U)

• Notice that on dist D is same as
  on uniform
• We can learn on U!!
• Need MQ oracle for g gt MQ oracle for D
• L?(g) should not be too large
• D close to uniform

6
An appropriate Boosting algorithm

• Final hypothesis majority of his
• Di D(x)?i(x) ? has to be normalized
• ?i(x) ? prob that hypotheses are almost equally
  divided over x
• Stop if Dis become too small
• Notice easily computable
• Close to uniform within a small factor