Nightmare at Test Time: Robust Learning by Feature Deletion

1
Nightmare at Test Time Robust Learning by
Feature Deletion

• Amir Globerson
• Computer Science and Articial Intelligence
  Laboratory, MIT, Cambridge, MA, USA
• Sam Roweis
• Department of Computer Science, University of
  Toronto, Canada
• All the equation images are clipped from this
  paper.

????????? (IBM???????/?????????1?)
2
???????

• ????(?)????
• Nightmare at Test Time
• ? ???????????????
• ??????Robust??
• ??????Over weighting?????
• ???????
• ???????(????)??????????????
• ??????????????????

3
???????????????

• ????????????????????????
• Spam filtering ????????
• Image processing pixel?? (pepper noise)
• Sensor network neurons die
• ?????????????????
• MinMax???? 2????????
• Player 1 feature removal mechanism
• Classifier????????????????
• Player 2 classifier builder
• ????????(??)?????

4
SVM (hinge loss)????2player game????Player 1
feature removal mechanism

• worst case???????????(xi)?Loss?????????????
• ??????? K (feature?????)

Worst case hinge loss
K???????????
Loss max??????
aij ??????(xi)??????????(j)??? (1-ai)??????(xi)?
??????????? ????????????????????
5
SVM (hinge loss)????2player game????Player 2
classifier builder

• ????????????loss(hwc)????????w????(SVM)

6
Player 1 feature removal mechanism????

• ????????loss??????????????????

??????????????Relax
si??????????????loss
????
7
?????????FDROP

• ???? ???
• ???????
• ??????O(??????)
• SVM?O(??????)

8
??? (??????)

• Multi-class??????
• Hinge loss?????(log loss??)??????????(challenge)
• Kernel?????
• ????????????????(challenge)
• ?????????????????????
• ??????a(?????????????)????relax???????????????????
  

9
????????(1)??????????????

• ????????
• loss??????feature?K???????
• ??????
• FDROP?different computational class
• ???? ????????????

K??????????
10
????????(2)????????????

• FDROP????????????????????????????????
• ?????????????????????

Clipped from A. Globerson S. Roweis,
Nightmare at Test Time Robust Learning by
Feature Deletion, ICML 2006
??????????FDROP????????????? ?????????????????????
??????????????????
11
???????SVM??????

• ???????????????SVM????
• ??????????????????????

Clipped from A. Globerson S. Roweis,
Nightmare at Test Time Robust Learning by
Feature Deletion, ICML 2006
????? ??????????
????? ?????????
??????? ?????????