Robust Optimization and Applications

1
Robust Optimization andApplications in Machine
Learning
2
Part 4 Sparsity in Unsupervised Learning
3
Unsupervised learning
4
Sparse PCA outline
5
Principal Component Analysis
6
PCA for visualization
7
First principal component
8
Sparse PCA outline
9
Why sparse factors?
10
PCA rank-one case
11
sparse PCA rank-one case
12
Sparse PCA outline
13
SDP relaxation
14
SDP relaxation
15
Dual problem
16
Sparsity and robustness
17
Sparse PCA decomposition?
18
Sparse PCA outline
19
First-order algorithm
20
Sparse PCA outline
21
PITPROPS data
22
PITPROPS data numerical results
23
Financial example
24
Covariance matrix
25
Second factor
26
Gene expression data
27
Clustering of gene expression data
28
Conclusions on sparse PCA
29
Part 4 Sparsity in Unsupervised Learning
30
Sparse Gaussian networks outline
31
Gaussian network problem
32
Correlation-based approach
33
Approach based on the precision matrix
34
Example
35
Relevance network vs. graphical model
36
Can we check this?
37
Sparse inverse covariance and conditional
independence
38
Related work
39
Maximum-likelihood estimation
40
Problems with ordinary MLE
41
MLE with cardinality penalty
42
Convex relaxation
43
Link with robustness
44
Properties of estimate
45
Algorithms challenges
46
First- vs. second-order algorithms
47
Black- vs. grey-box first-order algorithms
48
Algorithms problem structure
49
Nesterovs smooth minimization algorithm
50
Nesterovs method
51
Putting the problem in Nesterovs format
52
Making the problem smooth
53
Optimal scheme for smooth minimization
54
Application to our problem
55
Dual block-coordinate descent
56
Properties of dual block-coordinate descent
57
Link with LASSO
58
Example
59
Inverse covariance estimates
60
Average error on zeros
61
Computing time
62
Classification error
63
Recovering structure
64
Part 4 summary
65
References