MIDAG Tutorials All About Statistical Models

1
MIDAG TutorialsAll About Statistical Models

• Martin Styner
• Research Assistant ProfessorDepartments of
  Computer Science and PsychiatryNeurodevelopmental
  Disorders Research CenterUniversity of North
  Carolina at Chapel Hill

2
Topics

• All about statistical models
• Modeling - Dimensionality reduction
• Principal Component Analysis (PCA)
• Independent Component Analysis (ICA)
• Principal Geodesic Analysis (PGA) for m-reps
• Multiple object models object ensembles

3
Topics
PCA ICA
PGA of m-reps
Object Ensembles
4
Statistical Shape Model

• Probabilistic model built on training population
• Mean shape and likely modes of deformation away
  from the mean ? Origin and axis of a shape space
  coordinate system

5
Modeling Dimensionality Reduction

• High number of parameter, low number of samples
• A training population is itself a model
• Discrete set of templates, samples
• Very inefficient, correlation between parameters
• True dimensionality of parameters is smaller than
  size of training population

2D example
6
Assumptions

• Gaussian vs Non-Gaussian
• Many statistical model are based on Gaussianity
  of the features
• Euclidean vs Non-Euclidean
• Almost all statistical models are based on
  Euclidean geometry

7
Principal Component Analysis (PCA)
8
PCA
9
PCA

• PCA on landmark point coordinates was proposed by
  Cootes and Taylor and is called a Point
  Distribution Model (PDM)
• In 2D and 3D
• Used for segmentation Active Shape and Active
  Appearance Models
• Simply compute the Eigenvectors and Eigenvalues
  of the Covariance matrix

10
Eigenmodes

• Hippocampus-amygdala population
• Coordinates of boundary points from SPHARM
• Eigenmodes /- 2 sqrt(?i)

PC 1
PC 2
PC 3
11
PCA in High Dimensional Space
12
PCA for Dimensionality Reduction

• Linear approximation with only the first t
  eigenmodes
• Reduction of the Dimensionality of the shape
  space
• t is threshold

13
How to choose threshold?

• Choose threshold on variance to be explained in
  the model, rest is assumed to be noise (e.g. 95)
• Choose noise threshold on variance contribution
  of each eigenmode
• Threshold on reconstruction error
• Leave-One-Out Analysis

14
PCA Threshold Cumulative Variance

• Choose threshold on variance to be explained in
  the model, rest is assumed to be noise (e.g. 95)

Cumulative variance
15
PCA Threshold Noise Level

• Choose a noise level, e.g. 1 of total variation
• Estimate noise level ?2
• All Eigenmodes with Eigenvalue lt 1 are
  considered noise

16
PCA Threshold via Reconstruction Error

• Divide training set again into training and test
  set
• Train model on training set
• Fit shapes of test set to PCA shape space (simple
  projection)
• Evaluate approximation error
• Directly in the parameters
• Other measures, e.g. Volume overlap
• Iterate over subdivision into training/test set
• Test size 1 leave-one-out, computed exhaustively
• Test size n leave-n-out, computed with random
  sampling
• maximize log(P(test shapesmodel))

17
Evaluation of Shape Model

• Assumption of Gaussianity?
• Plot modes
• Correspondence evaluation
• Compactness
• How compact is a PCA model? Cumulative variance
• Generalization How general is a model?
• Leave-one-out tests
• Select a case and remove from training samples
• Check approximation error of case to PCA model
• Threshold from reconstruction error method
• Specificity

18
Shape Model Evaluation

• Specificity
• The ability to represent only valid instances of
  the object
• Create new object in PCA shape space
• Approximation error to closest sample in training
  set
• Create many objects with probabilistic Gaussian
  sampling in PCA shape space
• Average error and standard deviation

19
PCA Shape Model Application

• Orthopaedic plate implant manufacturer
• Plates for different ethnicities need to be
  differently shaped
• So far Design and test it manually on a series
  of cadaveric bones
• Bone database, PCA model
• Sample in PCA shape space
• Fit ok?
• Statistical statement
• Fits ok 68/95/99 of training
• Assuming Gaussianity
• University of Bern, MEM center

20
PCA Shape Model Application

• Ear implant manufacturer
• Similar to orthopaedic implant
• Model ear canal shape using a PCA model
• Find best set of shapes that fit the most people
• Paulsen, University of Denmark

21
PCA Robustness

• PCA is sensitive, especially in high dimensional
  space with low sample size
• Mean estimation, covariance matrix are not robust
• Robust estimators for mean covariance matrix
• Outlier rejection
• Median
• Iterative mean based on distance-weighting
• Sphere-ing Projection to sphere

22
Independent Component Analysis (ICA)

• Mixture signal from different sources

23
ICA

• This is a well known problem and can be solved
  using methods for blind source separation
• For instance independent component analysis (ICA)
• A shape instance can be considered a mixture of
  independent deformation components
• Try to isolate these components

24
PCA vs. ICA
25
Simple Gaussian Example in 2D
26
Simple Non-Gaussian Example
27
Criterions to find axis in ICA?

• Central limit theorem tells us that the
  distribution of a sum of independent random
  variables tends to a Gaussian distribution
• Sum of two independent random variables is closer
  to a Gaussian than any of the original variables
• We can find the independent components by
  maximizing non-Gaussianity of each component

28
Criterions to find axis in ICA?

• Mutual information is a measure for the
  dependence between random variables
• Information theory
• We can find the independent components by
  minimizing mutual information of each component

29
Comparison Lung Shape

• ICA modes of variation can be easier to interpret
• Modes are not orthogonal
• No obvious ordering

30
Day 3 Topics
PCA ICA
PGA of m-reps
Object Ensembles
31
Non-Euclidean Features

• How can we compute PCA if we have non-Euclidean
  features?
• Magnitude of vectors ? µ exp(1/n S
  log(fi))
• Tensor, Matrices ?n x ?n
• Angles S (in 2D), S2 (in 3D)
• Scale ? µ (? fi )1/n Geometric mean
• Medial m-rep representation per atom
• Position Euclidean
• Thickness Magnitude, non-Euclidean
• Angles non-Euclidean
• T ? ?3 ? S2 S2

32
Non-Euclidean Features

• So far we only looked at Euclidean Features
• Mean µ 1/n S fi
• Sample variance ?N2 1/n S (fi µ)2
• Use only if µ is known (rarely the case)
• Unbiased variance ?N-12 1/(n-1) S (fi µ)2
• Vectors, Position, Volumes ?n
• What if our features are non-Euclidean
• Magnitude of vectors ? µ exp(1/n S
  log(fi))
• Tensor, Matrices ?n x ?n
• Angles S (in 2D), S2 (in 3D)
• Scale ? µ (? fi )1/n Geometric mean

33
Euclidean vs Non-Euclidean Features

• Mean closest to data in sum of square distance.
• Project into log-space and compute PCA there.
• Allows sum or interpolation for non-Euclidean
  features

f ? ?3 ? S2 S2
f ? ?n
Curved Statistics (PGA)
Linear Statistics (PCA)
34
Advantages of Geodesic Geometry

• PGA has strikingly fewer principal components
  than PCA (LDLSS)
• Naturally avoids geometric illegals
• Geodesic interpolation in time space is natural

Pos A
Pos B
Pos A
Pos A
Pos C
35
Day 3 Topics
PCA ICA
PGA of m-reps
Object Ensembles
36
Object Ensembles

• Often we have not single object, but a whole
  series of objects with inter-relations

37
Shape Model of Object Ensembles

• How can we compute shape models of ensembles?
• So far, each object was individually aligned
• Local shape space
• Simple options
• First objects are jointly aligned and are all in
  the same coordinate system
• PCA for each object individually
• PCA for all objects parameters jointly

38
Object Ensembles Individual PCA

• All object are jointly aligned in a global
  coordinate system
• Each object has then its own PCA shape space
• Problems
• Shape space of each object does not align with
  shape space of other object
• No interconnections between objects possible
• We want to be able to express how shape of object
  1 relates to the shape of object 2

39
Object Ensembles Global PCA

• Objects aligned globally
• Concatenate all parameters
• All objects in global shape
• Single large parameter vector
• E.g. Vector x (hippocampus, amygdala, caudate)
• Compute PCA over this parameter vector
• Currently this is used often
• Unfortunately incorrect
• This PCA relate both shape, as well as position
  and rotation
• E.g. object 1 growth results in a rotation of
  object 2
• Rotation cannot be captured by PCA, as it is a
  non-linear operation

40
PCA and Rotation

• Single object normalize pose before PCA
• No rotation between the object
• PCA is a linear combination of the parameters
• PCA can move a single parameter only linearly
• No combinations of parameters other than addition
• E.g. Parameters are (x,y) coordinates in 2D
• Lets look at a single point
• Rotation is non-linear
• Multiplication between parameters
• PCA cannot express this rotation
• If rotation is small
• Ok since approximate linear

41
Object Ensembles Global PGA

• Objects aligned globally (with scale)
• Align each structure individually (without scale)
• Rotation translation relative to global
  coordinates are parameters of each object
• Concatenate all parameters
• local shape transformation into a single vector
• E.g. Vector x (hippocampus shape, transform,
  amygdala shape, transform)
• Compute PGA over full parameter vector
• No hierarchy or multi-scale
• Less suitable for segmentation
• Medial M-reps use global PGA with multiscale

42
M-rep Multiple Scale Levels
Ensemble
Object
Figures
Atom
Vertex/Voxel
43
M-rep Multi Scale Model

• Scale Hierarchy
• Global object ensemble
• By object
• By figure (atom mesh)
• By atom (interior section)
• By voxel or boundary vertex
• Always incl. relation to neighbors
• Express models at level k as residuals from
  level k-1

44
Representation of multiple objects via residues
from global variation

• Interscale residues
• E.g., global to per-object
• Provides localization
• Inter-relations between objects (or figures)
• Augmentation via highly correlated (near) atoms
• Prediction of remainder via augmenting atoms

45
M-rep Ensembles Figures and Objects

• Meshes of medial atoms
• Objects connected as host, subfigures
• Hinge atoms of subfigure on boundary of parent
  figure
• Blend in hinge regions
• Special coordinate system (u,w,t) for blend
  region
• Multi objects inter-related via proximity