Guido Gerig UNC, September 2002 1

1
Building of statistical models

• Guido Gerig
• Department of Computer Science, UNC, Chapel Hill

2
Statistical Shape Models

• Drive deformable model segmentation
• statistical geometric model
• statistical image boundary model
• Analysis of shape deformation (evolution,
  development, degeneration, disease)

3
Manual Image Segmentation

• Manual segmentation in all three orthogonal slice
  orientations.
• Instant 3D display of segmented structures.
• Cursor interaction between 2D/ 3D.
• Painting and cutting in 3D display.
• Open standard s (C, openGL, Fltk, VTK).

IRIS segmentation tool Segmentation of
hippocampus/amygdala from 3D MRI data.
4
SNAP Segmentation by level set evolution

• SNAP (prototype)
• 3D level-set evolution
• Preprocessing pipeline and manual editing
• Boundary-driven and region-competition snakes

5
Segmentation by level set evolution
(midag.cs.unc.edu)
6
Extraction of anatomical models
SNAP Tool 3D Geodesic Snake

• Segmentation by 3D level set evolution
• region-competition boundary driven snake
• manual interaction for initialization and
  postprocessing (IRIS)

free dowload midag.cs.unc.edu
7
Modeling of Caudate Shape
Surface Parametrization
8
Parametrized 3D surface models
Smoothed object
Raw 3D voxel model
Parametrized surface
Ch. Brechbuehler, G. Gerig and O. Kuebler,
Parametrization of closed surfaces for 3-D shape
description, CVIU, Vol. 61, No. 2, pp. 154-170,
March 1995 A. Kelemen, G. Székely, and G.
Gerig, Three-dimensional Model-based
Segmentation, IEEE TMI, 18(10)828-839, Oct. 1999
9
Surface Parametrization
Mapping single faces to spherical quadrilaterals
Latitude and longitude from diffusion
10
Initial Parametrization
a) Spherical parameter space with surface net, b)
cylindrical projection, c) object with coordinate
grid. Problem Distortion / Inhomogeneous
distribution
11
Parametrization after Optimization
a) Spherical parameter space with surface net, b)
cylindrical projection, c) object with coordinate
grid. After optimization Equal parameter area of
elementary surface facets, reduced distortion.
12
Optimization Nonlinear / Constraints
13
(No Transcript)
14
Shape Representation by Spherical Harmonics
(SPHARM)
15
Calculation of SPHARM coefficients
16
Reconstruction from coefficients
Global shape description by expansion into
spherical harmonics Reconstruction of the
partial spherical harmonic series, using
coefficients up to degree 1 (a), to degree 3 (b)
and 7 (c).
17
Importance of uniform parametrization
18
Parametrization with spherical harmonics
Surface Parametrization Expansion into
spherical harmonics. Normalization of surface
mesh (alignment to first ellipsoid). Correspondenc
e Homology of 3D mesh points.
19
Parametrization with spherical harmonics
20
Correspondence through Normalization

• Normalization using first order ellipsoid
• Spatial alignment to major axes
• Rotation of parameter space.

21
Parametrized Surface Models
1

• Parametrized object surfaces expanded into
  spherical harmonics.
• Hierarchical shape description (coarse to fine).
• Surface correspondence.
• Sampling of parameter space -gt PDM models
• A. Kelemen, G. Székely, and G. Gerig,
  Three-dimensional Model-based Segmentation, IEEE
  Transactions on Medical Imaging (IEEE TMI),
  Oct99, 18(10)828-839

3
6
10
22
3D Natural Shape Variability Left Hippocampus of
90 Subjects
23
Computing the statistical model PCA
24
Natural Shape Variability
25
Notion of Shape Space
Alignment Parametrization (arc-length) Principal
component analysis ? Average and major
deformation modes
26
Major Eigenmodes of Deformation by PCA

• PCA of parametric shapes ? Average Shape, Major
  Eigenmodes
• Major Eigenmodes of Deformation define shape
  space ? expected variability.

27
3D Eigenmodes of Deformation
28
Set of Statistical Anatomical Models
29
Alignment, Correspondence?

• Choice of alignment coordinate system
• Establishing correspondence is a key issue for
  building statistical shape models
• Various methods for definition of correspondence
  exist
• Resulting eigenmodes of deformation depend on
  these definitions

30
Object Alignment / Surface Homology
MZ pair
DZ pair
Surface Correspondence
31
Object Alignment before Shape Analysis
1stelli TR, no scal
1stelli TR, vol scal
Procrustes TRS
side
top
top
side
32
Correspondence through parameter space rotation
Parameters rotated to first order ellipsoids

• Normalization using first order ellipsoid
• Rotation of parameter space to align major axis
• Spatial alignment to major axes

33
Correspondence ctd.

• Rhodri Davies and Chris Taylor
• MDL criterion applied to shape population
• Refinement of correspondence to yield minimal
  description
• 83 left and right hippocampal surfaces
• Initial correspondence via SPHARM normalization
• IEEE TMI August 2002

34
Correspondence ctd.
Homologous points before (blue) and after MDL
refinement (red).
MSE of reconstructed vs. original shapes using n
Eigenmodes (leave one out). SPHARM vs. MDL
correspondence.
35
Model Building
VSkelTool
Medial representation for shape population
Styner, Gerig et al. , MMBIA00 / IPMI 2001 /
MICCAI 2001 / CVPR 2001/ MEDIA 2002 / IJCV 2003 /
36
VSkelToolPhD Martin Styner
Surface
M-rep
PDM
M-rep
Caudate
Voronoi
M-repRadii

• Population models
• PDM
• M-rep

VoronoiM-rep
Implied Bdr
37
II Medial Models for Shape Analysis
Medial representation for shape population
Styner and Gerig, MMBIA00 / IPMI 2001 / MICCAI
2001 / CVPR 2001/ ICPR 2002
38
Common model generation
Study population
Common model
...
Model building
Training population
Boundary SPHARM Medial m-rep
Two Shape Analyses - New insights, findings
39
A medial m-rep model incorporating shape
variability in 3 steps
Step 3 Computation of optimal sampling
Step 2 Computation of medial branching topology
Step 1 Definition of shape space
40
1. Shape space from training population

• Variability from training population
• Major PCA deformations define shape space
  covering 95
• Variability is smoothed
• Sample objects from shape space

1. 2. 3.
41
2. Common medial branching topology

• a. Compute individual medial branching topologies
  in shape space
• b. Combine medial branching topologies into one
  common branching topology

42
2a. Single branching topology

• Fine sampling of boundary
• Compute inner Voronoi diagram
• Group vertices into medial sheets (Naef)
• Remove unimportant medial sheets (Pruning)
• 98 vol. overlap

43
2a. Pruning of medial sheets
Amygdala-hippocampus From 1200 to 2 sheets
Left lateral ventricle From 1600 to 3 sheets
Volumetric overlap between reconstruction from
pruned skeleton and original object gt 98
44
2b. Common branching topology
For all objects in shape space

• Define common frame for spatial comparison
• TPS-warp objects into common frame using boundary
  correspondence
• Spatial match of sheets, paired Mahalanobis
  distance
• No structural (graph) topology match

Warp topology using SPHARM correspondence on
boundary
Match whole shape space
Final topology
Initial topology (average case)
Match
Match
45
3. Optimal grid sampling of medial sheets

• Appropriate sampling for model
• How to sample a sheet ?
• Compute minimal grid parameters for sampling
  given predefined approximation error in shape
  space

46
3a. Sampling of medial sheet

• Smoothing of sheet edge
• Determine medial axis of sheet
• Sample axis
• Find grid edge
• Interpolate rest
• m-rep fit to object (Joshi)

47
3b. Minimal sampling of medial sheet

• Find minimal sampling given a predefined
  approximation error

3x6
3x7
3x12
2x6
4x12
48
Medial models of subcortical structures
Shapes with common m-rep model and implied
boundaries of putamen, hippocampus, and lateral
ventricles. Each structure has a single-sheet
branching topology. Medial representations
calculated automatically.
49
Medial model generation scheme
Step 3 Compute minimal sampling
Step 2 Extract common topology
Step 4 Determine model statistics
Step 1 Define shape space
Goal To build 3D medial model which represents
shape population
50
Simplification VD single figure

• Compute inner VD of fine sampled boundary
• Group vertices into medial sheets (Naef)
• Remove nonsalient medial sheets (Pruning)
• Accuracy 98 volume overlap original vs.
  reconstruction

51
Pruning via sheet criterions
Number of Voronoi faces per sheet area
contribution of the Voronoi sheet to the whole
skeleton (Naef). Used only as fast, very
conservative ( 0.1), initial pruning. Size of a
medial manifold has no direct link to the
importance of the manifold to the shape.
1211 sheets
61 sheets
52
Pruning via sheet scheme
Hippocampus From 1200 to 2 sheets
Left lateral ventricle From 1600 to 3 sheets
volumetric overlap between reconstruction and
original shape gt 98
53
Common branching topology

• Surface sampled uniformly via parameterization.
• Full 3D Voronoi skeleton generated from the
  sampled boundary.
• Grouping/Merging/Pruning of skeleton sheets.
• Individual sheet extraction.
• Common Sheet Model via spatial comparison.

(Styner IPMI)
54
Pruning via sheet criterions
Volumetric contribution of sheet to overall
volume 1. conservative threshold (0.1). 2.
Non-conservative threshold (1)
61 sheets
6 sheets
6 sheets
2 sheets
Human Hippocampus data from M. Shenton, R.
Kikinis, Boston
55
Sampling of Medial Manifold
2x6
2x7
3x6
3x7
3x12
4x12
56
Medial models of subcortical structures
Shapes with common topology M-rep and implied
boundaries of putamen, hippocampus, and lateral
ventricles. Medial representations calculated
automatically (goodness of fit criterion).
57
Preprocessing of skeleton

• Points outside object ? remove
• Single points ? remove
• more than one manifold ? remove all but largest
• since spherical topology, no closed sheets ?
  punch hole

Human Hippocampus data from M. Shenton, R.
Kikinis, Boston
58
Topology preserving pruning

• Classification/Deletion scheme of D. Attali

59
Grouping of Voronoi faces to planar sheets

• Group the set of all Voronoi faces into a set of
  medial planar sheets
• First examined by Naef, 96. Our implementation is
  influenced by his ideas (2 step scheme).
• A) an initial grouping step most sheets are
  non-planar
• B) a refinement step partition initial sheets
  further, so that all are planar

60
Refinement step for grouping

• 1. Take a Voronoi edge that is adjacent two 3
  faces of the same group/sheet (non-planar
  location).
• 2. Assign a new group-id to each adjacent face
  and propagate all 3 ids on the non-planar sheet.
• ? partition of the non-planar sheet into 3 new
  sheets.
• Propagation is guided by geometric continuity
  criterion. The Voronoi face with best geometric
  continuity (angular difference of the normal)
  propagates first.
• 3. If any non-planar sheets, then repeat this
  process.

61
Pruning of Voronoi skeletons

• After grouping step similar groups are merged
  while maintaining planar sheets, merging based on
  combined geometric/radial continuity criterion
• Every pruning/deletion step changes the branching
  topology ? grouping has to be recalculated and
  the result of the grouping algorithm is again
  pruned. This is done until no parts of the
  Voronoi skeleton is deleted
• Concentrate on Global sheet/group criterions
• Number of Voronoi vertices per sheet, M. Naef
• Volumetric contribution per sheet

62
2a. Pruning via sheet criterions
Number of Voronoi faces per sheet area
contribution of the Voronoi sheet to the whole
skeleton (Naef, 1997). Used only as fast, very
conservative ( 0.1), initial pruning. Size of a
medial sheet has no direct link to the importance
of the sheet to the shape.
1211 sheets
61 sheets
63
2a. Pruning via sheet criterions
Volumetric contribution of sheet to overall
volume 1. conservative threshold (0.1). 2.
Non-conservative threshold (1)
61 sheets
6 sheets
6 sheets
2 sheets
Human Hippocampus data from M. Shenton, R.
Kikinis, Boston
64
2b. Common branching topologyWarp

• Define common frame for comparisons of branching
  topology (average case)
• TPS-Warp of all topologies into common frame
  using SPHARM boundary correspondence

65
3c. Minimal sampling over shape space

• Find minimal sampling in average case
• Test approximation to object set in shape space
• If there is an object with bad approximation
  error, then compute minimal sampling for that
  object and retest.
• If all objects have small approximation error,
  then common m-rep model with minimal sampling is
  found.