Shape Modeling in Medical Imaging

1
Shape Modeling in Medical Imaging

• James S. Duncan, Ph.D.
• Hemant Tagare, Ph.D.
• Image Processing and Analysis Group
• Section of Bioimaging Sciences
• Yale University

2
Outline

• Definition of Shape and Shape Modeling
• Data Models
• - Data Primitives
• - Surface Models (local/global strategies)
  Medial models
• Multiple surfaces atlases Volumetric
  Models
• Transformation Models
• - Rigid, affine
• - Nonrigid (including physical analogs,
  free-form deformation,
• actual biomechanical models)
• Shape Distribution Models - Shape
  Statistics of Landmarks (e.g. Bookstein, Kendall)
  
• - Point Distribution Models (Cootes
  Taylor)
• Shape modeling state-of-the-art
• Some remaining challenges

3
Definitions

• Shape (Webster) the visible makeup
  characteristic of a particular item or kind of
  item.
• Model (Webster) a description or analogy used
  to help visualize something that cannot be
  directly observed
• Shape Models efficient representations of data,
  transformations, shape distributions, useful in
  Medical Imaging for
• - segmentation/measurement of anatomy and
  function
• - structure-based registration/comparison of
  images
• - visualization of structure and function

4
Shape Modeling in Medical Image Analysis

• Lots of ideas in a variety of areas
• Little has been done to look across different
  modeling strategies, to classify and compare
  approaches
• One way to begin is to take what mathematicians
  (i.e. Kendall) define as shape and then look
  at how shape models in medical imaging relate

5
Shape (taken in part from The Statistical
Theory of Shape, C.G. Small, Springer, 1996)
Transformation
Data
Data
6
Shape Descriptors, Spaces and Distributions
- Points in each orbit configurations that can
be mapped to each other via transformation -
Each orbit corresponds to one shape - Any
function in this space that is constant on the
orbits is a Shape descriptor
Original data
- Shape Space the manifold of orbits - Shape
distribution probability distribution on this
space - Shape Metric any comparison metric
in shape space
7
Simple Example of a Shape Space
Shape space manifold of dim 2 (2N-4)
data 3 labeled points in a plane forming a
triangle Use Similarity and normalize, such
that the base of each triangle is mapped to a
fixed line of length 1
Compare by some metric (e.g.Procrustes)
8
I. Data Models
9
Data Primitives
- Points - Curves/ Surfaces - Volumes
E.g. Talairach landmarks
E.g. point clouds
10
Local Smoothness Models
Level Set-Based Strategies
F data adherance curvature smoothness (
local shape model)
From Vemuri, et al, IEEE PAMI, 1995
Many Other Deformable Modeling efforts with local
smoothness constraints
- Prince, Davatzikos - Terzopoulos (snakes
) - Metaxas
11
Global Curve/Surface Models(e.g. Staib and
Duncan, IEEE TMI, 1996)
12
Medial Models m-Reps (Pizer, et al., IPMI99
from www.unc.cs.edu)
3D m-rep of kidney ureter
M-rep model of kidney (2 linked figures)
x position r radius of p and s
Medial atom m (x, r, F, q)
Figural segments/objects formed from many atoms
arranged in a mesh/curve (approx a continuous
manifold)
x
F a local frame parameterized by
q
- q
q angle within atom
13
Multiple Structures Shape Atlases
E.g. ANIMAL MNI from L. Collins
(http//www.bic.mni.mcgill.ca/users/louis/mri_segm
entation/) 1. define labeled model 2.
match underlying gray scale info 3. carry
model to target image (see also earlier work by
Bajcsy)
Kennedy-Rademacher hand - labeled brain atlas
14
II. Transformation Modeling
15
Nonrigid Curve Transformation Models

• Similarity (many investigators) model nonrigid
  mappings as only translation rotation scaling
• Smooth nonrigid mappings (Tagare, TMI99)

16
Fluid Flow Modeling of Nonrigid Transformation
(Christensen, Miller, Grenander, Vannier,
Computer, 1996)
Atlas data fluidly deformed to match study MRI
data
From Atlas MRI data
Fluid Flow Model (alternate form also uses
Elastic mapping)
Where m and b viscosity coefficients and
velocity is given by
with
17
Free Form Deformation (originally from
Sederberg and Parry, 1986)

• a volumetric mapping based on trivariate
  Bernstein polynomials

Where P is the volumetric grid of control points
(l,m,n) are polynomial degrees (s,t,u) are
object points X deformed model points
From http//www.cs.unc.edu/geom/ffd Fast
Volume-Preserving Free Form Deformation Using
Multi-Level Optimization, by G. Hirota, R.
Mashewari, M. Lin, Dept of CS, UNC-CH related
work in Medical Image Analysis community E.
Bardinet, N. Ayache, CVRMed, 1995, Nice. (and
others).
18
Biomechanical (Finite Element) Models of
Physical Objects Combination of Data and
Transformation Modeling
19
LV Material Model (Papademetris, et al., Med.
Image Analysis, 2001)
20
III. Shape Distribution Models
21
Procrustes Landmark Analysis(Bookstein, Kendall,
Mardia etc.)
1
1
2
6
2
3
3
6
5
4
Similarity
4
5
Basic primitive Finite labeled points in the
plane (or R3) Transformation Similarity
(Translation, rotation, scale)
22
Procrustes Distribution
1
1
1
2
2
3
2
3
6
3
6
6
4
4
4
5
5
5

• Procedure
• Find the mean, move it to (0,0) (Eliminate
  translation)
• Scale it so sum of distances to mean 1
  (Eliminate scale)
• Rotate the resulting configuration so sum of
  squares is minimized.

Modeling examples -
PCA Normal Distribution
- Complex Bingham Distribution (Mardia)
23
Point Distribution Models (PDM) (Cootes
Taylor, IPMI 93, http//www.wiau.man.ac.uk/bim/Mo
dels/pdms.html)

• Each shape in a training set defined by the SAME
  n labeled points
• put into common coordinate frame using
  Procrustes analysis (shape space)
• represent each shape by 2n vector x
  (x_1,,x_n, y_1,, y_n)
• represent aligned training data by Gaussian
• pick out primary axes of variation using PCA
• the PDM shape model is now x xmean Pb

Training set 72 pts/example
t element vector of shape parameters
Mean of aligned training samples
a 2n x t matrix whose columns are unit vectors
along the principal axes of the cloud
24
Adding Appearance to PDMs (Cootes, IPMI99)

• For each example

• shape model x xmean Psbs
• texture model g gmean Pgbg
• Ps , Pg derived from training bi control modes
  of variation

25
Approximate Shape Distribution Models from
Parametrized Surfaces (Kelemen, Szekely, Gerig,
IEEE TMI, Oct, 1999)
- Basic Primitive surfaces parametrized by
spherical harmonics - Transformation
area-preserving mapping to standardized pose
(note not clearly a tranformation group) -Shape
Model PCA analysis of covariance matrix of
surface parameters transformed to shape space
(similar to Cootes/Taylor PDM) x
xmean Pshbsh with Psh derived from training
bi control modes of variation
22 left hippocampal structures from training set
Rows represent largest two modes of variation
(eigenvectors in shape space)
26
Shape Modeling State of the Art
27
Some Remaining Challenges in Shape Modeling

• when are two models representing the same thing
  ? (especially difficult in shape distribution
  modeling) - can two shape spaces be compared ?
• how do we make decisions regarding which model
  is best for a particular task ?
• How can models best be constructed from training
  data ?

28
Shape Modeling IPMI 2001

• Four papers in this session, each addresses one
  or more of the model types defined
• Papademetris Active Elastic Models (Example
  of Transformation Model)
• Davies Minimum Description Length Approach to
  Statistical Shape Modeling ( Finding an optimal
  Data Model in the context of a particular Shape
  Distribution/Shape Space)
• Joshi Multiscale 3D Deformable Model
  Segmentation Based on Medial Description (Data
  Model Transformation Model)
• Styner Medial Models Incorporating Object
  Variability for 3D Shape Analysis (Development
  of combined 3D Data Model in context of a
  particular Shape Distribution Model)