Stephen Pizer, Sarang Joshi, Guido Gerig

1
Tutorial Anatomic Object Ensemble
Representations for Segmentation Statistical
Characterization

• Stephen Pizer, Sarang Joshi, Guido Gerig
• Medical Image Display Analysis Group (MIDAG)
• University of North Carolina, USA
• with credit to
• P. T. Fletcher, C. Lu, M. Styner, A. Thall,
  P. Yushkevich
• And others in MIDAG
• This set of slides can be found at the website
  midag.cs.unc.edu/pubs/presentations/SPIE_tut.htm

17 February 2003
2
Segmentation

• Objective Extract the most probable target
  object geometric conformation z given the image
  data I
• Requires prior on object geometry p(z)
• Requires a measure of match p(Iz) of the image
  to a particular object conformation, so the image
  must be represented in reference to the object
  geometric conformation

3
Statistical Geometric Characterization

• Requires priors p(class) and likelihoods
  p(zclass)
• Uses
• Medical science determine geometric ways in
  which pathological and normal classes differ
• Diagnostic determine if a particular patients
  geometry is in the pathological or the healthy
  class
• Educational communicate anatomic variability in
  atlases
• Priors p(z) for segmentation
• Monte Carlo generation of images

4
Tutorial Anatomic Object Ensemble
Representations for Segmentation Statistical
Characterization

• Part I
• Multiscale Geometric Primitives,
• Especially M-reps
• Multiscale Deformable Model Segmentation
• Stephen Pizer

17 February 2003
5
Object Representation Objectives

• Relation of this object instance to other
  instances
• Representing the real world
• Basic entities object ensembles single objects
• Deformation while staying in statistical entity
  class
• Discrimination by shape class and by locality
• Mechanical deformation within a patient interior
  primitive
• Relation to Euclidean space/projective Euclidean
  space
• Matching image data
• Multiple object-oriented scale levels
• Yields efficiency in segmentation coarse to fine
• Yields efficiency in number of training samples
  for probabilities

6
Object Ensembles Single Objects

• Object descriptions
• Intuitive, related to anatomic understanding
• Mathematically correct
• Object interrelation descriptions
• Abutment and non-interpenetration

Large scale Smaller scale
7
Multiple Object-oriented Scale Levels -- For
Efficiency

• Scale based parents and neighbors
• Intuitive scale levels
• Ensemble Object Main figure
  Subfigure
• Slab through-section Boundary vertex

8
Multiple Object-oriented Scale Levels -- For
Efficiency

• Scale based parents and neighbors
• Statistics via Markov random fields Lu
• Residue from parent zki ith residue at
  scale level k
• Difference from neighbors prediction
• p(zki relative to P(zki), zki relative to N(zki))
• Efficiency of training from low dimension per
  probability
• Features with position and level of locality
  (scale)
• Feature selection Yushkevich

9
Discussion of Scale

• Spatial aspects of a geometric feature
• Position
• Scale 3 different types
• Spatial extent
• Region summarized
• Level of detail captured
• Residues from larger scales
• Distances to neighbors with which
  it has a statistical
  relationship
• Markov random field
• Consider point distribution model, landmarks,
  spherical harmonics, dense Euclidean positions,
  m-reps

Large scale Smaller scale
10
Scale Situations in Various Statistical Geometric
Analysis Approaches
Global coef for Multidetail
feature Detail residues each level of
detail Examples boundary spherical
boundary points, m-rep object
harmonics, global dense position hierarchy,
principal components
displacements wavelets
Level of Detail
Fine
Coarse
Location
Location
Location
11
Object Representations Atoms

• Atlas voxels with a displacement at each voxel
  Dx(x), label(x)
• Set of distinguished points xi with a
  displacement at each
• Landmarks
• Boundary points in a mesh
• With normal b (x,n)
• Loci of medial atoms m
  (x,F,r,q) or
  end atom (x,F,r,q,h)
• (show on Pablo)

12
Multiscale Object Representation via Interiors
M-reps

• Interiors (medial) at all but smallest scale
  levels
• Boundary displacement at smallest scale level
• Allows fixed structure in medial part
• Residues from previous scale level
• At each level recognizes invariances
  associated with shape
• Provide correspondence
• Across population Across comparable
  structures
• Provides prediction by neighbors
• Translation, rotation, magnification
• Structure trained from population Styner
• Basis for deformable model segmentation

boundary tradl medial medial atom
Continuous vs. sampled repns
13
M-rep Gives Multiscale Intrinsic Coords for
Nonspherical Nontubular Objects

• Here single-figure
• On medial locus
• (u,v) in r-proportional metric
• v along medial curve of medial sheet
• u across medial sheet
• t around crest
• Across narrow object dimension
• t along medial spokes
• Proportion of medial width r

14
Discrete M-rep Multifigure Objects and
Multiobject Ensembles

• Meshes of medial atoms
• Objects connected as host, subfigures
• Hinge atoms of subfigure on boundary
  of parent figure
• Blend in hinge regions
• Special coordinate system (u,w,t) for blend
  region
• Multiple such objects, inter-related via
  neighbors figural coords

w
15
M-rep Intrinsic Coordinates

• Within figure
• One medial atom provides a coordinate system for
  its neighbor atoms
• Position, Orientation, Metric
• Between subfigure and figure
• Host atoms coordinate systems provides
  coordinate system for protrusion or indentation
  hinge
• Between figures or between objects
• One object provides coordinate system for
  neighbor object boundary

16
Interpolating Boundaries in a Figure

• Interpolate x, r via B-splines Yushkevich
• Trimming curve via rlt0 at outside control points
• Avoids corner problems of quadmesh
• Yields continuous boundary
• Via modified subdivision surface Thall
• Approximate orthogonality at spoke ends
• Interpolated atoms via boundary and distance
• At ends elongation h needs also to be
  interpolated
• Need to use synthetic medial geometry Damon

Medial sheet
Implied boundary
17
Sampled medial shape representation M-rep tube
figures

• Same atoms as for slabs
• r is radius of tube
• spokes are rotated about b
• Chain rather than mesh

x rRb,n(q)b
xrRb,n(-q)b
18
Segmentation by Deformable M-reps

• For each scale level k, coarse to fine
• For all residues i at scale level k zki
• Maximize log p(zki relative to
  P(zki), zki relative to N(zki)) log
  p(Imagezji, jgtk, all i)
• i.e., maximize geometric typicality
• geometry to image match
• (show on Pablo)

w
19
Intensity Profiles Template Used in Geometry to
Image Match
Template to target image correspondence via
figural coordinates
Inside
Outside
Mean profile image along red meridian line,
from training or as analytic function of ?t/r
20
3-Scale Deformation of M-reps Pizer, Joshi,
Chaney, et al. Segmentation of Kidney from CT
Optimal warp
Optimal movement
Refined boundary
21
Three Stage - Single Figure Segmentation of
Kidney from CT
Axial, sagittal, and coronal target image
slices Grey curve before step. White curve
after step
Optimal movement
Optimal warp
Refined boundary
22
Segmentation by Deformable M-repsControlled
Validations

• Kidneys
• Human segmented
• Robust over all 12 kidney pairs
• Avg distance to human segns boundary lt1.7mm
• Clinically acceptable agreement with humans
• Monte Carlo produced
• Robust against initialization
• Other anecdotal validations
• Liver, male pelvis ensemble, caudate, hippocampus

w
23
For a copy of the slides in this talk see
website midag.cs.unc.edu/pubs/presentations/SPIE
_tut.htm For background to this talk see tutorial
at website midag.cs.unc.edu/projects/object-shap
e/tutorial/index.htm or papers at
midag.cs.unc.edu
24
References Non-M-reps

• Voxel displacements and labels Grenander, U and
  M Miller (1998). Computational anatomy an
  emerging discipline. Quarterly of Applied
  Mathematics, 56 617-694.
  Christensen, G, S Joshi, and M Miller (1997).
  Volumetric transformation of brain anatomy. IEEE
  Transactions on Medical Imaging, 16(6) 864-877.
• Landmarks Dryden, I K Mardia, (1998).
  Statistical Shape Analysis. John Wiley and Sons
  (Chichester).
• Point distribution models T Cootes, A Hill, CJ
  Taylor (1994). Use of active shape models for
  locating structures in medical images. Image
  Vision Computing 12 355-366.
• Spherical harmonic models Kelemen, A, G Székely,
  G Gerig (1999). Elastic model-based segmentation
  of 3D neuroradiological data sets. IEEE
  Transactions of Medical Imaging, 18 828-839.

25
References M-reps

• Overview Pizer, S, G Gerig, S Joshi, S Aylward
  (2002). Multiscale medial shape-based analysis of
  image objects. Proc. IEEE, to appear.
  http//midag.cs.unc.edu/pubs/papers/IEEEproc03_Piz
  er_multimed.pdf
• Deformable m-reps segmentation Pizer, S, et al.
  (2002). Deformable m-reps for 3D medical image
  segmentation. Subm. for IJCV special UNC-MIDAG
  issue. http//midag.cs.unc.edu/pubs/papers/IJCV01-
  Pizer-mreps.pdf
• Figural coordinates Pizer S, et al. (2002).
  Object models in multiscale intrinsic coordinates
  via m-reps. Image Vision Computing special
  issue on generative model-based vision, to
  appear. http//midag.cs.unc.edu/pubs/papers/GMBV02
  _Pizer.pdf
• Forming m-rep models Styner, M et al.,
  Statistical shape analysis of neuroanatomical
  structures based on medial models.
  Medical Image Analysis, to appear spring 2003.
  http//midag.cs.unc.edu/pubs/papers/MEDIA01-styner
  -submit.pdf

26
References M-reps

• Continuous m-reps Yushkevich, P et al. (2002).
  Continuous Medial Representations for Geometric
  Object Modeling in 2D and 3D. Image Vision
  Computing special issue on generative model-based
  vision, to appear. http//midag.cs.unc.edu/pubs/pa
  pers/IVC02-Yushkevich
• Implied boundaries via subdivision surfaces
  Thall, A (2002). Fast C2 interpolating
  subdivision surfaces using iterative inversion of
  stationary subdivision rules. UNC Comp. Sci.
  Tech. Rep. TR02-001. http//midag.cs.unc.
  edu/pubs/papers/Thall_TR02-001.pdf
• Markov random fields Lu, C, S Pizer, S Joshi
  (2003). A Markov Random Field approach to
  multi-scale shape analysis. Subm. to Scale Space.
  http//midag.cs.unc.edu/pubs/papers/ScaleSpace03_C
  onglin_shape.pdf
• Math of m-reps --gt boundaries Damon, J (2002),
  Determining the geometry of boundaries of
  objects from medial data. UNC Math. Dept.
  http//midag.cs.unc.edu/pubs/papers/Damon_SkelStr_
  III.pdf