Jerry L' Prince

1
Cortical Surface Segmentation and Topology

• Jerry L. Prince
• Image Analysis and Communications Laboratory
• Dept. of Electrical and Computer Engineering
• Johns Hopkins University

2
Acknowledgments

• Chenyang Xu
• Dzung Pham
• Xiao Han
• Duygu Tosun
• Bai Ying
• Daphne Yu
• Kirsten Behnke
• Xiaodong Tao

• Susan Resnick
• Mike Kraut
• Maryam Rettmann
• Christos Davatzikos
• Nick Bryan
• Aaron Carass
• Ulisses Braga-Neto

Funding sources NSF, NIH/NINDS, NIH/NIA
3
Outline

• Introduction
• Fuzzy Classification
• Nested Surface Segmentation
• Spherical Mapping and Partial Inflation
• Sulcal Segmentation
• Applications

4
Outline

• Introduction
• Fuzzy Classification
• Nested Surface Segmentation
• Spherical Mapping and Partial Inflation
• Sulcal Segmentation
• Applications

5
Brain Cortex Reconstruction
Magnetic Resonance Images (MRI)
Cortical Surface
6
Why Cortex Reconstruction?

• Study geometry of cortex
• relation to function
• changes in aging and disease
• Use in function mapping
• EEG/MEG/PET signals
• localization on surface instead of volume
• Surgical planning
• Automatic labels
• geometric plan

7
Nested Surfaces
Inner Central Outer
8
Some Difficulties

• Highly convoluted cortical folds
• Image noise
• Image intensity inhomogeneity
• Partial volume effect

9
Some Requirements

• Valid 2D manifold

• Topology correctness

10
Four Steps

• Fuzzy classification
• Nested surface segmentation
• Spherical mapping and partial inflation
• Sulcal segmentation

11
Outline

• Introduction
• Fuzzy Classification
• Nested Surface Segmentation
• Spherical Mapping and Partial Inflation
• Sulcal Segmentation
• Applications

12
Preprocessing
13
Fuzzy Segmentation
Pham Prince TMI 1999

• Yields continuous-valued fuzzy membership
• functions, with values in the range of 0,
  1

Gray matter
White matter
Cerebrospinal fluid
14
Published Algorithms
Pham and Prince

• AFCM Adaptive fuzzy c-means
• smooth gain field fuzzy clusters yields pseudo
  partial volume segmentation
• AGEM Adaptive generalized Expectation
  Maximization
• smooth gain field MRF label smoothness
  posterior density is fuzzy segmentation
• FANTASM
• Fuzzy segmentation with smooth membership
  functions and gain field

15
Membership Improvements

• White Matter
• Modifications to fill interior, remove extraneous
  surfaces, remove connectivity errors, and correct
  topology
• Gray Matter
• Modification to provide evidence of CSF in tight
  sulci

16
WM Isosurface

• Approximates WM/GM boundary
• Problems
• undesired surfaces
• connectivity errors
• handles

17
Autofill

• WM isosurface should represent the GM/WM
  interface of the cortex only

isosurface of WM segmentation before filling
isosurface of WM segmentation after filling
18
Autofill WM Volume
19
WM Isosurface Principle

• 0.5 of WM membership approximates WM/GM interface
• 0.5 of WMGM membership approximates GM/CSF
  interface

20
Marching Cubes Isosurface
Voxel values

• Consider values on corners of voxel
• Label as
• above isovalue
• below isovalue
• Determine position of triangular mesh surface
  passing through voxel
• Linear interpolation

21
Connectivity Errors

• Multiple meshes
• select the largest mesh
• Touching vertices, edges, and faces
• isovalue choice, or
• adjust pixel values by epsilon
• Ambiguous faces and cubes
• use saddle point methods, or
• use connectivity consistent MC algorithm

Most isosurface algorithms use rules that lead to
connectivity errors
22
Ambiguous Faces
Two possible tilings
23
Ambiguous Cubes
Two possible tilings
24
Digital Connectivity

• Consistent pairs
• (foreground,background) ? (6,18), (6,26),
  (18,6), (26,6)

25
Connectivity Consistent MC Algorithm
Ambiguous Face
Ambiguous Cube

• (black,white)
• (18,6) ? choose b, f
• (26,6) ? choose b, e

• (6,18) ? choose c, f
• (6,26) ? choose c, f

26
Remaining Problem Handles

• multiple surfaces
• shared vertices
• shared edges
• shared faces
• connectivity errors
• handles

Taken from actual white matter
27
Removes Handles by Editing WM
OR
28
Euler Number

• Handles easy to detect by computing the Euler
  number of the surface mesh

• Euler number of a triangular mesh
• A simple closed surface is topologically
  equivalent to a sphere iff
• genus is

A surface handle

• Euler number provides no information about the
  location of the handles

Illustration
29
GTCA Flow Diagram
30
Morphological Opening
structuring element
31
After Opening

• Divides object into two components
• body
• residue
• Build graph? Throw out residue pieces? NO!
• residue are often very large, but thin sheets
• opening may create holes that did not exist before

32
Conditional Topological Expansion

• Grow body by adding nice points from residue
  prohibits creation of handles allows filling of
  holes

33
Build a Graph
connected components
3
2
3
2
5
5
6
4
6
4
1
1
7
7
connectivity
34
Detect and Remove Cycles

• Find a cycle using depth-first search
• Find the smallest residue connected component in
  the cycle and remove it
• Repeat until no more cycles remain

35
Restore Residue

• Add remaining residue connected components back
  to body
• Run conditional topological expansion again.
• restores some points that were discarded prior to
  graph construction.

36
Success?

• Compute isosurface of binary volume
• Compute Euler number
• If less than 2 repeat on background
• Compute Euler number again
• If less than 2 repeat with larger structuring
  element, and so on
• Is isosurface algorithm consistent with digital
  topology?
• wrong algorithm ? connectivity paradoxes

37
Topology Correction Result
Before Topology Correction
After Topology Correction
¹WM
38
Results Quantitative
Genus of resulting volume.
Number of voxels changed in volume.
Ratio of voxels changed to original genus is
around 2
39
GM/WM Interface

• Topologically correct
• No self intersections
• Sub-voxel resolution
• Close to
• WM/GM surface
• GM central surface
• pial surface
• Represented by
• triangle mesh, or
• level set function

40
Gray Matter Isosurface

• Misses tight sulci

41
Partial Volume Effect
GM
CSF
Imaging
WM
partial volume averaging
42
Weighted Distance Skeleton
43
Anatomically Consistent Enhancement (ACE)


Outside
44
ACE Result
Original GM
ACE GM
45
Outline

• Introduction
• Fuzzy Classification
• Nested Surface Segmentation
• Spherical Mapping and Partial Inflation
• Sulcal Segmentation
• Applications

46
Deformable Surface Model

• Want to move the initial WM/GM mesh

47
Nested Deformable Surfaces
Pial Surface
Inner Surface
Central Surface
Initial WM Isosurface
48
Deformable Models

• Parametric deformable models (PDMs)
• Represent curves or surfaces through explicit
  parameterization
• e.g. curves tessellated with nodes,
• surfaces tessellated with triangles
• Geometric deformable models (GDMs)
• Implicit implementation
• uses level set numerical
• method

49
Parametric Deformable Models
Kass, Witkin, Terzopolous, 1987

• Curves/surfaces that deform with a speed law
  derived from image information and prior
  knowledge about object shape (e.g. boundary
  smoothness and continuity)

p location on contour
50
Level Set Method
Osher and Sethian 1988
One Extra Dimension
y
x
51
Advantages of GDMs

• Produce closed, non-self-intersecting contours
• Independent of contour parameterization
• Easy to implement numerical solution of PDEs on
  regular computational grid
• Stable computation

52
Parametric to Geometric
Osher Sethian 1988
Level Set PDE
53
Topology Behavior of Deformable Contour Models
Parametric
Geometric
TGDM

• Parametric ? self intersection problem
• Geometric ? cannot control topology
• TGDM (ours) ? preserves topology

54
Digital Embedding of Contour Topology

• Contour topology is determined by signs of the
  level set function at pixel locations
• Topology of the implicit contour is the same as
  the topology of the digital object

White Points
Black Points
55
Connectivity Rule of Contour

• Topology of digital contour determined by
  connectivity rule

Same digital object, different topologies
56
Topology Preservation Principle

• Preserving contour topology is equivalent to
  maintaining the topology of the digital object
• The digital object can only change topology when
  the level set function changes sign at a grid
  point
• Which sign changes can be allowed, and which
  cannot?
• To prevent the digital object from changing
  topology, the level set function should only be
  allowed to change sign at simple points

57
Simple Point

• Definition a point is simple if adding or
  removing the point from a binary object will not
  change the object topology
• Determination can be characterized locally by
  the configuration of its neighborhood (8- in 2D,
  26- in 3D) Bertrand Malandain 1994

Non- Simple
Simple
58
x is a Simple Point
59
x is Not a Simple Point
X
X
60
Topology Preserving Geometric Deformable Model
(TGDM)

• Evolve level set function according to GDM
• If level set function is going to change sign,
  check whether the point is a simple point
• If simple, permit the sign-change
• If not simple, prohibit the sign-change
• (replace the grid value by epsilon with same
  sign)
• (Roughly, this step adds 7 computation time.)
• Extract the final contour using a connectivity
  consistent isocontour algorithm

61
A 2D Demonstration
SGDM
TGDM
62
No Self-intersections
PDM Result
TGDM Result
63
A 3D TGDM Demonstration
SGDM Init 1
Original Object
2
1
TDGM Init 1
SGDM Init 2
TDGM Init 2
64
TGDM for Inner Surface
Final GM/WM Interface
65
TGDM for Inner Surface

• Evolution Equation

Region Force
66
TGDM for Central Surface
Final Central Surface
67
TGDM for Central Surface

• Gradient Vector Flow Xu Prince TIP98

68
TGDM for Central Surface

• Evolution Equation

Gradient Vector Flow Force
69
Nesting Constraint

• Nested surfaces
• Central is outside GM/WM
• Pial is outside central
• If level set function wants to go negative to
  positive
• allow if inner level set function is positive
• otherwise set to small positive epsilon

70
TGDM for Outer Surface
Final Pial Surface
Start from Central Surface
71
TGDM for Outer Surface

• Evolution Equation

Gradient Vector Flow Force
72
Results Visual Inspection

• Slice views of three surfaces overlaid on
  cross-sections of the original image

73
Repeatability Analysis

• 3 subjects, each scanned twice
• Surface pairs rigidly registered
• Average errors
• signed distance
• absolute distance

74
Repeatability Results (mm)
75
Landmark Validation Study
76
Landmark Validation Analysis

• Raters 12
• Brains 2
• Landmarks 10 per region
• Sulci 33 / brain
• Geometry 11 fundi, 11 gyri, 11 banks
• Surface Inner Pial
• Statistical software R version 1.8.1

• CRUISE surfaces are reference surfaces yield
  landmark offset
• signed and absolute
• Membership values
• white matter
• gray matter
• Statistical factors
• Brain
• Geometry
• Sulci

77
Landmark Validation Results

• MANOVA revealed significant factors
• geometry sulci, but not brain
• Landmark offset
• mean - 0.35 mm
• std 0.65 mm
• 16 farther than 1 mm from reference
• ACE regions show smaller offsets

• Signed distance consistently negative
• outward bias of CRUISE
• differs for geometry (largest for fundi)
• differs for surface
• Note we are optimizing parameters

78
Nested Surface Segmentation
Han et al, 2004

• Nearly fully automated
• skull-stripping is semi-automated (10 minutes)
• AC PC need to be picked manually (5 minutes)
• The rest is fully automated
• Less than 25 minutes for each brain
• (Previous PDM version takes 2-3 hours)
• More than 200 brain datasets processed so far
• average error is about 1/3 voxel
• highly repeatable ? scanner errors dominate

79
Outline

• Introduction
• Fuzzy Classification
• Nested Surface Segmentation
• Spherical Mapping and Partial Inflation
• Sulcal Segmentation
• Applications

80
Spherical and Partial Flattening
Tosun et al, 2003
81
Surface Inflation

• Coarsen shape
• More regular mesh structure
• Use relaxation operator
• Check norm of mean curvature

82
Atlas Registration
Subject
Atlas

• Simpler surface registered using modified ICP
• Atlas labels transfer easily

(a)
(b)
(c)
(d)
83
Spherical Mapping

• Single conformal map from atlas
• Inverse stereographic projection

84
Automatic Labelling

• Brains mapped to sphere
• Segmented sulci compared to labelled atlas
• Simple voting scheme leads to gt90 accuracy

85
Outline

• Introduction
• Fuzzy Classification
• Nested Surface Segmentation
• Spherical Mapping and Partial Inflation
• Sulcal Segmentation
• Applications

86
Sulcal Segmentation

• Goals
• Automatically segment sulci
• carry out cortical parcellation

• Principle
• Based on depth from outer surface

• Applications
• Localizing activation sites in functional images
• Brain registration
• Understanding morphological changes
• in normal aging and disease

87
Sulcal Regions
Defined as buried cortical regions that surround
sulcal spaces
88
Classifying Gyral and Sulcal Regions

• Generate a shrink-wrap surface
• Sulcal regions distinguished from gyral regions
  based on distance to shrink-wrap surface

89
Sulcal/Gyral Classification
sulcal regions (red) and gyral regions (blue)
Euclidean distance to outer surface
sulci gt 2 mm from outer surface
90
Watershed Segmentation

• Classification does not separate sulci
• Further segmentation is required
• Watershed by immersion is intuitive idea

91
Geodesic Distance Computation

• use Fast Marching (Kimmel and Sethian, 98)
• initial contour at time zero is gyral/sulcal
  boundary
• Propagation at unit speed in normal direction on
  mesh
• geodesic distance is arrival time of evolving
  contour

92
Watershed Computation

• Each local minimum
• produces a
• catchment basin (CB).
• Critique
• true sulci are
• separated
• single sulci are
• over-segmented.

93
Merging Algorithm

• Addresses over-segmentation problem
• Small ridges in sulcal regions result in
  formation of separate CBs
• Criterion for merging CBs
• 1) height of ridge
• 2) size of CB
• Provides different levels of merging

94
Sulcal Segmentation Results
Height threshold 1 cm Size threshold 3 cm2
Rettmann et al. MMBIA 2000
95
Sulcal Segmentation Results
96
Cross-Sections
97
(No Transcript)
98
Outline

• Introduction
• Fuzzy Classification
• Nested Surface Segmentation
• Spherical Mapping and Partial Inflation
• Sulcal Segmentation
• Applications

99
Repeat Scan Validation
Superior frontal sulcus
scan 1
scan 2
scan 3
100
Shape Analysis
Left
Subject 1
Subject 2
Right
Cingulate
101
Geometric Features
mean curvature
geodesic depth
102
Cortical Thickness
Yezzi et al, 2003
103
Baltimore Longitudinal Study of Aging

• PI Susan Resnick (NIA)
• 1994-2003
• Ages 55-85, 158 participants
• gt1000 separate scans, 1 per year per subject
• volumetric SPGR brain scans
• 0.9375x0.9375x1.5mm voxel size

104
Typical Thickness Map
Thickness Map from CRUISE
105
Cross-sectional Study of Cortical Thickness

• Preliminary study on 35 subjects

106
The END