Generative models for automated brain MRI segmentation

1
Generative models for automated brain MRI
segmentation

• Koen Van Leemput
• Athinoula A. Martinos Center for Biomedical
  Imaging
• Department of Radiology, MGH
• Harvard Medical School, USA
• Computer Science and Artificial Intelligence
  Laboratory
• Massachusetts Institute of Technology, USA

2
MRI of the brain

• Magnetic resonance imaging
• Harmless
• Three dimensional (3-D)
• High soft tissue contrast
• High spatial resolution
• Extremely versatile
• Possibly multi-spectral

voxel
Ideal for studying the living human brain
3
Segmentation of brain MRI

• Delineating structures of interest in the images
• Segmentation is important
• Basic neuroscience
• Uncovering disease mechanisms
• Diagnosis, treatment planning, and follow-up
• Clinical drug trials
• Automated computational methods are needed

4
Overview

• Segmentation basics modeling and inference
• Modeling MRI bias fields
• Mesh-based brain atlases
• Whole-brain segmentation

5
Overview

• Segmentation basics modeling and inference
• Modeling MRI bias fields
• Mesh-based brain atlases
• Whole-brain segmentation

6
The problem to be solved
MRI image
7
The problem to be solved
MRI image
Label image
8
One solution generative modeling

• Formulate a statistical model of how an MRI image
  is formed
• The model depends on some parameters

labeling model
imaging model
Label image
MRI image
9
Segmentation inverse problem
MRI image
Label image
10
Segmentation inverse problem
MRI image
Label image

• Bayesian inference
• Start from our statistical model of image
  formation
• Play with the mathematical rules of probability

11
Bayesian inference

• Practical approximation
• Involves two optimizations
• First estimate the optimal model parameters
• Then find the optimal segmentation based on those
  parameter estimates

12
Example Gaussian mixture model
labeling model
imaging model
MRI image
Label image

• The label in each voxel is drawn independently
  with a probability for tissue type k
• Assume a uniform prior for the
  labeling model parameters

13
Example Gaussian mixture model
labeling model
imaging model
MRI image
Label image

• The intensity in each voxel is drawn
  independently from a Gaussian distribution
  associated with its label
• The imaging model parameters are the mean
  and variance of each Gaussian
• Assume a uniform prior

14
Example Gaussian mixture model
three labels
Model parameters
are unknown
Mean and variance of each Gaussian
Relative weight of each Gaussian
15
Optimization 1 parameter estimation

• Given an MRI image to be segmented, what is the
  MAP parameter estimate ?
• Parameter optimization with an Expectation
  Maximization (EM) algorithm

• Repeatedly maximize a lower bound to the
  objective function
• Iterative parameter optimizer using only
  closed-form parameter updates!

current estimate
16
Optimization 1 parameter estimation
17
Optimization 1 parameter estimation
18
Optimization 2 segmentation
white matter
Upon completion of the parameter estimation
algorithm, assign each voxel to the MAP label
CSF
gray matter
19
Overview

• Segmentation basics modeling and inference
• Modeling MRI bias fields
• Mesh-based brain atlases
• Whole-brain segmentation

20
MRI bias field artifact

• Intensity inhomogeneities across the image area
• Imaging artifact in MRI
• equipment limitations
• patient-induced electrodynamic interactions

MRI data
after intensity windowing
21
MRI bias field artifact

• Causes segmentation errors with our segmentation
  procedure so far

22
MRI bias field artifact
Causes segmentation errors with our segmentation
procedure so far
23
Improved imaging model
labeling model
imaging model
MRI image
Label image
24
Improved imaging model
labeling model
imaging model
MRI image
Label image
old model
25
Improved imaging model
labeling model
imaging model
MRI image
Label image

polynomial bias field model
old model
26
Model parameter estimation

• Polynomial coefficients are part of the model
  parameters
• Parameter optimization with a Generalized
  Expectation Maximization (GEM) algorithm

• Repeatedly improve a lower bound to the objective
  function
• Iterative parameter optimizer using only
  closed-form parameter updates! Van Leemput et
  al., IEEE TMI 1999

current estimate
27
Example
28
Example
MRI data
White matter without bias field model
White matter with bias field model
Estimated bias field
29
Example
MRI data
White matter without bias field model
White matter with bias field model
Estimated bias field
30
Overview

• Segmentation basics modeling and inference
• Modeling MRI bias fields
• Mesh-based brain atlases
• Whole-brain segmentation

31
Improving the labeling model
labeling model
imaging model
MRI image
Label image

• So far our labeling model just expresses the
  relative frequency of occurrence of different
  labels
• Too simplistic for segmenting the brain into 30
  subregions

A more realistic labeling model is needed!
32
Improving the labeling model
33
Improving the labeling model
Try to find the underlying probability
distribution
Manual segmentations in N individuals (training
data)
34
Modeling the training data (2-D)
Triangular mesh representation
35
Modeling the training data (2-D)
atlas

• Assign label probabilities to each mesh node
• Flat prior
• Label probabilities are linearly interpolated
  over triangle areas

36
Modeling the training data (2-D)
atlas
Mesh node positions are sampled from a
topology-preserving Markov random field prior
warped atlases
knob that controls the flexibility of the atlas
warp
37
Modeling the training data (2-D)
atlas
Example segmentations are sampled according to
the deformed atlases
warped atlases
example segmentations
38
Bayesian inference Van Leemput, IEEE TMI 2009

• Given a collection of manual segmentations
• what is the most probable atlas?
• what is the most likely value of the parameter
  controlling the flexibility of the deformations?
• what is the most likely mesh
• representation?
• Good models explain regularities in the manual
  segmentations
• Automatically yields sparse representations that
  explicitly avoid overfitting to the training data
• cf. Minimum Description Length

39
Example atlas
Derived from manual segmentations of 36 brain
substructures in 4 individuals
Has average shape
40
Overview

• Segmentation basics modeling and inference
• Modeling MRI bias fields
• Mesh-based brain atlases
• Whole-brain segmentation

41
Whole-brain segmentation
labeling model
imaging model
MRI image
Label image

• Tetrahedral mesh-based atlas
• The labeling model parameters are the
  location of the mesh nodes
• The prior is the topology-preserving
  MRF model (penalizes deformations)

42
Whole-brain segmentation
labeling model
imaging model
MRI image
Label image

polynomial bias field model
Gaussian mixture model
43
Whole-brain segmentation

• Model parameter estimation
• Fully automated segmentation procedure
• No need for pre-processing (skull stripping, bias
  field corr., )
• Automatically adapts to different scanners and
  acquisition sequences!
• Fast!

Improve the imaging model parameters
(Generalized Expectation-Maximization closed-for
m expressions) Improve the atlas warp
(registration gradient in analytical form)
44
Examples (validation under way)
45
Examples (validation under way)
46
Examples (validation under way)
47
Examples (validation under way)
48
Examples (validation under way)
49
Examples (validation under way)
50

• Thanks!