Multimodal Registration of Medical Data

1
Multimodal Registration of Medical Data

• Prof. Leo Joskowicz
• School of Computer Science and Engineering
• The Hebrew University of Jerusalem

2
Intensity-based rigid registration (1)

• Use intensity information to define the measure
  of similarity between two data sets
• Rationale the closer the data sets are, the more
  similar their intensity values are.
• No segmentation is necessary! The entire data
  set is used. Slow, especially for 3D data sets.
• The parametric space of transformations is
  searched incrementally from an initial
  configuration. The search space is
  six-dimensional (3 rotations and 3 translations)

3
Intensity-based registration (2)

• Similarity measures
• cross correlation
• histogram correlation
• mutual information
• intensity values
• Uses brain CT/MRI, Xray/CT
• Example fluoroscopic Xray to CT

4
Xray/CT registration

• Problem definition given
• preoperative CT data set of rigid structure
• intraoperative Xray images from a calibrated
  camera at relatively known spatial
  configurations
• Find a rigid transformation that matches the CT
  data set to the intraoperative Xrays so that if
  Xrays of the CT were taken from the transformed
  camera positions, the resulting Xray images would
  be identical to the intraoperative ones.

5
Xray/CT registration setup
Fluoroscopic image
Ref C-arm
DRR
Ref patient
Ref ct volume
2D/3D registration problem!
6
Xray to CT registration algorithm

• Input preop CT, intraop Xray Ifluoro , intrinsic
  Xray camera parameters, initial guess p0 for
  camera pose
• 1. generate simulated Xray IDRR (called digitally
  reconstructed radiograph, or DRR) at camera
  pose pi
• 2. Compute dissimilarity between IDRR and Ifluoro
  by comparing their intensities
• 3. Compute a new camera pose pi1 pi d that
  best reduces the dissimilarity between IDRR ,and
  Ifluoro)
• repeat until no progress can be made

7
Digitally reconstructed radiographs
8
Generating DRRs

• For each pixel in the DRR
• plane, construct the ray
• emanating from the camera
• focal point. Sum up the
• intensities of the CT voxel
• values according to the Xray
• attenuation formula to obtain
• the gray level value of the
• DRR pixel.

DRR
CT
Camera
9
Generated DRRs
10
Real X-ray vs DRR
11
Similarity measure

• Pairwise comparison of normalized pixel
• intensity values
• IDRR(i,j) and Ifluoro (i,j) are the pixel
  values
• IDRR and Ifluoro are the average image
  values
• T is the region of interest

12
Examples of initial poses registrations (DRRs
only)
13
Actual use radiation therapy with the Cyberknife
(radiation therapy)
14
Cyberknife system setup
15
Frameless radiation therapy
Stereotactic setup
Track head with Xrays before each dose application
16
Matching skull X-ray and DRR
Match only regions
17
CyberKnife system Description

• The acquired radiographs are masked to isolate
  the same regions of interest.
• Sobel Edge detection filter finds the point where
  the radial ray through the center of the region
  crosses the skull edge.
• Interpolating over several pixels ,to better
  resolve the maximum.
• All feature vector components carried equal
  weight.

18
CyberKnife system Description(4)

• The iteration are well describes by Eulerian
  rotation convertion.
• Rotation of the skull , are modeled by rotating
  the camera.
• Using Semiempirical algorithms to find next
  iteration.

19
CyberKnife system Description(5)

• Resolving outer edged of the skull by adjusting
  its integration step length according to the
  local gradient of the Hounsfield numbers.
• Compensating Residual differences in contrast
  between DRRs and radiographs by fitting a gamma
  function that matches brightness hystograms , and
  applying this function to subsequent DRRs.

20
CyberKnife system Results (1)

• The tests were performed using an anthropomorphic
  head phantom consisting of a human skull encased
  in plastic.
• The phantom was held by a fixture, that allowed
  it to be translated and rotated with six degrees
  of freedom.

21
CyberKnife system Results (2)
22
CyberKnife system Results (3)
23
CyberKnife system Results (4)
24
CyberKnife system Results (5)
25
CyberKnife system Conclusions(1)

• The numerical offset of a point in the skull may
  be large due to large target sites distance from
  the rotational axes.
• Empirical mean radial error was only 0.7 mm ,
  indicating that the uncertainties in the six
  degrees of freedom are correlated (expected).

26
CyberKnife system Conclusions(2)

• No systematic errors.
• No linkage (except edge cases), between the least
  square statistic and the angle error.
• No simulation or real trial has suggested any
  possibility that LSS could mistakenly converge to
  a good minimum , where we are far from the true
  position.

27
Intensity-based registration

• Advantages
• no segmentation, automatic
• selective regions
• potentially accurate
• Disadvantages
• large seach space, many local minima
• slow

28
Deformable registration scope

• Necessary for soft tissue organs and for
  cross-patient comparisons
• brain images before and during surgery
• anatomical structures at different times or from
  patients tumor growth, heart beating, compare
• matching to atlases
• Much more difficult than rigid registration!
• problem is ill-posed solution is not unique
• error measurements and comparisons are difficult
• local vs. global deformations?

29
Deformable registration properties

• Mapping transformation can be
• global, e.g., a bi- or tri-variate polynomial
• local, e.g.a fine grid with displacement vectors
  
• Define an energy function that should be
  minimized to make the data sets match.
• Usually comes after rigid registration to get an
  approximate position estimate.
• Both geometry based and intensity-based
  techniques exist.

30
Mathematics of deformations
Global transformations
rigid
quadratic
affine
triliear
31
Global deformation transformations
32
Local grid-based deformable registration
image 1 image 2
33
Example MRI slice matching
image 1
image 2
after registration
difference image with deformation
difference image without deformation
34
Brain tumor matching - 2D map
35
Brain tumor matching - 3D map
match
source
target
36
Example spine matching
Initial configuration
After rigid registration
After deformable registration with local splines
37
Deformable registration techniques

• Too many to list here!
• Optical flow model
• Physics-based elastic and fluid models
• Use an elastic or deformable model
• Validation is difficult

38
Commercial products

• Medical image processing software packages
  include some registration capabilities (manual or
  semi-automatic feature selection)
• Contact-based rigid registration of CT and
  optical tracker in orthopaedics and neurosurgery
  (half a dozen companies)
• Intraoperative Open MR to tracker rigid
  registration

39
The future research directions

• In many areas, the problem is far from solved
  similar to image segmentation!
• Much clinical validation is needed. More
  coverage of other anatomy (60 focus on brains!)
• Interleave segmentation and registration
• Difficult data sets 2D and 3D ultrasound images,
  video sequences, portal images
• Model-based techniques are the most likely to be
  sufficiently robust for clinical use
• Integration requirements are very important.

40
Bibliography (1)

• Two chapters on registration in
  Computer-Integrated Surgery, Taylor et al, MIT
  Press, 1995.
• Medical Image Registration, Hajnal et al, CRC
  Press 2001
• A survey of medical image registration, Maintz
  and Viergever, Medical Image Analysis Journal,
  2(1), Oxford University Press 1998 (over 150
  references!)
• A method for registration of 3D shapes, Besl
  and McKay, IEEE Trans. on Pattern Analysis,
  14(2), 1992.
• Special issue on Biomedical Image Registration,
  Image and Vision Computing, Vol 19(1-2), 2001.
• Deformable models in medical image analysis a
  survey, McInerney and Terzopolous, Medical Image
  Analysis 1(2), 1996.

41
Bibliography (2)

• Retrospective registration of tomographic brain
  images, J. Mainz, PhD Thesis, Utrecht U., 1996
  www.cs.ruu.nl/people/twan/personal/list.html
• Localy affine registration of free-form
  surfaces, J. Feldmar and N. Ayache, Proc. IEEE
  CVPR , 1994.
• Matching 3D anatomical surfaces with non-rigid
  deformations using octree splines, R. Szeliski
  and S. Lavallee, Int. Journal of Computer Vision
  18(2), 1996.
• Fast intensity-based non-rigid matching, P.
  Thirion, Proc. Conf. Medical Robotics and CAS,
  1995.
• Multimodal volume registration by maximization
  of mutual information, W.Wells, P.Viola,
  R.Kikinis. (idem)