Medical Imaging Projects @ DePaul CDM

1
Medical Imaging Projects_at_ DePaul CDM

• Daniela S. Raicu, PhD
• Associate Professor
• Email draicu_at_cs.depaul.edu
• Lab URL http//facweb.cs.depaul.edu/research/vc/

2
Outline

• Medical Imaging (Computed Tomography)
• Content-based and semantic-based image retrieval
• Projects 1 and 2
• Mappings from low-level image features to
  semantic concepts
• Projects 3 and 4
• Liver segmentation
• Project 5

3
Content-based medical image retrieval (CBMS)
systems

-
Definition of Content-based Image
Retrieval Content-based image retrieval is a
technique for retrieving images on the basis of
automatically derived image features such as
texture and shape.

• Applications of Content-based Image Retrieval
• Teaching
• Research
• Diagnosis
• PACS and Electronic Patient Records

4
Diagram of a CBIR
5
CBIR as a Diagnosis Aid


An image retrieval system can help when the
diagnosis depends strongly on direct visual
properties of images in the context of
evidence-based medicine or case-based reasoning.

6
CBIR as a Teaching Tool
An image retrieval system will allow
students/teachers to browse available data
themselves in an easy and straightforward fashion
by clicking on show me similar images.
Advantages - stimulate self-learning and a
comparison of similar cases - find optimal cases
for teaching

• Teaching files
• Casimage http//www.casimage.com
• myPACS http//www.mypacs.net

7
CBIR as a Research Tool

• Image retrieval systems can be used
• to complement text-based retrieval methods
• for visual knowledge management whereby the
  images and associated textual data can be
  analyzed together
• multimedia data mining can be applied to learn
  the unknown links between visual features and
  diagnosis or other patient information
• for quality control to find images that might
  have been misclassified

8
CBIR as a tool for lookup and reference in CT
chest/abdomen

• Case Study lung nodules retrieval
• Lung Imaging Database Resource for Imaging
  Research http//imaging.cancer.gov/programsandres
  ources/Inf ormationSystems/LIDC/page7
• 29 cases, 5,756 DICOM images/slices, 1,143 nodule
  images
• 4 radiologists annotated the images using 9
  nodule characteristics calcification, internal
  structure, lobulation, malignancy, margin,
  sphericity, spiculation, subtlety, and texture
• Goals
• Retrieve nodules based on image features
• Texture, Shape, and Size
• Find the correlations between the image features
  and the radiologists annotations

9
LIDC Semantic Concepts
Calcification Popcorn Laminated Solid Non-central Central Absent Sphericity Linear . Ovoid . Round
Internal structure Soft Tissue Fluid Fat Air Spiculation Marked . . . None
Lobulation Marked . . . None Subtlety Extremely Subtle Moderately Subtle Fairly Subtle Moderately Obvious Obvious
Malignancy Highly Unlikely Moderately Unlikely Indeterminate Moderately Suspicious Highly Suspicious Texture Non-Solid . Part Solid/(Mixed) . Solid
Margin Poorly Defined . . . Sharp
10
Extracted Image Features
Shape Features Size Features Intensity Features Texture Features
Circularity Area MinIntensity 11 Haralick features calculated from co-occurrence matrices (Contrast, Correlation, Entropy, Energy, Homogeneity, 3rd Order Moment, Inverse Differential Moment, Variance, Sum Average, Cluster Tendency, Maximum Probability)
Roughness ConvexArea MaxIntensity 11 Haralick features calculated from co-occurrence matrices (Contrast, Correlation, Entropy, Energy, Homogeneity, 3rd Order Moment, Inverse Differential Moment, Variance, Sum Average, Cluster Tendency, Maximum Probability)
Elongation Perimeter MeanIntensity 11 Haralick features calculated from co-occurrence matrices (Contrast, Correlation, Entropy, Energy, Homogeneity, 3rd Order Moment, Inverse Differential Moment, Variance, Sum Average, Cluster Tendency, Maximum Probability)
Compactness ConvexPerimeter SDIntensity 11 Haralick features calculated from co-occurrence matrices (Contrast, Correlation, Entropy, Energy, Homogeneity, 3rd Order Moment, Inverse Differential Moment, Variance, Sum Average, Cluster Tendency, Maximum Probability)
Eccentricity EquivDiameter MinIntensityBG 11 Haralick features calculated from co-occurrence matrices (Contrast, Correlation, Entropy, Energy, Homogeneity, 3rd Order Moment, Inverse Differential Moment, Variance, Sum Average, Cluster Tendency, Maximum Probability)
Solidity MajorAxisLength MaxIntensityBG 11 Haralick features calculated from co-occurrence matrices (Contrast, Correlation, Entropy, Energy, Homogeneity, 3rd Order Moment, Inverse Differential Moment, Variance, Sum Average, Cluster Tendency, Maximum Probability)
Extent MinorAxisLength MeanIntensityBG 24 Gabor features - mean and standard deviation of Gabor filters consistency of four orientations and three scales.
RadialDistanceSD SDIntensityBG 24 Gabor features - mean and standard deviation of Gabor filters consistency of four orientations and three scales.
IntensityDifference 24 Gabor features - mean and standard deviation of Gabor filters consistency of four orientations and three scales.
11
Lung nodule representation
12
(No Transcript)
13
(No Transcript)
14
Retrieved Images
15
CBIR systems challenges REU projects

• Type of features
• image features
• - texture features statistical, structural,
  model and filter-based
• - shape features
• textual features (such as physician annotations)
• Project 1 Feature reduction for medical image
  processing
• Investigate how many features with respect to
  the number of unique nodules
• Investigate what the most important low-level
  image features are with respect to the retrieval
  process
• Investigate the uniformity of the features with
  respect to the same type of nodules.

16
CBIR systems challenges REU projects (cont.)

• Similarity measures
• -point-based and distribution based metrics
• Retrieval performance
• precision and recall
• clinical evaluation

• Project 2 Evaluation of CBIR and SBIR systems
• Perform a literature review on the current
  techniques used to evaluate CBIR systems both for
  the general and medical domain
• Investigate ways to include radiologists
  feedback in the retrieval process
• Investigate ways to evaluate the retrieval
  process by varying various numbers of parameters
  such as number of images retrieved, cutoff value
  for acceptable precision and recall, and minimum
  number of radiologists/observers needed to
  evaluate the system.

17
Correlations between Image Features and Concepts
18
Automatic Mappings Extraction

• Step-wise multiple regression analysis was
  applied to generate prediction models for each
  characteristic ci based on all image features fk

where p is the of used image features, are
the regression coefficients, and are the
prediction errors per model.
Goodness of fit for the regression model
19
Regression Models
Characteristics Entire dataset (1106 images, 73 nodules) At least 2 radiologists agreed At least 3 radiologists agreed
Calcification 0.397 0.578 (884, 41) 0.645 (644, 21)
Internal Structure 0.417 - (855, 40) - (659, 22)
Lobulation 0.282 0.559 (448, 24) 0.877 (137, 6)
Malignancy 0.310 0.641 (489, 23) 0.990 (107, 5)
Margin 0.403 0.376 (519, 28) - (245, 7)
Sphericity 0.239 0.481 (575, 27) 0.682 (207, 9)
Spiculation 0.320 0.563 (621, 29) 0.840 (228, 9)
Subtlety 0.301 0.282 (659, 25) 0.491 (360, 10)
Texture 0.181 0.473 (736, 33) 0.843 (437, 15)
20
Texture Regression Model
21
Malignancy Regression Model
22
Lobulation Regression Model
23
Spiculation Regression Model
24
Image Features Semantics Mappings challenges
REU projects

• Project 3 Multi-view learning classifier for
  lung nodule classification
• Investigate which image features are the best
  for individual semantic characteristics, build
  classifiers for each one of the individual
  classifiers, and combine the individual
  classifies for optimal learning/classification of
  lung nodules
• Project 4 Bridging the semantic gap in lung
  nodule interpretation
• Investigate ways to clinically evaluate the
  mappings from low-level image features to
  semantic characteristics
• Investigate the effect of the imaging
  acquisition parameters (such as pitch, FOV, and
  reconstruction kernel) on the proposed mappings

25
Liver Segmentation in CT images

-
Pixel-level Classification - tissue
segmentation - context-sensitive tools for
radiology reporting

Organ Segmentation
26
Liver Segmentation in CT images
Example of Liver Segmentation (J.D. Furst, R.
Susomboon, and D.S. Raicu, "Single Organ
Segmentation Filters for Multiple Organ
Segmentation", IEEE 2006 International Conference
of the Engineering in Medicine and Biology
Society (EMBS'06))
Region growing at 70
Region growing at 60
Segmentation Result
27
Liver Segmentation using Automatic Snake
a)
d)
b)
c)
a)
Figure a) Gradient vector flow segmentation b)
Traditional vector field segmentation c) and,d)
Respective segmentations overlaid on ground
truth (white).
Project 5 Automatic selection of initial points
for snake-based segmentation
28
uestions ?