Medical Imaging Projects

1
Medical Imaging Projects

• Daniela S. Raicu, PhD
• Assistant Professor
• Email draicu_at_cs.depaul.edu
• Lab URL http//facweb.cs.depaul.edu/research/vc/

2
IMP MediX Labs _at_ DePaul

• Faculty
• GM. Besana, L. Dettori, J. Furst, G. Gordon, S.
  Jost, D. Raicu, N. Tomuro
• CTI Students
• W. Horsthemke, C. Philips, R. Susomboon, J.
  Zhang E. Varutbangkul, S.G. Valencia

• IMP Collaborators Funding Agencies
• National Science Foundation (NSF) - Research
  Experience for Undergraduates (REU)
• Northwestern University - Department of
  Radiology, Imaging Informatics Section
• University of Chicago Medical Physics
  Department
• Argonne National Laboratory - Biochip Technology
  Center
• MacArthur Foundation

3
Outline

• Medical Imaging and Computed Tomography
• Soft Tissue Segmentation in Computed Tomography
• Project 1 Region-based classification
• Project 2 Texture-based snake approach
• Content-based Image Retrieval and Annotation
• Project 3 Lung Nodule Retrieval based on image
  content and radiologists feedback
• Project 4 Associations discovery between image
  content and radiologists assessment

4
What is Medical Imaging (MI)?


The study of medical imaging is concerned with
the interaction of all forms of radiation with
tissue and the development of appropriate
technology to extract clinically useful
information from observation of this technology.

5
Computed Tomography (CT)


G. Hounsfield (computer expert) and A.M.
Cormack (physicist) (Nobel Prize in Medicine in
1979) CT overcomes limitations of plain
radiography CT doesnt superimpose structures
(like X-ray) CT is an imaging based on a
mathematical formalism that states that if an
object is viewed from a number of different
angles than a cross-sectional image of it can be
computed (reconstructed)


_______________________________________________
6
CT Data

• Stages of construction of a voxel dataset from
  CT data
• CT data capture works by taking many one
  dimensional
• projections through a slice (scanning)
• (b) CT reconstruction pipeline

7
CT Data Acquisition


Slice-by-slice acquisition X-ray tube is
rotating around patient to acquire a slice
patient is moved to acquire the next
slice Volume acquisition X-ray tube is moving
continuously along a spiral (helical) path and
the data is acquired continuously


_______________________________________________
8
CT Data Acquisition

• slice-by-slice scanning
• (b) Spiral (volume) scanning

9
CT SPIRAL SCANNING


a patient is moved 10mm/s (24cm / single
scan) slice thickness 1mm-1cm faster than
slice-by-slice CT no shifting of anatomical
structures slice can be reconstructed with an
arbitrary orientation with (a single breath)
volume

• CT multi-slice systems
• parallel system of detectors
• 4/8/16 slices at a time
• generates a large data of thin slices
• better spatial resolution (? better
  reconstruction)

10
CT - DATA PROCESSING
CT numbers (Hounsfield units) HU computed via
reconstruction algorithm (tissue density/
X-ray absorption) most attenuation (bone)
least attenuation (air) blood/calcium increases
tissue density


Understanding Visual Information Technical,
Cognitive and Social Factors
11
CT - DATA PROCESSING


Relationship between CT numbers and
brightness level
Understanding Visual Information Technical,
Cognitive and Social Factors
12
CT - IMAGE DISPLAY


Human eye can perceive only a limited range
gray-scale values

Thoracic image a) width 400HU/level 40HU (no
lung detail is seen) b) width 1000HU/level
700HU (lung detail is well seen bone and soft
tissue detail is lost)

13
CT Medical Imaging (MI)_at_ CTI
Analysis
Visualization
Classification
Retrieval
14
Outline

• Medical Imaging and Computed Tomography
• Soft Tissue Segmentation in Computed Tomography
• Project 1 Region-based classification approach
• Project 2 Texture-based snake approach
• Content-based Image Retrieval and Annotation
• Project 3 Lung Nodule Retrieval based on image
  content and radiologists feedback
• Project 4 Associations discovery between image
  content and radiologists assessment

15
Soft-tissue Segmentation in Computed Tomography

Goal context-sensitive tools for radiology
reporting Approach pixel-based texture
classification
Organ Segmentation
16
Soft-tissue Segmentation in Computed Tomography
Pixel-based texture extraction

• Challenges
• Storage
• Input 0.5 terabyte of raw data dispersed over
  about 100K images
• Output 90 terabytes of low-level features in a
  180 dimensional feature space
• Compute
• 24 hours of compute time 180 features for a
  single image on a modern 3GHz workstation

17
Project 1 Challenges and opportunities

• Calculate image features at region-level instead
  of pixel-level
• Include Gabor features in the feature extraction
  phase in addition to the co-occurrence texture
  features
• Explore different approaches for region
  classification in addition to the decision tree
  approach
• Current Implementation Matlab

18
Liver Segmentation Example
J.D. Furst, R. Susomboon, and D.S. Raicu, "Single
Organ Segmentation Filters for Multiple Organ
Segmentation", IEEE 2006 International Conference
of the Engineering in Medicine and Biology
Society (EMBS'06)
Region growing at 70
Region growing at 60
Segmentation Result
19
Snake Application Demo
Soft-tissue Segmentation in Computed Tomography
Next figures are demonstrated how to
automatically classify the CT images of heart and
liver.
20
Demo For HEART
There are 4 main menu to operate this application.
SEGMENT To automatically segment the region of
interest organ
OPEN To open a new Image.
TEXTURE To calculate the texture models
co-occurrence/run-length
CLASSIFICATION To automatically classify the
segmented organ
21
HEART Segmentation
The application allows users to
change Snake/ Active contour algorithm parameters
22
HEART Segmentation (cont.)
Button is clicked
User selects points around the region of interest
23
HEART Segmentation (result)
Show segmented organ
If the user likes the result of the
segmentation, then the user will go to the
classification step
24
HEART Classification
Selection of texture models Co-occurrence, Run-le
ngth, Or Combine both models
Texture features corresponding to the selected
texture model are calculated and shown here
25
HEART Classification Result
Results are shown as follows. Predicted organ
Heart Probability0.86 And also rule which is
used to predict that this segmented organ is HEART
26
Demo For LIVER
Start application by open and load the image.
27
LIVER Segmentation
The application allows users to
change Snake/ Active contour algorithm parameters
28
LIVER Segmentation (cont.)
Segmentation Button is clicked
User selects points around the region of interest
29
LIVER Segmentation Result
Show segmented organ
If user is satisfied with the result, then it
will go to the classification step
30
LIVER Classification
Select texture models Co-occurrence, Run-length,
Or Combine both models
Texture features is calculated for the selected
model
31
LIVER Classification Result
Results are shown as follows. Predicted organ
Liver Probability1.00 And also rule which is
used to predict that this segmented organ is LIVER
32
Project 2 Challenges and opportunities

• Calculate texture image features at the pixel
  level instead of using the gray-levels
• Apply snake on the texture features
• Investigate different ways to objectively
  compare two segmentation algorithms, in
  particular the snake and the classification-based
  approach
• Current Implementation Matlab

33
Outline

• Medical Imaging and Computed Tomography
• Soft Tissue Segmentation in Computed Tomography
• Project 1 Region-based classification approach
• Project 2 Texture-based snake approach
• Content-based Image Retrieval and Annotation
• Project 3 Lung Nodule Retrieval based on image
  content and radiologists feedback
• Project 4 Associations discovery between image
  content and radiologists assessment

34
Outline

• Medical Imaging and Computed Tomography
• Soft Tissue Segmentation in Computed Tomography
• Project 1 Region-based classification approach
• Project 2 Texture-based snake approach
• Content-based Image Retrieval and Annotation
• Project 3 Lung Nodule Retrieval based on image
  content and radiologists feedback
• Project 4 Associations discovery between image
  content and radiologists assessment

35
Content-based medical image retrieval (CBMS)
systems

-
Definition of Content-based Image
Retrieval Content-based image retrieval is a
technique for retrieving images on the basis of
automatically derived image features such as
texture and shape.

• Applications of Content-based Image Retrieval
• Teaching
• Case-base reasoning
• Evidence-based medicine

36
Diagram of a CBIR
37
CBIR as a tool for lookup and reference

• Case Study lung nodules retrieval
• Lung Imaging Database Resource for Imaging
  Research http//imaging.cancer.gov/programsandres
  ources/Inf ormationSystems/LIDC/page7
• 29 cases, 5,756 DICOM images/slices, 1,143 nodule
  images
• 4 radiologists annotated the images using 9
  nodule characteristics calcification, internal
  structure, lobulation, malignancy, margin,
  sphericity, spiculation, subtlety, and texture
• Goals
• Retrieve nodules based on image features
• Texture, Shape, and Size
• Find the correlations between the image features
  and the radiologists annotations

38
Examples of nodule images
39
CBIR as a tool for lung nodule lookup and
reference
Low-level feature extraction
40
Nodule Characteristics

• Calcification
• (1. Popcorn, 2. Laminated, 3. Solid,
• 4. Non-Central, 5. Central, 6. Absent)
• Internal Structure
• (1. soft tissue, 2. fluid, 3. fat, 4. air)
• Subtlety
• (1. extremely subtle,..................., 5.
  obvious)
• Sphericity
• (1. Linear, 2. ......, 3. Ovoid, 4. ....., 5.
  Round)
• Texture
• (1. Non-Solid, 2. ....., 3. Part Solid, 4.
  ......., 5. Solid)

41
Nodule Characteristics

• Margin
• (1. Poorly, ......................., 5. Sharp)
• Lobulation
• (1. Marked, ....................., 5. No
  Lobulation)
• Spiculation
• (1. Marked, ....................., 5. No
  Spiculation)
• Malignancy
• (1. Highly Unlikely for Cancer, ...............,
• 5. Highly Suspicious for Cancer)

42
(No Transcript)
43
M. Lam, T. Disney, M. Pham, D. Raicu, J. Furst,
Content-Based Image Retrieval for Pulmonary
Computed Tomography Nodule Images, SPIE Medical
Imaging Conference, San Diego, CA, February 2007
44
Retrieved Images
45
Project 3 Challenges and opportunities

• Calculate co-occurrence texture features at the
  local level instead of global level
• Incorporate shape and size features in the
  retrieval process in addition to texture features
• Integrate radiologists assessments/feedback
  into the retrieval process
• Investigate different approaches for retrieval
  in addition to similarity measures
• Report the retrieval results with a certain
  confidence level (probability) instead of just a
  binary output (similar/not similar)
• Current implementation C
• Available Open Source at http//brisc.sourceforge
  .net/

46
Outline

• Medical Imaging and Computed Tomography
• Soft Tissue Segmentation in Computed Tomography
• Project 1 Region-based classification approach
• Project 2 Texture-based snake approach
• Content-based Image Retrieval and Annotation
• Project 3 Lung Nodule Retrieval based on image
  content and radiologists feedback
• Project 4 Associations discovery between image
  content and radiologists assessment

47
Associations between image content and semantics
48
Project 4 Challenges and opportunities

• Investigate other approaches for finding the
  associations between image features and
  radiologists assessment in addition to logistic
  regression and decision trees
• from image content to semantics
• from semantics to semantics
• from image features and semantics to semantics
• Create GUIs to display examples of images for
  each semantic concept
• Investigate how the current associations
  discovery approaches apply to mammography
  assessment (Northwestern project)
• Current implementation Matlab, Weka, SPSS

49
Questions? Thank you!