Stephen R. Aylward, Director of Medical Imaging, Kitware

1

• Stephen R. Aylward, Director of Medical Imaging,
  Kitware

2
Overview

• Open-source toolkits
• Medical image processing (ITK)
• Image-guided surgery (IGSTK)
• Microscopy (FARSIGHT)
• Image display (VTK)
• Information visualization (VTK)
• Scientific computing (VTK)
• Computer vision (VidTK)
• Open-API
• Databases (MIDAS)

• Open-source applications
• Medical images (VolView)
• Scientific data (ParaView)
• NEW!
• Open-source application building
• ITK, VTK, Qt (Maverick)

3
Overview

• Open-source toolkits
• Medical image processing (ITK)
• Image-guided surgery (IGSTK)
• Microscopy (FARSIGHT)
• Image display (VTK)
• Information visualization (VTK)
• Scientific computing (VTK)
• Computer vision (VidTK)
• Open-API
• Databases (MIDAS)

• Open-source applications
• Medical images (VolView)
• Scientific data (ParaView)
• NEW!
• Open-source application building
• ITK, VTK, Qt (Maverick)

4
IGSTK Features

• PI Kevin Clary, Georgetown (NIH)
• Toolkit for application builders
• Building blocks for C programmers
• Trackers, display layouts, scenes,
• Scene graph based on ITKs SpatialObjects
• Objects are tied to trackers
• Displays show objects in the scenes
• State machine for application control
• All events, not just GUI
• Tracker coordinates, type of data loaded, tracker
  unplugged
• Patient safety is a top priority
• State machine can be visualized and validated
• Confirm application never enters an invalid state
• Some cost added for application developers.

5
FARSIGHT Features

• PI Badri Roysam, RPI (NIH)
• Toolkit for biologists
• Python scripting
• 3D/4D Image analysis and visualization
• Scene graph based on ITKs SpatialObjects
• Arrangement of cells wrt vasculature
• Scene spans time and space
• Flexibility is a top priority
• New methods for processing and display
• Created by biologists (scripts) and computer
  scientists (new ITK/VTK methods)

6
Information Visualization using VTK
(OverView/ParaView)
Figure 3 - The OverView application displaying a
large graph linking miRNA sequences with the
proteins they affect.
Figure 4 - The OverView informatics application
is shown retrieving PubMed articles from a search
string ("HIV"), clustering them, and displaying
the articles in geospatial, graph, landscape, and
tabular views.
7
Maverick

• PI Kitware (AFRL)
• Toolkit for application development
• Qt modules for C programmers
• Qt designer to define applications
• Signals / slots (GUI-driven)
• Scene graph
• Memory (mavScene SpatialObjects / MRML)
• Disk (MRML)
• Display (mavSceneViewers)
• Extensible
• Slicer Execution Model
• GUI (User) is the priority
• Workflow of interdependent modules (maximize
  user-interaction re-use)
• Consistent look-and-feel (GUI design standard)

8
Roadmap (dreams / milestones)

• Maverick is released (weeks)
• FARSIGHT and IGSTK momentum continues
• Slicer converted to Qt and supports multiple
  workflows
• CTK evolves and is integrated into above systems

9
What's most important for open-source?

• NO DUPLICATION!
• - Specialization for an audience
• Community support
• License
• Transparent
• Value added
• User support
• Funding beyond delivery

• Pick ONE audience
• Clinical / biomedical research
• 1. Extensible application
• 2. Custom workflow applications
• Application builder
• Algorithms research
• Scripting
• Pharma
• Integration into existing workflows

10
What's most important for open-source medical
imaging applications?

• Medical imaging
• We are processing organs, tumors, vessels not
  images.
• Portable data on disk
• Crosses applications
• Patient record
• Data representation in memory
• ITK Spatial Objects (scene graph)
• IsInside
• ValueAt
• Transform
• GUI scene
• Present using intuitive names, hierarchy, etc.

• Why dont commercial CAD companies fear
  open-source CAD efforts
• Integration / turn-key
• GUI
• Patents
• Clinical testing
• Outreach
• Cross-institution distribution
• These are out targets

11
Dreams

• Imagine
• Custom application from my institute walks me
  through a workflow for planning a brain tumor
  biopsy procedure
• Save scene
• Load scene into Slicer to merge with new DTI data
• Save scene
• Load scene into surgical suite running IGSTK
  (robotics)
• Biopsy pre-chosen sites
• Send biopsy to lab
• Lab processes using a new research module loaded
  into their FARSIGHT-based application
• Multiple biomarkers (multiple scales MRI, DTI,
  cell, proteomics) are compared with a database of
  cases to predict best course of treatment.
• Radiation oncology program loads scene and biopsy
  results...

12
Reality

• Legacy applications, formats, data, expertise,
• Legacy funding
• Good news different audiences
• Opportunity for sharing
• Solution CTK
• Scene definition (Disk and Memory)
• IO Library for scene
• Scene converters for other types of scenes/data
• API and classes for methods that will run in
  Slicer, MITK, OpenMAF, Maverick, XIP, ...

13
CTK Implementation

• Scenes (Disk, Memory, Display)
• IGSTK / ITK / MITK (SpatialObjects)
• Slicer (MRML)
• XIP (OpenInventor)
• OpenSG
• XNAT
• Modular methods
• Slicer Execution Model
• Implement as ITK/VTK command-line programs
• Automatically come available as shared libraries
• Passes events to parent program
• Missing interaction with parent GUI (mouse
  clicks)
• Can be done, but no standard set
• GUI Modules?