ITK Lecture 4 3D image visualization with myITKgui

1
ITK Lecture 43D image visualization with myITKgui

• Methods in Image Analysis
• CMU Robotics Institute 16-725
• U. Pitt Bioengineering 2630
• Spring Term, 2006

2
Goals for this lecture

• Short introduction to the tools used to visualize
  3D images in ITK
• Discussion of myITKgui
• What it is
• How it works
• How to modify it

3
Why visualization now?

• ITK was intentionally designed without
  visualization tools
• Visualization is very helpful for novice users
• Dont worry about the details (yet) - your job
  for the moment is to write code here

4
The Visualization Toolkit (VTK)

• A very large open source project run by Kitware
• Many visualization tools surface extraction,
  volume rendering, data display, etc.
• Were interested in its ability to integrate with
  ITK to display 3D images

5
Linking ITK to VTK

• Use itkVTKImageExport (an ITK class) and
  vtkImageImport (a VTK class)
• itkVTKExporter itkVTKImageExportTypeNew()
• vtkImporter vtkImageImportNew()
• itkVTKExporter-gtSetInput( m_InputImage )
• ConnectPipelines(itkVTKExporter, vtkImporter)

6
The Pipeline

• Ill be using a term today called the pipeline
• A pipeline is a series of process objects that
  operate on one or more data objects
• The data objects flow along the pipeline

7
Whats ConnectPipelines(...)?

• Both ITK and VTK use pipelines
• You can interconvert ITK and VTK update calls so
  that modifying your image (in ITK) will cause the
  visualization to update (in VTK)
• If youre curious, look at this function once I
  distribute the example code next week

8
Rendering images in VTK

• There are numerous ways to show 3D image data in
  VTK
• My favorite is vtkImagePlaneWidget

9
To summarize process flow
ITK Pipeline
itkVTKImageExport
vtkImageImport
VTK Pipeline
10
The Fast Light Toolkit (FLTK)

• What we have so far is naked image processing
  and rendering code
• We need a GUI wrapped around it so that we can
  interact with the program
• FLTK is an easy-to-use cross platform GUI

11
Designing a GUI in FLTK

• You can write GUIs completely by hand, but its
  easier to use FLTKs tool Fluid
• Fluid lets you design an interface graphically
  and has a native file format, .fl
• It generates two files, a .h and a .cxx
  containing the GUI code

12
Program layout in FLTK

• The order in which classes are derived in an FLTK
  based program may seem non-intuitive
• It arises because the GUI code is generated
  automatically (by Fluid), and because the core
  functionality should be abstracted away from the
  GUI

13
Program layout in FLTK, cont.

• Top MyAppBase - The base class for your
  application, the majority of your code goes here.
• Middle MyAppGUI - Generated by Fluid, contains
  only GUI related code.
• Bottom MyApp - Contains very little code,
  typically only functions concerning both GUI and
  core functionality.

14
How to link FLTK and VTK

• vtkFlRenderWindowInteractor allows you to place a
  VTK render window inside a FLTK based GUI
• Mouse events, key presses, etc. are passed to the
  VTK window

15
Linking FLTK and ITK

• Although ITK does not render anything directly,
  you can use FLTK to monitor the progress of
  pipeline updates
• fltkLightButton changes colors to show modified
  status
• fltkProgressBar moves to show filter progress
  between 0 and 100 completion

16
What will you need to know?

• You should know that all of these pieces exist
  and what they do
• Dont worry about the specifics of implementing
  them just yet
• Youll be provided with base code where your job
  is to fill in the ITK pipeline - you are
  encouraged to experiment

17
Software

• FLTK http//www.fltk.org
• VTK http//www.vtk.org

18
What to do?

• Build FLTK
• FLTK contains Visual Studio project files for
  Windows and makefiles for most everyone else
• -OR- use Cmake!
• Follow the instructions and you shouldnt have
  any problems
• There is a /test directory containing a bunch of
  demo code if youre curious

19
What to do, cont.

• Build VTK
• The VTK build process is very similar to ITK,
  since both use CMake
• You can turn off BUILD_EXAMPLES and BUILD_TESTING
• Make sure VTK_USE_RENDERING is ON

20
What to do, cont.

• Bear in mind that building VTK is a time
  consuming process have a cup of coffee or four
• Fortunately, building FLTK is quite fast
• If you run into problems building VTK, please
  subscribe to the VTK users mailing list and ask
  for help

21
myITKgui

• myITKgui is the basecode that you will use for
  the remainder of the class
• It provides an application framework that allows
  you to rapidly add new functionality without
  getting wrapped up in GUI related issues

22
Classes

• migApp
• migAppBase - contains the pipeline
• migAppGUI - created by Fluid
• migApp - virtual implementation class
• migWindow
• migWindowBase - renders a 3D image
• migWindowGUI - created by Fluid
• migWindow - doesnt do much

23
Execution walkthrough

• Application starts
• The code in main() looks like
• int main(int, char)
• migApp theApp
• theApp.Show()
• Flrun()
• return 0

Create a new app
Make the app visible
Start the FLTK event loop
24
App creation

• The app initializes two render windows, one for
  the before pipeline image and one for the
  after pipeline image
• m_BeforeWindow new migWindow
• m_BeforeWindow-gtrenderWindow-gt label("myITKgui -
  'Before' Image")
• m_BeforeWindow-gtShow()

25
Window creation

• Window creation isnt too interesting
• Member variables are initialized and a renderer /
  renderWindow pair is created

26
Some terminology

• Event loop - a program executes in a timed loop
  and checks each cycle to see if anything
  interesting has happened
• Callback - a function (generically) that is
  called when an event occurs

27
After initialization

• Now we have
• One migApp
• Two migWindows
• Each migWindow has a VTK renderWindow and
  renderer tucked inside it
• No image is present
• FLTK sits in its event loop waiting for something
  to happen

28
The user pushes load image

• The callback for the load image button is set to
  migAppBaseReadImage - this is possible since
  the GUI derives from the Base
• You can see where the callback is set by editing
  migAppGUI.fl in Fluid

29
migAppBaseReadImage()

• Get a .mha filename from a dialog box
• Load this file
• Set the input image windows image to what we
  just loaded and create image plane widget objects
• Construct the ITK pipeline (what happens here
  depends on what youre doing)

30
Now what?

• We have everything we did before, plus
• An image in memory
• An ITK pipeline
• The input image is displayed using 3 image planes
• The output image does not exist yet
• FLTK goes back into waiting mode

31
User pushes Update Pipeline

• The callback for the update button is set to
  migAppBaseUpdatePipelineCallback
• Exactly what happens here is app dependent, but
  the intent is to call Update() on the last object
  (filter) in the pipeline
• Finally, the output image window has its image
  set, and image planes are created

32
Thats it

• Thats the limit of myITKguis functionality at
  this point
• Some easy extensions
• Add additional FLTK controls and use their
  callbacks to specify pipeline parameters in
  migAppBase
• Add additional VTK objects to the output window
  (points/lines/etc.)

33
Insert clever demo here
34
Error handling

• There really isnt any - doh!
• Dont read more than one image
• Dont push the update button more than once
• We handle these cases in our research code its
  probably useful for you to solve them on your own
  (its not hard, esp. case 2)

35
Stick a fork in it...

• If youre using a Mac, you need to attach a FLTK
  resource fork to the app prior to trying to run
  it (do this once per compile, not each time you
  run it)
• /Developer/Tools/Rez -t APPL -o ./myITKgui
  /Users/beowulf/fltk-1.1.3/FL/mac.r

This path is probably different on your machine
36
Extending myITKgui

• Replace CreateITKPipeline() with code that does
  something more interesting than binary
  thresholding
• Edit UpdatePipeline() so that your pipeline is
  updated correctly

37
By the end of next week

• Build ITK, VTK, and FLTK
• Build at least one of the examples under
  /Insight/Examples (HelloWorld)
• Build myITKgui and verify that it works correctly
  on your machine

should be done already
38
Optionally

• Change the filter in myITKgui to something else
  of your choice
• Start playing with FLTK to learn how to add
  buttons, change GUI behavior, etc.