Slicer 3 Update for Core 1

1
Slicer 3 Update for Core 1

• NA-MIC Algorithms Meeting
• UNC May 23, 2006

2
Why Slicer3?

• Address Longstanding Legacy Issues
• Cores 1, 3 Simplify coordinate systems used in
  processing, interaction, and display
• Core 3 Increase UI Flexibility, Improve Look
  Feel
• Core 2 Modularize the Base for Reusability
• Simplify Module Writing
• Core 1 Define interface to data model
• Core 1 Allow variety in programming styles
• Core 1, 3 Allow Modules to be usable with
  scripts, grid, pipeline

3
Slicer3 Opportunities

• Core 1
• Provide Platform for Next Generation Algorithms
• Core 2
• Leverage NA-MIC Kit and Engineering Methodology
• Core 3
• Work with DBP Users on Functionality and Usability

4
Slicer3 Constraints

• Limited Resources for Implementation
• Preserve Investment by Users and Developers in
  Existing Slicer Modules
• Adhere to NA-MIC Licensing Goals

5
Slicer3 Collaborating Resources

• NA-MIC Core 2
• NAC (P41 at BWH)
• Neuroimaging Analysis Center
• BIRN
• Biomedical Informatics Research Network (function
  and morphometry)
• NCIGT (U41 at BWH)
• National Center for Image Guided Therapy
• Several other collaborating projects contributing
  to Base and Modules

6
Slicer3 Plan

• Re-implement Base Layer
• Slice/Volume/Model Viewing
• Interaction, Events, Usability
• Module Interface
• Data Model
• Port Slicer2 Modules
• EMSegmenter, fMRIEngine/Ibrowser, DTMRI, ITK
  Registration
• Integrate New Modules

7
Slicer3 Timeline

• July 2005
• Kickoff, Concept/Planning at Programmer Week
• January 2006
• Beginning of New Repository
• March 2006
• Alpha Prototype of New Architecture
• July 2006
• Alpha2/Beta1 testing with Core 3
• November 2006
• Beta2
• January 2007
• 3.0 Release to NA-MIC at AHM

8
Slicer3 Software Environment

• CMake / CTest / CPack (Version 2.4)
• Dart (Version 2)
• VTK (5)
• ITK (2.6)
• KWWidgets (CVS Head)
• Teem (1.9)

9
Slicer3 Engineering Tools

• Dart (version 2)
• CMake (version 2.4)
• CTest
• CPack
• subversion
• phpBugTracker
• MediaWiki
• Your Favorite Editor and Compiler

10
Slicer 3.0 Architecure
User Desktop
Algorithms
ITK
VTK
Slicer Modules
Slicer 3.0
Scripts of Slicer Mods
VTK Apps Using ITK
Batch Programs
Non-NAMIC Cmd tools
LONI Pipeline
Birn Grid Data/Compute
11
Slicer3 Base Programming

• MRML, Logic, and KWWidgets all C to Maximize
  Reusability (callable from C)
• Base classes are all wrapped and accessible from
  Tcl at runtime
• MRML is a Library that can be used Independent of
  Slicer3
• Uses VTK Object Model (Reference Counting,
  Observer Pattern, Wrapping)
• Relies on ITK ImageIO and VTK PolyData IO
• GUI KWWidgets are VTK-Like C wrapping of Tcl/Tk
• KWWidgets are wrapped and callable from C, Tcl,
  Python
• KWWidgets provide Application level support (Main
  Window, Menus, Registry)
• VTK-specific Widgets (RenderWidget, Transfer
  Function Editor, Material Properties Editor)
• GUI separated from MRML and Logic to support
  possible alternate GUIs

12
Slicer3 Module Programming

• Many Options to Implement Modules
• Pure C Libraries
• vtkITK Library
• Pure Tcl code
• C/Tcl packages (Slicer2 Modules)
• External Executables (more later on this topic)
• Loosely Coupled Modules Built on Other Platforms
  (teem, FreeSurfer, MATLAB, etc)

13
Slicer3 Observer MVC Pattern

• MRML (Model)
• For Scene Description and Application State
• MRML Nodes are Persistent and Undoable
• Scene and Nodes are Observable
• Logic Encapsulate VTK and ITK Pipelines
  (Controller)
• Observe MRML to Configure Pipelines
• No UI Components (no Widgets, Actors, Mappers,
  Renderers or RenderWindows)
• GUI (View)
• Observes and Edits MRML
• Interacts with User and Displays Logic Outputs

14
Slicer 3 Architecture Diagram (2006-01-26)
Base
Execution Model Manager
(Managed by Base and communicats like other
Modules)
Auto-Generated GUIs from XML Descriptions
Logic Links to ITK MRML VTK (except Rendering)
GUI Links to VTK Rendering KWWidgets
Logic Links to Base Logic
GUI Links to KWWidgets
Mediators
Mediators
Observe Modified Events
MRML

• Provides
• Frames for Widgets,
• Routing of User Events

Observe Modified Events on Application State
Edit Mrml Scene/Nodes
Formulates Command Line and possibly stages data

• Registers GUI Event Observers

Interactive Module
Describe Input options
Observe Modified Events
Logic Links to Base Logic ITK MRML VTK (except
Rendering)
GUI Links to VTK Rendering KWWidgets
Mediators
Edit Mrml Scene/Nodes

• External
• Processes (Grid/Pipeline)
• Speak MRML
• (link to libmrml
• or have data staged by slicer)
• Uses the XML interface

MRML Aware Processes Can Edit Mrml
Scene/Nodes and Observe Modified Events

• Notes
• All classes in the Logic directory should be able
  to run headless without OpenGL or window system
  for scripting and testing
• Base/Logic contains transient application state
  (cursor location, focus, mrml scene connection)
• Interactive Modules are ones which interact with
  the VTK scene and/or User events
• Interactive Modules interact with 3D scene by
  creating objects in MRML scene (not by direct
  manipulation of the Renderer)
• Logic classes encapsulate and manage internal
  vtk/itk pipelines
• GUI classes are implemented as KWWidget
  subclasses
• Each Logic class defines a set of Get/Set methods
  for internal state and Modified Events that GUI
  classes can Observe

15
Main program Bootstrap application logic,
Create application gui
data layer
data mediator
logic layer
gui mediator
gui layer
(VTK, ITK, no KWWidgets, no VTKRendering) vtkObjec
t vtkSlicerLogic - Undo manager, -
tracing vtkSlicerApplicationLogic - MRML
slice - transient state - open
views, slices, modules
(and active) - cursor location -
version - focus, event routing vtkSlicerVie
wLogic vtkSlicerLinkedSliceLogic vtkSliceLogic
- vtkImageReslice - vtkImageComposite vtk
SlicerSliceLayerLogic - MRML node -
Opacity - Compositing -
Window/level vtkSlicerViewLogic vtkSlicerModuleLog
ic - Undo/tracing helpers vtkSlicerEditorLo
gic vtkSlicerMRMLLogic vtkSlicerCustomModuleLogic

vtkKWWidget vtkSlicerGUI - look feel?
- set command helpers? vtkSlicerApplicationGUI
- connect to window system -
vtkKWApplication - vtkKWWindow vtkSlicerView
GUI - Render windows - mappers,
actors vtkSlicerSliceGUI - vtkImageMapper
- vtkKWRenderWindow vtkSlicerSliceLogicG
UI vtkSlicerModuleGUI vtkSlicerCustomModu
leGUI
GUI code adds observers to modified events
generated by/to logic
Logic code adds modified event observers to MRML
scene and MRML nodes.
MRML data model
Apply methods from GUI classes call set method in
logic to change state
Logic methods apply changes to the MRML scene.
VTK pipeline
VTK pipeline
16
Slicer3 New Features

• ITK IO Framework (Archetype Based)
• Multiple Undo/Redo
• Modules as Executables
• For Grid and Pipeline Use
• Object Oriented Module Framework
• New UI Framework
• Extensive Testing Framework
• Much More

17
Core 3 Feature Requests

• Image Editor!!!
• Efficient Manual ROI Creation
• Unified Save
• Summary view of the state of the scene
• Checkbox the items you want to save
• Photoshop Layers Style Volume Viewing
• Module Dock of Frequently Used Modules
• Core 1 ideas?

18
Slicer3 Projects

• Architecture/Features (Steve Pieper, Group Bill
  Lorensen, Ron Kikinis, Mike Halle, Noby Hata)
• Data Model (Steve Pieper, Alex Yarmarkovich,
  Xiaodong Tao, Mathieu Malaterre)
• Execution Model (Dan Blezek, Bill Lorensen, Mike
  Halle, Jim Miller)
• Visualization and UI Infrastructure (Mathieu
  Malaterre with Sebastien Barre, Wendy Plesniak,
  Mike Halle)
• Interface Design and Usability (Wendy Plesniak,
  Mike Halle, Sebastien Barre)
• Transition of Selected Slicer2.x Modules to
  Slicer3 (Nicole Aucoin, Katie Hayes, Wendy
  Plesniak, Mathieu Malaterre)
• Build/Test/Deploy System (Andy Cedilnik)
• Pipeline Integration (Mike Pan, Jags Rajendiran)
• Grid Integration (Jeff Grethe, Brendan Faherty)

19
Transition of Slicer 2.x Modules Goals

• Slicer 2
• Port C classes to compile against VTK 5.x
• Maintain backward compatibility with VTK 4.x
• Edit TCL interface to properly call new class
  versions
• Write test cases to define gold standard
• Slicer 3
• Port threads of control from TCL to C
• Implement dynamic discovery and loading of modules

20
Transition of Slicer 2.x Modules Progress

• Slicer 2
• Nightly builds against VTK5
• TCL revisions underway
• Many tests written and run nightly

21
Slicer 3 Execution Model Goals

• Provide interface to batch programs
• Simplify and unify command line processing
• Auto generate C command line processing
• Simplify and unify GUI
• Auto generate GUI

22
Execution Model Progress

• XML Interface Description
• Describes command line options
• Describes suggestions for GUI
• GenerateCLP
• Generates C command line processing code from
  XML
• Reports XML description

23
Program Flow
User Desktop
Slicer Modules
Slicer 3.0
prog.xml
GenerateCLP
progCLP.h
prog.cxx
Non-NAMIC Cmd tools
LONI Pipeline
Birn Grid Data/Compute
24
Resample --xml
ltlabelgtIOlt/labelgt ltdescriptiongtInput/output
parameterslt/descriptiongt ltimagegt
ltnamegtInputVolumelt/namegt ltlabelgtinput
Volumelt/labelgt ltchannelgtinputlt/channelgt
ltindexgt0lt/indexgt ltdescriptiongtInput
volume to be resampledlt/descriptiongt
lt/imagegt ltimagegt ltnamegtOutputVolumelt/nam
egt ltlabelgtOutput Volumelt/labelgt
ltchannelgtoutputlt/channelgt ltindexgt1lt/indexgt
ltdescriptiongtResampled Volumelt/descriptiongt
lt/imagegt lt/parametersgt lt/executablegt

• lt?xml version"1.0" encoding"utf-8"?gt
• ltexecutablegt
• ltcategorygtfilteringlt/categorygt
• lttitlegtResample Volumelt/titlegt
• ltdescriptiongtResamples a volumelt/descriptiongt
• ltversiongt1.0lt/versiongt
• ltdocumentationurlgtlt/documentationurlgt
• ltlicensegtlt/licensegt
• ltcontributorgtBill Lorensenlt/contributorgt
• ltparametersgt
• ltlabelgtResampling Parameterslt/labelgt
• ltdescriptiongtParameters used for
  resamplinglt/descriptiongt
• ltfloat-vectorgt
• ltnamegtoutputPixelSpacinglt/namegt
• ltflaggt-slt/flaggt
• ltlongflaggt--spacinglt/longflaggt
• ltdescriptiongtSpacing along each dimension
  (0 means use input spacing)lt/descriptiongt
• ltlabelgtSpacinglt/labelgt
• ltdefaultgt0,0,0lt/defaultgt

25
Resample.cxx

• .
• .
• .
• include ResampleCLP.h
• Main (int argc, char argv)
• PARSE_ARGS
• algorithm code

26
Resample --help

• USAGE
• ResampleVolume
• --xml
  --echo -s ltstdvectorltfloatgtgt
• --
• 
  --version -h ltstdstringgt
• 
  ltstdstringgt
• Where
• --xml
• Produce xml description of command line
  arguments (default 0)
• --echo
• Echo the command line arguments (default 0)
• -s ltstdvectorltfloatgtgt, --spacing
  ltstdvectorltfloatgtgt
• Spacing along each dimension (0 means use
  input spacing) (default 0,0,0)
• --, --ignore_rest
• Ignores the rest of the labeled arguments
  following this flag.
• --version
• Displays version information and exits.
• -h, --help
• Displays usage information and exits.

27
Slicer Daemon

• Slicerd listens on local socket
• slicerget ltidgt
• gets volume and writes to stdout in nrrd
• slicerput name
• reads nrrd on stdin and puts in slicer

slicerget 0 unu 1op exp - slicerput
28
ITK IO and Slicer Daemon
AntiAliasBinaryImageFilter.exe bunny.z.nrrd " sh
slicerput.tcl"
29
Undo Architecture for Slicer3

• Versioning Commit Aside Strategy Encapsulated
  within MRML
• Store Initial Scene (from file) and Delta
  Scenes (scenes containing undoable changes)
• Delta Scenes are MRML Scenes
• Some Nodes are Reference Nodes
• Setting the MRML scene in the Application Logic
  causes the cascade of observer callbacks
• gt All Undoable operations must store their data
  as MRML nodes

30
Undo 1st Pass
Application Logic
Undo Stack
Initial Scene
Delta1 Scene
Delta2 Scene
Saved Scene
N1
R
R
N1
N2
N4
R
N4
N3
R
N5
N5
31
Undo Current
MRML Scene
Undo Stack
Initial Scene
Delta1 Scene
Delta2 Scene
Saved Scene
N1
N1
N2
N4
N4
N3
N5
N5
32
Slicer3 Slice Coordinates
RASToRAS Transforms
SliceToRAS Matrix
XYToSlice Matrix
RASToIJK Matrix
RAS to RAS Coordinates
Slice Coordinates
XY Image Space Coordinates In RenderWindow
IJK XYToIJK XY
IJK RASToIJK RASToRAS SliceToRAS
XYToSlice XY
IJK Volume Coordinates
33
Slice Management
Volume Display Node
Volume Node
Foreground Slice Layer Logic
Volume Display Node
Volume Node
Background Slice Layer Logic
Slice Node
Slice GUI
Slice Logic
Slice Composite Node
34
Slice Management
Volume Display Node
Stores Window/Level Lookup Table Transparency
Threshold
Volume Node
Foreground Slice Layer Logic
Volume Display Node
Volume Node
Background Slice Layer Logic
Slice Node
Slice GUI
Slice Logic
Slice Composite Node
35
Slice Management
Volume Display Node
Stores RAS Location of Slice (Origin and
Orientation) Field of View Pixel Dimensions
Volume Node
Foreground Slice Layer Logic
Volume Display Node
Volume Node
Background Slice Layer Logic
Slice Node
Slice GUI
Slice Logic
Slice Composite Node
36
Slice Management
Volume Display Node
Stores Volume IDs per Layer Opacity of Overlay
Volume Node
Foreground Slice Layer Logic
Volume Display Node
Volume Node
Background Slice Layer Logic
Slice Node
Slice GUI
Slice Logic
Slice Composite Node
37
Slice Management
Volume Display Node
Stores Image Data IJKToRAS Matrix Name
Volume Node
Foreground Slice Layer Logic
Volume Display Node
Volume Node
Background Slice Layer Logic
Slice Node
Slice GUI
Slice Logic
Slice Composite Node
38
Slice Management
Volume Display Node
Volume Node
Foreground Slice Layer Logic
Volume Display Node
Volume Node
Background Slice Layer Logic
Slice Node
Slice GUI
Slice Logic
Slice Composite Node
39
Slice Management
Volume Display Node
Volume Node
Pipeline with vtkImageBlend (Annotations,
Crosshairs)
Foreground Slice Layer Logic
Volume Display Node
Volume Node
Background Slice Layer Logic
Slice Node
Slice GUI
Slice Logic
Slice Composite Node
40
Slice Management
Volume Display Node
Volume Node
GUI SliceControllerWidget (Menus, Scale,
Buttons) SliceViewer (RenderWidget, ImageMaper,
ImageActor)
Foreground Slice Layer Logic
Volume Display Node
Volume Node
Background Slice Layer Logic
Slice Node
Slice GUI
Slice Logic
Slice Composite Node
41
Core 3 Feature Requests

• Image Editor!!!
• Efficient Manual ROI Creation
• Unified Save
• Summary view of the state of the scene
• Checkbox the items you want to save
• Photoshop Layers Style Volume Viewing
• Module Dock of Frequently Used Modules
• Core 1 ideas?

42
Unified Save Template

• Would show currently loaded data
• Show save state of each element
• User option to save each element

Save Page from NA-MIC / Kitware Wiki To Web
Project
43

• Slicer3 user interface and usability
• http//www.na-mic.org/Wiki/index.php/Slicer3Inter
  face_Design
• Task Design engineer Slicer3 GUI. Employ
  user-centered approach where possible, and
  establish persistent usability guidelines for
  developers.
• 3 goals focused on users and developers
• Enable a user to understand and effectively use
  the content and tools being presented
• Enable a user to accomplish a principle task by
  following an appropriate and satisfying workflow
  or a curiosity-driven exploration, at an
  appropriate pace.
• Enhance and support the developers experience
  with GUI infrastructure and guidelines that
  simplify and clarify their work.

44

• Engineering and design requirements
• Separate GUI from application layer
• Preserve application testability
• Use consistent event handling methods and API to
  application logic and MRML
• Provide mechanism for undo
• Centrally specify application look feel (
  permitting overrides)
• Rethink GUI layout keep good elements, brand
  elements, make improvements
• Develop Slicer3 brand logo, visual icons, splash
  screen, style sheets for www
• Develop dialog with key user communities
• Test designs of key modules with members of the
  user community and refine
• Monitor mechanisms for user feedback, bug
  reporting and feature requests
• Publish design guidelines for all new
  development.

engineering
design
usability
45

• Engineering and design requirements
• Separate GUI from application layer
• Preserve application testability
• Use consistent event handling methods and API to
  application logic and MRML
• Provide mechanism for undo
• Centrally specify application look feel (
  permitting overrides)
• Rethink GUI layout keep good elements, brand
  elements, make improvements
• Develop Slicer3 brand logo, visual icons, splash
  screen, style sheets for www
• Develop dialog with key user communities
• Test designs of key modules with members of the
  user community and refine
• Monitor mechanisms for user feedback, bug
  reporting and feature requests
• Publish design guidelines for all new
  development.

engineering
46

• GUI layer architecture and interface to
  application layer
• Base classes defined and implemented.
• Main application GUI development underway.
• Module GUI development underway.

47

• Engineering and design requirements
• Separate GUI from application layer
• Preserve application testability
• Use consistent event handling methods and API to
  application logic and MRML
• Provide mechanism for undo
• Centrally specify application look feel (
  permitting overrides)
• Rethink GUI layout keep good elements, brand
  elements, make improvements
• Develop Slicer3 brand logo, visual icons, splash
  screen, style sheets for www
• Develop dialog with key user communities
• Test designs of key modules with members of the
  user community and refine
• Monitor mechanisms for user feedback, bug
  reporting and feature requests
• Publish design guidelines for all new
  development.

design
48

• GUI design sketches
• Keep default view
• Keep slicer-blue 3D view
• Simplify and clarify module navigation
• Group slice window controls,
• Group 3D view controls.

49
(No Transcript)
50

• spec mockup
• identify and arrange common-use top-level
  functionality and anticipate Tk-look
• prescribe widget to display 3D nav by default,
  and slice-zoom during slice-window mouseover.

51

• current status
• added toolbar and re-designing module navigation
• populating GUI panel with KWWidgets and custom
  widgets
• adjusting layout to clarify functional groupings
  of widgets
• adding scrollbar to module GUI panel.
• designing 3D navigation / slice zoom widget in
  view control panel.

52
Toolbar icon design, (ongoing) design studies for
slicer3 logo
53

• Engineering and design requirements
• Separate GUI from application layer
• Preserve application testability
• Use consistent event handling methods and API to
  application logic and MRML
• Provide mechanism for undo
• Centrally specify application look feel (
  permitting overrides)
• Rethink GUI layout keep good elements, brand
  elements, make improvements
• Develop Slicer3 brand logo, visual icons, splash
  screen, style sheets for www
• Develop dialog with key user communities
• Test designs of key modules with members of the
  user community and refine
• Monitor mechanisms for user feedback, bug
  reporting and feature requests
• Publish design guidelines for all new
  development.

usability
54

• ongoing dialog with local user communities
• for now, keeping wiki page repository of
  feature and resource requests (which also
  incorporates appropriate slicer2 feature
  requests)
• http//www.na-mic.org/Wiki/index.php/Slicer3
  FeatureRequests
• will schedule early user testing sessions after
  beta