Visualization Architectures

1
VisualizationArchitectures
VisualizationArchitectures

• Alexandru C. Telea

Alexandru C. Telea
2
Visualization Users

• Problem
• we need visualization tools
• user requirements are very diverse

• data types dimensionality, attributes

• time real-time, interactive, batch applications

• product type research, industry, education

• user interfaces graphic, command-line

many visualization architectures
3
Classification

• Question
• a user needs a visualization
• which architecture/software to use?

classify architectures according to user roles
4
User Roles
Basic visualization model (one user role)
changes
views
visualization input
The simulation-visualization loop
5
Architecture
Basic visualization architecture
SYSTEM KERNEL
read/ write
produce
controls
6
User Roles
In the most general case

• We wish to customize all elementsof the
  visualization
• customize data
  operations user interfaces

User roles
7
User Roles
- uses visualization via Uis and output
images - is not a programmer
End User
(EU)
- customizes/builds visualization out of
predefined components - may program a little
(e.g. scripting)
(AD)
- customizes/builds the components themselves -
is a developer/programmer
(CD)
8
User Roles
End User
builds application
requirements and feedback
requirements and feedback
builds new components
9
Architecture Model
ARCHITECTURE
End User
facilities
facilities
facilities
Extensibility / customizability
Ease of use / implementation complexity
10
Architecture Types
Three major architectures exist
Ease of use / Implementation complexity / Price
Extensibility / Customizability / Genericity
Framework
Library
Best for CD
Best for AD
Best for EU
11
Architecture Choice
Questions
How do we decide which is the best architecture
for a given user? How do we find an
implementation of that architecture?
12
Libraries
1. Libraries

• stem from basic computer graphics libraries
• graphic data types and drawing operations

STRUCTURE
ATRRIBUTES
VIEWING
OPERATIONS
13
Libraries
OpenGL

• best known graphics library
• procedural model user explicitly calls
  OpenGL functions to

set/get state parameters

draw primitives using the current state
14
Example
Example

• drawing a line (C code)

Result
glBegin(GL_LINE_STRIP) glVertex2i(20,10) glVerte
x2i(50,10) glVertex2i(20,80) glVertex2i(50,80)
glEnd()
coordinates given as 2 integers (x,y)
x,y coordinates
function
15
Example

• several geometric primitives available

GL_POINTS GL_LINES,GL_LINE_STRIP GL_TRIANGLES,GL_T
RIANGLE_STRIP GL_QUADS,GL_QUAD_STRIP GL_POLYGON

• attributes (color,normal,etc) are given per
  primitive vertex

16
Viewing Pipeline

• OpenGL graphics viewing pipeline

current transformation matrix
world coordinates
transformed coordinates
projection matrix
projected coordinates
clipped coordinates
clip
viewport matix
pixel coordinates
17
Performance

• OpenGL is optimized for most platforms and
  OSs
• hardware rendering
• ideal for fast, portable graphics
• windowing system independent
  (graphics-context based)

Example 3D textured, semitransparent cube (30
lines C code)
18
Conclusion

• portable, fast, 3D graphics library
• requires significant programming
• no high-level abstractions needed for
  visualization (datasets, algorithms)
• used in most rendering back-ends of
  visualization systems

More Info http//www.opengl.org
19
Toolkits
3D Master Suite

• visualization toolkit atop of Open Inventor
• offers specialized components for
  graphing, charting web graphics
  vector hardcopy large data
  visualization specialized
  FEA,CAD/CAM, financial visualization

20
Architecture
Visualization application
3D Master Suite
Open Inventor
OpenGL
Windowing system
21
Components
3D Master Suite components
High-level 2D / 3D graphing and charting
Extends Inventors PostScript scene output with a
vectored output (new Inventor action)
Merges the Inventor and ISO VRML 97 formatsto
enable applications interchange them
Adds some 2D / 3D FEA, CAD/CAM, and financial
visualization operations
Adds decimation and culling algorithms toOpen
Inventor-based applications
22
Examples
3D Master Suite sample visualizations
23
Examples
LMVMaster decimation
GraphMaster histogram
3DDataMaster visualizations
24
Conclusions
3D Master Suite

• extends Open Inventor with higher level
  classes
• excellent OO software design
• good toolkit for to start building
  visualization applications CDs
  ADs
• however, no complex visualization algorithms
  and/or data structures (e.g. grids, fields, etc)
• commercial (so far)

More Info http//www.tgs.com
25
Turnkey Applications
2. Turnkey Applications

• dedicated to specific problem domains
• highly customized tools (viewers, GUIs, etc)
• narrow focus, so simple to use and learn
• usually built atop of a library/toolkit

Turnkey Application
End Users
Visualization library (OpenGL,Inventor,etc)
26
Example
VolVis a volume visualization medical application
Control menus
3D interactive viewer
Data probes
Transfer function GUIs
2D slice views
27
Architecture

• turnkey applications add
  specific control specific
  GUIs to algorithms and data structures built
  atop standard graphics libraries

Turnkey Application
Custom Part
Generic Part
Visualization library (OpenGL,Inventor,etc)
28
Conclusions

• easy to learn since focused
• best for specialized EUs
• usually hard to customize by ADs (not always
  open / modular architectures)

researchers may need more flexibility in quickly
customizing their visualizations new
architecture needed
29
Frameworks
3. Frameworks

• combine the flexibility of libraries with the
  usability of turnkey applications
• less domain-specific than turnkey apps, but
  more specialized than libraries
• less flexible than libraries, but more
  customizable than turnkey applications

30
Architecture
component2
component2

• communication
• attachment
• detachment
• synchronization

domain-specific semantics
component2
component2
application-specific functionality
31
Visualization FrameworksThe Computational
Steering Environment (CSE)
32
Overview
CSE

• A set of software components for combining
  scientific simulation applications with a
  visualization and interaction back-end
• focuses on providing a visual way to examine and
  modify simulation parameters

33
User Roles
Uniform, powerful modelling and design concepts
Programming tools
S y s t e m
CSE
Steering capabilities
Comprehensive and generic GUI
34
Architecture
Where do we place CSE?
Best for CD
Best for AD,EU
Library
Framework
Ease to extend/customize
CSE
More a framework than a library
35
Architecture
Researcher
graphical input
directmanipulation
animation
Computational Steering Environment
statevariables
parameters
results
Simulation
36
Requirements

• manipulate input parameters of simulations in
  a graphical, interactive way
• visualize simulation results graphically and
  interactively
• simple way to couple parameters to graphical
  (visual) elements
• simple to add visualization/steering to
  existing simulation code

37
Architecture
Researcher
text/ drag/ pick
visualize
selection
expression
.
PGO Editor
Selection satellite
Calculator satellite
.. satellite
data
data
data
data
Data Manager (kernel)
data
Simulation
38
Architecture
Satellite Parametrized Graphics Object (PGO)
Editor
visualize
Graphics objects
Degrees of freedom
State variables
Data manager
39
Architecture
Satellite Parametrized Graphics Object (PGO)
Editor
Graphics objects
Degrees of freedom
State variables
Data manager
40
Applications
Amospheric simulation for smog prediction
41
Applications
3D bouncing balls simulation
42
Applications
Astrophysical simulations
43
Conclusions
CSE

• simple to use client-server architecture for
  adding computational steering to existing
  applications
• adding visualization/steering is very fast
  (add a few API calls / design a few PGOs)
• must have access to/modify simulations source
  code
• no real visualization operators provided
  (i.e. datasets and algorithms)
• designing new types of PGOs might be difficult
• PGOs can control/view simple state variables
  only (viewing e.g. an isosurfce might be very
  complex)

44
Ongoing Directions
More information
on the CSE
http//www.cwi.nl/projects/cse
45
Visualization FrameworksThe Visualization
Toolkit (VTK)
46
Overview
VTK

• A C class library for scientific data
  representation and manipulation
• focused on visualization, but can be usedfor
  other purposes too (e.g. as infrastructure for
  scientific application development)

47
User Roles
Uniform, powerful modelling and design concepts
Programming tools
VTK
S y s t e m
Steering capabilities
Comprehensive and generic GUI
48
Architecture
Where do we place VTK?
Best for CD
Best for AD,EU
Library
Framework
Ease to extend/customize
More a library than a framework, but...
VTK
49
Architecture
C Class hierarchy
VTKObject
Other components
VTKMapper
Tcl/Tk, Java bindings
VTKFilter
VTKDataSet
VTKActor
VTKIntArray
VTKPolyData
VTKRenderer
Datasets - structure - attributes
Operations - read / write - filter, map - render
Various - basic data types - bindings
50
Architecture
VTK implements a demand-driven visualization
pipeline
VTKReader
operation
read data
update execution
update request
VTKDataSet
dataset
transform data
VTKFilter
underlying graphics and windowing libraries
VTKDataSet
map to graphics primitives
OpenGL
X11/Windows
VTKMapper
VTKActor
VTKRenderer
VTKRenderWindow
render primtives
display rendering and get events
51
Architecture
VTK application example
vtkQuadric q vtkQuadricNew()
q-gtSetCoefficients(0,1.2,3.5) vtkSampleFunction
f vtkSampleFunctionNew()
f-gtSetSampleDimensions(30,30,30)
f-gtSetImplicitFunction(q) vtkContourFilter c
vtkContourFilterNew() c-gtSetInput(f-gtGetOutpu
t()) c-gtGenerateValues(5,0,1.2) vtkPolyDataMap
per m vtkPolyDataMapperNew()
m-gtSetInput(c-gtGetOutput()) vtkActor a
vtkActorNew() a-gtSetMapper(m) vtkRenderer
r vtkRendererNew() r-gtAddActor(a)
VTKQuadric q
VTKSampleFunction f
VTKContourFilter c
VTKPolyDataMapper m
VTKActor a
VTKRenderer r
52
Example
The result of the previous VTK program
53
Visualization Examples
Flow visualization
Stream tubes
Isosurfacing
Hyper streamlines
54
Functionality

• VTK coverage
• scalar/vector visualizations
• medical visualization
• volume rendering
• image processing (not exhaustive)
• 3D geometric visualization
• basic computer graphics/rendering
• not much for time support/animations

55
Conclusions
VTK

• comprehensive C library for scientific
  visualization
• no real competitor of similar size on the market
• shareware
• must understand OOP basics to use it (AD)
• must be advanced in OOP to extend it (CD)
• no real EU role (no GUIs, no pipeline editor)
• documentation is quite limited
• many new releases improvements BUT also
  modifications

56
Ongoing Directions
Which are the possible extensions of VTK?
Convenience
Conceptual

• new classes
• multithreading/streaming
• better documentation
• simpler installation
• better language bindings

• pipeline editor?
• simpler architecture for CDs?

57
Visual Application Builders
58
Application Builders
Application Builders

• Are frameworks with a visual programming
  interface
• Merge the AD and EU roles better
• Appeared since building new applicationsis a
  major time-consuming task

59
User Roles
Uniform, powerful modelling and design concepts
Visual programming tools
Visual representation of concepts
S y s t e m
Steering capabilities
Comprehensive and generic GUI
60
The Visual Element
Best for AD,EU,CD (ideally)
Brings a visual representation to EU,AD, and CD
concepts
Visual Builder
Best for CD
Best for AD,EU
Library
Framework
Ease to extend/customize
61
Requirements
Visual Application Builders
Merge library and turnkey application
advantages for all user groups
End User

• easy code development and integration

• visual programming
• rich component set
• easy pipeline
• design

• steering
• GUI control
• interrogation

62
Application Builders
AVS http//www.avs.com
63
Application Builders
Oorange TU Berlin, Germany
64
Application Builders
MayaVi http//mayavi.sourceforge.net
65
MayaVi

• Built atop the VTK library
• Offers only subset of VTK functionality
• Menu-based pipeline construction

Pipeline construction
VTK viewer window
VTK object parameter GUI
66
MayaVi GUI
Grid plane
Contour grid plane
Isosurface
Scalar bar
67
MayaVi Pipeline
Data file
DataViz Manager
Module Manager
Module1
Module Manager
Module1
Module2
Module2
Module3
Module3
VTK Viewer
68
Pipeline Construction

• Menu-based
• Offers only subset of VTK functionality

Display of VTK pipeline behind a MayaVi module
(IsoSurface)
69
MayaVi Architecture
VTK core libraries
vtkLibrary.dll
vtkLibrary.dll
vtkLibrary.dll
VTK Python wrappers
vtkSomeLibraryPython.dll
vtkSomeLibraryPython.dll
vtkSomeLibraryPython.dll
MayaVi modules
SomeModules.py
SomeModules.py
Tcl/Tk core libraries
tcl.dll
tk.dll
MayaVi initialization
__init__.py
tkInter.pyd
Tk-to-Python bindings
python.dll
Python interpreter
MayaVi executable
70
Extending MayaVi

• Can (must) be done at several levels
• - VTK core level write new C class

- Python wrapper wrap C class in Python API
- MayaVi module level write new Python class
MayaVi module This contains both the
visualization stuff and the GUI stuff
- Library level place all files in the right
directories

• Quite difficult
• - multi-language C, Tcl, Python

- multi-model compiled, interpreted, dynamic
linking / loading
- dual hierarchy C and Python class
hierarchies to maintain
- not automated wrappers and GUIs must be
written manually
71
MayaVi Conclusions

• Simple-to-use visualization tool
• Great for simple scenarios
• Single-branch, short pipelines
• Uses only main VTK modules
• Effective mostly for end users, not application
  designers or component developers
• Free, cross-platform
• Simple to install
• However, not entirely bug-free

72
Application Builders
VISSION Visualization and Simulation with
Object-Oriented Networks
http//www.win.tue.nl/alext/ALEX/PAPERS
73
Vission
The Vission Environment
74
Preliminaries
Aim remove most/all component developer
restrictions present in other
environments Means use a novel, more complex
architecture
Include various libraries (VTK,Inventor,etc)No
restrictions on what can be included
Directly accept C classesNative
compiled/intepreted C support
Automatically built out of component specsVisual
pipeline construction
75
Preliminaries
Why are most visualization environments
limited? Because they are based on a too specific
library/code model
System C API, data and functionsUser code should
be based on this API
AVS
Given class hierarchy/execution model Users
should subclass and recompile
VTK
Python wrapping of VTK C libraryUsers should
subclass, recompile,wrap, build GUIs,
MayaVi
No need to subclass given hierarchyIntegrate any
C classesAutomatic GUI construction
VISSION
76
Overview
Component browser
Data viewer
Component user interface
Dataflow network
77
Components
How does VISSION integrate code? C classes are
wrapped by MC modules
GUI view
MC module
Data inputs
M
Output
Input
Operation
Data outputs
Module name
Instance name
C user code / data
78
System Internals
End User
Dataflow Manager
Interaction Manager
Application
Library Manager
Application Libraries
79
Architecture Overview
Vission and its users
80
System Internals
User Interface Write
Data Transfer
User Interface Read
MC module n1
MC module n2
obj1.set(obj2.get())
obj1.set()
obj2.get()
C object obj2
void set(T)
U get()
C interpreter
C compiled library
81
Vission and Types
What must the end-user know about types?
Port types
Module types
When can I connect two ports ?
Where do I find my desired component ?
?
Connection rules

• compatible C types
• one input, one output
• one read, one write
• automatic learning

?
82
Learning Mechanisms
SmartLink Mechanism that assists the network
construction based on behaviour learning
(IEEE VisSym 00)
83
Interactors
Vission automatically builds GUIs for components
Component Icon
Component Interactor
Float port
RGB port
Boolean port
vector port
84
Vission Interactors
Best-match process used to choose widgets
Widget Library
Component Port Types
float
Vector
Color
User preferences
text
float
Boolean
int, float
Color
Component Interactor
. . . extendable by component developer
85
Interactors Example
What interactors can we build for a float ?
For a float component Vission builds...
Type-ins
Sliders
Thumbwheels
86
Interactors Example II
What interactors can we build for a Color ?
For a Color component...
Class Color float r,g,b
Vission builds
Colorwheels
Type-ins
Color sliders
87
Interactors Summary
Interactors are automatically built for
arbitrarily complex components containing any
user-defined types
88
Vission and Libraries
White-box component integration (TOOLS 99)
callback
load

• Compiled C
• usual dynamic library
• comes as binary code

• Interpreted MC
• add dataflow info to raw C classes
• comes as source code

instantiate
Vission independent
Vission dependent
89
Putting It All Together
Vission and its users
90
Visualization Examples
Flow visualization
Stream tubes
Isosurfacing
Hyper streamlines
91
Visualization Examples II
Medical visualization
92
Visualization Examples III
numerical pipeline
vtk pipeline
Inventor pipeline
93
Visualization Examples IV
Lighting simulations (ACM JGT 97)
94
Vission Libraries
Vission integrates the Open Inventor library
95
Vission Libraries II
Vission integrates the Visualization Toolkit (VTK)
vtk visualization
Inventor visualization
96
Visualization Examples V
Vector field simplified visualizations (IEEE
Visualization 99 IEEE Visualization 00)
97
Visualization Examples VI
Vector field simplified visualizations
2D example
simplification pipeline
98
Animation Production
Easy production of animations by using looping
networks

• online animation / time-dependent simulation
  (e.g. iterative solvers)
• movie production (e.g. MPEG)

99
Computational Steering
Vission visualization
data sensor
usual Vission pipeline
external application (Showcase)
100
Online Assistance
Integrated HTML browser
HTML hyperlinks
101
Online Assistance II
Documentation for the VTK library
HTML hyperlinks to VTK components
102
Statistics
What is the complexity of Vission ? How long did
its development take ?
Subsystem
Classes
Creation time
kernel 3D graphics visualization radiosity numeric
al examples
307 82 267 21 19 139
16 months 5 weeks 2 months 6 days 1 hour hours
103
Conclusion

• Pros
• integrate any C class lib (not e.g. just VTK)
• execution model in system, not in library
• user code 100 system-API free (no subclassing,
  types usage, or API calls needed)
• many automated tasks (GUIs, visual modules,
  docs)
• Cons
• not (yet) publicly available
• too much development freedom can be confusing?
• not (yet) multi-platform

104
Comparison
Compare the various architectures discussed
visual programming
datasets,data processing
C O M P L E X I T Y
scene model,interaction
basic rendering
105
The End