Revolution

1
Revolution

• Enabling Large-Scale Collaborative Science

2
Outline

• Introduction
• Visualization Applications
• Distributed, Parallel, Grid-based, and
  Collaborative Visualization
• Collaborative Scientific Visualization
  Environments (CSVE)
• Future Directions

3
Revolution In Science

• Pre-Internet
• Theorize /or experiment, alone or in small
  teams publish paper.
• Post-Internet
• Construct and mine large databases of
  observational or simulation data.
• Develop simulations, analyses, synthesis.
• Access specialized devices remotely.
• Exchange information within multidisciplinary
  teams.

Image from www.aip.org
Image from CERN
4
Why Visualization?

• Visualization is now seen as an integral part of
  modern computing
• High performance computing generates vast
  quantities of data
• High resolution measurement technologies generate
  vast quantities of data
• Information systems incorporate large data sets
  and complex relations
• We simply must harness our visual systems to aid
  us in understanding our data

5
What Is Visualization?
6
Medical Applications

• From MRI, CT, Confocal Microscopes,
• We can visualize human anatomy at various scales

Curved Surface through the aorta tree. Visible
Human Server, from R.D. Hersch at Ecole
Polytechnique Fédérale de Lausanne
Optical nerve in the retina. Imaris software from
B. Ehinger, Department of Ophthalmology, Lund
University Hospital
Torn ACL. Anonymous image.
7
Climate Applications

• From simulators, satellites, measurement
  stations,
• We can visualize events, climate, and current
  weather

These images show a comparison between two large
El Niño events. The first begins in Oct '81 and
the second in Oct '96. Image from NCAR.
Top-south view of 3-D volume of the simulated
Andrew's radar reflectivity. Image from Y. Liu,
McGill University.
Satellite and surface image for January 19, 2004.
Image from Unisys Weather.
8
Oil and Gas Applications

• From simulators, seismic data sets, field
  measurements,
• We can visualize production, management, and
  exploration

Immersive visualization of horizons, faults,
wells, and salt dome. Image from BP
Visualization Center, University of Colorado.
Real-time cross-section planes where opacity is
reduced in order to show values of interest.
Image from HueSpace of Norway.
Streamlines emanating from a virtual well show a
three-dimensional oil flux. Image from Lawrence
Berkeley National Laboratory.
9
Molecular Applications

• From simulators, experiments, measurements,
• We can visualize molecules, simulated values, and
  statistical measurements

Main chain hydrogen bonds and peptide bonds
deviating more than some degree from
planarity Image from Dirk Walther, UCSF.
Fancy CPK model. Atoms are made of various metals
(C gold, H chrome, N bronze, O silver, S
brass). The ellipsoid (made of red glass) is the
one with the smallest volume containing 70 of
all atoms. The molecule is Trypsin Inhibitor.
Image from L. Chiche .
The image depicts the electrostatic potential at
each point of the Van der Waal's dot surface
around aspirin. Image from Roger Sayle
10
Environmental Applications

• From observations, experiments, measurements,
• We can visualize terrain, database information,
  and measurements

Monitoring wind profile in Monterey Bay. Image
from A. Pang, UCSC.
Populations of trees using a range of rendering
techniques. Image from USDA Forest Service,
Pacific Northwest Research Station .
Patterns of recent forest management activities
in the Northwest. Image from J. S. Nighbert,
Oregon BLM.
11
Scientific Visualization

• 1987 NSF Report B.H. McCormick, T.A. DeFanti, and
  M.D. Brown, "Visualization in Scientific
  Computing," in Computer Graphics, Vol. 21, No. 6,
  (special issue).
• Turning firehoses of data into a visual
  representation
• Enabling the scientist to see the unseen
• Argued that investment in high performance
  computing in US was wasted unless there was
  corresponding investment in visualization
• Led to the development of several visualization
  software systems

One of the many visualization software systems
created during this time. Developed by B.
Hibbard. vis5d.sourceforge.net
12
Dataflow Visualization

• Visualization represented as a pipeline
• Read data
• Filter data
• Map data
• Render data
• Display data
• System realized in at least two ways
• Modular Visualization Environment
• Toolkits or Libraries

13
Modular Visualization Environment

• Modular Visualization Environments
• IRIS Explorer, OpenDX, AVS,
• Visual programming paradigm - allows easy
  experimentation which is what one needs in
  visualization
• Extensible add your own modules
• Scientist uses visual programming to connect
  modules together

IRIS Explorer www.nag.com
OpenDX www.opendx.org
AVS5 www.avs.com
14
Visualization Libraries and Toolkits

• Visualization libraries and toolkits
• OpenGL, Java3D, VTK, OpenRM, Java3D,
• Provides the application programmer an API
• Scientist uses applications or incorporates
  visualization code in own software
• Open source
• OpenGL
• Industry standard
• Hardware acceleration
• Basis for VTK, OpenRM, Java3D
• Java3D
• A mapping of OpenGL
• OpenRM
• Direct volume rendering
• VTK
• Bindings to Tcl, Python, Java

vmd from NCSA using OpenGL www.opengl.org
Cave using Java3D from the University of Calgary
Java3D java.sun.com/products/java-media/3D/
Visapult from LBNL using OpenRM www.openrm.org
Virtual creatures from Stanford University using
VTK www.vtk.org
15
Visualization and Simulation

• Visualization is a key tool in understanding the
  results of numerical simulations of complex
  phenomena
• Use cases of visualization for simulation
• Pre-processing
• Treat dataflow visualization environment and
  simulation as separate activities
• Tracking
• Replace data in visualization pipeline with the
  simulation
• Track behavior
• Steering
• Include control module in visualization pipeline
• Simulation responds to visualization environment
• Post-processing
• Again, treat visualization and simulation as
  separate activities

Reservoir simulation using VTK from Geocap
Pre-process
Track
Steer
Post-process
16
Visualization and Observation

• Visualization is a key tool in understanding
  observational data
• Use cases of visualization for observational data
• Monitor
• Monitor incoming observations
• Post-processing
• Treat visualization and observations as separate
  activities
• Integration
• Accept multiple input streams

Monteray Bay monitoring from REINAS, UCSC
Monitor
Post-process
Integrate
17
Distributed Visualization

• Distributed visualization
• Offload some computationally intensive tasks
• Couple the simulation with the visualization
• Typically, a single processor is not powerful
  enough to run both the simulation and
  visualization
• Control and, in most cases, rendering will remain
  local
• Types
• Single-processor
• Multi-processor
• Networked processors
• These types can be used in combination
• Visualization pipeline can be distributed in a
  number of ways

Single-processor, possibly multi-processor
Multi-processor - Parallel
Loosely-coupled
18
Issues

• Multi-processor issues
• Load balance
• Latency
• Decomposition,
• Control
• Launching remote parts
• Interacting with remote parts (steering problem)
• Authorization
• Authentication
• Resource discovery
• Data
• Format
• Proprietary
• Open Standards
• Compression
• General purpose
• Special purpose

19
Visual Network Computing/VizserverTM

• Multi-processor loosely coupled
• Access to SGI high performance computing/graphics
  over network
• Renders on remote devices
• Remote framebuffer compressed and distributed via
  TCP/IP over network
• Control over compression
• Features
• Application transparent
• Shared-control
• Platform/independent
• Advanced visualization environments
• Scalable

20
Grid

• Grid Development and promotion of standard
  protocols to enable interoperability and shared
  infrastructure
• Globus toolkitTM Open source reference
  implementation for building grid infrastructure
  and applications
• Global Grid Forum Development of standard
  protocols and APIs for Grid computing
• Layered Architecture
• Collective Managing multiple resources to
  provide a ubiquitous infrastructure and services
• Resource Sharing single resources, negotiating
  access, controlling use
• Connectivity Talking to things securely
• Fabric Controlling access and resources locally

Real-time visualization of advanced photon source
data, Image from Argonne National Laboratory
21
Grid Service

• Idea A service with well-defined interface
  advertises itself in a distributed directory
  service
• Application queries directory service on how to
  interact with the service
• Web Service
• URI
• Discovered by XML artifacts
• Interactions through XML-based messages
• Standards WSDL, SOAP,
• Grid Service
• Extends Web services
• Standards OSGA, OSGI

22
Grid Visualization

• Use Grid Services to discover
• Grid Visualization Service
• Simulation Running on Grid
• Data Stores on Grid
• Grid Middleware
• Compression
• Native / XML Data
• Grid Visualization Service
• Simulation can register parameters and data with
  the service
• Data stores or databases can be registered with
  the the service
• Supports multiple clients
• Service manages connections from external clients
• External clients can connect and interact with
  data streams
• Synchronizes connected clients

23
Parallel Visualization

• Chromium
• Open Source
• Enables parallel rendering
• Replaces systems OpenGL driver
• Industry standard API
• Supports existing applications
• Streams of API
• Alters/Discards/Injects
• Routes commands
• Geometry is moved across network
• Rendered remotely
• Visapult - LBNL
• Parallel Volume Rendering
• Uses OpenRM an industry standard

(a)
(b)
Chromium was created by Greg Humphreys,
chromium.sourceforge.net
Visapult, Image from LBNL
24
Collaborative Visualization
Need to move away from seeing collaborative
visualization as a group crowded around a display
screen

• Radical collocation has proved highly successful
• Manhattan Project
• Space missions
• Software development
• Productivity Doubled!
• Teasley et al, Michigan
• But it requires
• Social disruption
• Advance planning
• Goal of Computer Supported Cooperative Work
  (CSCW)
• Gain in productivity, but reduce collocation
  requirement using electronic collaboration

Towards collaboration over network
25
CSCW Model

• CSCW Model associates applications with
  approaches
• Based on
• When?
• Where?
• Visualization
• Real Time
• Same Place
• AVS, Amira,
• Different Place
• What do we share?
• Display
• Visualization
• Process
• How many users/location?

26
Sharing Screen

• Simple model
• Broadcast display of application to a set of
  passive users
• Number of available technologies
• IRIS Explorer, AVS,
• VNC Virtual Network Computing
• RealVNC www.realvnc.com
• tightVNC www.tightvnc.com

VNC, from ATT
27
Sharing Visualization

• Share the visualization
• Geometry is exchanged
• Master/Slaves
• Number of available applications
• COVISE, IRIS Explorer
• Advantages
• Greater involvement of collaborators
• Shared Control Token Passing
• Disadvantages
• Cant determine what collaborators are doing
• Limited collaboration

COVISE, from Dr. Ulrich Lang Computing Center
University of Stuttgart Visualisation Department
28
Sharing Process

• Each collaborator may participate in producing
  the visualization
• Two variations
• Replicated
• Initial data sharing
• Parameters are interlinked
• Small network traffic
• Application tailored to individuals expertise
• CSVE
• Interlinked
• Separate pipelines
• Cross wiring pipelines enables collaboration
• Greater flexibilty
• Varying network traffic
• COVISA

29
Issues

• Portable
• Different OS
• Different Libraries/Toolkits/MVEs
• Functionality
• Data
• Parameters
• Algorithms
• Applications
• Participation
• Joining/Leaving
• Floor control
• Privacy
• WYSIWYTIS
• Authentication
• System
• Performance
• Scaling
• Reliability
• Robust

CSVE, from Patrick OLeary
COVISA, from Jason Wood, Visualization Scientist,
University of Leeds
30
Access Grid

• The Access Grid
• Ensemble of resources
• Multimedia large-format displays,
• Presentation and interactive environments,
• Interfaces to Grid middleware and to
  visualization environments
• VRVS
• Desktop Web-based alternative
• Advantages
• Greater sense of involvement
• Lower geek threshold
• Used in combination with VNC

Access Grid, Image from www.accessgrid.org
VRVS, www.vrvs.org
31
CSVE

• Collaborative Scientific Visualization
  Environment (CSVE)
• Facilitate Scientist - Computer Scientist or
  Small Group Interaction
• Open Source
• Java
• JMF
• VTK
• Features
• Interactive 3D Visualization
• Streaming Audio/Video
• Streaming Media
• Desktop Capture
• Chat
• Whiteboard
• Telepointer
• Remote Control Client
• Data Management

A visualization expert interacts with
a research area expert
32
CSVE

• Anastasia Mironova Vis 2003
• Interactive 3D Visualization
• Handles several data formats
• Create/Manage isosurfaces, slices,
• Simple tools for interacting with visualization
• Seamless network propagation of visualization
  parameters

Create visualization objects
Manage visualization objects
33
CSVE

• Brian Mullen Vis 2003
• Streaming Media
• Stream any mpeg, avi, mov file to collaborators
• Streaming Audio/Video
• Stream audio/video from two to collaborators
• Desktop Capture
• WYSIWIS not WYSIWYTIS

Stream scientific videos
Stream audio/video to collaborators
See what they are looking at
34
CSVE

• Scientific Database
• Currently
• Relational Database MySQL, Oracle,
• Flat files
• Moving to Meta Catalogue
• Based on an extension of XML
• Why XML?
• Accepted way of describing things for the Web and
  the Grid.
• Good at describing things because
• Wide range of concepts can be captured in this
  way.
• It provides a basis for validators, transformers,
  parsers, analyzers, displayers,
• So simple
• This is why HTML became so widely used.
• Can teach anyone to use it in a short period of
  time.

lt?xml version'1.0'?gt ltlistgt ltrecipegt
ltrecipe_namegtChocolate Chip Barslt/recipe_namegt
ltauthorgtCarol Schmidtlt/authorgt
ltmealgtDinner ltcoursegtDessertlt/coursegt
lt/mealgt ltingredientsgt
ltitemgt2/3 C butterlt/itemgt ltitemgt2 C
brown sugarlt/itemgt ltitemgt1 tsp
vanillalt/itemgt ltitemgt1 3/4 C unsifted
all-purpose flourlt/itemgt ltitemgt1 1/2
tsp baking powderlt/itemgt ltitemgt1/2 tsp
saltlt/itemgt ltitemgt3 eggslt/itemgt
ltitemgt1/2 C chopped nutslt/itemgt
ltitemgt2 cups (12-oz pkg.) semi-sweet choc.
chipslt/itemgt lt/ingredientsgt
ltdirectionsgt Preheat oven to 350
degrees. Melt butter combine with brown sugar
and vanilla in large mixing bowl. Set aside to
cool. Combine flour, baking powder, and salt
set aside. Add eggs to cooled sugar mixture beat
well. Stir in reserved dry ingredients, nuts,
and chips. Spread in greased 13-by-9-inch pan.
Bake for 25 to 30 minutes until golden brown
cool. Cut into squares. lt/directionsgt
lt/recipegt lt/listgt
35
CSVE

• Message Passing
• Objects through bit-stream
• Same underlying principles as remote object
  broker or RMI
• No parsing
• Flexible
• Extensible
• Efficient
• No parsing!
• Moving to XML messages
• The way messages are passed by Grid- and
  Web-services
• Slower
• Standard format
• Requires parsing messages built into Java

3 Tier Architecture
36
Application Neuroscience

• Pain
• Quality of Life
• Neurochemical Changes
• Image Reconstruction
• Removal of Noise and Artifacts
• Deconvolution of Light Source
• Segmentation of Data
• Visualization Techniques
• Maximum Intensity Projection (MIP)
• Volume Visualization

37
Application Neuroscience
38
Application Cancer

• Bone Cancer
• Bone Destruction
• Tumor Burdon
• Image Reconstruction
• Removal of Noise and Artifacts
• Edge Detection
• Automation
• Segmentation of Data
• Visualization Techniques
• Isosurfaces
• Volume Visualization

39
CSVE
Streaming Media
Interactive Visualization
Desktop Capture

• Portable
• Windows
• Apple
• Linux

Additional Applications
40
CSVE
41
CSVE

• Future Work
• Grid Protocol Based
• Resource discovery
• Databases
• Simulations
• Smart Instruments
• Visualization Resources
• Data exchange
• Message passing
• Server as a Grid-service
• Remote Control
• OpenRM Direct Volume Visualization Version
• More Visualization Techniques
• More sophisticated data management

42
Acknowledgements

• NSF MRI grant, 0215583, and a NSF REU Supplement
  to the grant
• NSF EPSCoR Alaska for funding both Anastasia
  Mironovas and Brian Mullens summer research
  internships
• The University of Alaska Anchorage (UAA) Office
  of Undergraduate Research and Scholarship, Office
  of Research and Graduate Studies, and Dr. Hilary
  Davies, whom through Discovery Grants and travel
  funds made it possible for both Mironova and
  Mullen to present their work at Visualization
  2003
• Jonathan Snelling, supported by a NSF REU
  Supplement, for his work on a multi-document
  graphical interfaces
• Brian Mullen for his development of streaming
  audio/video tools (he put the C in CSVE)
• Anastasia Mironova for her development of volume
  visualization tools, integrating additional data
  formats, and winning Best Poster at Visualization
  2003
• My CS 401 Software Engineering class at UAA
  (Nicholas Armstrong-Crews, Jan Reitspies, Kevin
  Dickerson, William Sistar, John Vicente, Jeffrey
  Woods, Daniel Stokley, Justin Dieters,
  Christopher Johnson, Mark Blum, Shannon Smith,
  Brandon Douthit-Wood, Shane Ursani, Nathaniel
  Freeburg, and Christopher Ulmer).