Scientific Visualization Ume University

1
Scientific VisualizationUmeå University
Lecturers Roger Crawfis Ion Barason Bo
Kågström Eric Elmroth
The Ohio State University
2
Introduction

• Getting acquainted - teams
• Current studies or major
• Hometown
• A interesting data problem

3
Outline

• 1D and 2D Visualization Algorithms
• Data Topologies and Data Sources
• 2D Computer Graphics
• 3D Computer Graphics and Viewing
• Illumination and Shading

4
Outline (cont)

• 3D Visualization techniques
• Iso-contour surfaces
• Volume Rendering
• Flow Visualization
• Volume Rendering - Optical Models

5
Outline (cont)

• Transfer Functions
• Volume Rendering
• Drebin, Carpenter and Hanrahan
• Volume Rendering
• Projected Polyhedron
• Volume Rendering
• Splatting

6
Outline (cont)

• Medical Visualization
• Flow Volumes
• Global Vector Field Visualization
• Virtual Environments
• The CAVE

7
1D Visualization

• y f(x)
• Line Charts
• Curve Fitting
• Smoothing or Approximation

8
1D Visualization

• Non-cartesion coordinate systems

9
Basic 2D Visualizations

• Scalar Data on a Regular Grid
• Surface plots(2?D graphics)

10
2D Visualizations

• Contour Lines - f(x,y) constant

11
2D Visualizations

• Psuedo Coloring

12
2D Computer Graphics

• Image formats and pixel limitations
• Color Tables
• grey-scale
• hot to cold
• perceptual

13
Transfer Functions

• Besides the basics, most tools allow you to
  define your own color mappings.

14
2D Visualization

• Vector Fields
• Hedgehogs
• Streamlines

15
1D Visualization

• Vector?

16
Terrain Visualization

• Exaggerated surface height
• color-coded by
• height
• additional variable

17
Working with 2D Regular data

• Reconstruction is critical
• Useful Image Processing operations
• Histogram
• Data mappers
• Region of interest selection
• Edge detection
• Noise removal or blurring

18
Data Topologies - Structured

• Cartesian
• x j1 xi ?
• Regular or Uniform
• x j1 xi ?x
• Rectilinear or Perimeter
• x j1 x(i)

19
Data Topologies - Structured

• Curvilinear
• x j1 x(i,j)
• y j1 y(i,j)
• Curves may intersect in i or j
• Curves may not cross in i or j

20
Data Topologies - Unstructured

• Unstructured or cell data or finite-element data
• Tetrahedral

21
Data Topologies - Unstructured

• Hexahedral

22
Data Topologies - Unstructured

• Finite-element zoo

23
New Data Topologies

• Improved data topologies offer huge potential for
  savings in computational science
• Hierarchical models are becoming more common

24
New Data Topologies

• Hierarchical
• Multi-Block Curvilinear
• N-sided Polyhedron where ngt6
• Multi-Grid or Adaptive Mesh Refinement

25
Data Sources

• Computational Science
• Data Acquisition / Imaging
• Historical Observation
• Survey, Census, etc.

26
What is Computer Graphics?

• Given three points, what to draw
• The points
• A wire-frame triangle
• A solid triangle

27
2D Graphics - Rasterization
28
2D Graphics - Aliasing
29
2D Graphics - Line Drawing

• DDA Algorithm
• Midpoint (Breshenham) Algorithm

30
2D Graphics - Color limitations

• 8-bit color displays
• 24-bit
• Gamma correction
• Printing versus displaying

31
Color Tables

• If we have 3 bit buffer we can turn on and off
  each gun and display on the screen 8 different
  color simultaneously.
• But each of our guns have 256 intensities! What a
  waste!

32
Color Tables
33
2D Contouring

• Continuous f(x,y)
• Use steepest decent to find zero crossing (root)
  of the function f(x,y)-c
• Follow contour from this seed point until we
  reach a boundary or loop back.
• Direction close to ?f ? z
• Problems?

34
2D Contouring

• Discrete Data
• Assume the Mean Value Thereom
• Assume monoticity?
• 1D Analogy
• 5 Points

35
2D Contouring

• Given a quadrilateral
• f(x,y) 0.5

lt0.5
gt0.5
36
3D Graphics - Primitives

• Points
• Lines
• Triangles
• Polygons
• Surfaces (cones, spheres, etc.)

37
3D Viewing Transformations

• Coordinate Systems
• Object-centered to World-centered
• Projection to image plane
• Windowing Systems

38
Illumination and Shading

• Light sources
• Flat shading
• Diffuse reflection (Gourand)
• Specular reflection (Phong)

39
Hidden Surfaces

• Backface Removal
• Z-buffer
• Ray tracing

40
Transparency

• Opacity
• A material property the prevents light from
  passing through the object (a 1).
• Transparency
• A material property that allows light to pass
  through the object (a 0).
• Translucency, semi-transparency
• Graded or blurred transp. (0 lt a lt 1).

41
Compositing
42
Visualization Algorithms

• 2D Slice planes in 3D

43
2D Slice Planes

• Orthogonal to a coordinate axis
• Uniform Grids gt image
• Arbitrary
• Specify the normal and a point
• Produces a 2D unstructured grid

44
2D Slice Planes

• Mesh with colors at vertices
• 128x128x128 volume can produce over 50,000
  triangles.
• Mesh with 2D texture coordinates
• Very slow if no hardware support
• More precise transitions
• Mesh with 3D texture coordinates

45
2D Flip book of slices

• Rather than view the 2D slice in 3D, rapidly play
  a sequence of orthogonal slice planes in a movie
  loop.
• Sometimes difficult to build a complete mental
  model.

46
3D Visualizations

• Point plots
• Animation can bring out the surface or pattern
  (MacSpin)
• Depth Cueing aids in the depth perception.

47
3D Visualizations

• Spheres or cubes dispersed throughout the volume
• color-coded
• optional shape-controlled.

48
3D Visualization

• Iso-contour surfaces

49
3D Visualization

• Can add information about an additional variable
• Here, two additional variables control the color.

50
3D Visualization

• Volume Rendering

51
3D Visualization
52
3D Visualization

• Volume Rendering can mimic contouring.
• Use a transfer function with an impulse at the
  contour level.

53
Assignment

• Create the image on the right
• BubbleViz module
• crop to back-top
• downsize by 4
• semi-gloss refl.
• Orthogonal slice
• Iso-contour
• specular
• semi-transparent
• Bounding Box

54
Contouring in 3D

• Treat volume as a set of 2D slices
• Apply 2D Contouring algorithm on each slice.
• Or given as a set of hand-drawn contours
• Stitch the slices together.

55
Contour Stitching

• Problem
• Given 2 two-dimensional closed curves
• Curve 1 has m points
• Curve 2 has n points
• Which point(s) does vertex
• i on curve one correspond to
• on curve two?

i
56
A Solution

• Fuchs, et. al.
• Optimization problem
• 1 stitch consists of
• 2 spans between curves
• 1 contour segment
• Triangles of Pi,Qj,Pi1 or Qj1,Pi,Qj
• Consistent normal directions

Qj
Pi
Pi1
57
Fuchs, et. al.

• Left span
• PiQj gt go up
• Right span (either)
• Pi1Qj gt go down
• PiQj1 gt go down

58
Fuchs, et. al.

• Constraints
• Each contour segment is used once and only once.
• If a span appears as a left span, then it must
  also appear as a right span.
• If a span appears as a right span, then it must
  also appear as a left span.

59
Fuchs, et. al.

• This produces an acceptable surface (from a
  topological point of view)
• No holes
• We would like an optimal one in some sense.

60
Fuchs et. al.

• Graph problem
• Vertices Vij span between Pi and Qj
• Edges are constructed from a left span to a right
  span.
• Only two valid right spans for a left span.

Qj
Qj1
Pi
Pi1
61
Fuchs, et. al.

• Organize these edges as a grid or matrix.

Q
j
i
P
QjPiQj1
PiQjPi1
62
Fuchs, et. al.

• Acceptable graphs
• Exactly one vertical arc between Pi and Pi1
• Exactly one horiz. arc between Qj and Qj1
• Either
• indegree(Vij) outdegree(Vij) 0
• both gt 0
• (if a right, also has to be a left)

63
Fuchs, et. al.

• Claim
• An acceptable graph, S, is weakly connected.
• Lemma 2
• Only 0 or 1 vertex of S has an indegree 2.
• E.g., Two cones touching in the center.
• All other vertices have indegree1
• (recall indegree outdegree)

64
Fuchs, et. al.

• Thereom 1 S is an acceptable surface if and only
  if
• S has one and only one horizontal arc between
  adjacent columns.
• S has one and only one horizontal arc between
  adjacent rows.
• S is Eulerian (closed walk with every arc only
  once).

65
Fuchs, et. al.

• Number of arcs is thus mn.
• Many possible solutions still!!!
• Associate costs with each edge
• Area of resulting triangle
• Aspect ratio of resulting triangle

66
Fuchs, et. al.

• Note that Vi0 is in S for some i.
• Note that V0j is in S for some j.
• With the weights (costs), we can compute the
  minimum path from a starting node Vi0.
• Since we do not know which Vi0, we compute the
  paths for all of them and take the minimum.
• Some cost saving are achievable.

67
Marching Cubes

• Lorensen and Cline, SIGGRAPH 87
• Predominant method used today.
• Efficient and simple

68
Marching Cubes

• Treat each cube individual
• No 2D contour curves
• Allow intersections only on the edges or at
  vertices.
• Pre-calculate all of the necessary information to
  construct a surface.

69
Marching Cubes

• Consider a single cube
• All vertices above the contour threshold
• All vertices below
• Mixed above and below

70
Marching Cubes

• Binary label each node gt (above/below)
• Examine all possible cases of above or below for
  each vertex.
• 8 vertices implies 256 possible cases.

71
Marching Cubes

• Only 15 distinct cases
• not really
• Order vertices into an 8-bit index

72
Marching Cubes

• Animating the contour value
• Special functions for contouring
• Varying speeds and numbers of triangles

73
Marching Cubes

• Data Structures/Tables
• static int const HexaEdges122 0,1,
  1,2, 2,3, 3,0,
• 4,5,
  5,6, 6,7, 7,4,
• 0,4,
  1,5, 3,7, 2,6
• typedef struct
• EDGE_LIST HexaEdges16
• HEXA_TRIANGLE_CASES
• / Edges to intersect. Three at a time form a
  triangle. /
• static const HEXA_TRIANGLE_CASES HexaTriCases
  
• -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
  -1, -1, -1, -1, / 0 /
• 0, 8, 3, -1, -1, -1, -1, -1, -1, -1, -1, -1,
  -1, -1, -1, -1, / 1 /
• 0, 1, 9, -1, -1, -1, -1, -1, -1, -1, -1, -1,
  -1, -1, -1, -1, / 2 /
• 1, 8, 3, 9, 8, 1, -1, -1, -1, -1, -1, -1,
  -1, -1, -1, -1, / 3 /

74
Marching Cubes - How simple

• / Determine the marching cubes index /
• for ( i0, index 0 i lt 8 i)
• if (val1nodesi gt thresh) / If the
  nodal value is above the /
• index CASE_MASKi / threshold,
  set the appropriate bit. /
• triCase HexaTriCases index / triCase
  indexes into the MC table. /
• edge triCase-gtHexaEdges / edge points to
  the list of intersected edges /
• for ( edge0 gt -1 edge 3 )
• for (i0 ilt3 i) / Calculate and store
  the three edge intersections /
• vert HexaEdgesedgei
• n0 nodesvert0
• n1 nodesvert1
• t (thresh - val1n0) / (val1n1 -
  val1n0)
• tri_ptri add_intersection( n0, n1, t
  ) / Save an index to the pt. /
• add_triangle( tri_ptr0, tri_ptr1,
  tri_ptr2, zoneID ) / Store the triangle /

75
Marching Cubes

• Topological inconsistencies in the 15 cases
• Turns out positive and negative are not symmetric.

-

-
-
-
-
-


-


76
Ambiguities

• Right or wrong?

77
Efficient Searching

• With lt 10 of the voxels contributing to the
  surface, it is a waste to look at every voxel.
• A voxel can be specified in terms of its
  interval, its minimum and maximum values.

78
Span Space

• Maximum

Minimum
79
Span Space - Representing

• K-d Trees

• Maximum

Minimum
80
Assignment

• Use an AVS/Express network to create and test the
  Marching cubes cases (guest4/MarchingCubes.v).
• Answer the following question
• Does AVS/Expresss Isosurface module use the
  modified contouring tables to avoid holes?
• Give an example to (with two cubes), to prove
  your point.
• Show an instructor when completed

81
Material Classification

• Drebin, Carpenter and Hanrahan
• Material Probabilities
• Levoy
• Contour surfaces
• MRI data
• Skin versus Brain
• Using Texture mapping

82
Transfer Function

• Maps raw voxel value into presentable entities
  color, intensity, opacity...
• Raw-data ? material (R, G, B, a, Ka, Kd, Ks,...).
• Material ? shaded material.
• High gradient in the data values detects a
  surface and is used as a measure of its
  orientation.

83
Shading and Gradients

• For surface properties and reflections, we need a
  normal.
• Gradient implies a normal.

84
Drebin, et. al.

• Classify volume into material percentages
• several materials per voxel are possible
• Calculate colors and opacities for each voxel
• Calculate a density for each voxel.
• Determine the gradient from the density volume.

85
Drebin, et. al.

• Determine a surface strength for each voxel from
  the density volume
• Using the color and opacity volume, the gradient
  volume and the strength volume, compute a shaded
  volume.
• Transform and view this volume.

86
Drebin, et. al.
Original CT Data
Fat
Tissue
Bone
Density
Color and Opacity
Strength
Gradient NX NY NZ
Transformed
Shaded
Final Image
87
Drebin, et. al.

• Material Classification
• Use a probability, rather than a threshold.
• Bayesian estimate
• Zone centered
• We know the x-ray absorptions of the materials
  (bone, ...)

88
Drebin, et. al.

• Work backwards, given x-ray absorption, I, what
  is the probability of it being air? Of bone?,
• P(I) ??iPi(I)
• ?I gt percent of material (our goal)
• Pi(I) gt probability distribution that material
  i has an intensity I.
• The x-ray absorption of a homogenous material.
• Known a priori

89
Drebin, et. al.

• Bayesian Estimate
• ?i(I) Pi(I) / ?Pj(I) for all n materials.
• ? ?I (I) 1

90
Drebin, et. al.

• Matte volumes
• Cut-outs (world-space matte)
• Depth Cueing (image space matte)

91
Drebin, et. al.

• Surface Extraction
• Each material has a density, both a real and an
  artificial one for visualization.
• D(x,y,z) ?Pi(I)(x,y,z) ?I
• N ?D, and normalize
• For noisy data, blur D.

92
Drebin, et. al.

• Lighting
• absorption
• emitting
• surface scatteringand absorption
• Compositing
• Front over Surface over Back

93
Levoy

• Contour Surfaces with Ray tracing
• If step is too large,will miss the surface.
• If step is too small,it is too slow.

94
Contour Detection

• Control step, or iteratively search until you hit
  the surface.
• Similar to root finding

95
Levoy

• Use a simple transfer function.

96
Medical Visualization

• Data sources
• CAT
• PET
• MRI
• Ultrasound
• Electrical signals
• Computational Science

97
Medical Visualization

• Uses
• Education / Biology
• Surgical Training / Simulation
• Pre-surgical planning
• Diagnosis

98
Virtual Environments

• Immersive
• Interactive
• User Centered

VR
Stereo
Interactive
UserInterface
Graphics
99
Virtual Environments

• Draw at 120Hz
• Track user position/orientation at 120Hz
• Provide Haptic feedback at gt 200Hz

100
Virtual Environments

• Display Technologies
• HMDs - Head Mounted Displays
• Large theater - Imax, Omnimax
• Stereo displays
• HUDs - Heads Up Displays
• windshields
• goggles
• CAVE - Surround video projections

101
Virtual Environments

• Interactive user navigation devices
• Head tracker
• Treadmill
• Bicycle
• Wheelchair
• Boom
• Video detection

102
Virtual Environments

• Interactive user control devices
• Mouse
• 3D pointer - Polymeus, Microscribe,
• Spaceball
• Hand-held wand
• Gesture
• Custom

103
Augmented Reality

• Merged real imagery and computer generated
  imagery.
• Video capture into visualization system
• See-thru glasses

University of North Carolina, Chapel Hill
104
Augmented Reality
University of North Carolina, Chapel Hill
105
Augmented Reality
University of North Carolina, Chapel Hill
106
Augmented Reality

• Also useful for non-medical
• Mechanics drawing super-imposed over the actual
  machinery.
• Guided tours.

107
Haptics

• Force feedback is needed at very fast rates.
• Gloves
• force resistant
• nerve stimulated
• Theme Park rides

108
Stereo Lithography

• Build real models of the visualizations
• Example, molecular docking

109
The CAVE Architecture

• Four projection screens
• Four graphics rendering engines
• Stereo displays
• Head-tracking of one user

110
The CAVE Architecture
Floor
Right
Front
111
The CAVE Architecture

• Several people can view at once
• The projections are only correct for one person.
• Lasers synch the stereo displays with Liquid
  Crystal shutter glasses on each viewer.

112
The CAVE Architecture

• Benefits
• Eye movement problems are avoided!!!
• Users orientation does not matter.
• Can see and examine real people and objects
  within the room

113
The CAVE Architecture

• Problems
• The light intensity on each projector varies
• Precise alignment of the projectors is necessary
  for a smooth seam.
• Viewing does not change for the other viewers.
• Expensive.

114
Making VR Work

• To ensure latency, many of the visualization
  techniques need to be streamlined or
  pre-computed.
• Examples, pre-computed iso-contours, precomputed
  stream lines and particle traces.