Buffers and Mappings

1
Buffers and Mappings
UBI 516 Advanced Computer Graphics
Ar.Gör.Cengiz Güngör gungor_at_ube.ege.edu.tr
2
Historical Background

• Vector displays
• Anybody remember Battlezone? Tempest?

3
Historical Background

• 1970 Local illumination models
• Phong shading Plastic models
• 1980 Computer realism,
• PCs and home computers
• Global illumination models
• High cost, but very realistic
• Texture mapping
• Low cost, but reasonable results

4
Historical Background

• Until past decade application programmers have
  had only indirect access to the frame buffer.
• Today Texture mapping, antialiasing,
  compositing and alpha blending are only a few of
  the techniques that become possibble when the API
  (OpenGL) allows us to work with discrete buffers.

5
Buffers and Mappings
6
Buffers

• We have already used two types of standart
  buffers
• Color buffers and depth buffers.
• A block of memory
• n m k bit elements
• Bit plane
• Any of the k n m planes in a buffer
• Pixel
• All k of the elements at a position

7
OpenGL Buffers
8
OpenGL Buffers and the Pixel Pipeline

• Color buffers (including frame buffer)
• Double buffering front / back buffers
• Stereo buffering right / left buffers
• Depth buffer ( Z-buffer)
• For hidden-surface removal
• Accumulation buffer Stencil buffer
• Motion blur, anti-aliasing Used for
  masking operation

9
Read from / Write into Buffers

• In a modern graphics system, a user program can
  both write into and read from the buffers.
• We occasionally want to read or write a single
  pixel or bit.
• Rather, we tend to read and write rectangular
  blocks of pixels (or bits), known as bit blocks.
• bit-blt (bit-block transfer) raster-ops,
  raster operations
• Geometric object operations is different pixel
  operations.
• OpenGL contains both a geometric pipeline and a
  pixel pipeline.

write_block( source, n, m, x, y, destination, u,
v ) Source is at (x,y), Destination is at
(u,v), Operation writes m n bit-block.
10
Pixel Pipeline

• OpenGL support a seperate pixel pipeline and a
  variety of buffers.
• Data can be moved among these buffers.
• And data can be moved between buffers and memory.
• There are 16 operations for data transfer.

11
Writing Modes

• Source ? Destination writing operation has
• 16 different modes
• Using modes
• mode 3 copy dest ? source
• mode 7 logical OR dest ? source dest
• mode 6 XOR dest ? source ? dest

12
XOR mode

• mode 6 twice XOR operation returns the original
  pixel.
• (x ? y) ? y x
• If we use menus
• A menu appears when mouse clicked,
• And area of screen that it covered, must be
  returned orginal state when menu disappeares.
• S frame buffer, M back buffer (backing
  storage)
• S1 ? S0 ? M0 (S1 is xored back buffer)
• M1 ? S1 ? M0 (S0 ? M0) ? M0 S0
• S2 ? S1 ? M1 (S0 ? M0) ? M1 (S0 ? M0) ? S0
  M0

13
Pixels and Images in OpenGL

• Read, draw, copy operations start at raster
  position
• void glReadPixels( GLint x, GLint y, GLsizei
  width, GLsizei height, GLenum format, GLenum
  type, GLvoid pixels )
• void glDrawPixels( GLsizei width, GLsizei
  height, GLenum format, GLenum type, const
  GLvoid pixels )
• void glCopyPixels( GLint x, GLint y, GLsizei
  width, GLsizei height, GLenum type )

14
Mapping Methods

• One of the most powerful uses of buffers is for
  surface rendering.
• Texture mapping Surface patterns. In the real
  world,we can distinguish among object of similar
  size and shape by their textures. It can be
  1,2,3 or 4 dimensional.
• Bump mapping Distort of normal vectors like an
  orange object.
• Environment mapping Reflections.

15
Texture Mapping
16
Two-Dimensional Texture Mapping

• Texture space (s, t) Image 0,1 0,1
• Texel Texture element. pixel in the texture
• Object space (xw, yw, zw) is 3 dimensional
• Screen space (xs, ys) is 2D

17
Texture Mapping Methods
texture space (s, t)
object space (xw, yw, zw)
screen space (xs, ys)
forward mapping
backward mapping
pixels
texture space
18
Example of Texture Mapping
19
Example of Texture Mapping
glVertex3d (s, s, s) glTexCoord2d(5, 5)
glVertex3d (s, s, s) glTexCoord2d(1, 1)
20
The Art of 3D Computer Animation and Effects by
Isaac Kerlow
21
Texture Mapping

• Mapping from texture space to object space
• Linear mapping for a parametric surface
• Linear mapping function (s,t) ? (u,v)
• u fu(s,t) a s b t c
• v fv(s,t) d s e t f
• Invertible mapping when a e ? b d

22
Two-part texture mapping

• Bier and Sloan, Two-part texture mapping, IEEE
  CGA, 6(9), 1986.
• First step maps the texture to a simple
  three-dimensional intermediate surface, such as
  sphere, cylinder or cube.
• 2D texture
• 2D ? simple 3D mapping
• Second, 3D intermediate surface mapped to object
  surface.
• Simple 3D ? complex 3D mapping
• 3D object (maybe complex)

23
Two-part texture mapping

• Two-stage forward mapping
• S mapping (surface mapping)
• T(u, v) ? T(xi, yi, zi) intermediate surface
• O mapping (object mapping)
• T(xi, yi, zi) ? O(xw, yw, zw)
• intermediate surface plane, cylinder, cube,
  sphere

cube? ??
cylinder? ??
24
Two-part texture mapping

• S and O mapping

texture
S mapping
O mapping
Using object normals
Rays from centerof object
Using intermediate object normals
25
Texture Mapping in OpenGL

• OpenGL textures
• From one- or two-dimensional texture
• To 1D, 2D, 3D, 4D object
• 2D texture to 3D object

26
glTexImage2D()

• void glTexImage2D(GLenum target, GL_TEXTURE_2D
  target texture bufferGLint level, 0, 1, 2,
  level-of-detail for Mip-MapGLint components, 0,
  1, 2, 3 componentsGLsizei width, width of
  textureGLsizei height, height of textureGLint
  border, 0 border widthGLenum format, color
  format of pixels (R, G, B, A)GLenum type, type
  of pixel data (int, short, long, )const GLvoid
  pixels source color array)

27
glTexImage2D Arg 1

• GLenum target
• GL_TEXTURE_2D
• GL_PROXY_TEXTURE_2D
• Provides queries for texture resources
• Proceed with hypothetical texture use (GL wont
  apply the texture)
• After query, call GLGetTexLevelParamter to verify
  presence of required system components
• Doesnt check possibility of multiple texture
  interference

28
glTexImage2D Arg 2

• GLint level
• Used for Level of Detail (LOD)
• LOD stores multiple versions of texture that can
  be used at runtime (set of sizes)
• Runtime algorithms select appropriate version of
  texture
• Pixel size of polygon used to select best texture
• Eliminates need for error-prone filtering
  algorithms

29
glTexImage2D Arg 3

• GLint internalFormat
• Describes which of R, G, B, and A are used in
  internal representation of texels
• Provides control over things texture can do
• High bit depth alpha blending
• High bit depth intensity mapping
• General purpose RGB
• GL doesnt guarantee all options are available on
  given hardware

30
glTexImage2D Args 4-6

• GLsizei width, GLsizei height
• Dimensions of texture image
• Must be 2m 2b (b0 or 1 depending on border)
• min, 64 x 64
• GLint border
• Width of border (1 or 0)
• Border allows linear blending between overlapping
  textures
• Useful when manually tiling textures

31
glTexImage2D Args 7-9

• GLenum format
• Describe how texture data is stored in input
  array
• GL_RGB, GL_RGBA, GL_BLUE
• GLenum type
• Data size of array components
• GL_SHORT, GL_BYTE, GL_INT
• Const GLvoid texels
• Pointer to data describing texture map

32
glGetTexImage()

• void glGetTexImage(GLenum target, GL_TEXTURE_2D
  target texture bufferGLint level, 0, 1, 2,
  level-of-detail for Mip-MapGLenum format, color
  format of pixels (R, G, B, A)GLenum type, type
  of pixel data (int, short, long, )GLvoid
  pixels texture buffer)

33
Example of Texture Mapping

• In display() function
• glEnable(GL_TEXTURE_2D)glBegin(GL_QUADS) glTex
  Coord2f(0.0, 0.0) glVertex2f(x1, y1,
  z1) glTexCoord2f(1.0, 0.0) glVertex2f(x2, y2,
  z2) glTexCoord2f(1.0, 1.0) glVertex2f(x3, y3,
  z3) glTexCoord2f(0.0, 1.0) glVertex3f(x4, y4,
  z4)glEnd( )

34
Repeat or Clamp

• First problem is what happens if we specify a
  value of s t outside the range (0.0, 1.0).
• Values below this interval cause the texture to
  repeat.
• Values above this interval cause the texture to
  clamp.
• For repeated textures we
• glTexParameter(GL_TEXTURE_WRAP_S, GL_REPEAT)
• For t we use GL_TEXTURE_WRAP_T
• For clamping we use GL_CLAMP

35
Aliasing (1/3)

• Aliasing is a second problem.
• Sometimes the texel is larger or smaller than a
  pixel.
• To magnify the texel
• glTexParameter(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTE
  R, GL_NEAREST)
• To minify the texel
• glTexParameter(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILT
  ER, GL_NEAREST)
• To get a more visuallypleasing image we use
  filtering to obtainan average texel value.
• OpenGL can do this if we specify GL_LINEAR
  instead of GL_NEAREST.

36
Aliasing (2/3)

• If 2nd param GL_TEXTURE_MIN_FILTER
• 3th param can be GL_NEAREST, point
  sampling GL_LINEAR, linear interpolationor
  MIPMAPs (texture pyramids) GL_NEAREST_MIPMAP
  _NEAREST
• GL_LINEAR_MIPMAP_NEAREST
• GL_NEAREST_MIPMAP_LINEAR
• GL_LINEAR_MIPMAP_LINEAR

½ ½
¼ ¼
... Size reduced until 1x1
37
Aliasing (3/3)
38
Aliasing with Filtering

• OpenGL tries to pick best mipmap level
• Question Which texel corresponds to a particular
  pixel?
• GL_NEAREST (Point Sampling)
• Pick the texel with center nearest pixel
• GL_LINEAR (Bilinear Sampling)
• Weighted average of 2x2 closest texels
• GL_NEAREST_MIPMAP_LINEAR
• Average nearest texels from two mipmap levels
• GL_LINEAR_MIPMAP_LINEAR (Trilinear)
• Average two averaged texels from two mipmaps

39
gluBuild2DMipMaps

• int gluBuild2DMipmaps(GLenum target, GL_TEXTURE_
  2D target texture bufferGLint level, 0, 1, 2,
  level-of-detail for Mip-MapGLint
  components, 0, 1, 2, 3 componentsGLsizei
  width, width of textureGLsizei height, height
  of textureGLint border, 0 border widthGLenum
  format, color format of pixels (R, G, B,
  A)GLenum type, type of pixel data (int, short,
  long, )const GLvoid pixels source color
  array)
• For a 64 x 64 original array we can set up 32 x
  32, 16 x 16, 8 x 8, 4 x 4, 2 x2, 1 x 1 arrays
  through
• gluBuild2DMipMaps ( GL_TEXTURE_2D, 3, 64,
  64,GL_RGB, GL_UNSIGNED_BYTE, my_texels )

40
MIPMAPS

• With versus without MIPMAP

41
Texture and Shading

• A forth problem is the interaction between
  texture and shading.
• Modulation Solution
• void glTexEnvi(
• GLenum target, GL_TEXTURE_2D target texture
  buffer
• GLenum pname, GL_TEX_ENV_MODE
• GLint param GL_MODULATE, GL_DECAL
• )
• Example
• glTexEnv(GL_TEX_ENV, GL_ENV_MODE, GL_MODULATE)
• Decaling Solution
• If we replace GL_MODULATE by GL_DECAL, the color
  of the texture determines the color of the object.

42
Perspective Projection

• Proper texture mapping also depends on what type
  of projection is used.
• Default is Linear Interpolation.
• It does not work well with perspective
  projections.
• We can use a better interpolation by
• glHint(GL_PERPECTIVE_CORRECTION, GL_NICEST)

43
Environmental Maps
44
Environment Mapping

• Blinn, Models of light reflection for computer
  synthesized pictures, Computer Graphics,
  11(2)192198, 1977.
• Greene, Environment mapping and other
  applications of world projections, IEEE CGA,
  6(11)2129, 1986.
• Reflection mapping
• It does not use ray tracing methods.

environment texture mapping
shiny object
shiny object
object
object
camera
camera
ray tracing
environment mapping
45
Environment Mapping
46
Ray Tracing vs. Environment Map
environment map
ray tracing
47
Environment Mapping

• 2 stage algorithm

R reflection vector
Environment map Texture mapping
Environment map (intermediate object)
48
Environment Mapping

• Used to model a object that reflects surrounding
  textures to the eye
• Polished sphere reflects walls and ceiling
  textures
• Cyborg in Terminator 2 reflects flaming
  destruction
• Texture is distorted fish-eye view of environment
• Spherical texture mapping creates texture
  coordinates that correctly index into this
  texture map

49
Sphere Mapping
50
Blinn/Newell Lattitude Mapping
51
Cube Mapping
52
Bump Maps
53
Bump Mapping

• Blinn, Simulation of wrinkled surfaces,
  Computer Graphics, 1978.
• Use textures to modify surface geometry
• Use texel values to modify surface normals of
  polygon
• Texel values correspond to height field
• Height field models a rough surface
• Partial derivative of bump map specifies change
  to surface normal

54
Bump Mapping
55
Bump Mapping Process
N(u,v)
P(u,v)
B(u,v)
P(u,v)
N(u,v)
56
Bump Mapping Process
N
N
P
D
57
Two-pass Bump Mapping

• McReynolds and Blythe, Programming with OpenGL
  Advanced Rendering, SIGGRAPH97 Lecture Notes.
• Key Idea NL NL DL
• NL Gouraud Shading
• DL image of surface detail approximation

58
Multirendering and The Accumulation Buffer
59
Accumulation Buffer

• A buffer for multirendering
• Same as frame buffer resolution
• Before operation
• glClear(GL_ACCUM_BUFFER_BIT)
• AccumBuffer ? 0
• void glAccum( GLenum op, GLfloat value )
• GL_LOAD AccumBuffer ? FrameBuffer
• GL_ACCUM AccumBuffer ? AccumBuffer value
  FrameBuffer
• GL_MULT AccumBuffer ? AccumBuffer value
• GL_RETURN FrameBuffer ? AccumBuffer

60
Accumulation Buffer

• clear
• make FrameBuffer with camera at (x, y)
• AccumBuffer ? AccumBuffer 1.0 FrameBuffer
• make FrameBuffer with camera at (x1, y)
• AccumBuffer ? AccumBuffer 1.0 FrameBuffer
• make FrameBuffer with camera at (x1, y)
• AccumBuffer ? AccumBuffer 1.0 FrameBuffer
• AccumBuffer ? AccumBuffer 0.33
• FrameBuffer ? AccumBuffer

1/3
1/3
1/3
61
Accumulation Buffer

• glClear( GL_ACCUM_BUFFER_BIT)for (i0
  iltnum_images i) glClear( GL_COLOR_BUFFER_BIT
  GL_DEPTH_BUFFER_BIT
  ) display_image(i) glAccum( GL_ACCUM, 1.0/
  (float) num_images )
• glAccum( GL_RETURN, 1.0)

62
Other Multipass Methods Motion Blur
t 0.3
t 0.2
t 0.1
motion blurred image
t 0.0
another example
63
Depth of Field

• When camera focused an object, near and far
  objects are blured.
• Accumulation buffercan be used for to generate
  this effect.

focused view
camera 1
camera 2
camera 3
Accumulation
64
Sampling and Aliasing
65
Nyquist sampling theorem

• The ideal samples of a continuous function
  contain all the information in the original
  function if and only if the continuous function
  is sampled at a frequency at least twice the
  highest frequency in the function.
• We can reconstruct a continuous function f(x)
  from its samples fi by the formula

Aliasing of sinusoid
66
Images

• An image is a 2D function I(x, y) that specifies
  intensity for each point (x, y)

67
Sampling and Image

• Our goal is to convert the continuous image to a
  discrete set of samples
• The graphics systems display hardware will
  attempt to reconvert the samples into a
  continuous image reconstruction

68
Point Sampling an Image

• Simplest sampling is on a grid
• Sample dependssolely on valueat grid points

69
Point Sampling

• Multiply sample grid by image intensity to obtain
  a discrete set of points, or samples.

Sampling Geometry
70
Sampling Errors

• Some objects missed entirely, others poorly
  sampled

71
Fixing Sampling Errors

• Supersampling
• Take more than one sample for each pixel and
  combine them
• How many samples is enough?
• How do we know no features are lost?

150x15 to 100x10
200x20 to 100x10
300x30 to 100x10
400x40 to 100x10
72
Unweighted Area Sampling

• Average supersampled points
• All points are weighted equally

73
Weighted Area Sampling

• Points in pixel are weighted differently
• Flickering occurs as object movesacross display
• Overlapping regions eliminates flicker

74
How is this done today?Full Screen Antialiasing

• Nvidia GeForce2
• OpenGL render image 400 larger and supersample
• Direct3D render image 400 - 1600 larger
• Nvidia GeForce3
• Multisampling but with fancy overlaps
• Dont render at higher resolution
• Use one image, but combine values of neighboring
  pixels
• Beware of recognizable combination artifacts
• Human perception of patterns is too good

75
GeForce3

• Multisampling
• After each pixel is rendered, write pixel value
  to two different places in frame buffer

76
GeForce3 - Multisampling

• After rendering two copies of entire frame
• Shift pixels of Sample 2 left and up by ½ pixel
• Imagine laying Sample 2 (red) over Sample 1
  (black)

77
GeForce3 - Multisampling

• Resolve the two samples into one image by
  computingaverage between each pixel from Sample
  1 (black) andthe four pixels from Sample 2 (red)
  that are1/ sqrt(2) pixels away

78
GeForce3 - Multisampling

• No AA Multisampling

79
GeForce3 - Multisampling
4x Supersample Multisampling