Binary Compression Rates for ASCII Formats

1
Binary Compression RatesforASCII Formats
Martin Isenburg Jack Snoeyink University of
North Carolina at Chapel Hill
2
Take this home

• We can store compressed geometry as ASCIIand
  achieve benchmarkcompression rates.

Geometry compressiondoes not require binary.
This enables full conformancebetween
ASCII and binary X3D.
3
Overview

• Motivation
• Benchmark Compressor
• Storing Bits as ASCII
• Results
• More Motivation
• Demo
• Summary

4
Motivation
5
Why ASCII Scenes?

• authors like text-based 3D
• wide acceptance
• read, understand, modify, and save scene with
  any text editor
• Web3D Developer survey very important
• no binary standard
• rather readable ASCII than binary

6
Why Compression?

• authors want compressed 3D
• smaller files
• faster download
• Web3D Developer survey very important
• data-heavy nodes
• audio / images / video
• geometry

7
Approaches to Compression

• binary formats
• proprietary compression schemes
• automatically protects the content
• Shockwave3D, Viewpoint, Virtue3D, Cult3D ...
• gzipped text-based formats
• human readable ASCII content
• WRL, X3D, OBJ, PLY, SMF, OFF

8
Compression Standards

• software to read, edit, and write content is
  available and widely used
• binary compression standards
• audio data MP3
• image data JPEG, GIF
• movie data MPEG
• geometry data ?

9
Why no Standard yet? (1)

• ? structure of 3D data is complex
• audio sequence of numbers
• image block of numbers
• video sequence of blocks of numbers
• geometry vertex positions

triangles (or polygons ?) texcoords (1, 2,
4, 8 layers ?) normals (smoothing groups ?)
colors (per-face ? per-vertex ?) bones (1, 2,
3 attachments ?)
10
Why no Standard yet? (2)

• ? 3D data is much less used
• audio everybody
• image everybody
• video everybody
• geometry anybody ?
• ? 3D data is not a consumable product on its
  own
• ? profitability issues

11
Geometry compression X3D

• long wanted feature (96)
• compression requires binary VRML
• Compressed Binary Format workgroup
• binary VRML 5 new compressed nodes
• but CBF proposal refused (98)
• problem-child the binary specification
• second call for BF proposals (00)
• unanswered

12
Compression without Binary

• ? Web3D 2002 paper
• eliminate binary requirement
• only requires specification changes where it
  matters
• does not interfere with how authors publish
  content
• does not affect (overall) readability

Coding Polygon Meshes as Compressable
ASCII Isenburg Snoeyink, 02
13
Design Objective

• work with Shout3D API
• pure Java viewer (plugin-less)
• fast, light-weight decoding
• low computational complexity
• small size for decoder class
• be worthwhile

less than 400lines of code
5,381 bytes
1 6
14
ASCII coder binary coder

• How good is the compression?
• compare to benchmark coder
• oops
• New Design Objectivegtgtgt Maximum Compression
  ltltlt
• produce ASCII output
• compress with gzip -9
• be as good as benchmark

15
BenchmarkCompressor
16
Benchmark Compressor
Triangle Mesh Compression Touma Gotsman, 98

• Connectivity Coder
• codes connectivity as sequence of vertex degrees

• Geometry Coder
• codes geometry as sequence of corrective vectors

? only handles fully triangulated meshes
? no support for texture coordinates
17
Polygon Mesh Compressor
handling non-triangular meshes
18
Its available!

• a working Webimplementation
• Java applet
• based on node set of Shout3D
• WRL, OBJ, PLY,OFF, SMF,
• compress anymodel online

19
Binary vs. ASCII
binary
scene
difference
ASCII
lion wolf raptor horse cat dog
average
42.6 18.4 21.5 20.3 22.5 21.5
55 60 62 101 77 61 74
20
Mesh Compression Process

• grow region store symbols

21
Why is binary so much better?

• Because it
• is binary?
• stores symbols in binary?
• uses better predictiveencoding scheme?
• compresses symbols withentropy coding?
• allows to store a bit-stream?

No!
Not quite!
Sort of.
Yes, but
Yes!
22
Entropy Coders

• for a symbol sequence of t types

t
1
?
Entropy pi log2( ) bits
pi
i 1
of type t
pi
total
23
Contents of Binary File

• an array of bits
• 0 0 1 1 0 1 0 0 1 1 1 1 0 1 0 0 0 1 0 1 0
• six floats
• xmin -1.2323 ymin -2.1324 zmin -1.7123
• xmax 2.6731 ymax 0.8372 zmax 3.4971
• an integer
• precision 12

24
triceratops.wrl
Shape appearance Appearance material Material
diffuseColor 1 .5 0 geometry
IndexedFaceSet creaseAngle 0.9 coord
Coordinate point -0.0715 1.7609 ...
-0.4479 -1.5153 1.5304 coordIndex
7 6 209 204 -1 4 4577 4223 4222 -1
25
triceratops_compressed.wrl
Shape appearance Appearance material Material
diffuseColor 1 .5 0 geometry
CompressedIndexedFaceSet creaseAngle 0.9
code 0 0 1 1 1 0 1 1 0 1 1 1 0 1 1 0 1 1 1 0 1
1 1 0 1 1 0 1 1 1 1 0 1 1 1 0 1 0 0 0 1 1 1 0 1
1 0 1 1 0 1 1 0 bits 12 box -1.2323
-2.1324 -1.7123 2.6731 0.8372 3.4971
26
triceratops_compressed.wrl
Shape appearance Appearance material Material
diffuseColor 1 .5 0 geometry
CompressedIndexedFaceSet creaseAngle 0.9
code AQgN9gOdRhhlXdfzKEyz0CCZUJHLs3 fWpsTQF
xortsk8XW8j7jEZwL vXS7GVbHHUuX bits 12
box -1.2323 -2.1324 -1.7123 2.6731 0.8372
3.4971
27
Storing Bitsas ASCII
28
Design Goals

• resulting ASCII should be safe

• obvious choices
• uuencode
• base64
• Hexbin

• represent binary files with a set of 64 ASCII
  characters

• other possibilities 2 . 96 chars

29
Design Goals

• resulting ASCII should be safe
• conversion should be simple

• efficient implementation byusing 2k ASCII
  characters
• value of k bits
• look-up table

LT A, B, C,D, E, F,G, H,
, ,
0 0 1 1 0 1 0 0 1 1 1 1 0 1
30
Design Goals

• resulting ASCII should be safe
• conversion should be simple
• ASCII representation should be compact

32 chars 60.0 64 chars 33.0 96 chars
21.5
31
Design Goals

• resulting ASCII should be safe
• conversion should be simple
• ASCII representation should be compact
• size of ASCII representation of bits after
  gzipping should be close to original

32
Results
33
fish.wrl

• Shape appearance Appearance material
  Material modulateTextureWithDiffuse
  true diffuseColor 1 1 1 texture
  ImageTexture url fish.jpg geometry
  IndexedFaceSet creaseAngle 0.9 coord
  Coordinate point -0.0715 4.7609 6.3930 ...
  -0.4479 -4.5153 4.5304 coordIndex 7 6 209
  204 -1 4 217 4577 -1 4577 4223 4222 -1
  texCoord TextureCoordinate point 0.3735
  0.9441 0.3289 0.2666 0.4990 0.1082
  texCoordIndex 0 1 2 3 -1 4 5 6 7 4311
  -1 4311 4293 4683 -1

34
fish_compressed.wrl

• Shape appearance Appearance material
  Material modulateTextureWithDiffuse
  true diffuseColor 1 1 1 texture
  ImageTexture url fish.jpg geometry
  CompressedIndexedFaceSet creaseAngle
  0.9 code AQgN9gOdRhhlXdfzK20Eyz0CCZUJHMK8EKLs
  3QrP fWpsTQFxo4dPVVZ0pGK720hVQw3a9xXrtsk8XW4d8j7
  jEDfP7pC hVQw3a9xXrtPTsk8XW8j7jEDfP7pCgEofTPoTnfU
  YfTWzzAbc9eh Fji6UL0ZwL vXS7GVbHHUuX bits
  12 box -1.9725 -7.6861 -10 1.9725 7.6861
  10 bits_tex 10 box_tex 0.0146 0.0749
  0.9896 0.9834

35
resulting file sizes
without gzipping the text file
IFS
scene
ratio
comp IFS
lion wolf raptor horse cat dog
average
1,360 569 586 749 791 586
57 25 29 27 30 29
1 24 1 23 1 20 1 28 1 26 1 2
0 1 25
36
resulting file sizes
with gzipping the text file
IFS
scene
ratio
comp IFS
lion wolf raptor horse cat dog
average
442 183 200 266 267 186
43 19 22 21 23 22
1 10 1 10 1 9 1 13 1 11 1 8
1 11
37
Binary vs. ASCII
binary
scene
difference
ASCII
lion 42.6 43.2 1.3 wolf 18.4 18.7 1.8
raptor 21.5 21.9 1.7 horse 20.3 20.6
1.7 cat 22.5 22.8 1.6 dog 21.5 21.9 1.7
average 1.7
38
MoreMotivation
39
ASCII binary Compression
Scene.x3d
ASCII
binary
40
Complete Conformance

• compressed nodes exist in ASCII and binary
• going back and forth between the two is a very
  simple mapping
• same decompression algorithm used no matter if
  compressed node stored in ASCII or binary

41
Demo
gt local web lt
42
Summary
43
Summary (1)

• completely independent things
• ASCII format
• binary format
• use of compressed geometry nodes
• store compressed nodes in ASCII
• as compact as if stored in binary (after
  gzipping)

44
Summary (2)

• provide bit-field in ASCII formats
• same decompression algorithm for ASCII and binary
  version
• full conformance between ASCII and binary version
  of X3D --- --- including compressed nodes

? design/decide compression now without waiting
for a binary format !!!
45
Thank you.
46
(No Transcript)
47
Example XML node

• CompressedIndexedFaceSet IndexedFaceSet
  MFbyte out code NULL IC SFint32
  out bit 1..24 IC SFint32
  out bit_tex 1..16 IC SFint32
  out bit_col 1..16 IC SFint32
  out bit_nor 1..16 IC
• SFvec6f out box 1..24 IC
  SFvec4f out box_tex 0 0 1 1 IC
  SFvec6f out box_col 0 0 0 1 1 1 IC
  
• SFbool valid false IC