3D Model Streaming Based on JPEG 2000

1
3D Model Streaming Based on JPEG 2000

• ??? M9507327 ???

2
Introduction

• 3D mesh streaming technique base on JPEG 2K
• Geometry Image (3D -gt 2D)
• JPEG 2K ROI, Progressive Compression and
  multiple component.

3
Introduction

• View-Dependent, Quality-Scalable,
  Multi-Resolution
• Using the JPIP protocol.
• X3D
• Image JPEG 2K
• Video Motion JPEG 2K
• 3D content Geomtry Image

4
JPEG 2000-BASED MESH STREAMING
5
Parameterization

• Geometry Image use RGB to perform 3D mesh
  geometry properties.
• Vertex Position
• Normal Maps
• Texture coordinates ( unnecessary )

6
Parameterization

• Surface ParameterizationF(x,y,z)(u,v)

7
Parameterization

• If the 3D model is Closed Manifold, use the cut
  method in 5 to find the proger boundary.
• The model is flattened to a 2D surface.
• Resample the grid points in the image and
  caculate the attributes with bi-linear
  interpolation.(vertex positions, normal mapsetc)

8
JPEG 2000 Coding

• JPEG 2K has lossless compression mode.
• JPEG 2K support multiple layer compression. Thus
  it can compresses all attributes into one image.
• JPEG 2K support progressive compression,
  decompression (good to achieve over the Internet)
• Arbitrary image block can be retrieved from a
  JPEG 2k image.

9
JPEG 2000 Coding

• Vertex positions ( X, Y, Z )
• Normal map ( Nx, Ny, Nz )

10
JPEG 2000 Coding

• Because JPEG 2K uses wavelet transform to
  compress the image into multiple quality
  layers.Image resolution low-gthigh
  (progressive)
• View-dependent use JPEG 2K ROIfirst divided
  image into several blocks (with size 8X8),
  blocks which are close to the users viewpoint
  can be selected to transmit first.

11
JPEG 2000 Image Transmission

• Focus window the users corrent spatial region.
• Transmitted by blocks, not pixels.(if user is
  keep changing the view points, then it wont
  accrue many fragments.)

12
JPEG 2000 Image Transmission

• Using W(l,r,c) to indicate a focus window.
• l((x,y),(w,h)) (x,y) location , (w,h)
  size
• rimage resolution level
• cindex of the image layer
• We assume there is only one image layer for easy
  explanation, we can only use W(l) to indicate the
  focus window

13
View-dependent

• Every block is

14
View-dependent

• Low resolution first, when low resolution
  translated complete, next resolution will do the
  same way.

15
View-dependent

• When user changes view point, the system will
  ignore the block which has already transmitted.

16
Multi-Resolution

• (r0..rmax )
• (resolution
  ??)
• Since the pixel at lower resolution level is
  generated by averaging 4 pixels at higher
  resolution levels.

17
Multi-Resolution

• When focus window reach es the highest
  resolution, the sequence of the following focus
  iwndows is decided as the DFS manner.

18
Multi-Resolution

• If the user change the viewport, the sequence
  will be recomputed to fit users demand.

19
X3D Extension

• The new streaming method can be employed by X3D
  and serves as a replacement to the node
  IndexedFaceSet

20
X3D Extension

• A multimedia content which contain images, videos
  and 3D scenes can be encoded as JPEG 2K images,
  Motion JPEG 2K videos, and X3D scenes. Or other
  codec.

21
Deforming Meshes Streaming

• Since a 3D model can be encoded as a JPEG 2K code
  stream, a 3D animation model which can be treated
  as deforming meshes can then be encoded as a
  Motion JPEG 2K code stream.
• Hence, to convert a 3D animation model to be a
  Motion JPEG 2K geometry video.

22
Deforming Meshes Streaming

• Convert each frame of the animation to be one
  JPEG 2K geometry image.
• To keep the Consistency , the vertices at the
  four corners of the squares should be the same in
  all frames.
• Animation -gtMotion JPEG 2K geometry Videoand
  able to tranmite by JPIP server.

23
Transmission Architecture

• Client-Server

24
Transmission Architecture

• P2P ( not by JPIP)
• JPEG 2K geometry images -gt Data Pieces.
• Since the P2P transmission can not guarantee the
  sequence of the tranmitted blocks, when viewport
  changes, node send out the block list.

25
Result
26
Result
27
Result
28
(No Transcript)
29
Result

• Original Angel model 2.3 MB
• Geometry Image 99KB

30
Future Work

• Even the lossless mode of JPEG 2K can compress
  the data with out losing the quality, when
  convert the 3D model,we have to cut the model
  first to find the boundary and this actually may
  cause the distortion or data loss.