Full Body 3D Scanning

1
Full Body 3D Scanning

• Team KIPPA
• Sam Calabrese, Abhishek Gandhi, Changyin Zhou
• smc2171, asg2160, cz2166_at_columbia.edu

2
Outline

• Background
• Motivation
• Our Plan
• Data Capture
• Data Processing
• Result Comparison
• Discussion

3
Background
Background Motivation Our Plan Data Capture
Data Processing Result Comparison

• Laser Scanner using TOF
• High precision, long range, slow
• Laser Scanner using triangulation method
• High precision, smaller range, occlusion problem,
  slow
• Image-based method using triangulation method
  (motion, stereo, focus/defocus, shading)
• Relatively low precision and resolution
• Pose assumption to the surface
• Fast (can be real-time)
• .. With Structured Light
• Improve the precision and resolution
• Indoor

4
Motivation
Background Motivation Our Plan Data Capture
Data Processing Result Comparison

• Lots of people dream to have an accurate 3D model
  of themselves
• Lots of applications with the real 3D model
  (Animation, Augmented Realistic and even Clothes
  Design)
• Difficulties
• Laser scanner too slow to scan a live person
  (moving and non-rigid)
• Image-based method not enough resolution and
  precision (even with structured light)
• Laser may hurt the eyes

5
Our Plan
Background Motivation Our Plan Data Capture
Data Processing Result Comparison

• Image-based method to model the head
• Laser scanner to capture the body
• Proper experiment settings to minimize the model
  movement during the scanning
• Proper post-process to tolerant slight movement
• Software to merge them together
• Map skins and do animation afterward

6
Data Capture - Head
Background Motivation Our Plan Data Capture
Data Processing Result Comparison
FaceGen www.facegen.com Use face symmetric, a
large set of 3D Face models (gender, ages,
races..)
7
Data Capture -Body
Background Motivation Our Plan Data Capture
Data Processing Result Comparison

• A professional model a large private room 4
  hours
• Tripods to rest the arms (10 cm lower than the
  shoulders)
• Mark the feet position
• A camera to track the movement
• Scanner is as high as the shoulder
• The distance to scanner ranges from 2.5m to 3m

8
Data Capture Body (contd)
Background Motivation Our Plan Data Capture
Data Processing Result Comparison

• Four scan points (Frontal/Back x Left/Right)
• Ten targets around for further registration
• Precision set to 2mm
• 10 min for each scan, but a long time to move
  the scanner
• 90 130 K vertices for each range data

9
Data Capture Body (contd)
Background Motivation Our Plan Data Capture
Data Processing Result Comparison

• Registration with Cyclone
• The body movement causes big problems
• we need more stronger non-rigid mesh merging
  methods

10
MeshLab
Background Motivation Our Plan Data Capture
Data Processing Result Comparison

• Initially used to convert PTX to PLY
• Problems arose with the original PTX files from
  Cyclone
• Used MeshLab to reorient
• Used again to clean data and resurface after VRip
  surfacing

11
Scanalyze
Background Motivation Our Plan Data Capture
Data Processing Result Comparison

• Used to re-register the Range scans following the
  initial errors
• Prepares for VRip directly by saving out .conf
  and .xf files which VRip reads to orient the
  different scans to each other while maintaining
  their original orientation

12
VRip
Background Motivation Our Plan Data Capture
Data Processing Result Comparison

• Used to create a surface from the registered
  point cloud
• Uses view direction and ramp weights to get
  confidence about each vertex
• Sampled at .002m in each direction per voxel
• Used ramp weight of .004m with slight increase in
  standard weights

13
PlyCrunch and Meshlab (again)
Background Motivation Our Plan Data Capture
Data Processing Result Comparison

• PlyCrunch is Packaged with VRip
• Decimated the Mesh from 3.5 million polygons to
  just over 10,000
• Full of holes because Plycrunch deleted
  Triangles, but left proper vertices.
• Meshlab used to clean and resurface resulting
  mesh, filled most of the holes

14
3dsMax
Background Motivation Our Plan Data Capture
Data Processing Result Comparison

• Used to finalize mesh by capping the remaining
  holes and smoothing the result.
• Used to attach the head from facegen
• Attached a skin shader for Mental Ray, one of the
  built in Renderers which used Sub-Surface
  Scattering for realism, modified to match the
  texture from FaceGen
• Rigged with Biped object
• Animated with stock Motion Capture Data
• Rendered into animations

15
3dsMax
Background Motivation Our Plan Data Capture
Data Processing Result Comparison
16
Result Comparison
Background Motivation Our Plan Data Capture
Data Processing Result Comparison

• We compared our output with that of two
  professional companies
• Headus based in Australia
• AvatarMe (initially developed by University of
  Surrey)
• Criteria for Comparison
• Resolution of the output
• Accuracy of scanned data
• Error prevention during scanning
• Cost/Ease of setup

17
Resolution of the output
Background Motivation Our Plan Data Capture
Data Processing Result Comparison

• We made way for Quality via Quantity
• KIPPA
• Number of Polygons in original scan
• Scan 1 168,855
• Scan 2 179,868
• Scan 3 174,686
• Scan 4 101,010
• Total number of points 500,000
• Data Density 2mm
• Others
• Data Density 4 mm (Headus)
• Average number of points 300,000

18
Resolution of the output
Background Motivation Our Plan Data Capture
Data Processing Result Comparison

• AvatarMe KIPPA

19
Accuracy of Scanned Data(smoothness, details
captured)
Background Motivation Our Plan Data Capture
Data Processing Result Comparison

• Used more than just one good software

Vrip- 3,858,996
MeshLab- 3,858,783
PlyCruch- 10,019
MeshLab- 15,056
3ds Max- 15,080
93,108
20
Error prevention during scanning Cost/Ease of
setup
Background Motivation Our Plan Data Capture
Data Processing Result Comparison

• KIPPA
• We didnt have a pre-defined model/shape against
  which to map our data.(explain)
• Cheap/Simple set-up
• Separate scanning for head and body
• Limited body movement below the neck
• Others
• WBX Platform(explain)
• (movable platform, cables, etc.)
• http//www.cyberware.com/documentation/digisize/ww
  w/info/WBXPlatform.html

21
Discussion Conclusion

• Reduction and dealing with motion
• Repositioning the scanner was difficult
• Problems during transformation
• Efficient use of markers
• Ignored hair
• Missed details of hands and toes
• To conclude,
• 3D human scan was a very interesting problem to
  deal with. We managed to get a satisfying output
  in a very efficient manner.
• It was enjoyable learning and using softwares
  such as Vrip, Cyclone, MeshLab, PlyCrunch and 3ds
  Max.

22
References Resources

• ALLEN, P., 2007. 3d photography 2007 fall. Class
  notes on Active 3D Sensing.
• ALOIMONOS, Y., AND SPETSAKIS, M. 1989. A unified
  theory of structure from motion.
• BERALDIN, J., BLAIS, F., COURNOYER, L., GODIN,
  G., AND RIOUX, M. 2000. Active 3D Sensing.
  Modelli E Metodi per lo studio e la conservazione
  dellarchitettura storica, 2246.
• BLAIS, F., PICARD, M., AND GODIN, G. 2004.
  Accurate 3d acquisition of freely moving objects.
  In 3DPVT04, 422429.
• BLAIS, F. 2004. Review of 20 years of range
  sensor development. Journal of Electronic Imaging
  13, 231.
• DHOND, U., AND AGGARWAL, J. 1989. Structure from
  stereo-a review. Systems, Man and Cybernetics,
  IEEE Transactions on 19, 6, 14891510.
• DU, H., ZOU, D., AND CHEN, Y. Q. 2007. Relative
  epipolar motion of tracked features for
  correspondence in binocular stereo. In IEEE
  International Conference on Computer Vision
  (ICCV).
• NAYAR, S., WATANABE, M., AND NOGUCHI, M. 1996.
  Realtime focus range sensor. IEEE Transactions on
  Pattern Analysis and Machine Intelligence 18, 12,
  11861198.
• RUSINKIEWICZ, S., HALL-HOLT, O., AND LEVOY, M.
  2002. Real-time 3D model acquisition. Proceedings
  of the 29th annual conference on Computer
  graphics and interactive techniques, 438446.
• SCHECHNER, Y., AND KIRYATI, N. 2000. Depth from
  Defocus vs. Stereo How Different Really Are
  They? International Journal of Computer Vision
  39, 2, 141162.
• WATANABE, M., AND NAYAR, S. 1998. Rational
  Filters for Passive Depth from Defocus.
  International Journal of Computer Vision 27, 3,
  203225.
• ZHANG, R., TSAI, P., CRYER, J., AND SHAH, M.
  1999. Shape from shading A survey. IEEE
  Transactions on Pattern Analysis and Machine
  Intelligence 21, 8, 690706.
• Facegen http//www.facegen.com, Meshlab
  http//meshlab.sourceforge.net/
• Vrip, Scanalyze, Plycrunch http//graphics.stanfo
  rd.edu/software/vrip/, 3Ds Max
  www.autodesk.com/3dsmax
• Prometheus http//personal.ee.surrey.ac.uk/Perso
  nal/A.Hilton/research/PrometheusResults/index.html
  
• Headus www.headus.com/au/3D_scans/index.html
• TC Square http//www.tc2.com/what/bodyscan/index
  .html
• Cornell University Body Scan http//www.bodyscan
  .human.cornell.edu/scene0037.html

23

• Thanks to Prof. Allen, Karan, Matei and Paul for
  your kindly help and support.
• Special thanks to our model, Daniel, for his
  professional, passion and great cooperation.

24

• Any Questions
• Thank You
• (team KIPPA)