Introduction to Computer Vision

1
Introduction to Computer Vision

• CS223B, Winter 2005

2
Richard Szeliski Guest Lecturer

• Ph. D., Carnegie Mellon, 1988
• Researcher, Cambridge ResearchLab at DEC,
  1990-1995
• Senior Researcher, InteractiveVisual Media
  Group, Microsoft, 1995-
• Research interests
• computer vision (stereo, motion),computer
  graphics (image-based rendering), parallel
  programming

3
What is Computer Vision?
4
What is Computer Vision?

• Image Understanding (AI, behavior)
• A sensor modality for robotics
• Computer emulation of human vision
• Inverse of Computer Graphics

5
Intersection of Vision and Graphics
6
Computer Vision TruccoVerri98
7
Image-Based Modeling
8
Related disciplines

• Image Processing
• Scientific / medical imaging
• Pattern Recognition
• Computer Graphics
• Learning
• Artificial Intelligence
• Visual Neuroscience
• Applied Mathematics

9
Mathematics

• What kinds of mathematics are used?
• Signal and image processing
• Euclidean and projective geometry
• Vector calculus
• Partial differential equations
• Optimization
• Probabilistic estimation

10
Syllabus

• What we will be covering in this course

11
Syllabus

• Image Transforms / Representations
• filters, pyramids, steerable filters
• warping and resampling
• blending
• image statistics, denoising/coding
• edge and feature detection

12
Image Pyramid
Lowpass Images

• Bandpass Images

13
Pyramid Blending
14
Parametric (global) warping

• Examples of parametric warps

aspect
rotation
translation
perspective
cylindrical
affine
15
Syllabus

• Optical Flow
• least squares regression
• iterative, coarse-to-fine
• parametric
• robust flow and mixture models
• layers, EM

16
Image Morphing
17
Syllabus

• Projective geometry
• points, lines, planes, transforms
• Camera calibration and pose
• point matching and tracking
• lens distortion
• Image registration
• mosaics

18
Panoramic Mosaics

19
Syllabus

• 3D structure from motion
• two frame techniques
• factorization of shape and motion
• bundle adjustment

20
3D Shape Reconstruction
Debevec, Taylor, and Malik, SIGGRAPH 1996
21
Face Modeling
22
Syllabus

• Stereo
• correspondence
• local methods
• global optimization

23
View Morphing

• Morph between pair of images using epipolar
  geometry Seitz Dyer, SIGGRAPH96

24
Z-keying mix live and synthetic

• Takeo Kanade, CMU (Stereo Machine)

25
Virtualized RealityTM

• Takeo Kanade, CMU
• collect video from 50 stream
• reconstruct 3D model sequenceshttp//www.cs
  .cmu.edu/afs/cs/project/VirtualizedR/www/Virtualiz
  edR.html

26
Virtualized RealityTM

• Takeo Kanade, CMU
• generate new video
• steerable version used for SuperBowl XXVeye
  vision system

27
Syllabus

• Tracking
• eigen-tracking
• on-line EM
• Kalman filter
• particle filtering
• appearance models

28
Syllabus

• Recognition
• subspaces and local invariance features
• face recognition
• color histograms
• textures
• Image editing
• segmentation
• curve tracking

29
Edge detection and editing
Elder, J. H. and R. M. Goldberg. "Image Editing
in the Contour Domain," Proc. IEEE Computer
Vision and Pattern Recognition, pp. 374-381,
June, 1998.
30
Image Enhancement

• High dynamic range photographyDebevec et
  al.97 Mitsunaga Nayar99
• combine several different exposures together

31
Syllabus

• Image-based rendering
• Lightfields and Lumigraphs
• concentric mosaics
• layered models
• video-based rendering

32
Concentric Mosaics

• Interpolate between several panoramas to give a
  3D depth effect
• Shum He, SIGGRAPH99

33
Applications

• Geometric reconstruction modeling, forensics,
  special effects (ILM, RealVis,2D3)
• Image and video editing (Avid, Adobe)
• Webcasting and Indexing Digital Video (Virage)
• Scientific / medical applications (GE)

34
Applications

• Tracking and surveillance (Sarnoff)
• Fingerprint recognition (Digital Persona)
• Biometrics / iris scans (Iridian Technologies)
• Vehicle safety (MobilEye)
• Drowning people (VisionIQ Inc)
• Optical motion capture (Vicon)

35
Projects

• Lets look at what students have done in previous
  years
• Stanford  http//www.stanford.edu/class/cs223b/win
  ter01-02/projects.html
• CMU  http//www-2.cs.cmu.edu/ph/869/www/869.html
  
• UW http//www.cs.washington.edu/education/courses/
  cse590ss/01wi/
• GA Tech  http//www.cc.gatech.edu/classes/AY2002/c
  s4480_spring/