Title: CS6534 Guided Studies Crowd Flow Analysis
1CS6534 Guided StudiesCrowd Flow Analysis
- Supervised by Dr. Hau-San WONG
- Prepared by Kam-fung YU
- (51150118)
2Background challenge
- Video Surveillance System are widely used for
monitoring - Performance is good as the number of object for
detection is small (Spatial variation is small)
and - The change over time is small (Temporal variation
is small) - BUT
- A challenge for crowd flow
- The number of objects is in the order of 102103
- The change of the scene is very fast
3Related works
- Based on Tracking of Individuals
- Shape and Color Model of Individuals
- Trajectories of Points
- Boundary Contour
- xt Slices of Spatio-temporal Video Volume
- People Counting in the Crowd
4Shape color model of individuals
3D human model
- Models human shape by using 3D model
- Data-driven Markov chain Monte Carlo (DDMCMC)
- Iterate an optimized solution
T. Zhao et. al., Bayesian Human Segmentation in
Crowded Situations, IEEE CVPR03, 2003.
5Trajectories of points
- Similar method with Shape Color Model
- Use some simple feature, such as corner of an
object, to extract points probabilistically - Clustering the points into independently moving
entities, cluster
Shape and Color Model
Trajectories of Points
G. Brostow et. al., Unsupervised Bayesian
Detetcion of Independent Motion in Crowds, IEEE
CVPR, 2006.
6Boundary contour
- Use of low-interest points to detect the object
clustering - Select by the high temporal and spatial
discontinuity - Outline the object by joining edges
Clustered object
Sample Scene
P. Tu et. al., Crowd Segmentation through
Emergent Labeling, In ECCV Workshop SMVP, 2004
7Xt sclies of spatio-temporal video volume
- Scan interesting lines over a certain frames,
xt-slice - Use the Hough transform to detect movement in the
xt slices
5 corresponding xt slices
Sample Scene with 5 lines
Hough transform
P. Reisman, Crowd Detection in Video Sequences,
IEEE Intelligent Vehicles Symposium, 2004.
8People counting in the crowd
- Clustering of some feature points by their motion
- Estimate the number of people by the number of
cluster
A result of clustering on two video scene
V. Rabaud et. al., Counting Crowded Moving
Objects, IEEE CVPR, 2006
9Limitations on tracking of individuals
- Involes Iteration
- Convergence Decease as Number of Objects Increase
- Large Computational Time
- High of Computational Power
- Difficulty to implement on Real Time Monitoring
System
10Our approach
- Proposed by Saad Ali Mubarak Shah in 2007
- Individual Flow ? Global Optical Flow
- Tracking Individuals ? Measuring Global
Quantities - Using Fluid Dynamics to treat the problem
- Global QuantitiesFinite Time Lyapunov Exponents
Field (FTLE), Lagrangian Coherent Structures(LCS) - Expect a Higher Faster Algorithm in Performance
11Algorithmic outline
12Optical flow
- 16x16 size block
- Displacement vector x
- p frames for 1 mean field
- q mean field for 1 block mean field
S. Ali, M. Shah, A Lagrangian Particle Dynamics
Approach for Crowd Flow Segmentation and
Stability Analysis, CVPR May, 2007
13Algorithmic outline
14Flow map
- Launch a set of particles over the optical flow
field - Solve a flow map for a time period T p?q frames
- Interpolation a cubic velocity equation by 4th
order Runge-Kutta-Fehlberg algorithm (RK4) - ??x, ?y are used to record the x and y coordinate
at each initial position launched after time
Flow map of x-particle
Flow map of y-particle
S. Ali, M. Shah, A Lagrangian Particle Dynamics
Approach for Crowd Flow Segmentation and
Stability Analysis, CVPR May, 2007
15Algorithmic outline
16Ftle Field
- Compute the four spatial derivates
- Plug into the Cauchy-Green deformation tensor
- The largest finite time Lyapunov exponent with
the maxmum eigenvalue ?max of the tensor and the
period Tp?q frames
FTLE Field Plot
S. Ali, M. Shah, A Lagrangian Particle Dynamics
Approach for Crowd Flow Segmentation and
Stability Analysis, CVPR May, 2007
17Algorithmic outline
18lcs
FTLE Field Plot
S. Ali, M. Shah, A Lagrangian Particle Dynamics
Approach for Crowd Flow Segmentation and
Stability Analysis, CVPR May, 2007
19segmentation
- This process involves two stages
- Cut spatially into different region by the ridges
in FTLE - Use the Lyapunov divergence to decided two
segment merge or not
1st stage
2nd stage
S. Ali, M. Shah, A Lagrangian Particle Dynamics
Approach for Crowd Flow Segmentation and
Stability Analysis, CVPR May, 2007
20Algorithmic outline
21Flow instability detection
- Flow instability is defined as the change in the
number of flow segments with respect to time
New segment
Current segment
S. Ali, M. Shah, A Lagrangian Particle Dynamics
Approach for Crowd Flow Segmentation and
Stability Analysis, CVPR May, 2007
22Capabilities
- Capable for monitoring thousands of objects
simultaneously - Get rid of number of people constrain
- Capable for monitoring flow in any orientation
- Obtain same result under any rotation
- Capable for new segment detection over time
- Locate the increase or the decrease of segments
over time
23Potentials
- Potential for flow control or city design
- Making immediate decision for crowd flow
- Facilitate on the planning of city streets,
traffic flow, overhead, bridges and passageways - Potential for flow pattern recognition
- Extraction of various flow pattern
- Flow pattern solution space construction for a
given static scenery - Flow pattern bases finding
24limitations
- Limitation on crowd density
- Degraded as crowd density is low
- Worse at only a number of objects
- Limitation on a large number of many-fold
dynamics flow - Too many segments (too noisy) on the scene
- Hard to merge segment
- Limitation on a rapid unstable flow
- Hard to retrieval information from rapid changing
flow - Too slow to capture the information
25Further suggestions
- Find out the critical crowd density for an
acceptable performance - Finding out a method that can undergo
segmentation under a noisy domain - Designing a rapid flow capturing algorithm
- Finding out the possible flow patterns on given
static scenery - Find out the flow patterns solution space and
bases
26conclusion
- In this guided study, we studied about various
kinds of methods and the Lagrangian Dynamics in
solving the crowd segmentation problem. - We also realized the capabilities, potentials and
limitations . - We finally suggested some possible direction for
future studies.
27references
- Z.N. Li, M.S. Drew, Fundamentals of Multimedia,
NJ Pearson Education Hall, 2004 - P.E. Mattison, Practical Digital Video with
programming examples in C, NY John Wiley Sons
Inc, 1994 - L. Perko, Differential Equations and Dynamical
Systems 3rd Ed., NY Springer, 2001 - Intel Corporation, Open Source Computer Vision
Library, Reference Manual, USA Intel
Corporation, 2001 - S. Ali, M. Shah, A Lagrangian Particle Dynamics
Approach for Crowd Flow Segmentation and
Stability Analysis, CVPR May, 2007 - S. C. Shadden, Lagrangian Coherent Structures
Analysis of time-dependent dynamical systems
using finite-time Lyapunov exponents,Available
Online http//www.cds.caltech.edu/shawn/LCS-tut
orial/, Last update 15th April, 2005 - P. Reisman, Crowd Detection in Video Sequences,
IEEE Intelligent Vehicles Symposium, 2004. - T. Zhao et. al., Bayesian Human Segmentation in
Crowded Situations, IEEE CVPR03, 2003. - P. Tu et. al., Crowd Segmentation through
Emergent Labeling, In ECCV Workshop SMVP, 2004. - G. Brostow et. al., Unsupervised Bayesian
Detetcion of Independent Motion in Crowds, IEEE
CVPR, 2006. - D. Yang et. al., Counting People in Crowds with
a Real-Time Network of Simple Image Sensors,
ICCV, 2003. - V. Rabaud et. al., Counting Crowded Moving
Objects, IEEE CVPR, 2006. - E. Rosten and T. Drummond, Machine learning for
high-speed corner detection, Europe Conference
on Computer Vision, May 2006. - C. Tomasi and T. Kanade, Detection and tracking
of point features, Technical Report
CMU-CS-91-132, Carnegie Mellon University, April
1991. - S. Ali, Crowd Flow Segmentation Stability
Analysis, Available Online http//www.cs.ucf.ed
u/sali/Projects/CrowdSegmentation/index.html ,
Last visited 30th Nov, 2008
28Thank you
- Department of Computer Science
- City University of Hong Kong