Detecting usual and unusual events from video streams

1
Detecting usual and unusual events from video
streams

• Dan Kong
• 09/02/2005.

2
Outline.

• Introduction
• Model-based approach.
• Unsupervised approach.

3
What is event detection?

• An intelligent engine that can bridge image
  features and semantic description of meaningful
  activities.
• Event detection is built on
• Moving objects detection.
• Tracking (blobs, trajectories etc.)
• Applications
• Surveillance.
• Content-based video retrieval.
• Video segmentation.

4
What is event?

• something that happens at a given place and
  time
• Two types of events
• Object domain events
• Frame or shot domain events.
• In each domain, we can further define
• Usual events.
• Unusual events.

5
Why event detection is hard?

• Uncertainty in detection and tracking
• Insufficient discriminating information between
  moving objects.
• The number of events surpass what we can expect.
• It is hard to specify what is usual/unusual event
  in an unconstrained environment.

6
Two approaches

• Model-based approach
• Fitting model for normal activities using
  extracted features.
• Unusual events are detected by maching against
  learned models.
• Unsupervised approach
• Usual events high recurrence of events that are
  similar.
• Unusual events group of events that are not
  similar to the rest.

7
Learning the Distribution of Object Trajectories
for Event Detection. Nei Johnson and David
Hogg The university of Leeds.
8
Raw data

• 2D image trajectories of moving objects within
  the scene
• Obtained from object tracker.
• Trajectories representation
• A sequence of flow vectors
• Normalized between 0,1
• Object existed for frames is represented
  as

9
Raw Data example
10
Modelling pdf

• Vector Quantisation
• Modelling pdfs by the point distribution of
  prototype vectors.
• Implemented using competitive neural learning
  network.

11
Vector Quantisation

• The competitive neural network implements the
  following algorithm
• Randomly place prototypes within the feature
  space.
• Initialize learning rate parameter between
  (0,1).
• Let be the input feature vector for this
  epoch.
• Find the prototype which is nearest to
  this input by the Euclidean metric

12
Vector Quantisation (contd)

• Update prototypes as follows
• Decrease with a cooling schedule.
• Repeat steps 3-6 for many epochs.

13
Pdf of flow vectors
4 input nodes, 1000 output nodes
14
Modelling the Pdf of trajectories

• Sequences of different lengths are modeled.
• Sequences which are similar should be close in
  the vector space of the representation and vice
  versa.
• Using vector quantization by adding a leaky
  neuron layer.

15
Leaky Neurons

• Leaky neurons have a memory of previous
  activations.
• A leaky neuron has a single input and a single
  output
• A leaky neuron with a slow decay rate will retain
  a trace of its highest input.

16
Approximate pdf of trajectories

• Use vector quantisation to place prototypes
  within the vector space of the leaky neuron
  outputs.
• The second vector quantisation is implemented by
  attaching a second competitive learning network
  to the leaky neuron layer.

17
Experimental results
18
Experimental results
19
Event recognition

• Assessment of the typicality of instantaneous
  movements and trajectories statistically
• Label each prototype with its local probability
  density.
• Recognition of simple and complex events can be
  achieved by attaching semantics or meaning to
  area of the distributions
• Trajectory prediction

20
Event Detection by Eigenvector Decomposition
Using Object and Frame Features Fatih Porikli
Tetsuji Haga Mitsubishi Electronic Research
Laboratories
21
Motivation

• Utilize the hard to describe but easy to
  verify property of visual events.
• Compare each event with all other events observed
  to determine how many similar events exist.
• Event detection becomes a problem of compare two
  events and measure their similarity.

22
Classify tracking features

• Object based features.
• Histograms (aspect ration, orientation, speed,
  color, size).
• HMMs (coordinate, orientation, speed).
• Scalar (duration, length, global direction).
• Frame based features.
• Histograms (orientation, location, speed)
• Scalar (number of objects, size)

23
HMM Representation

• Replace trajectory information as the emitted
  observable output of HMM.
• The state sequence that maximize the probability
  becomes the corresponding model for the given
  trajectory.
• We denote a HMM model as
• The optimum of states depend on the complexity
  and duration of the trajectories.

24
Features to Events

• Construct affinity matrices.
• Usual events are detected by analyzing the
  affinity matrices and find the object clusters.
• Unusual events are detected by analyzing the
  affinity matrices and order the objects with
  respect to their conformity scores.

25
Affinity Matrix Construction

• For each feature, an affinity matrix A is
  constructed. The elements of this matrix are
  equal to the similarity of the corresponding
  objects and . The similarity is defined as
• In case of HMM parameter based feature

26
Usual events detection
27
Spectral Clustering Algorithm
28
Estimating Cluster Numbers

• After each eigenvalue computation, a validity
  score is computed using clustering results.
• Determine correct cluster number by evaluating
  the first local maximum of this score.

29
Unusual Events Detection
30
Unusual Events Detection

• The conformity score of an object for a given
  feature is the sum of the corresponding row of
  the affinity matrix that belong that feature.
• Different feature are combined using a simple
  weighted sum approach.
• Objects are ordered with respect to its total
  conformity score. The object that has the minimum
  score corresponds to most different, thus most
  unusual events.

31
(No Transcript)
32
(No Transcript)
33
(No Transcript)
34
(No Transcript)
35
Thanks!