Perceived video quality measurement

1
Perceived video quality measurement

• Muhammad Saqib Ilyas
• CS 584 Spring 2005

2
Agenda

• Motivation
• Subjective vs. objective measures
• Mean Square Error based metrics
• Recommended framework
• Perceived video quality metrics
• Structural distortion based metrics
• Other video quality metrics
• Conclusions

3
Motivation

• Background in voice over IP
• PSQM (Perceived Speech Quality Measurement)
• Improving voice over IP quality over slow WAN
  links

4
Subjective measures

• Mean opinion score
• Most reliable
• Slow
• Expensive

5
MSE based metrics

• Mean Square Error
• Peak SNR

6
Recommended framework
7
Pre-processing

• Temporal alignment
• Compression, processing and transmission
• Color space transformation
• Visual blur in HVS
• Linear space invariant lowpass characterized by
  PSF (Point Spread Function)
• Light adaptation
• Webers law

8
Contrast Sensitivity Function

• Models sensitivity to different spatial and
  temporal frequencies
• Bandpass in nature
• Implementation
• Filter
• Weighting factors after channel decomposition
• Mostly lowpass filters used
• More robust to changes in viewing distance
• Temporal filters

9
Channel decomposition

• Visual cortex neurons tuned to
• spatial and temporal frequencies
• Orientation
• Direction of motion
• Neuron modeled as 2D Gabor function
• Collection of neurons modeled as octave band
  Gabor filter bank
• Spatial spectrum sampled at
• Octave intervals in radial frequency dimension
• Uniform intervals in orientation dimension

10
Channel decomposition

• Visual cortex neuron output saturates with
  increase in contrast
• Typically modeled based on application and
  computational constraints
• Sophisticated channel decomposition
• Wavelet transforms
• DCT

11
Error normalization and masking

• Masking/facilitation
• Presence of one image component will
  decrease/increase visibility of another image
  component at the same spatial location
• Strongest when two signals have the same
  frequency components and orientation
• Implementation
• Gain control space-varying visibility threshold
  for the particular channel

12
Error normalization and masking

• Base error threshold for every channel is
  elevated to account for the presence of the
  reference signal
• Elevated threshold used to normalize error signal
  into JND

13
Error pooling

• Combines error values from various channels into
  one
• Typically Minkowski pooling is used

• ei,k is the normalized and masked error of the
  k-th coefficient in the i-th channel
• ß is a constant typically with a value between 1
  and 4

14
Video distortion meter

• Image quality assessment
• Cognitive emulator
• Asymmetric tracking
• Humans detect quality transition from good to
  poor more readily

15
Multi-metric MPEG quality

• Combines
• Error sensitivity based metric
• Blockiness detection

16
Digital Video Quality (DVQ)

• Simplicity is key consideration
• LC Local contrast is ratio of DCT amplitudes to
  DC amplitude for a block
• CM Contrast masking

17
Others

• Moving picture quality metric
• Color space based metric
• Blocking artifact based metric

18
Critique

• Computational complexity
• Memory requirement
• Viewing resolution
• Resolution of display device
• Digital pixel luminance value non-linear
  relationship
• Viewing distance
• Reliance on linear channel decomposition
• Correlation between channels modeled using
  sophisticated masking techniques
• Current masking models inaccurate

19
Structural Similarity Index Metric (SSIM)

• Different kinds of image distortion have
  different perceived quality
• Metrics discussed so far measure error
• Error and structural distortion agree quite often
• But the same amount of error may lead to
  different structural distortion
• Bottom-up approach
• Simulate the hypothesized functionality of the
  overall HVS

20
Reduced reference metrics

• Discussed in proposal presentation
• Based on temporal and spatial dissimilarity
  information

21
No reference / blind

• Complications
• Unquantifiable factors when reference is not
  available include but not limited to
• Aesthetics
• Cognitive relevance
• Learning
• Visual context
• Philosophy
• All images/videos are perfect unless distorted
  during
• Acquisition
• Processing
• Reproduction

22
No reference

• Determining the possible distortion introduced
  during these stages
• Reference is perfect natural images/videos
• Measured with respect to a model best suited to a
  given distortion type or application
• E.g., natural images/videos do not contain
  blocking artifacts
• To improve prediction
• Some HVS aspects are also modeled
• Texture and luminance masking

23
Other metrics

• Marker bits hidden in video frames
• Marker bits additionally transmitted on an aux
  channel
• Watermarking

24
Conclusion

• Perceptual quality measures not yet mature
• Computational complexity a big hurdle especially
  for real-time applications
• Accuracy of models also doubtful
• where pin-point accuracy is not required
• Less accurate RR or NR metrics may be used in