Advanced Multimedia Coding, Communication, and Processing

1
Advanced Multimedia Coding, Communication, and
Processing
Dr. Ming Yang Department of Mathematical and
Computer Sciences Jacksonville State University,
Jacksonville, AL 36265
2
Outline

• Video Streaming over IP Networks
• Motivation in this Research
• Proposed Methodology
• High Bitrate Information Embedding
• Network Modeling with Markov Model
• Video Data Loss Recovery
• Summary and Future Research Plan

2
3
Video-over-IP
3
4
Digital Video Coding
4
5
Video Frame Damage
Lost Packet
Delivered Packet

• Mapping between packet loss and slice/frame
  damage
• Slice consecutive macroblocks (MB) in the
  raster scanning order

5
6
Error Propagation
B
B
P
B
B
P
B
B
P
B
B
I
B
B
P
B
B
P
B
B
P
B
B
I
Coding/Decoding Dependencies
Group Of Pictures
Group Of Pictures
Damaged Frame due to Packet Loss
Complete Frame
Infected Frame
P
B
P
B
B
B
B
I
B
P
B
B
B
P
B
6
7
Proposed Methodology

• Lost Packets Need to be Concealed/Recovered
• Retransmission ? not a good candidate for
  Real-Time/Interactive Applications
• Adding Redundancy ? Lower Compression Ratio and
  Additional Bandwidth
• Encoder-based Algorithms ? Modifications on
  Encoder/Decoder
• Decoder-based Algorithms ? Limited Performance
• Proposed Methodology
• Embed Redundant Information within Video Frames
• Avoid Lower Compression Ratio or Additional
  Bandwidth
• No Need to Modify Encoder/Decoder
• A Hybrid Methodology which Combines both
  Encoder-based Decoder-based Algorithms

7
8
System Architecture
8
9
Steganography Model

• Hiding Both the Content and the Very Existence
  of the Message
• Low Bit-rate Information Hiding Digital
  Watermarking
• High Bit-rate Information Hiding Covert
  Communication

• Cover-Object (I) Original Video Frames
• Message (M) Redundant Information
• Stego-Object (I) Video Frames with Embedded
  Information
• Communication Channel Video Encoder/Decoder, IP
  Networks

9
10
Text Image Hiding

• APPARENTLY NEUTRALS PROTEST IS THOROUGHLY
  DISCOUNTED AND IGNORED. ISMAN HARD HIT. BLOCKADE
  ISSUE AFFECTS PRETEXT FOR EMBARGO ON BY-PRODUCTS,
  EJECTING SUETS AND VEGETABLE OILS.
• A message sent by a spy in World War II.
• PERSHING SAILS FROM NY JUNE 1.
• Extract the second letter from each word.

10
11
Multimedia Steganography
11
Multimedia Information Security Cryptography
and Steganography
12
Visual Distortion

• Maximize the Benefits from Webers Law (HVS)
• Luminance Masking (Watson) Human Eyes are Less
  Sensitive to Changes in the Presence of Large
  Mean Luminance
• Contrast Masking (Watson) Human Eyes are not
  Sensitive to Certain Image Components in the
  Presence Similar Frequency and Orientation

12
Multimedia Information Security Cryptography
and Steganography
13
Embedding Strategy
A GOP (Group Of Pictures)
Video Codec
Extracting Algorithm (E)
Message (M)
I
B
B
P
B
B
P
B
B
P
B
B
Performance Analysis Comparison (BER)
Embedding Algorithm (E)
Message (M)
Luma Sample
Chroma Sample
13
14
Embedding Algorithm

• Channel Capacity 1 bit per 4x4 Block
• Minimizing Visual Distortion on Host Content
  (Subjective Objective Analysis)
• Robustness to Lossy Compression H.264/AVC Codec
• Blind Retrieval Information Retrieval without
  Original Content

14
15
Original vs. Stego Frames
Original Frames
Stego- Frames
Original Frames
Stego- Frames
15
16
Data Delivery Rate
16
17
Rate-Distortion Analysis
17
18
Network Modeling
18
19
Network Late Loss
Lost Packet
Delivered Packet
Delayed Packet
Packet Delay
Final Loss Pattern
19
20
VoD Transmission
RTP/UDP/IP Protocol Packet Size 1500 Bytes
Internet with Background Traffic
Ethernet
DSL
Ethernet
Remote Client
DSL
VoD Server
Local Client
Local Client
Experiments will be based on this trace (around
14 loss)
Negilible loss
20
21
Transition Matrix - Markov

• Six-Order General Markov Model
• of States 26 64
• of Transitions 64x2 128

21
22
Loss-Run and FEC
22
23
Video Frame Recovery
23
24
Hiding Schemes
Channel Capacity of Information Embedding 1 Bit
per 4x4 Block For a CIF Frame (352x288) 6336
(352x288/4x4) Bits can be Delivered
Time Axis
Protected Frame (also as Host Frame for other
Protected Frames)
1-bit Protection
2-bit Protection
Purely Host Frame
Embedding Dependency
4-bit Protection
8-bit Protection
24
25
Video Down-Sampling
Recovered 4x4 Block
n number of protection bits
Down-Sampling
Up-Sampling
Delivery
Redundant Information Generation
In the reconstruction, each 4x4 block is
estimated by replacing each pixel with the value d
4x4 Block

• An Estimation of Protected Frame has to be
  Generated
• Estimation is done through Down-Sampling and
  Quantization
• Sender Side Frame Down-Sampling (4x4 Block)
• Receiver Side Up-Sampling (4x4 Block) to Recover
  Protected Block

25
26
Error Localization
4 Bits Protection
4 Bits Protection
A Macroblock within Host Frame 1
4x4 Blocks within Protected-frame
4x4 Blocks within Host-frames
26
27
Subjective Quality
Damaged Frame
1-bit Protection
2-bit Protection
4-bit Protection
8-bit Protection
27
28
Objective Quality
coastguard-qcif.yuv
foreman-qcif.yuv
28
29
foreman-qcif _at_ 475 bps
Maximum Gain at 2-Bit Protection
Minimum Gain at 8-Bit Protection
As Packet Loss Increases, Performance Gain
Increases
As Packet Loss Increases, Subjective Qualities
Decreases
Maximum Gain 1.0 dB
Minimum Gain 0.35 dB
As Number of Protection Bits Increases Protected
Frame can be Recovered with a Higher Quality
Less Frames can be Protected.
0-Bit Protection Damaged Macroblock Recovered
with Spatial Interpolation in H.264/AVC Reference
Software
29
30
Intensity Adjustment
Macroblock (MB) Indentified from Reference Frame
Adjusted Macroblock (MB)
Adjust Average Intensity for each 4x4 Block
Motion Vector Recovery based Methodologies
Intensity Adjustment
4x4 Block

• Combination with Motion Vector Recovery based
  Algorithms
• FC Frame Copy based error concealment
• FC-ADJ FC combined with Y-samples intensity
  adjustment
• MVR Motion Vector Recovery based error
  concealment
• MVR-ADJ MVR combined with Y-samples intensity
  adjustment

30
31
R-D Analysis (10 Loss)
31
32
R-D Analysis (20 Loss)
32
33
Summary

• Information Hiding Causes No Visual Distortion
• Embedded Information Can be Reliably Transmitted
  and Used for Video Recovery
• Compression Performance is not Affected
• Modification on Encoder/Decoder is Avoided
• Video Data Can be Recovered with a Higher Quality
  Compared to Existing Methodologies
• Developed Methodology has been Successfully
  Applied to Medical Imaging Security and Privacy,
  Audio/Video Synchronization

33
34
Audio/Video Synchronization
34
Advanced Multimedia Coding, Processing,
Communication
35
Medical Imaging Security
35
Advanced Multimedia Coding, Processing,
Communication
36
THANK YOU!April-11-2009
36