Title: Challenges in Wireless Multimedia
1Challenges in Wireless Multimedia
- CSE Department Seminar Series
- September 26, 2003
- Borko Furht
2OUTLINE
- Scalable Video Source Coding
- Channel Coding and Error Control
- Power-Aware Coding and Transmission Techniques
- Networking Issues
- Rate control
- Multimedia Security
- Application Virtual Workplace
3Mobile Internet Access
Internet Subscribers (millions)
Source Ericsson
Year
4Wireless Multimedia Architecture
5Bandwidth Problem
Bandwidth is like money and sex - only too much
seems to be enough. Arnold Penzias, former
chief scientist of Bell Labs
6Generations of WAN Air Interfaces Based on
Access Technologies
- 1G FDMA (Frequency Division Multiple Access)
- 1980s - each caller has a dedicated frequency
channel (3 callers use 3 channels) - 2G TDMA (Time Division Multiple Access) and GSM
(Groupe Speciale Mobile) - 1990s - callers timeshare a frequency channel (9
callers use 3 channels) - 3G CDMA (Code Division Multiple Access) and
WCDMA (Wide Code Division Multiple Access) - 1990s - callers use a shorter bandwidth
- 2000s - spread spectrum. Each code is spread,
randomly broken down and mixed (14 callers use
the full bandwidth of 1 channel)
7Data Services
2,000
8M-Commerce Applications
- Transaction Management
- Digital Content Delivery
- Telemetry Services
- Searching for Killer Applications!
9Transaction Management
- On-line shopping tailored to mobile phones and
PDAs - on-line catalogs
- shopping carts
- back office functions
- Initiate and pay for purchases and services
- Micro-transactions - subway fees, digital cash
10Digital Content Delivery
- Information browsing
- weather
- transit schedules
- sport scores
- ticket availability
- market prices
- Downloading entertainment products
- Transferring software, high-resolution images,
and full-motion video - Innovative video applications
11Telemetry Services
- Wide range of new applications
- Transmission of receipt of status, sensing, and
measurement information - Communication with various devices from homes,
offices, or in the field - Activation of remote recording devices or service
systems
12ATT Wireless Welcome to mlife
Get the latest weather forecasts
Get the latest weather forecasts
Find breaking news, flight information, entertainm
ent..
Get the business and investments news
Get the business and investments news
13Future of Wireless Technology
- Mobile networks have already begun the migration
to IP-based networks - IP as the routing protocol
- 4G, New spectrum, and Emerging wireless air
interfaces (very high bandwidth 10 Mbps) - It may entirely be IP-based and packet-switched
- Increasing usage of wireless spectrum
- On average, the number of channels has doubled
every 30 months since 1985 (Coopers law)
14Wireless Multimedia Challenges
- Adaptive Decoding - Optimizing rich digital media
for mobile information devices with limited
processing power, limited battery life and
varying display sizes - Error Resilience - Delivering rich digital media
over wireless networks that have high error rates
and low and varying transmission speeds - Network Access - Delivering rich digital media
without adversely affecting the delivery of voice
and data services - Negotiable QoS for IP multimedia sessions as well
as for individual media components
15Components of a Wireless Video System
Input Video
Transport Network Layer
Video Encoder
Packetizer
Modulator
Channel Encoder
Wireless Channel
Output Video
Video Decoder
Depacketizer
Demodulator
Channel Decoder
Tradeoff Throughput, Reliability, Delay
16Source and Channel CodingTrade-off
- Classic goal of source coding
- Achieve the lowest possible distortion for a
given target bit rate - Classic goal of channel coding
- Deliver reliable information at a rate that is as
close as possible to the channel capacity - Shannons separation principle
- It is possible to independently consider source
and channel coding without loss in performance - The separation principle applies only to
point-to-point communications and it is not valid
for multiuser or broadcast scenarios
17Pragmatic Approach
- Keep the source coder and channel coder separate,
but optimize their parameters jointly - Key problem in this optimization is the bit
allocation between the source and channel coder - Joint source-channel coding schemes
- In the infancy today
- Exploit the redundancy in the source signal for
channel decoding (Source-controlled channel
decoding) - Designing the source codec for a given channel
characteristic (Channel-optimized source coding)
18Characteristics of a Wireless Video System
- The capacity of wireless channel is limited by
the available bandwidth of the radio spectrum and
various types of noise and interference - The wireless channel is the weakest link of
multimedia networks mobility causes fading and
error bursts - Resulting transmission errors require error
control techniques (such as FEC - forward error
control and ARQ automatic repeat request)
19The Case for Scalable Video Coding
- In emerging wireless applications, multimedia
data will be streamed - over various access networks (GPRS, UMTS, WLANs,
etc.) - to a variety of devices (PCs, TVs, PDAs, cellular
phones, etc.) - The transmission of multimedia data need to cope
with unpredictable bandwidth variations - due to heterogeneous access technologies of
receivers (3G, 802.11a, etc.) or - due to dynamic changes of network conditions
(interference, etc.)
20Scalable Video Coding Techniques
- Scalable video coding methods can adapt in real
time to the bandwidth variations over
heterogeneous networks and to the terminal
capabilities while using the same pre-encoded
system. - Scalable video coding uses multiple bit streams
layered video coding - For example, in a two-layer coding, the codec
generates two bit streams - Base layer the most vital video information
- Enhancement layer the residual information to
enhance the quality of the base layer image - This form of two-layer coding is known as SNR
scalability
21Scalability Techniques
- Data partitioning
- SNR scalability
- Spatial scalability
- Temporal scalability
- Hybrid scalability
22Data Partitioning
- Data partitioning is used when two channels are
available for transmission (it is not true
scalable coding) - Divides the bitstream of a single layer into two
parts, or layers.
Multiplexer
Base-layer bitstream
Video in
Output bitstream
Single layer encoder
Data Partitioner
Enhancement- layer bitstream
23Block DiagramTwo-Layer SNR Scalable Coder
Base layer bitstream
Video in
Base layer Encoder (MPEG 1)
Multiplexer
Output bitstream
Base layer Decoder (MPEG 1)
-
Enhancement layer Encoder (MPEG 2)
Enhancement layer bitstream
24Adaptive Video Coder Based on 3D-DCT
- Original video cube 8x8x8
- 3D Discrete Cosine Transform
25Motion Analysis for Various Blocks
- Partition image into NxN inspection areas
- Examine each area for motion content based on
Normalized Pixel Difference (NPD) between frames
1 and 8 - Three motion types defined
- No Motion
- Low Motion, and
- High Motion
- 3D-DCT block size adapts based on determined
motion content
26Example of a Video Hallway Clip
- 8 Frames of luminance (Y) component
- Inspection area size 16x16
- Inspection areas used to determine NPD thresholds
27Video Example, 420
Original
Cr120
Cr190
Cr408
28Architecture of 3D-DCT Adaptive Encoder
29Example of a Scalable CodingAdaptive 3D-DCT Coder
Original
30Adaptive 3D-DCT Coder
Layer 1 Cr164 (in vehicles, 144
Kbps)
31Adaptive 3D-DCT Coder
Adding Enhancement Layer 2 Cr96 (For
pedestrians, 384 Kbps)
32Adaptive 3D-DCT Coder
Adding Enhancement Layer 3 Cr54 (for indoor
use, 2 Mbps)
33Channel Coding and Error ControlEffects of
Transmission Errors
- Example 1 The extra insertion bit causing the
loss of the first GOB - Example 3 Corruption of the group quantizer
parameter that resulted in employing the wrong
quantizer in decoder
- Error-free frame
-
- Example 2 Corrupted group number
- causing a GOB misplacement
34Channel Coding and Error Control
- Trade-off between throughput, reliability, and
delay - Forward Error Correction (FEC)
- Automatic Repeat Request (ARQ)
- Error Resilience Techniques for Low Bit Rate
Video - Techniques that reduce the amount of introduced
errors for a given error event (Resynchronization)
- Techniques that limit interframe error propagation
35Recovery From Packet Loss FEC scheme
- Piggyback lower quality stream
- Send lower resolutionaudio stream as
theredundant information - For example, nominal stream PCM at 64 kbpsand
redundant streamGSM at 13 kbps. - Sender creates packet by taking the nth chunk
from nominal stream and appending to it the
(n-1)st chunk from redundant stream.
- Whenever there is non-consecutive loss,
thereceiver can conceal the loss. - Only two packets need to be received before
playback - Can also append (n-1)st and (n-2)nd low-bit
ratechunk
36Joint Source Coding and Transmission Power
Management
- Goal to limit the amount of distortion in the
received video sequence, while minimizing
transmission energy - Combines
- Error resilience and concealment techniques at
the source coding level, and - Transmission power management at the physical
layer - Optimization problem Minimizing the energy
required to transmit video under distortion and
delay constraints
37Joint Source Coding andTransmission Power
Management
Decoder Concealment Strategy
Channel State Information
Controller
Control power
Control coding parameters
Modulator
Channel Encoder
Video Encoder
Video in
Wireless Channel
Demodulator
Channel Decoder
Video out
Video Decoder
Goal to limit the amount of distortion in the
received video sequence,
while minimizing transmission
energy
38Transmission Energy
- Total energy to transmit all the packets in a
frame - The algorithm calculates the power needed to
achieve the desired probability of loss
39Controlling the Bit Rate
- Most video codecs use variable-length coding
techniques - Most existing mobile radio systems transmit at a
fixed bit rate - Goal Constant signaling rate leading to a
different constant bit rate for each modulation
scheme - Rate Control Techniques - determine the sending
rate of video traffic based on the estimated
bandwidth in the network - Source-Based Rate Control
- Receiver-Based Rate Control
- Hybrid-Based Rate Control
40Rate Shaping Techniques
- Techniques that adapt the rate of pre-compressed
video stream to a target rate constraint - Rate shaper is an interface (or filter) between
the compression layer and the network transport
layer
Variable rate
Constant bit rate
Network Transport Layer
Video in
Compression Layer
Rate Shaper
41Rate Shapers
- Codec filters
- Frame-dropping filters (dropping B,P, or I
frames) - Layer-dropping filters (in scalable video coding
schemes) - Frequency filters (discard DCT coefficients of
the highest frequency) - Requantization filters (reqauntizes the DCT
coefficients with a larger quantizers, resulting
in rate reduction)
42Multimedia Content Security
- Access control in applications such as
video-on-demand and videoconferencing, so only
selected users can access the data - Established encryption algorithms (DES or AES)
are very complicated and involve large number of
computations. - Software implementations of these schemes are not
fast enough to process the large amount of
multimedia data - Hardware implementations require additional costs
to both data generation and receivers
43General ArchitectureSelective Encryption System
44Example of Video EncryptionMPEG Encoder
Secret Key Selective encryption algorithm That
operates on sign bits of DC coefficients
Secret Key Permutation of the Huffman codeword
list
Secret Key Randomly change the sign bits of
motion vectors
45Example Encrypting Frames of a MPEG-4 Video
Sequence
Original frame Encrypted VLC only Encrypted
FLC only Encrypted VLC and FLC
46Virtues of the Virtual Workplace
- Universal access to information, applications,
services, processes, and people, from any device,
over any network connection - wired, wireless, or
Web
47Virtual Workplace Video Clip
- Wireless Internet and Web
- Wireless appliances
- Security
- Redundant systems
- Wireless applications videoconferencing
48(No Transcript)
49The Portable Office
Take the office with you, wherever you go
50Secure Authentication
High Security Authentication, including
Bio-Authentication
51IntegratedMessaging and Communication
Integrated messaging (eg. voice, chat), voice to
text, with intelligent alerting
52Information Portability
Access information over any connection wired
or wireless, regardless of form factor
53Business Collaboration
Collaborative capabilities allow on-line
information sharing and communication
54Business Continuity
Resilient to network interruptions
55Further Readings
- Hanzo, Cherriman, and Streit, Wireless Video
Communications, IEEE Press, 2001. - IEEE Trans. On Circuits and Systems for Video
Technology, Special Issue on Wireless Video, June
2002. - Sun and Reibman, Compressed Video over
Networks, Marcel Dekker, 2001 - Wang, Ostermann, and Zhang, Video Processing and
Communications, Prentice Hall, 2002. - Furht and Ilyas, Wireless Internet Handbook,
CRC Press, 2003.