Multimedia Systems AMS505 8'1 2001

1
Multimedia SystemsAMS505 8.1 2001

• Major Greg Phillips
• Royal Military College of Canada
• Electrical and Computer Engineering
• greg.phillips_at_rmc.ca
• 1-613-541-6000 ext. 6190

2
What is multimedia?

• Traditionally applied to artworks in which more
  than one medium of expression is used. E.g.,
• sculpting in clay single media
• painting on canvas single media
• sculpting in clay, then painting the sculpture
  multimedia
• In an information systems context, refers to a
  combination of communication media, including any
  of
• text
• static graphics
• audio
• video
• virtual-reality

3
Graphics

• Key parameters are image dimensions (in pixels)
  and colour depth (in bits/pixel)
• For example
• 336 x 480 x 24 b/pixel
• 3870720 b
• 483840 B
• 473 kB
• Compressed with PNG
• 177 kB (2.71 compression)

http//www.w3c.org
4
Compression Technologies

• lossless, lossy, hybrid
• Discrete Cosine Transform (DCT), e.g. JPEG
• wavelets
• fractals

5
8 bits per pixel uncompressed 159 kB GIF 73
kB, PNG 68 kB
24 bits per pixel uncompressed 473 kB
6
8 bits per pixel uncompressed 159 kB GIF 73
kB, PNG 68 kB
24 bits per pixel uncompressed 473 kB
7
Lossy Compression

• Lossy compression sacrifices image quality to
  achieve higher levels of compression
• Most common technique is JPEG (Joint Photographic
  Experts Group)
• Compression algorithm based on discrete cosine
  transform (DCT) which is alleged to reduce the
  perceived loss in image quality
• Algorithm is parameterised by a quality factor
  which controls the degree of loss
• more loss gives a much smaller file size, but at
  the expense of considerable quality

8
24 bits per pixel uncompressed 473 kB
24 bits per pixel JPEG (q2) 89 kB
9
24 bits per pixel uncompressed 473 kB
24 bits per pixel JPEG (q2) 89 kB
10
24 bits per pixel JPEG (q50) 14 kB
24 bits per pixel JPEG (q255) 7 kB
11
24 bits per pixel JPEG (q50) 14 kB
24 bits per pixel JPEG (q255) 7 kB
12
Compression Effectiveness
13
Vector Graphics

• so far, only discussed raster (pixel-oriented)
  graphics
• graphics can also be stored in vector
  (line-oriented) format
• commonly used for map storage
• PostScript is a common vector graphics format
  used in many printers
• For the WWW, new format called Scalable Vector
  Graphics on the horizon

14
Audio Digitization (PCM)
15
Audio Storage Requirements

• CD quality
• 44.1 ksamples/s 16 bits/sample 705 kbps
• One CD stores up to 72 minutes of audio
• 72 min 60 s/min 705 kbps (8 b/B) 380.7 MB
• But the CD format also includes error correction
  with about a 1.8 overhead factor
• 380.7 MB 1.8 685 MB (or, in real MB, 650)

16
Audio Encoding Techniques

• DAT (PCM) 768 kbps
• CD 705 kbps
• CCITT telephony (PCM) 128 or 64 kbps
• EUROCOM military telephony (CVSD) 16 or 32 kbps
• HF digital telephony (LPC-10) 2.4 or 4.8 kbps

17
Audio Compression

• Makes use of psychoacoustic knowledge to reduce
  the amount of information required to achieve the
  same perceived quality (lossy compression)
• MPEG audio achieves CD quality in about 192 kbps
  (a 3.71 compression ratio)
• Sony MiniDisc uses Adaptive TRAnsform Coding
  (ATRAC) to achieve a 51 compression ration
  (about 141 kbps)

http//www.bok.net/tristan/MPEG/MPEG-content.html
http//www.minidisc.org/aes_atrac.html
18
Audio Compression Comparison
Mendelsohn's Electric Guitar Concerto MPEG-3
Audio, 128 kbps 1166 kB (10 min _at_ 16 kbps)
Mendelsohn's Electric Guitar Concerto MPEG-3
Audio, 8 kbps 74 kB (37 sec _at_ 16 kbps)
19
Video Storage Requirements

• NTSC video (North American TV)
• 352 pixels/line 240 lines/frame 29.97
  frames/sec 24 b/pixel 60.76 Mbps
• At this rate, a CD could hold (discounting error
  correction)
• 685 MB 8 b/B / 60.76 Mbps 90 seconds of video
• A DVD could hold
• 4.7 GB 8 b/B / 0.06076 Gbps 10 min of video
• Actually, its much worse than this, especially
  since DVD is typically stored at 720 by 480
  pixels, which gives 249 Mbps
• 4.7 GB 8 b/B / 0.249 Gbps 2.5 min of video

Actually, Im lying video isnt stored in
bits/pixel. Its much more complicated than that.
20
MPEG Video Compression

• MPEG is a nickname (Motion Picture Experts Group)
  for a series of ISO standards for video
  compression (MPEG-1, 2, and 4)
• The standards are extremely flexible, supporting
  an almost infinite range of image sizes, frame
  rates and encoding mechanisms
• MPEG-2 (used with DVD) uses eighteen separate
  mechanisms to achieve compression
• compression ratios are sometimes claimed in the
  1001 range, but 81 to 301 is more realistic
• compression is very computationally demanding
  decompression less so (not good for real-time)

http//www.bok.net/tristan/MPEG/MPEG-content.html
21
MPEG motion prediction
Current frame
Previous frame motion
Compensated difference
Current - previous
22
Video Quality Comparison
Mark Nizer Promo Video Apple Quicktime 483 kB (4
min _at_ 16 kbps)
Mark Nizer Promo Video Apple Quicktime 3126
kB (26 min _at_ 16 kbps)
23
Video Conferencing

• CTX, CTX Proprietary algorithms developed by
  Compression Labs.
• SG3 Proprietary algorithm developed by
  Picturetel.
• H.130 CCITT standards for working at 2.048Mbit/s
• H.261 CCITT Video coding, also known as PX64
• H.320 CCITT umbrella standard for narrow
  bandwidth audio- visual systems
• G.728 CCITT audio compression at 16kbit/s.

24
Synchronization

• essential that sound and image be synchronized
  for video presentation
• this turns out to be quite technically
  challenging, largely since sound is presented in
  samples (44,100 samples/s) and video is presented
  in frames (24-30 frames/s)
• the problem is even worse if the two are being
  transmitted across distinct communications media

25
Communications Issues

• multimedia is very bandwidth intensive
• DVD-style MPEG-2 video requires about 5 Mbps
• current CNR provides about 8 kbps continuous
• therefore one MPEG-2 video stream requires about
  625 CNR channels
• for communication, latency matters
• latency is the time between transmission and
  reception
• codecs (encoder/decoder) add significant latency
• for live broadcast, heartbeat matters
• voice typically packetized into 40 ms segments
• must arrive very close to 40 ms intervals,
  otherwise quality suffers
• can buffer at the receiving end, but this adds
  latency