Multimedia Programming Techniques for Teachers

1
Multimedia Programming Techniques for Teachers

• Curriculum Designer Vincent Ng
• Total 24 hours
• Session 1/8

2
Objectives/Aims

• Understand the basic concepts of multimedia
  technology
• Appreciate the concepts of object-oriented
  designs
• Write multimedia program modules with a suitable
  computer language and
• Develop teaching and/or learning packages with
  multimedia and interactivities.

3
Agenda

• Lesson 1
• Basic multimedia technology
• Lesson 2
• Web interactions and HTML forms
• Lesson 3
• Object-oriented design concepts and multimedia
  project development
• Lesson 4
• Multimedia elements in Webpages, simple CGI
  programming
• Lesson 5-8
• ASP programming

4
Lesson Objectives

• Different Media Types available
• What are Multimedia Systems
• Image and Video compressions

5
Motivation

• How can we capture, store and touch-up an image?
• How can we capture, store and make a MP3 file for
  a talk?

6
What are multimedia?

• Audio, still image or motion video - in addition
  of text and graphics.
• What is a multimedia system?
• A computer platform, communications network or
  software tool that supports the interactive use
  of at-least one of the above types of information

7
Media Types Still Images

• Facsimile (low quality image)
• first practical means of transmitting document
  images over a telephone line
• standardized to CCITT group 3 compression
  standards
• using run-length encoding (medium level of
  compression)
• pixel density 100 to 200 dpi
• Document image (medium quality image)
• for storing business documents
• basic requirement 300 dpi
• an uncompressed A4 size page over 1 Mbyte
• group 3 compression 300 Kbyte
• group 4 compression 75 Kbyte

8
Media Types Still Images (cont.)

• Photographic image (high quality image)
• for web displays
• digital albums
• photo identification systems
• patient medical histories (digital X-ray films)
• satellite images
• basic requirement 300 dpi
• compression requirement high

9
Media Types Audio

• Voice commands (input) and voice synthesis
  (output)
• used for hand-free operation of a computer
  program
• voice synthesis is used for presenting results of
  an action to the user in a synthesized voice
• voice emails, internet phone, etc.
• bandwidth requirement for telephone voice 8 bit
  x 8000/s 64kbps
• Music
• radio on the web, studio on the web, etc.
• need compression
• speed of decompression when playback is crucial
• storage requirement for a typical MP3 song
  3-5Mbytes

10
Media Types Video

• Full-motion stored and live video
• for lots of applications
• e.g., video conferencing
• CD-ROM DVD-ROM technology provided the basis
  for the development
• requirements
• large bandwidths for communication
• massive storage
• high-density, high-performance compression
  technology
• decompression requirement very high and
  time-constrained

11
Force Behind Multimedia -Technology Thrust

• High Performance PC/workstation with audio-visual
  processing capability
• Fast paced improvement in CPU performance (GHz
  level)
• High performance compression techniques developed
  for video, image and audio (JPEG, MP3,
  MPEG-1,2,4,7)
• High capacity, high performance storage devices
• Availability of magnetic disks with capacity of
  gigabyte magnitude
• Tremendous increase in data transfer rate due to
  disk array technology
• High Speed fiber optic networks and fast packet
  switch technology
• Network with ?100 Mbit/s already in place,
  prototype Gbit networks also being deployed
• Broad bandwidth
• broadband internet services

12
Class Practice Activities 1

• What is the difference between GIF and JPEG?
• Try to list as many as we remember
• Each member has a floppy which has several jpeg
  images
• View the images and note the differences
• Convert the first.jpg into a thumbnail image by a
  given software

13
Types of Lossy Compression Techniques

• Prediction-oriented techniques
• predicting subsequent values by observing
  previous ones, and transmitting only the
  differences between actual and predicted data
• e.g. motion compensation in MPEG, etc.
• Frequency-oriented techniques
• taking the advantages of humans perceptual
  properties, e.g., humans have different
  sensitivities to various spatial and temporal
  frequency combinations
• Importance-oriented techniques
• taking the advantages of other characteristics
  (besides frequency) of media, e.g., allocate more
  bits to encode important parts of images (edges)
• statistical based techniques

14
Types of Lossy Compression Techniques (cont.)

• Hybrid Compression Approaches
• good combination of various compression
  approaches
• generally adopted by systems and standards
• e.g., in MPEG
• motion compensation ? achieve temporal
  compression
• transform coding (DCT) ? achieve spatial
  compression
• Huffman coding ? achieve statistical compression

15
JPEG Image Compression Standard

• JPEG stands for Joint Photographic Experts Group
  CCITT ISO
• 1st international standard for continuous-tone
  still images
• JPEG Objectives
• to be at or near the state-of-the-art for degree
  of compression versus image quality
• to be parameterizable so that the user can select
  the desired compression versus quality tradeoff
• to be applicable to practically any kind of
  source image, without regard to dimensions, image
  content, aspect ratio, etc.
• to have computational requirements that are
  reasonable for both hardware or software
  implementations,

16
JPEG (cont.)

• Modes of Operations
• sequential encoding, where each image component
  is encoded in the same order that it was scanned
• progressive encoding, where the image is encoded
  in multiple passes so that a coarse image is
  presented rapidly, followed by repeated images
  showing greater and greater detail
• lossless encoding, where the encoding guarantees
  exact reproduction of all the data in the source
  image
• hierarchical encoding, where the image is encoded
  at multiple resolutions.
• Design Approach
• Hybrid of different techniques, e.g. Huffman and
  Arithmetic coding (entropy coding), DCT
  (transform coding), Quantization, etc.

17
Block Diagram of JPEG Architecture
18
Progressive Modes of JPEG
19
MPEG Video Compression Standard

• MPEG stands for Moving Picture Experts Group
• JPEG, or even motion JPEG, does not exploit
  temporal (frame-to-frame) redundancy in video
  sequences
• MPEG-1 is intended for the storage of VCR-quality
  audio-visual sequences on CD-ROMs
• The official name of MPEG-1 is Coding of Moving
  Pictures and Associated Audio for Digital Storage
  Media at up to about 1.5Mbps
• The video compression ratio is in the order of
  261
• It is lossy and asymmetric, with the encoding
  more expensive than decoding
• MPEG is an evolving family of standards for
  recording and transmitting digital audio-video
  information, including MPEG-2, MPEG-4, or more ...

20
Target Features of MPEG-1

• Random Access requires that a compressed video
  bit stream be accessible in its middle and any
  frame of the video be decodable in a limited
  amount of time
• VCR Paradigm, i.e., fast forward/backward
  searching possibly with the help of an
  application-specific directory structure and,
  using the appropriate access points (more
  demanding than random access)
• Robustness to Errors catastrophic behavior in
  the presence of errors should be avoidable
• Coding/Decoding Delay maintain the
  conversational and face-to-face nature of the
  application
• Format Flexibility support playback of video in
  windows
• Audio-Video Synchronization
• Editability

21
MPEG-1 An Overview

• Conflicting requirements
• high compression ratio for the target 1.5Mbps ?
  inter-frame encoding
• random access, etc. ? intra-frame encoding
• A good balance is needed!!
• Two basic coding schemes
• DCT-based compression for exploiting spatial
  redundancy
• Block-based motion compensation for exploiting
  temporal redundancy. It takes use of both
  predictive coding and interpolative coding.
• Consists of three main parts/documents
• video based on H.261, JPEG, etc.
• Audio based on MUSICAM technology
• System specify how the two bit streams, i.e.,
  audio and video, should be multiplexed over a
  single transmission channel or over a single
  storage device.

22
Frames in MPEG

• I-frames (Intracoded frame)
• use basically JPEG
• provide entry point for random access
• yield moderate compression
• P-frames (Predicted frame)
• encoded with previous reference frame (I or P
  frames) using motion compensation
• unidirectional prediction
• B-frames (Bidirectional frames)
• encoded using the previous and
• the next reference frame using motion
  compensation
• achieve maximum compression

Both I- P-frames are reference frames. Typical
sequenceI B B P B B P B B I...
23
Use of Prediction Interpolation in MPEG

• Prediction
• used by P-frames
• assume that the current picture can be modeled as
  a translation of the picture at some previous
  time
• displacements need not be the same everywhere in
  the frame ? code the motion information (motion
  vector)
• Interpolation (also known as bi-directional
  prediction)
• used by B-frames
• key feature of MPEG to achieve high degree of
  compression
• areas just uncovered are not only predictable
  from the past, but can be predicted from the
  future.
• trade-off (between higher compression and better
  prediction) associated with the frequency of
  B-frames
• ? place references at about 1/10th sec.
  interval, e.g. IBBBPBBBI, IBBPBBPBBI (PAL),
  IBBPBBPBBPBBI(NTSC)
• Both are based on motion compensation techniques!

24
Motion Compensation
I, B, P Frames in Action

• I-frame is encoded without reference to any other
  frame
• P-frame is predicted from I-frame or P-frame
• B-frame is interpolated from the two references,
  i.e. I-frame P-frame here.

I-frame (Intracoded)
B-frame (Interpolated coded)
P-frame (Predicted coded)
25
More on MPEG

• We know the intention of MPEG-1 It has been
  optimized for medium resolution
• MPEG-2
• The official name of MPEG-2 is Generic Coding of
  Moving Pictures and Associated Audio
• intended for the recording and transmission of
  studio-quality motion video at bit rates in the
  order of 4 to 6 Mbps
• designed to optimally handle ITU-R 601
  recommendation, I.e., 720x480 pixels for the
  luminance signal and half for the color space
• MPEG-2 can operate either in progressive or
  interlaced scan mode
• The emerging standard for HDTV, consequently
  MPEG-3 is abandoned

26
Class Practice Activities 2

• Form small groups of 3-4 members
• Record a short speech of about a minute
• Convert the file into the MP3 format
• www.okstate.edu/ind-engr/step /WEBFILES/Informatio
  n/instructions.html

27
MPEG Audio Layers - MP3

• In 1987, the IIS started to work on perceptual
  audio encoding in the framework of digital audio
  broadcasting.
• The IIS finally devised a very powerful algorithm
  that is standardized as ISO-MPEG Audio Layer-3
  (the compression referred to the popular MP3
  music)
• Without data reduction (as in PCM), you have to
  end up with more than 1.4 Mbits (44.1k x 2 x 16
  bits) to represent just one second of stereo
  music in CD-quality.
• By using MPEG audio coding, you may shrink down
  the original sound data from a CD by a factor of
  12, without losing sound quality.
• There are three layers defined. Encoder
  complexity and performance are increasing with
  the layer number. The three layers are
  compatible in a hierarchical way, i.e., a Layer-N
  decoder may be able to decode bitstream data
  encoded in Layer-N and all layers below N.

28
MPEG Audio Layers - MP3 (cont.)

• Compression achieved by different layers
• What does the MPEG standard defined, exactly?
• For each layer, the standard specifies the
  bitstream format and the decoder. To allow for
  future improvements, it does not specify the
  encoder (but an information chapter giving an
  example for an encoder for each layer is provided
• What have the three audio layers in common?
• All layers use the same basic structure
• the coding scheme can be described as perceptual
  noise shapingor perceptual subband/transform
  coding
• For such a scheme, the decoder is much less
  complex than the encoder

29
A New Audio Compression Standard MPEG-2 AAC

• Standing for MPEG Advanced Audio Coding
• A data compression standard for the 21st century
• declared by MPEG as an international standard by
  the end of April 1997
• The driving force to develop AAC was the quest
  for an efficient coding method for surround
  sounds, like 5-channel signals
• The appropriate incorporation of high coding gain
  and great flexibility opens up a wide field of
  applications.
• With sampling frequencies between 8kHz and 96kHz
  and any number of channels between 1 and 48, the
  method is well-prepared for future developments
  in the audio sector
• For future details, please take a look at
  153.96.172.2/amm/techinf/aac/index.html

30
What you would know after?

• What is multimedia?
• Different Media Types
• Technology Thrust of Multimedia Systems
• Image compression, JPEG
• MPEG

31
Something to bring back next time

• Evaluate the multimedia set up in your school
• Multimedia machine numbers, speed, capacity,
  sound card, graphics card
• Network bandwidth, VPN, video servers (HW and
  management tools)
• Multimedia software