??????????MPEG

1
??????????MPEG

• 2005?fall

2
1 ??

• MPEG(Motion Picture Experts Group)????????????????
  ??????????????????????????
• MPEG?????1988??
• JPEG?MPEG???ISO????????,???????????JPEG???????????
  ?,?MPEG???????????????,????????????????

3
MPEG

• ???????(International Organization for
  Standardization,ISO)????????(International
  Electro-technical Commission,IEC)????ISO/IEC
  JTC1/SC29/WG11,??????????????????????????????
• MPEG?????MPEG-1?MPEG-2?MPEG-4??????MPEG-7??

4
MPEG?????????

• ????(Working Draft,WD)
• ???(Working Group,WG)???????
• ?????(Committee Draft,CD)
• ????WG????????WD??????????ISO???????,?ISO?????????
  ????
• ??????(Draft International Standard,DIS)
• ??????CD????????????????CD???????
• ????(International Standard,IS)
• ???????ISO????????????????????????

5

• MPEG??????MPEG-1?1992???,?VCD?????????352288?????
  ,MPEG-1????????,???????????????
• 1995??MPEG-2?720576?????????????????????
• 1999?12????MPEG-4?
• MPEG-7?????????????
• ?MPEG??????,???????????????MPEG-1?MPEG-2????????
  ??,??????????MPEG-4 ? MPEG-7??

6
MPEG??

• MPEG-1ISO/IEC 11172
• MPEG-2ISO/IEC 13818
• MPEG-4ISO/IEC 14496
• MPEG-7ISO/IEC 15938
• MPEG-21ISO/IEC 21000

7
??

• video coding
• audio coding
• system definition
• which describes the combination of individual
  data streams into a common stream.

8
2 ????

• An image must consist of three components.
• luminance Y
• two color difference signals Cr and Cb
• color subsampling
• 14 different pixel aspect ratios
• 11
• 169
• 43

9

• refresh frequency
• 23.976Hz, 24Hz, 25Hz, 29.97Hz, 30Hz, 50Hz,
  59.94Hz, and 60Hz
• An MPEG macro block is partitioned into 1616
  pixels for the luminance component and 88 pixels
  for each of the two chrominance components.
• A macro block is formed of six blocks of 88
  pixels first four blocks for the luminance
  component then the two chrominance blocks.

10
??

• ???????????????????,?????????,MPEG?JPEG???????????
  ??????YUV???Y??????16?16???,U?V??????8?8??,?16?16
  ???????4?8?8?,??8?8??????DCT???
• ???16?16??????????8?8?????????????????????????????
  ????????????352?240?NTSC???????,?22?15330????????
  ????352?288?PAL???????,?22?18396??????

11
?????
12

• efficient coding
• temporal redundancies of successive images
• random access
• images are coded individually.
• MPEG supports four types of image coding.
• I
• P
• B
• D

13

• I?(????intra frame),????????JPEG?????,???????,????
  ??????,????????????,?P??B??????
• P?(?????Predicted Picture),?????I?P??????????
• B??(???? Bidirectional Prediction),??????????I?P??
  ??????????

14
I frames (intra coded pictures)

• coded without using information about other
  frames (intraframe coding).
• An I frame is treated as a still image. Here MPEG
  falls back on the results of JPEG. Unlike JPEG,
  real-time compression must be possible. The
  compression rate is thus the lowest within MPEG.
• I frames form the anchors for random access.

15

• I frames are encoded as in JPEG.
• A DCT on the 88 blocks defined within the macro
  blocks
• The DC-coefficients are then DPCM coded, the
  differences between consecutive blocks of each
  component are calculated and transformed into
  variable-length code words.
• AC-coefficients are run-length encoded and then
  transformed into variable-length code words.
• MPEG distinguishes two types of macro blocks
• those contain only coded data
• those additionally contain a parameter used for
  scaling the characteristic curve used for
  subsequent quantization.

16

• I???????????,????????????????,?????????????I??????
  ?,??????????,??????????
• I??????????????????,??????????,I?????????????
• I????????????????,????????????

17
P frames (predictive coded pictures)

• require information about previous I and/or P
  frames for encoding and decoding.
• Decoding a P frame requires decompression of the
  last I frame and any intervening P frames.
• The compression ratio is considerably higher than
  for I frames.
• A P frame allows the following P frame to be
  accessed if there are no intervening I frames.

18

• the most similar macro block in the preceding
  image must be determined
• MPEG does not specify an algorithm for motion
  estimation, but rather specifies the coding of
  the result.
• motion vector (the spatial difference between the
  two macro blocks) and the small difference
  between the macro blocks need to be encoded.
• The search range, that is, the maximum length of
  the motion vector, is not defined by the
  standard. As the search range is increased, the
  motion estimation becomes better, although the
  computation becomes slower.

19
????

• ?????????????????????????????????????????????????,
  ??????????????????????????,???????????????????
• ???????????,???????????????????,????????????????,?
  ?????????,???16?16??????????
• ?????????????????????????,??(????)???????????????

20
???????????
21
Block Motion Estimation
22
Video sequence Tennis frame 0
23
Video sequence Tennis frame 1
24
Frame Difference
25
Motion Vector-Motion Estimation
26

• P???I??????????,???????????????I???16?16???,?I
  ????????,??????????????P??I???,?????????
• ????????????,????P??????,????,?????????????????,
  P?????????????????

27

• 16?16???????????,??????????????????????
• ???????????,???????,??????????????????????,???????
  ????,????????????????,???16?16????????????????????
  ,???????????(?????????)?

28

• P frames can consist of macro blocks as in I
  frames, as well as six different predictive macro
  blocks.
• In coding P-frame-specific macro blocks
• differences between macro blocks as well as the
  motion vector need to be considered.
• The difference values between all six 88 pixel
  blocks of a macro block being coded and the best
  matching macro block are transformed using a
  two-dimensional DCT.

29

• Further data reduction is achieved by not further
  processing blocks where all DCT coefficients are
  zero. This is coded by inserting a six-bit value
  into the encoded data stream.
• Otherwise, the DC- and AC-coefficients are then
  encoded using the same technique.
• Next, run-length encoding is applied and a
  variable length coding is determined according to
  an algorithm similar to Huffman.
• motion vectors of adjacent macro blocks are DPCM
  coded. The result is again transformed into
  variable-length coded words using a table.

30
B frames

• B frames(bidirectionally predictive coded
  pictures) require information from previous and
  following I and/or P frames.
• B frames yield the highest compression ratio
  attainable in MPEG.
• A B frame is defined as the difference from a
  prediction based on a previous and a following I
  or P frame.
• It cannot ever serve as a reference for
  prediction coding of other pictures.

31

• A macro block can be derived from macroblocks of
  previous and following P and/or I frames.
• a prediction can interpolate two similar macro
  blocks.
• two motion vectors are encoded
• one difference block is determined between the
  macro block to be encoded and the interpolated
  macro block.
• Subsequent quantization and entropy encoding are
  performed as for P-frame-specific macro blocks.
• B frames need not be stored in the decoder.

32
D frames

• D frames (DC coded pictures) are intraframe-coded
  and can be used for efficient fast forward.
• During the DCT, only the DC-coefficients are
  coded the AC coefficients are ignored.

33

• D frames contain only the low-frequency
  components of an image.
• A D-frame always consists of one type of macro
  block and only the DC-coefficients of the DCT are
  coded.
• D frames are used for fast-forward display. This
  could also be realized by a suitable placement of
  I frames.

34
(No Transcript)
35

• P??B???????????,?????????????????
• P????????????,??????????????P???????????????,?P???
  ????????????,?????????
• B???????????,?????????????B?????????????,??MPEG-1?
  ??????????????????????
• ????????,I?????????P????????????????,??P?????????
  ?B?????????,????B?????????????

36
(No Transcript)
37
MPEG????

• ?MPEG???????????????????????????????????????,?????
  ?????????????JPEG????????????,?? MPEG?????????
• ?????,????????????????????????,??????????????,????
  ???????????????????????????????

38

• ???????????MPEG?????,??,MPEG??????
• ?????????????????????????????????,??????????????
• ??????????????????????,?????????????????????????,?
  ??????????????

39

• ??????????????????????
• ???I????????????????????????????????????????,?????
  ????????????????????????,????????,????????????????
  ???????????????????????????????,??????????????????
  ???????I?????????????????????????,?????????

40

• ????????P??B??
• ???????????????????P???????B?????,????I???????B???
  ?????P????I????????,????????B?????????????

41
Quantization

• AC-coefficients of B and P frames are usually
  very large values, whereas those of I frames are
  very small.
• MPEG quantization adjusts itself accordingly.
• If the data rate increases too much, quantization
  becomes more coarse.
• If the data rate falls, then quantization is
  performed with finer granularity.

42
3 ????

• MPEG audio coding is compatible with the coding
  of audio data used for Compact Disc Digital Audio
  (CD-DA) and Digital Audio Tape (DAT).
• The most important criterion is the choice of
  sample rate of 44.1kHz or 48kHz (additionally
  32kHz) at 16bits per sample value. Each audio
  signal is compressed to either 64, 96, 128, or
  192Kbit/s.

43

• Three quality levels (layers) are defined with
  different encoding and decoding complexity.
• An implementation of a higher layer must be able
  to decode the MPEG audio signals of lower layers
• FFT is applied for audio, and the spectrum is
  divided into 32 nonoverlapping subbands
• noise level in each subband is determined using
  a psychoacoustic model.

44

• In the first and second layers, the appropriately
  quantized spectral components are simply
  PCM-encoded.
• The third layer additionally performs Huffman
  coding.
• MPEG provides for two types of stereo sound.
• Two channels are processed completely
  independently.
• In the joint stereo mode, MPEG achieves a higher
  compression ratio by exploiting redundancies
  between the two channels

45
(No Transcript)
46

• The minimal value is always 32Kbit/s.
• The layers support different maximal bit rates
• layer 1 allows for a maximum of 448Kbit/s
• layer 2 for 384Kbit/s
• layer 3 for 320Kbit/s.
• For layers 1 and 2, not all combinations of bit
  rate and mode are allowed, and a decoder is not
  required to support a variable bit rate.
• In layer 3, a variable bit rate is specified by
  allowing the bit rate index to be switched.

47
4 ???

• An audio stream is comprised of frames, which are
  made up of audio access units, which in turn are
  divided into slots.
• An audio access unit is the smallest compressed
  audio sequence that can be completely decoded
  independently of all other data.

48
Video Stream

• A video stream is comprised of 6 layers
• sequence layer
• the beginning of the sequence layer includes two
  entries the constant bit rate of the sequence
  and the minimum storage capacity required during
  decoding.
• A video buffer verifier influences the quantizer
  and forms a type of control loop.
• group of pictures layer
• This layer contains at least an I frame, which
  must be one of the first images.
• the difference between decoding order and display
  order

49

• picture layer
• contains a whole still image.
• image number.
• slice layer
• Each slice consists of macro blocks
• A slice also includes the scaling used for DCT
  quantization of all its macro blocks.
• macro block layer
• block layer

50
System Definition

• specifies the combination of audio and video data
  streams
• the coordination of input data streams with
  output data streams, clock adjustment, and buffer
  management.
• One could define a protocol to supply the header
  upon request.
• MPEG does not prescribe compression in real-time.
• MPEG defines the decoding process but not the
  decoder itself.

51
5 MPEG-1

• MPEG-1?????ISO/IEC 11172,???????????????????1.5
  Mbit/s?????????????????(Information technology
  Coding of moving pictures and associated audio
  for digital storage media at up to about 1.5
  Mbit/s)?
• MPEG-1??1992??????????????????,????????????????1.5
  Mb/s???MPEG????
• MPEG-1????MPEG???MPEG???MPEG?????
• MPEG-1?????????,????????,?????????

52
MPEG-1

• Coding of moving pictures and associated audio at
  up to about 1.5 Mbit/s
• Part 1 Systems
• Part 2 Video
• Part 3 Audio
• Part 4 Conformance testing
• Part 5 Software simulation

53

• MPEG-1??????????????????,??????????/????????,?????
  ???????????????????????????????,?????????????????,
  ???MPEG-1??????????

54

• MPEG-l?????????????????
• ????????????,?????
• ???????????????
• ???DCT?????????
• ?DCT??????,????????,????DCT??????????
• ? DCT?????????
• ??????????(DC)????????

55

• ???????,MPEG-1?????????,?ISDN?LAN?????,???????????
  ???
• ???????,??MPEG-1??????????????????????????????????
  ????????VCD???
• VCD??MPEG-1????,?????25200?,????65?,??????????,??
  ???74??VCD??288???????,???????VHS????VCD??????????
  ????,??????????????????

56
MPEG-2

• MPEG-2?????ISO/IEC 13818,?????????
  ??????????????(Information technology Generic
  coding of moving pictures and associated audio
  information )?
• MPEG-2???1990?????,1995????????
• MPEG-2?MPEG-1???,??????????????MPEG-2?????MPEG-1??
  ????,??????????????,???????????(scalability)???

57
MPEG-2

• Generic coding of moving pictures and associated
  audio
• Part 1 Systems
• Part 2 Video
• Part 3 Audio
• Part 4 Conformance testing
• Part 5 Software simulation
• Part 6 System extensions - DSM-CC
• Part 7 Audio extension - NBC mode
• Part 8 VOID - (withdrawn)
• Part 9 System extension RTI
• Part 10 Conformance extension - DSM-CC
• Part 11 IPMP on MPEG-2 Systems

58

• ?????????????????????????????MPEG-2??????????(HDTV
  )???????????.
• MPEG-2??????????MPEG-1????????MPEG-2????????,?MPE
  G-1??????????MPEG-2?????????,????????????,??MPEG-2
  ????????????????,?MPEG-1????

59

• MPEG-2???????????(Profiles)?????(levels),?????????
  ????20????11????????,??????,??????????/???????????
  ??????????
• ??,DVD???????????????????????,?110Mb/s???????????
  ?????,???????????????????????,?????4.69Mb/s?DVD??M
  PEG-2??,????????????HDTV????????

60
(No Transcript)
61
MPEG-4

• 1999???ISO/IEC??MPEG-4?MPEG-4?????????????????
• ?????????????(?PC???????)?WWW(???)???????????MPEG-
  4????
• MPEG-4????????????????????????????????????????????
  ???????(??????)?
• MPEG-4????????(???????)??????????????????????,
  ??????????

62

• MPEG1?MPEG2???????????????,??????????????????????,
  ??????????????????,???????????????????????
• MPEG-4?????????????,??????AVO(Audiovisual
  Object)????AV??????,??AVO?????????????,???,??AV???
  ????????,MPEG-4????????????-???????(Content-based
  Interactivity)

63

• ???????,MPEG-4??????????????????????????????????,?
  ????????,?????????????????????????????????????????
  ,???????????????????????????
• MPEG-4???????????????,?????????,???????????,???-??
  ?????,???????????????????????????,MPEG-4???????VS?
  ????VO??????VOL??????VOP????

64
MPEG-4

• Coding of audio-visual objects
• Part 1 Systems
• Part 2 Visual
• Part 3 Audio
• Part 4 Conformance testing
• Part 5 Reference Software
• Part 6 Delivery Multimedia Integration Framework
• Part 7 Optimized software for MPEG-4 tools
• Part 8 MPEG4 on IP framework

65
MPEG-4

• Part 9 Reference Hardware Description
• Part 10 Advanced Video Coding
• Part 11 Scene Description and Application Engine
• Part 12 ISO Base Media File Format
• Part 13 IPMP Extensions
• Part 14 MP4 File Format
• Part 15 AVC File Format
• Part 16 Animation Framework eXtension (AFX)

66
MPEG-4 Versions

• Version 1December 1998
• Version 2December 1999
• more tools were added in subsequent amendments
  that could be qualified as versions, even though
  they are harder to recognize as such

67
Audiovisual Objects (AVOs) in MPEG-4

• AVOs are individually coded in order to achieve
  maximum efficiency.
• defining a syntax for storing information about
  Intellectual Property Rights (IPR) pertaining to
  MPEG-4 AVOs and developing tools to support IPR
  identification and IPR protection.

68
MPEG 4
69
Coding of Visual Objects

• Object-based scene layering and separate coding
  and decoding of layers
• Shape-adaptive DCT coding
• Object-based tool box for motion prediction

70
Important components of an MPEG-4 terminal
71
Example of Sprite Coding
72
More on MPEG4

• Textures and Still Images
• Synthetic Objects
• face and body animation
• Static and Dynamic Mesh Coding with texture
  mapping
• Texture Coding for View Dependent applications
• Natural Sound
• Synthesized Sound
• TTS?SA

73

• MPEG-4???????
• ????????
• ???????
• ??????,????????????
• ?????????,?????DVD?
• ????,????????????
• ?????,?????????????
• ?????????

74

• ???PC??DVD???MPEG-4??,???????????????(??DVD-ROM???
  )?
• ???????MPEG-4????Internet???????????????????????
  ???????16kbit/s??????????????2D?3D???????,????????
  ???Internet???????
• ?????????UMTS???MPEG-4?????

75

• ????????????????,MPEG-4??????????????
• MPEG-4????MPEG-1?MPEG-2????????????,MPEG-4????????
  ???????

76
MPEG-7

• ???????????,???????????????
• MPEG-7???????????MPEG-7???????????????,?????????
  ??????????????,?????????????????,???????????????
  ??????????
• MPEG-7???????????,???????????
• MPEG-7??????MPEG???????MPEG-1? MPEG-2?MPEG-4??????
  ??,?MPEG-7?????????

77
MPEG7

• Multimedia Content Description Interface
• Part 1 Systems
• Part 2 Description Definition Language
• Part 3 Visual
• Part 4 Audio
• Part 5 Description Schemes
• Part 6 Reference Software
• Part 7 Conformance
• Part 8 Extraction and Use of MPEG-7 Descriptions

78

• MPEG-7?????????????,??????????????????????????????
  ???
• ??????????????????????????(??)????????????????????
  ??????????????????
• ???????????????????????????????????????
• ???????????
• ??????????????????????????????,????????????????
• MPEG-7???????????????????,??????

79

• MPEG-7???????????????,???????N????????????????????
  ?????????????????,????????????????????????????????
  ?????

80
MPEG-7?????

• ???????????
• ???????(???????)
• ????????????
• ?????
• ???????
• ????
• ???????
• ????
• ??
• ??
• ??????
• ?????????????

81

• ???,?????AV???????????????????,AV?????????????????
  ?,????????MPEG-7??????
• ??????
• ?????????????????(???)???????????,??????????????
• ??????????????????????????(??)???????
• ??????,?????????,??????,??????????????????
• ??????????,?????????????,????????????????????

82

• ??????(????)??????,??????????????????????(????)??
  ?
• ???????????????????,?????????????,???????????????
  ?

83

• MPEG-7?????????????,?????????????????????????
• ??????MPEG-7????????
• ?MPEG-7???????????????????

84
???H.264

• ????????,???????????????????????????,?MPEG-1?MPEG-
  2?H.261?H.263??
• ?????????,2001?12????JVT(Joint Video
  Team,???????),?ITU-T?ISO????????????????????????
• JVT??????????????????,????????????????????????????
  ?
• ITU-T?2003?5???????????????????????H.264/AVC,?????
  ??????????????14496-10/MPEG-4AVC?

85
H.264???

• ???????????,??H.264???????????MPEG2?1/8,MPEG4?1/3
  ?
• ???????????????????
• ???????????????????????????????????
• ??????H.264????????(Network Adaptation Layer),
  ??H.264???????????????(?????,CDMA,GPRS,WCDMA,CDMA2
  000?)?

86

• H.264???????????,???????
• H.264(MPEG-4 Part 10)??????????????(CD
  ?DVD)?????????,????????????

87
(No Transcript)