Multimedia- and Web-based Information Systems

1
Multimedia- and Web-based Information Systems

• Lecture 5

2
Multimedia Color- and Video-technology
3
Video-Technology

• Television- and Video-Technology form the basis
  of the medium motion picture
• Generation
• Recording from the real world
• Synthesis on the basis of a description
• Analogous and digital technology

4
Representation of the video signal

• Representation of the video signal contains
• Visual representation
• Transmission
• Digitalization

5
Visual Representation

• Presentation of the video signal trough a CRT
  (Cathode Ray Tube)
• In television and computer screens
• Representation of a scene as realistic as
  possible
• Delivery of the space and time content of a scene

6
Fundamentals of visual representation

• Resolution
• Width W
• Height H
• E.g. W833, H625
• Width/height-relation
• 43 or 169
• Perception of depth
• In the natural preception trough the use of both
  eyes (different view angles onto one scene)
• Focus-depth of the camera, appearance of the
  material of an object

7
Fundamentals of visual representation

• Luminance / Chrominance
• Motion picture resolution / continuity
• Discreet sequence of single pictures can be
  perceived as a continually sequence
• Boundary of motion picture resolution
• 15 pictures/sec (video used 30 pictures/sec)
• No boundary with acoustic signals

8
Fundamentals of visual representation

• Flicker
• With small refresh rate
• Eg. 50 or 60 Hz
• Full and half pictures (interlacing)

9
RGB Color Coding

• RGB (Red Green Blue)
• Additive color blend
• Normalization of values (RGB1)

10
YUV Color Coding

• For the human eye, brightness is more important
  than color information
• Brightnessinformation (Luminance)
• 1 channel of luminance (Y)
• Color Information (Chrominance)
• 2 channels of chrominance (U and V)

11
Component Coding YUV

• Y 0.30 R 0.59 G 0.11 B
• U 0.493 (B-Y)
• V 0.877 (R-Y)
• Errors in Y are more severe
• Y to be encoded with high bandwidth
• YUV Coding often specified with a raito of the
  channels (422)

12
Component Coding YUV

• YIQ (similar to YUV)
• Derived from NTSC
• Y 0.30 R 0.59 G 0.11 B
• I 0.60 R 0.28 G 0.32 B
• Q 0.21 R 0.52 G 0.31 B

13
Shared Signal

• Individual components (RGB, YUV, YIQ) need to be
  combined to one signal
• Methods of modulation to avoid interference

14
Video formats

• Resolution of a picture (frame)
• Quantisation
• Framerate
• Video controller
• Dedicated video memory

15
Video formats

• CGA (Color Graphics Adapter)
• 320x200, 4 colors, 16.000 bytes
• EGA (Enhanced Graphic Adapter)
• 640x350, 16 colors, 112.000 bytes
• VGA (Video Graphic Array)
• 640x480, 256 colors, 307.200 bytes
• XVGA (eXtended Video Graphic Array)
• 1024x768, 256 colors, 768.423 bytes
• XGA (eXtended Graphic Array)
• 1024x768, 16M colors, 2304 kbytes
• Many more

16
Conventional Systems

• NTSC (National Television Systems Commitee)
• From the USA, oldest standard, widely used, 30
  Hz, 525 lines
• SECAM (Sequential Coleur avec Memoire)
• France, Eastern Europe, 25 Hz, 625 lines
• PAL (Phase Alternating Line)
• Western Europe, 25 Hz, 625 lines

17
High-Definition Television (HDTV)

• Resolution
• 1440x1152 / 1920x1152
• Frame rate
• 50 or 60 Hz
• No longer interlaced

18
Digitalisation of video signals

• Conversion into a digital representation
• Nyquist-Theorem (bandwidth half the sampling
  rate)
• Of the components
• Quantisation
• 2 Alternatives
• Shared Coding
• Component Coding

19
Shared Coding

• Scanning of the whole of the analogue video
  signal (e.g. composite video)
• Dependant on the standard
• Bandwidth the same for all components
• Disadvantage low in contrast

20
Component Coding

• Separate digitalisation of the components (e.g.
  YUV)
• Ratio 422
• 864 scan values for luminance
• 432 scan values for chrominancy

21
Digital Television

• Digital Television Broadcasting (DTVB)
• Digital Video Broadcasting (DVB)
• DVB-T (terrestric broadcast)
• System description
• Implementation of HDTV
• Employs MPEG-2
• Coding of Audio and Video

22
Advantages of DVB

• Increase in the number of TV-channels
• Adaptable picture and sound quality
• Encryption possible for Pay-TV
• New Services Data broadcast, Multimedia
  broadcast, Video-on-Demand
• Convergence of PC and TV

23
Multimedia Data Compression
24
Data Compression

• Audio and Video require lots of storage space
• Increasing Demand
• Text Single Pictures Audio Motion Picture
• Data rates influence
• Transmission
• Processing
• Efficient Compression
• Theory
• Standards

25
Storage Space / Bandwidth

• Considerable storage capacity for uncompressed
  pictures, audio and video data
• For uncompressed Video, even a DVD is not
  sufficient
• Uncompressed Audio-/Videodata requires very high
  bandwidth

26
Required Storage Space

• Text
• 80 x 60 2 bytes 9600 bytes 9,4 KByte
• Figures
• 500 primitives 5 Bytes for properties 2500
  bytes
• Voice
• 8 kHz, 8 bit quantisation 8 kByte / s
• Audio
• 2 x 4410016 bit / 8 bit 1 byte 172 Kbyte / s
• Video
• 640 x 480 3 x 25 frames 22,500 Kbyte /s

27
Important Methods

• JPEG (JPEG 2000)
• For single pictures
• H.261 and H.263
• Video sequences of small resolution
• MPEG 1,2 and 4
• Motion Picture and Audio (MPEG Layer 3)

28
Demands on Methods

• Good quality
• Small complexity
• Effective implementation
• Time boundaries with decompression (and
  compression)
• MPEG-1 high effort with compression

29
Demands in Dialogue mode

• End-to-End latency
• Part of the (De-)Compression lt 150 ms
• 50 ms -gt natural dialogue
• Additionally all latencies of the network,
  communication protocols and of the in- and output
  devices

30
Demands in Query mode

• Fast Forward / Rewind with simoultaneuos display
  of the data
• Random access to single frames
• lt 0.5 s
• Decompression of single pictures without
  interpretation of all the frames before them

31
Demands in Dialogue and Query mode

• Format independent of screen size and refresh
  rate
• Audio and video in different qualities (to adapt
  to the respective circumstances)
• Synchronisation of Audio and Video
• Implementation in software

32
Classification of compression methods

• Entropy coding
• Lossless methods
• Source coding
• Often lossy
• Hybrid coding
• Combined application of both of the methods above
  for a specific scenario

33
Entropy coding

• Independent of media specific properties
• Data to compress is a sequence of digital data
  values
• Losslessness
• Data before and after the compression/decompressio
  n are identical

34
Source coding

• Usage of the semantics of the information
• Compression ratio depends on the specific medium
• Data before and after the compressen/decompression
  are very similar to each other but no longer
  identical

35
Hybrid coding

• Combination of entroy and souce coding, used e.g.
  In
• JPEG
• MPEG
• H.263

36
Decompression

• Inverse function of the compression
• Decompression possible in real time?
• Symmetric methods
• Similar effort for coding and decoding
• Assymetric method
• Decoding possible with smaller effort

37
Run length encoding

• Sequence of identical bytes
• Number of repeating bytes
• Mark M (e.g. !)
• Stuffing if M is in the data space
• Example 1 0, !, 256
• Example 2 !, ! (Stuffing)
• In what cases does it help? Maximum saving?

38
Suppression of null values

• Special case of run length encoding
• Selection of a single character that is repeated
  often (e.g. 0)
• Mark M, after that number of repetitions
• In what cases does it help? Maximum saving?

39
Vector quantisation

• Splitting of the data stream into blocks of n
  bytes
• Table with patterns for blocks
• Index into the table to the entry most similar to
  the block
• Multi-dimensional table -gt vector
• Approximation of the original data stream
• Example

40
Pattern Substitution

• Patterns of frequent occurence replaced by one
  byte
• Mark M, then index into a table
• Well suited for text
• e.g. keywords in programming languages

41
Diatomic Encoding

• Putting together of two bytes of data at a time
• Determination of the byte-pairs occuring most
  frequently
• e.g. in the English language
• E, T, TH, RE, IN, ... (8 in total)
• Special bytes not occuring in the text used to
  represent 2 letters
• Reduction in data of ca. 10

42
Static encoding

• Frequency of occurence of a character
• Different coding length for characters
• Basis of the Morse code
• Important unambigous decompression

43
Huffmann coding

• Regards the probability of occurence
• Minimum number of bits for given probability of
  occurence
• Characters occuring most often get the shortest
  code words
• Binary tree (Nodes contain probabilities, edges
  bit 0 or 1)

44
Huffmann coding

• P(A)0.16, P(B)0.51, P(C)0.09, P(D)0.13 and
  P(E)0.11

45
Huffmann Coding
P(ADCEB)1.0
1
0
P(B)0.51
P(ADCE)
0
1
P(CE)0.20
P(AD)0.29
0
1
1
0
P(C)0.09
P(E)0.11
P(D)0.13
P(A)0.16

• w(A)001, w(B)1, w(C)011, w(D)000, w(E)010

46
Transformation coding

• Data transformed into a better suited
  mathematical space
• Inverse Transformation needs to be possible
• Discrete Cosine-Transformation (DCT)
• Fast-Fourier-Transformation (FFT)
• See example in the JPEG lecture

47
Prediction or relative encoding

• Forming the difference to the previous value
• Data do not differ much
• Combination of methods
• e.g. homogenous areas in pictures
• DPCM, DM and ADPCM

48
Further Methods

• Color tables
• with pictures (video)
• Muting
• Threshold for sound volume