IT212, How Computers Work Multimedia

1
IT-212, How Computers WorkMultimedia

• Electrical and Computer Engineering
• Spring 2002

2
Multimedia Requirements

• MP 3
• Minimum industry standard
• Must include
• Sound card (sounds played and recorded)
• Speakers
• Hardware and software support for displaying
  video and animations
• Minimum CD speed

3
Additional MM Elements

• Video Support for 3D Rendering
• MPEG for Full Screen Video and Animation
• DVD Drives
• Enhanced TV
• Interactive, replay/pause, etc.
• Random Access to Video/audio
• Virtual Reality
• Total immersion into created environment
• seeing, hearing, feeling, positional orientation

4
CD-ROMs

• Compact Disk, Read Only Memory
• Large Storage Capacity
• 650MB if both sides are used, usually not done
• Use for Large Data Not Needing Update
• Clip art
• Photographs
• Large programs
• Book volumes

5
CD-R, CD-RW

• CD-R
• Compact Disk, Recordable
• Record only once (WORM)
• CD-RW
• Compact Disk, ReWritable
• Record and erase multiple times

6
DVD

• Digital Video Disk
• Digital Versatile Disk
• Large Capacity, 8.5 GB
• Movies
• Huge data storage
• Maps
• Phone directories

7
CD-ROM Technology

• Data Stored in Spiral Sectors
• Hard and floppy drives use concentric tracks
• Variable Rotation Speed
• Constant tangential velocity, i.e., data moves at
  same speed over optical sensor
• Faster near center

8
CD-ROM Technology
Outer Circle 7 sectors
Inner Circle 4 sectors
When reading outer circle, need to rotate slower
so that time to read each sector is same
9
CD-ROM Technology

• Buried Surface Contains Lands and Pits
• Detector Shines Laser Beam Through Plastic
  Protective Layer to Buried Data Layer
• Light that strikes pits is scattered and not
  reflected
• Light that strikes lands is reflected directly
  back towards detector and redirected via prism to
  photo-diode
• Photo-detector converts light/no-light to
  electrical signal indicating 1 or 0

10
CD-R Technology

• Disk Layers
• Clear Polymer (laser side)
• Clear lacquer
• Gold reflective
• Dye (usually green)
• Polycarbonate (Lexan) plastic structural layer
  (label side)

11
CD-R Technology

• Disk Surface Pregrooved
• ATIP absolute timing in pregroove
• Groove is modulated
• Laser/photodetector measures modulation and
  adjusts motor to maintain constant
  medium/detector relative speed
• Laser Beam Heats Buried Layer and Produces Pits
  by Either
• Bleaching dye layer
• Distorting polycarbonate plastic layer
• Producing bubble in dye layer

12
DVD Technology

• Disk Layers
• Symmetric, read from both sides
• Multiple data layers on each side
• Laser focused through one data layer to next
• Spiral in on one layer, out on other
• No need to reposition head in middle of recording
• Shorter Wavelength Light Allows Focusing to
  Smaller Size

13
CD-RW and DVD RAM

• Recording Layer Is Made of a Material Which Can
  Be Either Crystalline (Reflective) or Amorphous
  (Non-reflective)
• Write
• Do nothing to make land
• To make pit, laser beam strikes buried layer and
  heats spot to 900-1300F
• Changes crystalline spot to amorphous
• Erase
• Amorphous spots can be annealed back to
  crystalline state using lower energy beam (400F)

14
Sound As Input

• Sound From Microphone or Line Enters Sound Card
  As an Analog Signal
• ADC Digitizes Sound Feeds to Digital Signal
  Processor (DSP) Chip
• DSP Analyzes/alters Digital
• Reduce noise
• Compress data
• Data Is Sent to CPU, Which Can Store Data on Hard
  Drive (in WAV or MP3 Formats)

15
Sound As Output

• Reverse Path
• Hard drive
• CPU
• DSP (to uncompress)
• DAC
• Speaker amplifier
• Speakers

16
MIDI and FM Synthesis

• Musical Instrument Digital Interface
• Need to Reduce Size of .WAV Files
• Predefine Selection of Musical Instruments
• Define music as collection of instruments playing
  specific notes, e.g., musical score

17
Synthesis alternatives

• Wavetable Synthesis
• Recorded versions of actual instruments (stored
  in Sound Card ROM) are used.
• Pitch may be shifted up or down by DSP if
  necessary
• FM Synthesis
• Separate chip on sound card (FM synthesis card)
  attempts to recreate sound of instrument by
  generic programming routine

18
Audio DSP, Sound Compression

• Alternative to MIDI Is Compression
• Simplify sounds to those that the human ear can
  detect
• Applies to changes in volume, pitch, and
  overtones
• Fletcher-Munsen curves
• Compressed file typically 10 of original
• Compression used in Dolby and MP3

19
Audio DSP, Noise Reduction

• Tape Recording Has High Frequency Noise Called
  Hiss
• Hiss reduces signal to noise ration at high audio
  frequencies
• Dolby Digital B
• During recording, maintain constant SNR by
  increasing amplitude of high frequencies
• During playback, reduce amplitude of high
  frequencies by amount they were increased

20
Dolby Digital Sound

• Six Channels Are Recorded
• Five channels in full frequency range
• Left, center, right stereo (front)
• Left and right surround (behind)
• Sixth channel for low frequency effect (LFE)
• Low frequencies are not directional
• Bandwidth of Each Channel Adjusted On-the-fly
  Center Channel Normally Allocated Most Bandwidth
• Fewer Than 6 Channels Can Be Used by Remixing

21
3D Audio

• How Do Our Ears Locate Where a Sound Is Coming
  From?
• Intensity (how loud), one ear vs. the other
• Timing closer ear hears note before other
• Position of ear relative to noise source (e.g.,
  is ear directly facing sound, or is sound
  coming in at an angle)

22
Creating 3D Sound

• Immersion Game
• Player virtually moves with virtual environment,
  sound affected by location
• For each sound source, two spheres are defined
• Region outside of which this sound source is not
  detected
• Within sphere, volume and nature of sound
  increases as player moves closer to source
• Inner sphere
• volume limited to maximum

23
MP-3

• MP-3 Approach Large Audio Files
• Reduced sampling
• 16,000 (music), 10,000 (voice) per second (vs.
  44,000 per second for CD
• Compress out differences ear unable to detect
• Compression methods reduced storage requirements
  to 1/10 original
• Conversion from CD to MP-3 performed by ripper
  (utility program)

24
Portable MP-3 Players

• Music Downloaded From PC Via Parallel Port Into
  32 to 64MB of Flash Memory
• 32 MB can store 30min of CD quality music, or 1
  hour of reduced quality (voice grade) music
• Field Programmable Gate Array Chip Converts MP-3
  Data Into Uncompressed Digital Form for DAC
• Lack of Moving Parts Minimizes Power Requirements

25
Multimedia Video

• PC Video vs. VCRs
• Random Access
• Quick access
• Record while playback
• Real-time pause and rewind
• Videoconferencing
• Digital vs. Analog Implementation

26
Compression Methods

• Delta Video (Delta Modulation)
• Transmit differences from previous frame only
• Unsuitable for noisy communications link
• MPEG
• Records only key frames and reconstructs
  intermediate frames by comparing transitions from
  one key frame to the next
• Lossy Compression
• Eliminate unessential differences within
  displayed images

27
Virtual Reality

• 3D Feedback (Audio, Video, Sensory) Guided by
  Personal Interaction
• No Precise Definition of VR Components
• Partial immersion (joystick, 3D mouse, monitor,
  software
• Total immersion (head mounted displays, data
  gloves, etc.)
• Total Immersion Typical Elements
• Software continuously calculating relative
  position
• Calculates binocular vision

28
3D Graphic Rendering

• In 3D, Each Point Is Described by Its Horizontal
  and Vertical Position and Depth
• Each Surface Represented by Triangles
• Geometry DSP computes
• Relative position of each triangle
• Effect of viewing angle
• Intensity, texture, reflections
• Requires extensive use of floating point math
• Need to redraw 15 to 20 times per second to make
  rotation look smooth

29
Additional Graphic Features

• Shading
• Texture Maps
• Repeated bitmaps used as wall paper
• Pixelation can be corrected by creating an
  additional variation of texture map designed
  specifically (and only) for close-up viewing (MIP
  Mapping)
• Perspective Correction alters form of texture map
  as map gets deeper into view (narrows, alters
  details)

30
Distant object alteration

• Accounts for Fact That Objects Further Away Are
  Less Distinct Than Close in
• Reduces Detail That Must Be Computed by Rendering
  Engine
• Fogging Blends White Into Distant Objects to
  Create Hazy Horizon
• Depth Cueing Adds Black at End of Halls

31
Graphic Features

• Alpha Blending
• Merges two different texture maps on a pixel by
  pixel basis by taking a percentage of each color
  at each point
• Stippling
• Draws background color and intersperses
  occasional texels from other texture map
• Bilinear Filtering
• Smoothes edges of textures by blending color
  transitions

32
3D Video Cards

• Most Recently Used Texture Maps Are Stored on
  Card to Accelerate Retrieval
• 3D Graphics Coprocessor Can Quickly Compute
  On-the-fly 3D Rendering
• 2D Graphics Coprocessor Can Accelerate
  Representation of 2D Objects Such As Bitmaps and
  Icons
• VRAM
• Simultaneous reading and writing