Wireless Video Conferencing

1
Wireless Video Conferencing

• EECS150 Spring 2007 - Lab Lecture 10
• Neil Warren
• Allen Lee

2
Checkpoint 3 Information(1)

• Check off Requirement
• 1) Be able to set 2 sets different source and
  destination addresses on 2 boards and
  communicate. (CP4 video streaming)
• 2) Be able to set both sides destination
  address to 0xFF and a unique source address on
  each and have communications occur. (CP4
  Handshake)

3
Checkpoint 3 Information

• If youre behind, you can forfeit checkpoint 3
  and receive a black box.
• You will receive a 0 for CP3 but will be able to
  move on to CP4 which is worth many more points.
• Talk to Neil if you want this option.
• You lose the points before you get the BB.

4
Welcome to Checkpoint 4

• Combine CP2 and CP3 to make two-way wireless
  video conferencing system
• Note Specification has changed to make your life
  much easier based on questions from students so
  far. Make sure you work with the non-draft
  version of the Checkpoint4.doc

5
Overview

• Revisit CP2 and CP3
• Communications protocol
• Video compression
• Picture-in-picture display

6
Recall from Checkpoint 2
7
Checkpoint 4 Datapath
8
Our Advice (1)

• Separate the work between you and your partner to
  make the best use of your time.
• Communications System
• Graphics System
• Agree on an interface and make sure you conform
  to it.

9
Our Advice (2)

• Debug Independently!
• Communications System
• Assume that the graphics system works perfectly
  (conforms to your interface).
• Hardcode in data and requests using timers and
  debug the system by using the packet sniffer.
• Graphics System
• Assume that the communications system works
  perfectly.
• Hardcode transmission inputs, and whatever
  interface method you use for send/receive
  acknowledgements.
• Much easier to debug each piece, then just put
  the 2 working pieces together.
• If you conform to your interface it should work!

10
Communications Module (1)

• Responsibilities
• Maintain communications link.
• Lost packet robustness
• Provide cross-compatibility
• Conform to communications protocol
• Encode and Decode packets and pass data to
  graphics engine
• Handle the initial handshake between master and
  slave.

11
Communications Module (2)

• Recommended Pieces
• State Machine
• Keeps track of the current state of the
  communications module.
• Handles handshaking and properly re-synchronizing
  the communications link after lost packet.
• Sends commands to Packet Encoder, receives
  updates from Packet Decoder
• Packet Encoder
• Generate payload data for each type of packet and
  handle handshake with transceiver.
• Take commands from state machine.
• Packet Decoder
• Passes data to the graphics engine and gives
  status updates to the state machine.

12
Communications Module (3)

• Initial Handshaking
• Purpose
• To allow any 2 boxes without knowledge of the
  others address to communicate.
• To set up the timing of master and slave send and
  receives.
• Note At the beginning both the master and slave
  begin with their destination addresses being 0xFF
  and their source addresses being unique

13
Communications Module (4)

• Handshake Method
• Master broadcasts Master-Seek packets once
  every 2.4ms. When a slave receives a
  Master-Seek packet it sends a Slave-ReqAck
• When the master receives the Slave-ReqAck
  packet it changes its destination address to the
  source address of the slave and then transmits a
  Master-Ready packet with the new destination
  address.
• When the slave receives a Master-Ready packet
  the slave sends a Slave-Ready packet and moves
  into a state where it can receive active video
  packets from the master. The Slave-Ready packet
  should be sent with its destination address as
  the source of the address of the master.
• When the master receives the Slave-Ready ready
  packet, it sends its first packet of active video
  data.
• Now the communication should alternate between
  the master and slave transmitting active video
  data.

14
Communications Module (5)

• Communications Protocol
• Purpose
• To allow cross-compatibility between different
  implementations.
• Abstracts away the communications aspect of the
  project for the graphics module.
• The 8-bit header designates the type of packet.
• The 8-bit line number indicates which line the
  respective active video data corresponds to.
• The 80-bit data contains the AV data for 32
  pixels. (16 pixel pairs)
• 8 bit headers and line numbers make it easy to
  view on the packet sniffer!

15
Communications Module (6)
Type 8-Bit Header 8-Bit Line Number 80-Bit Data Payload
Active Video 1 8d1 8bXmit Line SAMPLE 0-31
Active Video 2 8d2 8bXmit Line SAMPLE 32-63
Active Video 3 8d3 8bXmit Line SAMPLE 64-95
Active Video 4 8d4 8bXmit Line SAMPLE 96-127
Active Video 5 8d5 8bXmit Line SAMPLE 128-159
Active Video 6 8d6 8bXmit Line SAMPLE 160-176
Master-Seek 8d23 XX 80h2
Slave-ReqAck 8d24 XX 80h1
Master-Ready 8d25 XX 80h2
Slave-Ready 8d26 XX 80h1
16
Communications Module (7)

• Time Outs let you know if a communication loss
  has occurred.
• After the expiration of 10ms from the last AV
  packet transmission, if the slave not does
  receive a packet it should return to a state
  where it can receive AV packets from the master.
• If this occurs 25 consecutive times without ever
  receiving an AV packet, then a communication loss
  is declared and the system must be reset. A
  message should appear on screen indicating a
  communication loss has occurred.
• After the expiration of 10ms from the last AV
  packet received, if the master does not receive a
  packet it should send its next AV packet.
• If this occurs 25 consecutive times without ever
  receiving an AV packet, then a communication loss
  is declared and the system must be reset. A
  message should appear on screen indicating a
  communication loss has occurred.

17
Communications Module (8)

• Tips
• Make use of the packet sniffer to determine what
  is actually flying through the air.
• The communications protocol was re-designed
  specifically for this purpose.

18
Video Compression (1)

• Wireless transfer is the bottleneck
• Uncompressed video
• 507x720x16 5,840,640 bits per frame
• Thats 175 Mbits per second!
• Theoretical max bandwidth for wireless is
  256kbps, so 22.8 seconds per frame.

19
Video Compression (2)

• Simple lossy compression algorithm
• Resolution 176x127
• 5 bits grayscale per pixel pair

20
Video Compression (3)

• Compressed Video
• 176x127x5/2 55,880 bits per frame
• Approximately 1051 compression
• At theoretical max wireless bandwidth, thats 4.5
  frames per second

21
Video Compression (4)

• Vertical compression
• Sample every other odd active line
• (254 active odd lines)/2 127 lines
• Horizontal compression
• Sample every fourth pixel pair (except for the
  last eight)
• (360-8)/4 88 pixel pairs 176 pixels
• For each sampled pixel pair, keep only the upper
  five bits of the average luminance

22
Video Compression (5)

• Why 176x127 instead of 180x127?
• Good question. Theres a good answer, figure it
  out!

23
Video Decompression

• Reverse of compression, but lossy due to lack of
  chrominance
• 5-bit grayscale represents a pixel pair
• Decompresses to 32 bits, with appropriate padding
  for chrominance values

24
Writing Compressed Frames

• Set aside an area in SDRAM for sending and
  receiving compressed frames
• Update the stored frame to send only after the
  entire frame has been sent
• It is okay to lose a local frame to store a
  compressed one

25
Writing Compressed Frames

• Sequence for handling compressed frames
• Wait for SDRAM and ChipCon initialization
• Store a compressed frame
• Tell transmitter FSM that frame in SDRAM is ready
• Wait as compressed frame is sent
• Receive signal from transmitter that last packet
  for frame has been sent
• Once entire compressed frame is sent, store a new
  compressed frame

26
Writing Compressed Frames

• Two options for implementing compressed video
  controller
• Merge with VD processor, using the same FIFO
• Advantage Arbiter sees one write port for local
  video
• Drawback Need to cross clock boundaries
• Separate module with own FIFO
• Advantage May not need to cross clock boundaries
• Drawback Arbiter sees two write ports for local
  video
• Pick the one that makes more sense to you

27
Dual-Clock Communication

• Modules running on the camera clock and the
  system clock cannot safely communicate with
  direct control signals.
• If your modules need to communicate but are on
  different clocks, you must either implement a
  safe handshaking protocol or use the memory
  element we provide.
• For your convenience, weve included a
  dual-clocked one-bit-wide FIFO to safely exchange
  control signals, but you dont have to use it.

28
Sender and Receiver FSMs

• Sender/Transmitter
• Waits for frame to be available in SDRAM
• Reads from SDRAM and packages data into a packet
• Requests a new frame once last packet for frame
  is sent
• Receiver
• Decodes the compressed data
• Writes received data into SDRAM

29
Displaying Compressed Video
30
Displaying Compressed Video

• Constantly display compressed frames to avoid
  SDRAM decay
• This way, you dont have to auto-refresh!

31
Multi-Port Arbiter

• At least two more ports are needed for the sender
  and receiver FSMs.
• A fifth port may be needed depending on how you
  handle writing compressed frames.

32
Tips, Hints, Common Pitfalls

• Start early!
• If youre already done with CP3, start now!
• We are not providing any Verilog besides the
  optional one-bit-wide memory element.
• Run the TA solution to see what the final result
  should look like.

33
Tips, Hints, Common Pitfalls

• Split up the project into graphics and
  communications.
• These parts are fairly independent so you and
  your partner can work in parallel.
• Do not combine until youve verified each part
  works independently.
• It can take up to 25 minutes to push to board
  after combining!

34
Tips, Hints, Common Pitfalls

• Work Incrementally
• Display a sectioned screen
• Display only compressed video at full frame rate
• Simultaneously display local compressed and
  uncompressed video at full frame rate
• Display uncompressed video at full frame rate and
  local compressed at 1 FPS
• Test sender and receiver FSMs by sending local
  packets to yourself

35
Tips, Hints, Common Pitfalls

• Be careful how you handle the half-full packet
  (packet 6)
• Make sure the receiver knows which half of the
  payload to look at for the video data

36
Tips, Hints, Common Pitfalls

• 2 Clocks with phase offset
• Camera Clock ! Normal Clock
• Use the 1-bit wide dual-clocked fifo we provide
  to cross clock boundaries.

37
Requirements

• Must be TA Solution Compatible.
• EC can be non-compatible, but you must still
  develop a compatible solution first.
• 1 bit file. Both master and slave systems should
  contained on 1 bit file.
• Onscreen prompts. Should display to the user the
  communications status (see TA solution for more
  info)
• Some method of outputting the (max) time between
  new frames.

38
Extra Credit

• Maximum 20
• Early check off 10
• Some Ideas (EC TBD)
• Faster frame rate
• Larger packet sizes
• Faster transmission
• Non-full duplex communications
• UI options
• Menus
• Video output modifications (change contrast with
  buttons, etc.)
• Use N64 controller for extra buttons to add
  functionality
• We will provide a black box to interface
• Open-ended! Come up with something special, cool,
  clever, unique, and/or challenging.

39
Questions?