Multimedia Technology Research in HKUST

1
Multimedia Technology Research in HKUST

• Oscar Au (PhD, Princeton University)
• Dept. of ECE, HKUST
• Clearwater Bay, Hong Kong
• Tel [phone]
• Email eeau_at_ust.hk
• www.ee.ust.hk/eeau

2
Oscar Aus background

1. BA Sc - Toronto (82-86), MA/PhD/Postdoc -
   Princeton (86-92)
2. Professor, HKUST (1992 to now).
3. Director, Multimedia Technology Center, HKUST.
4. Associate Editor of 6 journals IEEE TCSVT, IEEE
   TIP, IEEE TCAS-1, JSPS, JMM, JFI
5. Chair of IEEE CAS MSATC, Vice Chair/Chair of IEEE
   SP MMSP TC, member of IEEE CAS VSPC/DSP TC, IEEE
   SP IVMSP TC, IEEE ComSoC MMTC
6. Steering Committee of IEEE TMM/ICME. IEEE
   Fellow/DLP.
7. General Chair, PCM 2007/PV 2010/ICME 2010.
8. Host MPEG in 1/2005.
9. 320 technical papers. 80 patents filed. 9
   granted. 3 book chapters.
10. 67 MPEG/HEVC/AVS contributions. Several accepted
    into MPEG4, AVS and HEVC.
11. Forensic expert. 7 press conferences. Talks/media
    interviews.

3
My multimedia expertise

• Video coding (MPEG-1/2/4, H.261/3/4, AVS),
  transcoding, enhancement, error
  resilience/concealment, denoising, deinterlacing,
  HDR, SVC, MVC, 3DTV, DVC, GPU video processing,
  depth estimation, view interpolation, subpixel
  rendering, white balancing, color demosaicking,
  compressive sensing, light weight encryption,
  watermarking, motion estimation, rate control,
  perceptual quantization
• Image coding (JPEG, JPEG2000), watermarking,
  restoration, enhancement, processing
• Audio/speech coding, watermarking, processing
• Special achievement Our fast motion estimation
  technology was accepted into MPEG-4. Up to 8000
  times faster with same quality. First ever in
  Hong Kong and China.

4
Video Processing

• Standard-based image/video compression for SDTV/
  HDTV MPEG-1/2/4, H.261/3/4, AVS, HEVC, etc.
• Patented technologies accepted into standard 1
  for MPEG-4, 2 for AVS, 2 for HEVC
• Video transcoding, ITF, HKD 4,000,000, contract
  research
• Watermarking, light-weight encryption, forensics
  10 patents, ITF, HKD 7,000,000.
• Fast motion estimation, 5 patent, HKTIIT, HKD
  10,000,000
• 3DTV, free viewpoint TV, distributed/scalable
  Video coding
• Denoising, deinterlacing, error resilience,
  enhancement
• Hardware-friendly algorithms for GPU and ASIC,
  Vision chip
• 80 patents, IEEE Fellow, Associate Editors

5
Image Processing
Capture / Display - subpixel processing (contract
GRF, HKD 1,000,000)

• High dynamic range (HDR) imaging
• Subpixel rendering
• Automatic white balancing (AWB)
• Color interpolation (demosaicing)
• Denoising, enhancement, restoration
• Watermarking, data hiding, forensic
• Halftone image
• Texture synthesis
• Segmentation,

6
Video coding (encoder optimization)

• Fast motion estimation, PMVFAST
• Fast subblock ME
• Fast multi-frame ME
• Subpixel ME
• Fast intra-prediction
• Luma/chroma coding
• Rate control (block level), R-D modeling
• Rate allocation (frame level)

7
Video coding related issues

• Denoising temporal, spatial, inter-color
• Deinterlacing
• Deblocking
• Color processing
• Transcoding
• Error resilience

8
GPU/Hardware

• GPU-based processing
• ME, intra prediction
• Hardware architecture (SW/HW co-design)
• ME, deblocking, demosaicking,
• Filtering, subpixel rendering
• Data dependency, data loading, data re-use, low
  power, low latency, reconfigurability

9
Beyond simple video coding

• Scalable video coding
• Distributed video coding
• Multiview video coding
• Intra-prediction
• Subblock motion estimation
• Multiple reference frame motion estimation
• Subpixel motion estimation
• Rate control
• JPEG2000 optimization

10
Imaging

• Demosaicking
• Automatic white balancing
• Color correction
• Subpixel rendering
• High dynamic range imaging
• Compressive sensing
• Audio variable speed playback
• Audio coding, MP3, AAC HE, SBR
• Speech recognitions

11
Security/forensics/others

• Image/halftone watermarking
• Attacks and steganography
• Capacity estimation
• Image forensics
• Light-weight encryption
• Encrypted domain processing