Avalon Imaging Codec Architecture Overview

1
Avalon Imaging Codec Architecture Overview

• Peggi Goodwin
• Lead Program Manager, ImagingWindows Client
  Platform
• Microsoft Corporation

2
Session Outline

• Problems with todays user experience
• Extensible Imaging Pipeline
• CODEC extensibility
• Pixel format extensibility
• Metadata extensibility
• From a Development Perspective
• Avalon and WinFX

3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
8 Bits Per Channel Isnt Sufficient
8
Metadata Can Be Used for Many Purposes
File Name CRW_8529.CRW Camera Model Name
Canon EOS 10D Shooting Date/Time 12/20/2004
25900 PM Shooting Mode Aperture-Priority
AE Tv( Shutter Speed ) 1/1000 Av( Aperture
Value ) 5.6 Metering Mode Center-weighted
averaging Exposure Compensation 0 ISO Speed
200 Lens 300.0mm Focal Length
420.0mm Image Size 3072x2048 Image Quality
RAW Flash Off White Balance Daylight AF
Mode One-Shot AF Parameters Contrast
Normal Sharpness Normal Color
saturation Normal Color tone
Normal Color Space sRGB File Size
6311KB Drive Mode Continuous shooting Owner's
Name Peggi Goodwin Camera Body No.
1120405343
Last Edit 2004-05-02 143709 Edited By
John Smith Type of Edit Rotate 0.7 Crop
0.116,0.166 12.444,8.418 Black Point 4 4 3
White Point 255 253 249 Curves 65,56 128,135
191,199 Color Balance S 6 -2 5 M 8 0 2 H
0 0 0 Brightness 2 Contrast 6
Unsharp Mask 50 0.9 2
Custom Function C.Fn01-1 C.Fn02-1
C.Fn03-0 C.Fn04-0 C.Fn05-3
C.Fn06-0 C.Fn07-4 C.Fn08-0
C.Fn09-2 C.Fn10-0 C.Fn11-0
C.Fn12-0 C.Fn13-1 C.Fn14-0
C.Fn15-0 C.Fn16-1 C.Fn17-0
Category Events Subcategory
Birthdays Year 2002 Month April Keywords
Party Family Friends
Funny Subject Jackie People Dad
Jackie Robin Tyrone Steve Sharing
Family Steve Maria
9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
The Solution Windows CODEC Pipeline

• Built-in CODEC support for most popular image
  formats and metadata schemas
• Extensible framework for CODECs, pixel formats,
  and metadata
• Integrated into Windows desktop for consistent
  user experience

15
CODEC Extensibility

• Platform provides CODECs for common formats
• TIFF, JPEG, PNG, BMP, GIF
• You provide CODECs for proprietary image formats
• Get same platform support as standard image
  formats
• Enable multiple applications to use your image
  format, while preserving your IP
• Integration with graphics acceleration and
  display pipeline for WinFX applications

16
Pixel Format Extensibility

• Preserve up to 32 bits per channel of image data
• Read, write, and print support (with next
  generation print path)
• Rendering support in future
• Platform provides common pixel formats
• You can develop your own pixel formats
• Provide converters to/from standard formats

17
Metadata Extensibility

• Platform supports common metadata schemas
• You define specialized metadata for your purposes
• Stored in image file
• Preservation of unrecognized metadata during
  editing

18
Imaging Architecture
Imaging APIs
CODEC Arbitrator
Image File
19
Scope of CODEC Discovery

• Global
• Register at installation for automatic run-time
  discovery
• Private
• Register at run-time, visible to current process
  only
• Application-specified
• Software specifies component by GUID or by Vendor
• Specified component must be registered

20
Participating in Automatic Discovery

• Image format specified by pattern encoded in
  image file
• GUIDs are recommended for new image formats to
  avoid pattern collisions
• Register at installation
• Specify pattern(s) of supported image format(s)
  in registry entry

21
CODEC Discovery
Install CODEC with driver or software
Register CODEC components
Application requests imaging operation
Read pattern from image file
Check registry for CODECs with matching pattern
More than one match?
Instantiate matching CODEC and perform requested
operation
No
Yes
Go to Arbitration
22
CODEC Arbitration
Go to Arbitration
Instantiate each decoder that matches pattern
Query each decoder for supported operations on
this image
Score each decoder based on sum of weights
Multiple decoders tied for high score?
No
Select highest scoring decoder
Yes
Select first in list of decoders tied for
highest score
23
DemoCODEC Registration and Discovery
24
Registering a Decoder for Discovery

• HKEY_CLASSES_ROOT\CLSID\CATID_WICBitmapDecoders\
  Instance\Your Decoder CLSID\CLSID Your
  Decoder CLSID
• HKEY_CLASSES_ROOT\CLSID\CATID_WICBitmapDecoders\
  Instance\Your Decoder CLSID\Friendly Name
  Your Decoders Name
• HKEY_CLASSES_ROOT\CLSID\Your Decoder
  CLSID\Patterns\0\Position 0
• HKEY_CLASSES_ROOT\CLSID\Your Decoder
  CLSID\Patterns\0\Length 3
• HKEY_CLASSES_ROOT\CLSID\Your Decoder
  CLSID\Patterns\0\Pattern FF D8 FF
• HKEY_CLASSES_ROOT\CLSID\Your Decoder
  CLSID\InProcServer32\(Default)
  drive\path\yourdll.dll
• HKEY_CLASSES_ROOT\CLSID\Your Decoder
  CLSID\InProcServer32\ThreadingModel Both

25
Registering a Pixel Format Converter for Discovery

• HKEY_CLASSES_ROOT\CLSID\CATID_WICFormatConverters
  \Instance\Your Format Converter
  CLSID\CLSIDYour Format Converter CLSID
• HKEY_CLASSES_ROOT\CLSID\CATID_WICFormatConverters
  \Instance\Your Format Converter
  CLSID\Friendly Name Your Format Converters
  Name
• HKEY_CLASSES_ROOT\CLSID\Your Format Converter
  CLSID\PixelFormats\ PixelFormatGuid1
• HKEY_CLASSES_ROOT\CLSID\Your Format Converter
  CLSID\PixelFormats\ PixelFormatGuid2
• . . .
• HKEY_CLASSES_ROOT\CLSID\Your Format Converter
  CLSID\PixelFormats\ PixelFormatGuidN
• HKEY_CLASSES_ROOT\CLSID\Your Format Converter
  CLSID\InProcServer32\(Default)
  drive\path\yourdll.dll
• HKEY_CLASSES_ROOT\CLSID\Your Format Converter
  CLSID\InProcServer32\ThreadingModel Both

26
Registering a Metadata Reader for Discovery

• HKEY_CLASSES_ROOT\CLSID\CATID_WICMetadataReaders
  \Instance\Your Metadata Handler CLSID\CLSID
  Your Metadata Handler CLSID
• HKEY_CLASSES_ROOT\CLSID\CATID_WICMetadataReaders
  \ Instance\Your Metadata Handler
  CLSID\Friendly Name Your Metadata Handlers
  Name
• HKEY_CLASSES_ROOT\CLSID\Your CLSID\Containers\
  ContainerFormatGUID\0\Position 0
• HKEY_CLASSES_ROOT\CLSID\Your CLSID\Containers\
  ContainerFormatGUID\0\Pattern 74 58 4D 50
• HKEY_CLASSES_ROOT\CLSID\Your CLSID\Containers\
  ContainerFormatGUID\0\Mask FF FF FF FF
• HKEY_CLASSES_ROOT\CLSID\Your CLSID\Containers\
  ContainerFormatGUID\0\DataOffset 4

27
Participating in Arbitration

• IWICBitmapDecoder.QueryCapability ( IStream
  pStream, DWORD pCapability )
• enum WICBitmapDecoderCapabilities
• WICBitmapDecoderCapabilitiesSameEncoder,
• WICBitmapDecoderCapabilityCanDecodeAllImages,
• WICBitmapDecoderCapabilityCanDecodeSomeImages,
• WICBitmapDecoderCapabilityCanDecodeThumbnail,
• WICBitmapDecoderCapabilityCanEnumerateMetadata

28
CODEC Class Hierarchy
IUnknown
IWIC Imaging Factory
IWIC Palette
IWIC Bitmap Decoder
IWIC Bitmap Source
IWIC Bitmap Encoder
IWIC Bitmap FrameEncode
IWIC Bitmap
IWIC Bitmap FlipRotator
IWIC Format Converter
IWIC Bitmap FrameDecode
IWIC Bitmap Scaler
IWIC Bitmap Clipper
29
CODEC Extensibility Interfaces

• Decoders implement
• IWICBitmapDecoder
• IWICBitmapFrameDecode
• IWICMetadataBlockReader
• IWICBitmapSource

• Encoders implement
• IWICBitmapEncoder
• IWICBitmapFrameEncode
• IWICMetadataBlockWriter

• Pixel Format Converters implement
• IWICFormatConverter
• IWICBitmapSource

30
Metadata Handler Class Hierarchy
IUnknown
IWIC MetadataQueryReader
IWIC StreamProvider
IPersist
IPersist Stream
IWIC MetadataQueryWriter
IWIC PersistStream
31
Metadata Handler Extensibility Interfaces

• Medata Readers implement
• IWICMetadataReader
• IWICPersistStream
• IWICStreamProvider
• Metadata Writers implement
• IWICMetadataWriter
• IWICPersistStream
• IWICMetadataReader
• Use UnknownMetadataHandler to read/write
  unrecognized blocks

32
Imaging as Part of the Developer Platform

• Avalon is the next generation platform for
  Windows client applications and content
• Part of WinFX, along with Indigo and WinFS
• Consistent programming model for user interface,
  graphics, media, and documents
• Uses Direct3D as its hardware acceleration
  abstraction
• Printing and remoting pipelines being evolved to
  match rendering model and provide greater
  fidelity
• Releasing as part of Windows codenamed Longhorn
• Concurrent release for Windows XP SP2 and
  Windows Server 2003

33
Summary

• Providing CODECs for proprietary image formats
  gives you platform support while preserving your
  IP
• Preservation of up to 32 bpc of image data
  supports broader color gamuts and greater dynamic
  range
• You can define new types of metadata and store it
  in the image file, just like standard metadata

34
Call to Action

• Use the platform provided CODECs for common image
  formats
• Migrate your proprietary image CODECS to the new
  platform
• Write metadata handlers to read, write, and
  update specialized metadata
• Take advantage of support for 32 bpc image data

35
Community Resources

• Community Sites
• http//www.microsoft.com/communities/default.mspx
  
• List of Newsgroups
• http//communities2.microsoft.com/communities/news
  groups/en-us/default.aspx
• Attend a free chat or webcast
• http//www.microsoft.com/communities/chats/default
  .mspx
• http//www.microsoft.com/seminar/events/webcasts/d
  efault.mspx
• Locate a local user group(s)
• http//www.microsoft.com/communities/usergroups/de
  fault.mspx
• Non-Microsoft Community Sites
• http//www.microsoft.com/communities/related/defau
  lt.mspx

36
Additional Resources

• Email codecs _at_ microsoft.com
• Web Resources
• Online Developer Center http//msdn.microsoft.com
  /longhorn/
• Community preview builds for Windows XP SP2 and
  Windows Server 2003 available now
• SDK http//winfx.msdn.microsoft.com/
• Avalon Newsgroup microsoft.public.windows.develop
  er.winfx.avalon
• Related Sessions
• Advances in Windows Printing Overview
• Windows Color System Overview
• Ask the Experts
• Windows Color
• Windows Printing
• Avalon Imaging