Introduction to Python, COM and PythonCOM

1
Introduction to Python, COM and PythonCOM

• Mark Hammond
• Skippi-Net, Melbourne, Australia
• mhammond_at_skippinet.com.au
• http//starship.python.net/crew/mhammond

2
Introduction to Python, COM and PythonCOM

• The plan
• Section I - Intro to Python
• 30-45 mins
• Section II - Intro to COM
• 30-45 mins
• Python and COM
• The rest!

3
Section I

• Introduction to Python

4
What Is Python?

• Created in 1990 by Guido van Rossum
• While at CWI, Amsterdam
• Now hosted by centre for national research
  initiatives, Reston, VA, USA
• Free, open source
• And with an amazing community
• Object oriented language
• Everything is an object

5
Why Python?

• Designed to be easy to learn and master
• Clean, clear syntax
• Very few keywords
• Highly portable
• Runs almost anywhere - high end servers and
  workstations, down to windows CE
• Compiled to machine independent byte-codes
• Extensible
• Designed to be extensible using C/C, thereby
  allowing access to many external libraries

6
Most obvious and notorious features

• Clean syntax plus high-level data types
• Leads to fast coding
• Uses white-space to delimit blocks
• Humans generally do, so why not the language?
• Try it, you will end up liking it
• Variables do not need declaration
• Although not a type-less language

7
Pythonwin

• We are using Pythonwin
• Only available on Windows
• GUI toolkit using Tkinter available for most
  platforms
• Standard console Python available on all
  platforms
• Has interactive mode for quick testing of code
• Includes debugger and Python editor

8
Interactive Python

• Starting Python.exe, or any of the GUI
  environments present an interactive mode
• gtgtgt prompt indicates start of a statement or
  expression
• If incomplete, ... prompt indicates second and
  subsequent lines
• All expression results printed back to
  interactive console

9
Variables and Types (1 of 3)

• Variables need no declaration
• gtgtgt a1gtgtgt
• As a variable assignment is a statement, there is
  no printed result
• gtgtgt a1
• Variable name alone is an expression, so the
  result is printed

10
Variables and Types (2 of 3)

• Variables must be created before they can be used
• gtgtgt bTraceback (innermost last) File
  "ltinteractive inputgt", line 1, in ?NameError
  bgtgtgt
• Python uses exceptions - more detail later

11
Variables and Types (3 of 3)

• Objects always have a type
• gtgtgt a 1gtgtgt type(a)lttype 'int'gt gtgtgt a
  "Hello"gtgtgt type(a)lttype 'string'gtgtgtgt
  type(1.0)lttype 'float'gt

12
Assignment versus Equality Testing

• Assignment performed with single
• Equality testing done with double ()
• Sensible type promotions are defined
• Identity tested with is operator.
• gtgtgt 111gtgtgt 1.011gtgtgt "1"10

13
Simple Data Types

• Strings
• May hold any data, including embedded NULLs
• Declared using either single, double, or triple
  quotes
• gtgtgt s "Hi there"gtgtgt s'Hi there'gtgtgt s
  "Embedded 'quote'"gtgtgt s"Embedded 'quote'"

14
Simple Data Types

• Triple quotes useful for multi-line strings
• gtgtgt s """ a long... string with "quotes" or
  anything else"""gtgtgt s' a long\012string with
  "quotes" or anything else' gtgtgt len(s)45

15
Simple Data Types

• Integer objects implemented using C longs
• Like C, integer division returns the floor
• gtgtgt 5/22
• Float types implemented using C doubles
• Long Integers have unlimited size
• Limited only by available memory
• gtgtgt long 1L ltlt 64gtgtgt long
  5213598703592091008239502170616955211460270452235
  6652769947041607822219725780640550022962086936576L

16
High Level Data Types

• Lists hold a sequence of items
• May hold any object
• Declared using square brackets
• gtgtgt l An empty listgtgtgt l.append(1)gtgtgt
  l.append("Hi there")gtgtgt len(l)2

17
High Level Data Types

• gtgtgt l1, 'Hi there'gtgtgtgtgtgt l "Hi there", 1,
  2gtgtgt l'Hi there', 1, 2gtgtgt l.sort()gtgtgt
  l1, 2, 'Hi there'

18
High Level Data Types

• Tuples are similar to lists
• Sequence of items
• Key difference is they are immutable
• Often used in place of simple structures
• Automatic unpacking
• gtgtgt point 2,3gtgtgt x, y pointgtgtgt x2

19
High Level Data Types

• Tuples are particularly useful to return multiple
  values from a function
• gtgtgt x, y GetPoint()
• As Python has no concept of byref parameters,
  this technique is used widely

20
High Level Data Types

• Dictionaries hold key-value pairs
• Often called maps or hashes. Implemented using
  hash-tables
• Keys may be any immutable object, values may be
  any object
• Declared using braces
• gtgtgt dgtgtgt d0 "Hi there"gtgtgt d"foo" 1

21
High Level Data Types

• Dictionaries (cont.)
• gtgtgt len(d)2gtgtgt d0'Hi there'gtgtgt d 0 "Hi
  there", 1 "Hello"gtgtgt len(d)2

22
Blocks

• Blocks are delimited by indentation
• Colon used to start a block
• Tabs or spaces may be used
• Maxing tabs and spaces works, but is discouraged
• gtgtgt if 1... print "True"... Truegtgtgt

23
Blocks

• Many people hate this when they first see it
• Almost all Python programmers come to love it
• Humans use indentation when reading code to
  determine block structure
• Ever been bitten by the C code?
• if (1) printf("True") CallSomething()

24
Looping

• The for statement loops over sequences
• gtgtgt for ch in "Hello"... print ch...
  Hellogtgtgt

25
Looping

• Built-in function range() used to build sequences
  of integers
• gtgtgt for i in range(3)... print i...
  012gtgtgt

26
Looping

• while statement for more traditional loops
• gtgtgt i 0gtgtgt while i lt 3... print i...
  i i 1... 012gtgtgt

27
Functions

• Functions are defined with the def statement
• gtgtgt def foo(bar)... return bargtgtgt
• This defines a trivial function named foo that
  takes a single parameter bar

28
Functions

• A function definition simply places a function
  object in the namespace
• gtgtgt fooltfunction foo at fac680gtgtgtgt
• And the function object can obviously be called
• gtgtgt foo(3)3gtgtgt

29
Classes

• Classes are defined using the class statement
• gtgtgt class Foo... def __init__(self)...
  self.member 1... def GetMember(self)...
  return self.member... gtgtgt

30
Classes

• A few things are worth pointing out in the
  previous example
• The constructor has a special name __init__,
  while a destructor (not shown) uses __del__
• The self parameter is the instance (ie, the this
  in C). In Python, the self parameter is
  explicit (c.f. C, where it is implicit)
• The name self is not required - simply a
  convention

31
Classes

• Like functions, a class statement simply adds a
  class object to the namespace
• gtgtgt Fooltclass __main__.Foo at 1000960gtgtgtgt
• Classes are instantiated using call syntax
• gtgtgt fFoo()gtgtgt f.GetMember()1

32
Modules

• Most of Pythons power comes from modules
• Modules can be implemented either in Python, or
  in C/C
• import statement makes a module available
• gtgtgt import stringgtgtgt string.join( "Hi",
  "there" )'Hi there'gtgtgt

33
Exceptions

• Python uses exceptions for errors
• try / except block can handle exceptions
• gtgtgt try... 1/0... except ZeroDivisionError.
  .. print "Eeek"... Eeekgtgtgt

34
Exceptions

• try / finally block can guarantee execute of code
  even in the face of exceptions
• gtgtgt try... 1/0... finally... print
  "Doing this anyway"... Doing this
  anywayTraceback (innermost last) File
  "ltinteractive inputgt", line 2, in
  ?ZeroDivisionError integer division or
  modulogtgtgt

35
Threads

• Number of ways to implement threads
• Highest level interface modelled after Java
• gtgtgt class DemoThread(threading.Thread)...
  def run(self)... for i in range(3)...
  time.sleep(3)... print
  i... gtgtgt t DemoThread()gtgtgt t.start()gtgtgt
  t.join()01 ltetcgt

36
Standard Library

• Python comes standard with a set of modules,
  known as the standard library
• Incredibly rich and diverse functionality
  available from the standard library
• All common internet protocols, sockets, CGI, OS
  services, GUI services (via Tcl/Tk), database,
  Berkeley style databases, calendar, Python
  parser, file globbing/searching, debugger,
  profiler, threading and synchronisation,
  persistency, etc

37
External library

• Many modules are available externally covering
  almost every piece of functionality you could
  ever desire
• Imaging, numerical analysis, OS specific
  functionality, SQL databases, Fortran interfaces,
  XML, Corba, COM, Win32 API, etc
• Way too many to give the list any justice

38
Python Programs

• Python programs and modules are written as text
  files with traditionally a .py extension
• Each Python module has its own discrete namespace
• Python modules and programs are differentiated
  only by the way they are called
• .py files executed directly are programs (often
  referred to as scripts)
• .py files referenced via the import statement are
  modules

39
Python Programs

• Thus, the same .py file can be a program/script,
  or a module
• This feature is often used to provide regression
  tests for modules
• When module is executed as a program, the
  regression test is executed
• When module is imported, test functionality is
  not executed

40
Python Protocols

• Objects can support a number of different
  protocols
• This allows your objects to be treated as
• a sequence - ie, indexed or iterated over
• A mapping - obtain/assign keys or values
• A number - perform arithmetic
• A container - perform dynamic attribute fetching
  and setting
• Callable - allow your object to be called

41
Python Protocols

• Sequence and container example
• gtgtgt class Protocols... def __getitem__(self,
  index)... return index 2... def
  __getattr__(self, attr)... return "A big "
  attr... gtgtgt pProtocols()gtgtgt p39gtgtgt
  p.Foo'A big Foogtgtgt

42
More Information on Python

• Cant do Python justice in this short time frame
• But hopefully have given you a taste of the
  language
• Comes with extensive documentation, including an
  excellent tutorial and library reference
• Also a number of Python books available
• Visit www.python.org for more details

43
Section II

• Introduction to COM

44
What is COM?

• Acronym for Component Object Model, a technology
  defined and implemented by Microsoft
• Allows objects to be shared among many
  applications, without applications knowing the
  implementation details of the objects
• A broad and complex technology
• We can only provide a brief overview here

45
What was COM

• COM can trace its lineage back to DDE
• DDE was expanded to Object Linking and Embedding
  (OLE)
• VBX (Visual Basic Extensions) enhanced OLE
  technology for visual components
• COM was finally derived as a general purpose
  mechanism
• Initially known as OLE2

46
COM Interfaces

• COM relies heavily on interfaces
• An interface defines functionality, but not
  implementation
• Each object (or more correctly, each class)
  defines implementation of the interface
• Each implementation must conform to the interface
• COM defines many interfaces
• But often does not provide implementation of
  these interfaces

47
COM Interfaces

• Interfaces do not support properties
• We will see how COM properties are typically
  defined later
• Interfaces are defined using a vtable scheme
  similar to how C defines virtual methods
• All interfaces have a unique ID (an IID)
• Uses a Universally Unique Identifer (UUID)
• UUIDs used for many COM IDs, including IIDs

48
Classes

• COM defines the concept of a class, used to
  create objects
• Conceptually identical to a C or Python class
• To create an object, COM locates the class
  factory, and asks it to create an instance
• Classes have two identifiers
• Class ID (CLSID) is a UUID, so looks similar to
  an IID
• ProgID is a friendly string, and therefore not
  guaranteed unique

49
IUnknown interface

• Base of all COM interfaces
• By definition, all interfaces also support the
  IUnknown interface
• Contains only three methods
• AddRef() and Release() for managing COM lifetimes
• COM lifetimes are based on reference counts
• QueryInterface() for obtaining a new interface
  from the object

50
Creating objects, and obtaining interfaces

• To create an object, the programmer specifies
  either the ProgID, or the CLSID
• This process always returns the requested
  interface
• or fails!
• New interfaces are obtained by using the
  IUnknownQueryInterface() method
• As each interface derives from IUnknown, each
  interface must support QI

51
Other standard interfaces

• COM defines many interfaces, for example
• IStream
• Defines file like operations
• IStorage
• Defines file system like semantics
• IPropertyPage
• Defines how a control exposes a property page
• etc - many many interfaces are defined
• But not many have implementations

52
Custom interfaces

• COM allows you to define your own interfaces
• Interfaces are defined using an Interface
  Definition Language (IDL)
• Tools available to assign unique IIDs for the
  interface
• Any object can then implement or consume these
  interfaces

53
IDispatch - Automation objects

• IDispatch interface is used to expose dynamic
  object models
• Designed explicitly for scripting languages, or
  for those languages that can not use normal COM
  interfaces
• eg, where the interface is not known at compile
  time, or there is no compile time at all
• IDispatch is used extensively
• Microsoft Office, Netscape, Outlook, VB, etc -
  almost anything designed to be scripted

54
IDispatch - Automation objects

• Methods and properties of the object model can be
  determined at runtime
• Concept of Type Libraries, where the object model
  can be exposed at compile time
• Methods and properties are used indirectly
• GetIDsOfNames() method is used to get an ID for a
  method or property
• Invoke() is used to make the call

55
IDispatch - Automation objects

• Languages usually hide these implementation
  details from the programmer
• Example object.SomeCall()
• Behind the scenes, your language willid
  GetIDsOfNames("SomeCall")Invoke(id,
  DISPATCH_METHOD)
• Example object.SomeProp
• id GetIDsOfNames("SomeProp")Invoke(id,
  DISPATCH_PROP_GET)

56
VARIANTs

• The IDispatch interface uses VARIANTs as its
  primary data type
• Simply a C union that supports the common data
  types
• and many helper functions for conversion etc
• Allows the single Invoke() call to accept almost
  any data type
• Many languages hide these details, performing
  automatic conversion as necessary

57
Implementation models

• Objects can be implemented in a number of ways
• InProc objects are implemented as DLLs, and
  loaded into the calling process
• Best performance, as no marshalling is required
• LocalServer/RemoteServer objects are implemented
  as stand-alone executables
• Safer due to process isolation, but slower due to
  remoting
• Can be both, and caller can decide, or let COM
  choose the best

58
Distributed COM

• DCOM allows objects to be remote from their
  caller
• DCOM handles all marshalling across machines and
  necessary security
• Configuration tools allow an administrator to
  configure objects so that neither the object nor
  the caller need any changes
• Although code changes can be used to explicitly
  control the source of objects

59
The Windows Registry

• Information on objects stored in the Windows
  registry
• ProgID to CLSID mapping
• Name of DLL for InProc objects
• Name of EXE for LocalServer objects
• Other misc details such as threading models
• Lots of other information also maintained in
  registry
• Remoting proxies, object security, etc.

60
Conclusion for Section II

• COM is a complex and broad beast
• Underlying it all is a fairly simple Interface
  model
• Although all the bits around the edges combine to
  make it very complex
• Hopefully we have given enough framework to put
  the final section into some context

61
Section III

• Python and COM

62
PythonCOM architecture

• Underpinning everything is an extension module
  written in C that provides the core COM
  integration
• Support for native COM interfaces exist in this
  module
• Python reference counting is married with COM
  reference counting
• Number of Python implemented modules that provide
  helpers for this core module

63
Interfaces supported by PythonCOM

• Over 40 standard interfaces supported by the
  framework
• Extension architecture where additional modules
  can add support for their own interfaces
• Tools supplied to automate this process
• Number of extension modules supplied, bringing
  total interfaces to over 100

64
Using Automation from Python

• Automation uses IDispatch to determine object
  model at runtime
• Python function win32com.client.Dispatch()
  provides this run-time facility
• Allows a native look and feel to these objects

65
Using Automation - Example

• We will use Excel for this demonstration
• Excel ProgId is Excel.Application
• gtgtgt from win32com.client import Dispatchgtgtgt
  xlDispatch("Excel.Application")gtgtgt
  xlltCOMObject Excel.Applicationgtgtgtgt

66
Using Automation - Example

• Now that we have an Excel object, we can call
  methods and set properties
• But we cant see Excel running?
• It has a visible property that may explain things
• gtgtgt xl.Visible0
• Excel is not visible, lets make it visiblegtgtgt
  xl.Visible 1gtgtgt

67
Automation - Late vs. Early Bound

• In the example we just saw, we have been using
  late bound COM
• Python has no idea what properties or methods are
  available
• As we attempt a method or property access, Python
  dynamically asks the object
• Slight performance penalty, as we must resolve
  the name to an ID (GetIDsOfNames()) before
  calling Invoke()

68
Automation - Late vs. Early Bound

• If an object provides type information via a Type
  Library, Python can use early bound COM
• Implemented by generating a Python source file
  with all method and property definitions
• Slight performance increase as all names have
  been resolved to IDs at generation time, rather
  than run-time

69
Automation - Late vs. Early Bound

• Key differences between the 2 techniques
• Late bound COM often does not know the specific
  types of the parameters
• Type of the Python object determines the VARIANT
  type created
• Does not know about ByRef parameters, so no
  parameters are presented as ByRef
• Early bound COM knows the types of the parameters
• All Python types are coerced to the correct type
• ByRef parameters work (returned as tuples)

70
Playing with Excel

• gtgtgt xl.Workbooks.Add()ltCOMObject ltunknowngtgtgtgtgt
  xl.Range("A1C1").Value "Hi", "From",
  "Python"gtgtgt xl.Range("A1C1").Value((L'Hi',
  L'From', L'Python'),)gtgtgt xl.Range("A1C1").PrintO
  ut()gtgtgt

71
How did we know the methods?

• Indeed, how did we know to use Excel.Application
  ?
• No easy answer - each application/object defines
  their own object model and ProgID
• Documentation is the best answer
• Note that MSOffice does not install the COM
  documentation by default - must explicitly select
  it during setup
• COM browsers can also help

72
Native Interfaces from Python

• Examples so far have been using using IDispatch
• win32com.client.Dispatch() function hides the
  gory details from us
• Now an example of using interfaces natively
• Need a simple example - Windows Shortcuts fits
  the bill
• Develop some code that shows information about a
  Windows shortcut

73
Native Interfaces from Python

• 4 main steps in this process
• Import the necessary Python modules
• Obtain an object that implements the IShellLink
  interface
• Obtain an IPersistFile interface from the object,
  and load the shortcut
• Use the IShellLink interface to get information
  about the shortcut

74
Native Interfaces from PythonStep 1 Import the
necessary Python modules

• We need the pythoncom module for core COM support
• We need the win32com.shell.shell module
• This is a PythonCOM extension that exposes
  additional interfaces
• gtgtgt import pythoncomgtgtgt from win32com.shell
  import shell

75
Native Interfaces from PythonStep 2 Obtain an
object that implements the IShellLink interface

• Use the COM function CoCreateInstance()
• Use the published CLSID for the shell
• The shell requires that the object request be for
  an InProc object.
• Request the IShellLink interface

76
Native Interfaces from PythonStep 2 Obtain an
object that implements the IShellLink interface

• gtgtgt sh pythoncom.CoCreateInstance(shell.CLSID_Sh
  ellLink, ... None,...
  pythoncom.CLSCTX_INPROC_SERVER,...
  shell.IID_IShellLink)gtgtgt shltPyIShellLink at
  0x1630b04 with obj at 0x14c9d8gtgtgtgt

77
Native Interfaces from PythonStep 3 Obtain an
IPersist interface from the object, and load the
shortcut

• Use QueryInterface to obtain the new interface
  from the object
• gtgtgt pe sh.QueryInterface(pythoncom.IID_IPersistFi
  le)gtgtgt pe.Load("c/winnt/profiles/skip/desktop/ID
  LE.lnk")

78
Native Interfaces from PythonStep 4 Use the
IShellLink interface to get information about the
shortcut

• gtgtgt sh.GetWorkingDirectory()'L\\src\\python-cvs\
  \tools\\idle'gtgtgt sh.GetArguments()'l\\src\\pyth
  on-cvs\\tools\\idle\\idle.pyw'gtgtgt

79
Implementing COM using Python.

• Final part of our tutorial is implementing COM
  objects using Python
• 3 main steps we must perform
• Implement a Python class that exposes the
  functionality
• Annotate the class with special attributes
  required by the COM framework.
• Register our COM object

80
Implementing COM using Python.Step 1 Implement
a Python class that exposes the functionality

• We will build on our last example, by providing a
  COM object that gets information about a Windows
  shortcut
• Useful to use from VB, as it does not have direct
  access to these interfaces.
• We will design a class that can be initialised to
  a shortcut, and provides methods for obtaining
  the info.

81
Implementing COM using PythonStep 1 Implement a
Python class that exposes the functionality

• Code is getting to big for the slides
• Code is basically identical to that presented
  before, except
• The code has moved into a class.
• We store the IShellInfo interface as an instance
  attribute

82
Implementing COM using PythonStep 1 Implement a
Python class that exposes the functionality

• Provide an Init method that takes the name of the
  shortcut.
• Also provide two trivial methods that simply
  delegate to the IShellInfo interface
• Note Python would allow us to automate the
  delegation, but that is beyond the scope of this!

83
Implementing COM using PythonStep 2 Annotate
the class with special attributes

• PythonCOM requires a number of special attributes
• The CLSID of the object (a UUID)
• Generate using print pythoncom.CreateGuid()
• The ProgID of the object (a friendly string)
• Make one up!
• The list of public methods
• All methods not listed as public are not exposed
  via COM

84
Implementing COM using PythonStep 2 Annotate
the class with special attributes

• class PyShellLink _public_methods_'Init',
  'GetWorkingDirectory',
  'GetArguments' _reg_clsid_"58DE4632-9323-11D3-
  85F9-00C04FEFD0A7" _reg_progid_"Python.ShellDem
  o" ...

85
Implementing COM using PythonStep 3 Register
our COM object

• General technique is that the object is
  registered when the module implementing the COM
  object is run as a script.
• Recall previous discussion on modules versus
  scripts
• Code is trivial
• Call the UseCommandLine() method, passing the
  class objects we wish to register

86
Implementing COM using PythonStep 3 Register
our COM object

• Code is simple
• note we are passing the class object, not a class
  instance.
• if __name__'__main__' UseCommandLine(PyShell
  Link)
• Running this script yields
• Registered Python.ShellDemo

87
Testing our COM object

• Our COM object can be used by any automation
  capable language
• VB, Delphi, Perl - even VC at a pinch!
• We will test with a simple VBScript program
• VBScript is free. Could use full blown VB, or
  VBA. Syntax is identical in all cases

88
Testing our COM object

• set shdemo CreateObject("Python.ShellDemo")shd
  emo.Init("c\winnt\profiles\skip\desktop\IDLE.lnk"
  )WScript.Echo "Working dir is "
  shdemo.GetWorkingDirectory()
• Yields
• Working dir is L\src\python-cvs\tools\idle

89
Summary

• Python can make COM simple to work with
• No reference count management
• Many interfaces supported.
• Natural use for IDispatch (automation) objects.
• Simple to use COM objects, or implement COM
  objects

90
Questions?
91
More Information

• Python itself
• http//www.python.org
• newscomp.lang.python
• Python on Windows
• http//www.python.org/windows
• Mark Hammonds Python extensions
• http//starship.python.net/crew/mhammond
• http//www.python.org/windows