Python: Design and Implementation

1

• Python Design and Implementation
• Guido van Rossumguido_at_google.com
• Intel Research Berkeley, 10/23/06

2
Overview

• Brief history lesson
• Pythons design and design process
• Pythons implementation
• Questions

3
Timeline

• Project started, name picked - Dec 1989
• First public release (USENET) - Feb 1991
• python.org website - 1996 or 1997
• 2.0 released - 2000
• Python Software Foundation - 2001
• 2.4 released - 2004
• 2.5 released - 2006

4
Why I Designed Python

• ABC - teaching language created at CWI
• designed in late 70s, early 80s (not me)
• I was on the implementation team until 86
• had my own thoughts about its failing
• Amoeba - distributed microkernel OS
• almost completely, but not quite, unlike Unix
• only languages available were C and sh
• I decided to try to bridge the gap
• Other influences C, Modula-3, Icon, A68

5
Original Design Process

• Consider a desirable feature (either user
  feedback or my own needs while using it)
• Only if there is absolutely no way to implement
  it as a library module or C extension, consider
  changing the compiler
• Make the absolutely minimal changes necessary to
  the compiler reuse existing functionality as
  much as possible
• This gave us explicit self, for example

6
Modern Design Process

• Pretty much the same, but also worrying about
  backwards compatibility
• Set the bar really high by only accepting new
  language features that will be of use to a wide
  variety of users
• Prevent repeating early mistakes, where sometimes
  shortcuts were taken that were too extreme, and
  had to be changed at great cost (e.g. int/long,
  classic classes)

7
Counterforce Readability

• No hack so clever or it must be readable!
• Require at least one of
• Does it read naturally, like English?
• Does it resemble well-known math notation?
• Does it resemble a similar feature in another
  (popular) programming language?
• Is it a logical extension of existing syntax?
• (Of course this is highly subjective)

8
Community Process

• PEP Python Enhancement Proposal
• Like JSR (Java), RFC (IETF)
• Though a bit less formal (smaller community)
• Not just for language changes
• Common APIs (e.g. db-API, WSGI)
• Sometimes library design (often not needed)
• Bookkeeping (e.g. release schedules)
• Typically not for performance enhancements
• Not a democracy!

9
Original Design Goals

• Simple implementation (1 person)
• Typical Very-High-Level Language goals
• Cross-platform (hardware OS)
• Readability and expressive power
• Easy to learn and remember predictability
• Safety bugs dont crash interpreter
• Dont compete with C complete it!
• Extensibility through C extension modules
• This makes Python an ideal glue language

10
Standard VHLL Features

• Automatic memory management
• Small number of powerful data types
• Bytecode interpreter
• Built-in serialization (marshal, pickle)
• KISS
• Even indentation was borrowed!
• Lots of other stuff, too
• Python doesnt have NIH syndrome -)

11
Pythons Big Ideas

• No type checking in the compiler
• Dynamic typing, static scoping
• Everything is an object
• Everything has a namespace (or several!)
• Everything can be introspected, printed
• System has few privileges over user
• Interactive prompt gtgtgt
• Simplicity of implementation still matters!

12
How Python Is Compiled

• Lexer -gt token stream
• Uses a stack to parse indentation
• Parser -gt concrete syntax tree
• Simple stupid LL(1) parsing engine
• Filter -gt abstract syntax tree (uses ASDL)
• Pass 1 -gt symbol table
• Pass 2 -gt bytecode
• Bytecode is represented as objects
• Saving to disk (.pyc file) is just a speed hack

13
How Python Is Executed

• Virtual machine executes bytecode
• Simple stack machine
• Stack frames allocated on the heap
• C stack frames point to heap stack frames
• Additional stack for try blocks
• VM calls out to abstract operations
• e.g. PyNumber_Add(a, b)
• Some opcodes represent support calls
• e.g. import, print (since these have syntax)

14
Example Bytecode

• def gcd(a, b)while b a, b b, abreturn
  a
• LOAD_FAST 1 (b)
• LOAD_FAST 0 (a)
• LOAD_FAST 1 (b)
• BINARY_MODULO
• ROT_TWO
• STORE_FAST 0 (a)
• STORE_FAST 1 (b)

• gtgtgt dis.dis(gcd)
• 2 0 SETUP_LOOP 29 (to
  32)
• gtgt 3 LOAD_FAST 1 (b)
• 6 JUMP_IF_FALSE 21 (to
  30)
• 9 POP_TOP
• 3 10 LOAD_FAST 1 (b)
• 13 LOAD_FAST 0 (a)
• 16 LOAD_FAST 1 (b)
• 19 BINARY_MODULO
• 20 ROT_TWO
• 21 STORE_FAST 0 (a)
• 24 STORE_FAST 1 (b)
• 27 JUMP_ABSOLUTE 3
• gtgt 30 POP_TOP
• 31 POP_BLOCK
• 4 gtgt 32 LOAD_FAST 0 (a)
• 35 RETURN_VALUE

15
Separation of Concerns

• VM knows very little about objects
• Just their abstract API
• Occasionally cheats for performance hacks
• e.g. intint, listint, function calls
• Some objects are part of the VM
• e.g. code, frame, traceback not function!
• Namespaces implemented as dictionaries
• Objects know nothing about VM
• Support infrastructure ties things together
• import bookkeeping, parser, interactive prompt

16
Object Header

• Fields shared by all objects
• typedef struct int ob_refcnt
  PyTypeObject ob_type PyObject
• Example (in reality this uses macros)
• typedef struct int ob_refcnt
  PyTypeObject ob_type long ob_ival
  PyIntObject

17
Type Object Struct

• Most members are function pointers
• typedef struct ltHEADgt char tp_name
  destructor tp_dealloc printfunc
  tp_print getattrfunc tp_getattr
  setattrfunc tp_setattr PyTypeObject

18
Other Python Implementations

• Jython - compiles to Java bytecode
• IronPython - compiles to .NET IL
• Why?
• Because we can!
• Keeps Python language definition honest
• No implementation-specific hacks in the spec
• Side bet in case Java or .NET wins
• Fills important gap on those platforms

19
Jython Details

• 100 Pure Java certified
• Language differs in few details
• e.g. strings are always Unicode (UTF-16)
• Library differs where C code is wrapped
• e.g. no Tkinter
• Fully automated wrapping of Java classes
• thanks to Javas reflection API
• Needs to load significant Jython Runtime
• Executes 2-5x slower than CPython

20
IronPython Details

• Very similar in architecture to Jython
• C instead of Java (duh)
• Compiles to IL instead of JVM bytecode
• Uses standard .NET objects where possible
• Can import any managed object class
• Sometimes faster than CPython
• Implements same language as Python 2.4
• Supported by Microsoft
• But open source license!

21
Question Time