Continuations and Stackless Python

1
Continuations and Stackless Python

• Where Do You Want To Jump Today?

2
Stacklessssshhh Python ?
... And the clueless implementor ...
3
Why Stackless?

• Recursion depth limits ?
• Platform dependant
• Generators ?
• Coroutines ?
• Micro-Threads ?
• Persistent State ?
• Pickle the running program
• Mobile Agents
• Continuations ?
• Modify running Program ?

4
Fact Sheet

• Minimum C Stack resources
• Unlimited recursion
• 100 binary compatible to Standard Python
• Except for left bugs ?
• 5 faster executable on Win32
• Full support for first-class continuations
• Continuations included as extension module
• As portable as Standard Python
• Breaks 4K stack limit on PalmIII

5
What Is a Continuation
x 2 y x 1 z x 2
The current continuation at any point in the
execution of a program is an abstraction of the
rest of the program
6
Continuation Objects

• Tiny interface objects to Python frames
• Continuations are callable objects
• Calling brings us immediately back to where we
  left off
• Can pass and return a value

7
Implementing a Loop
8
Building a Generator (1)
9
Building a Generator (2)
10
Easy? Almost, but...

• The given principle can implement coroutines
• We can switch to any other disjoint frame chain
• Trouble when more than one continuation owns the
  target frame!

11
Critical Situation
a-frame with refcount gt 1
b-frame still referenced
c-frame wants to return
12
Frame Ownership

• In order to jump to a frame, we must own it
• We own a frame iff its refcount is 1
• The program must ensure this by building clone
  frames (solution, first half).
• We cannot just make a copy, but keep the current
  fram in place... (solution, second half)

Note This rule is strong enough to build a
complete object system!
13
Push-back Frame
a-frame as continuation
new frame with contents of a-frame
b-frame still referenced
c-frame wants to return
14
3rd Half of Solution

• Upto version 0.3
• Managing growing chains of pushed-back frames
  becomes tedious. Continuation frames pile up.
  Runtime isnt O(1)
• Insert an extra node object that always walks
  with the real frame does the trick
• Runtime is O(1) now

15
Node Object
a-frame as continuation
new frame with contents of a-frame
PyCoNode
b-frame still referenced
c-frame wants to return
16
Final View
new frame with contents of a-frame
PyCoNode
b-frame still referenced
c-frame wants to return
a-frame as continuation
17
Standard stackfull Python
Frame Stack
C Stack (simplified)
def a(x) ... b(x1) ... def b(x)
... c(xx) ... def c(x) ...
print "x",x ... a(42)
18
Stackless Python
19
About Optimization

• Moved opcode/oparg into the instructions
• Shortened the big switch
• Mapped Error handling to pseudo-opcodes
• Shortened lifetime of state variables
• Inc/decref not macro but inline (win32)
• Would gain about 15 for standard Python
• (but this is not what I want ?)