Nick Benton

1

• Nick Benton
• Programming Principles and Tools Group
• Microsoft Research Cambridge
• http//research.microsoft.com/Comega/

2
Plan

• Overview of the PPT group
• Who we are and what we do
• Overview of C?
• Why another programming language?
• Data access in C?
• Concurrency in C?
• Summary

3
MSRC Programming Principles Tools
4
PPT People

• Luca Cardelli
• Nick Benton
• Karthik Bhargavan
• Gavin Bierman
• Cedric Fournet
• Georges Gonthier
• Andy Gordon
• Tony Hoare
• Andrew Kennedy
• Simon Marlow
• Simon Peyton Jones
• Claudio Russo
• Don Syme
• visitors, PhD students, interns,

5
Exciting times for language research

• C Java are strongly typed, garbage collected
  languages with language-level security mechanisms
• Ten years ago, this was the preserve of
  pointy-headed academic languages
• Mixed-language programming is really happening
• Advanced analysis tools for finding bugs and
  checking conformance to specifications are
  becoming normal at Microsoft
• Behavioural types (contracts) for concurrency
  are a hot topic in MS product groups and
  elsewhere
• This idea only goes back about six years in
  academia
• Parametric polymorphism in next version of C and
  CLR
• designed and implemented in the product in our
  group
• Formal techniques being successfully applied to
  (very) low-level languages (e.g. TAL)

6
PPT Some projects

• SML.NET
• Generics for C and .NET
• Polyphonic C
• Stack-Walking Security in .NET
• Formal Tools for Securing Web Services
• Query Languages for Semistructured Data
• First-class Functions for Excel
• Haskell
• Languages for Systems Biology

7
C? Overview
8
Why C??

• Application domain distributed, web-based
  scenarios
• Common architecture three tier architecture
• Data services tier (database)
• Middle tier (main application) C/Java
• User interface tier (XML/HTML)
• Common concurrency scenario asynchronous events
  and message passing
• Web services, workflow orchestrations,
• Currently built using C/Java with API support
  (particularly poor for concurrency)

9
What we want vs. what we get

• Its about both
• DATA
• XML, relations, objects
• CONTROL
• Asynchrony, concurrency
• Even though our languages are modern, support
  for other features is pretty primitive
• Data Much of access uses strings
• Control Monitors (1960s)
• Essence These important features are left to the
  API ?

10
The C? project in a slide

• Were building an experimental language with
• C at its core (of course ?)
• First-class support for common data models
• A better model of concurrency for both local and
  distributed concurrency

Our intent is that this language will make it
easier to write data-intensive, distributed
applications
11
The guiding principle
Put important features in the language itself,
rather than in libraries

• Make invariants and intentions more apparent
  (part of the interface)
• Nicer syntax
• Stronger compile-time guarantees (types)
• Compiler has more freedom to choose different
  implementations (optimize)
• Also helps other tools

12
Data access in C?
13
How bad is data access today?
14
Typical API access
15
API access further observation

• Storing queries as strings is not only ugly but
  also a security risk
• Imagine where input OR 11 --
• This will return the entire relation!!
• This is the source of many security loopholes in
  web-based databases
• XML APIs no better ?

16
Cant we do better?
17
A sneak preview
Cool type declarations!
public class card struct string name
string title string email
string? phone logo? logo
ltcardgt ltnamegtJohn Doelt/namegt lttitlegtCEO,
Widget Inc.lt/titlegt ltemailgtjohn.doe_at_widget.comlt
/emailgt ltlogogtwidget.giflt/logogt lt/cardgt
18
A sneak preview
XML in your code!!
card c ltcardgt ltnamegtJohn Doelt/namegt
lttitlegtCEO, Widget Inc.lt/titlegt
ltemailgtjohn.doe_at_widget.comlt/emailgt
ltlogogtwidget.giflt/logogt lt/cardgt
c.. Console.WriteLine(it)
Path expressions (like XPath) in your code!!
19
A sneak preview
SQL in your code!!
alts1 select Title,Artist from CDs
where Style CDStyle.Alt alts1.ConsoleWri
teLine(Title0,Artist1,
it.Title,it.Artist)
20
The three pillars of C? data access

• Streams (lazy lists)
• Anonymous structs (tuples)
• Either types (disjoint unions)

21
C? user scenario

• C programmer who
• Is XPATH-savvy
• Uses relational databases
• The first point is incredibly important it has
  had a profound effect on the design of C?
• (Other approaches possible)

22
Streams

• New type T (sequence of T)
• Close relative of IEnumerableltTgt
• (Distant relative of Haskells list(T))
• Streams generated by statement blocks that yield
  values
• Streams consumed by foreach loop

23
Streams Example
public static int FromTo(int s, int e)
for (i s i lt e i) yield return i
int OneToTen FromTo(1,10) foreach(int j
in OneToTen) Console.WriteLine(j)
24
Why streams?

• We use streams to represent
• The sequence of rows from a table
• The iteration operator from XML Schema/DTDs
• lt!ELEMENT foo (bar)gt

25
From the XQuery definition

• Sequences are never nestedfor example,
  combining the values 1, (2, 3), and ( ) into a
  single sequence results in the sequence (1, 2,
  3)
• Thus, streams in C? are never nested
• int doesnt exist

26
Anonymous structs

• New type structint i string s
• Similar to tuples in Haskell/ML
• Constructed in obvious way
• Project using dot notation

structint i string s tup new(i42,sGavin)
int temp tup.i
27
Why anonymous structs?

• We use anonymous structs to represent
• A row from a table
• Ordered sequences of XML Schema/DTD elements,
  e.g.
• lt!ELEMENT foo (bar,pop,pop)gt

28
Union types

• New type choiceTS
• A value of this type is either a value of type T
  or a value of type S

29
Union type example

• Why?
• To represent choice in XML Schema/DTD
• lt!ELEMENT foo (bar pop)gt

choicestring Button int foo 42
30
Summary DTDs to C?
31
Example encoding
lt!ELEMENT bib (book)gt lt!ELEMENT book (title,
(author editor), publisher, price)gt
32
Generalized member access

• A key design feature of C?
• Observe C? can encode XML-like values
• Want some query-like facility
• Solution Extend the dot notation behaviour (to
  match / from XPATH)

The power is in the dot!
33
Generalized member access

• Sequences
• Anonymous structs
• Either types

string ss ss.ToUpper()
string
structint i string s x x.s
string
choicestringPersonint y y.Length
choiceintInch?
34
Path expressions

• GMA allows us to write elegant, concise code, e.g.

virtual int From(int n) for () yield
return n From(0).return
itit.ToString().Console.WriteLine(it)
35
Data access demo
36
Christmas come early

• The type system and operational semantics of C?
  have been formally specified
• C? has loads of other goodies
• More XPath-like extensions
• More SQL-lite extensions

37
Concurrency in C?
38
Asynchrony is where its at

• Distribution gt concurrency latency
  gt asynchrony gt more
  concurrency
• Message-passing, event-based programming,
  dataflow models
• For programming languages, coordination
  (orchestration) languages frameworks, workflow

39
.NET today

• Java-style monitors
• OS shared memory primitives
• Clunky delegate-based asynchronous calling model
• Hard to understand, use and get right
• Different models at different scales
• Support for asynchrony all on the caller side
  little help building code to handle messages
  (must be thread-safe, reactive, and deadlock-free)

40
C? concurrency

• New concurrency model and language constructs
• Based on the join calculus
• A foundational process calculus like the
  p-calculus but better suited to asynchronous,
  distributed systems
• A single model which works both for
• local concurrency (multiple threads on a single
  machine)
• distributed concurrency (asynchronous messaging
  over LAN or WAN)
• It is different
• But its also simple

41
In one slide

• Objects have both synchronous and asynchronous
  methods.
• Values are passed by ordinary method calls
• If the method is synchronous, the caller blocks
  until the method returns some result (as usual).
• If the method is async, the call completes at
  once and returns void.
• A class defines a collection of chords
  (synchronization patterns), which define what
  happens once a particular set of methods have
  been invoked. One method may appear in several
  chords.
• When pending method calls match a pattern, its
  body runs.
• If there is no match, the invocations are queued
  up.
• If there are several matches, an unspecified
  pattern is selected.
• If a pattern containing only async methods fires,
  the body runs in a new thread.

42
A simple buffer

• class Buffer
• async put(String s)
• String get() put(String s)
• return s

43
A simple buffer

• class Buffer
• async put(String s)
• String get() put(String s)
• return s

• An ordinary (synchronous) method with no
  arguments, returning a string

44
A simple buffer

• class Buffer
• async put(String s)
• String get() put(String s)
• return s

• An ordinary (synchronous) method with no
  arguments, returning a string
• An asynchronous method (hence returning no
  result), with a string argument

45
A simple buffer

• class Buffer
• async put(String s)
• String get() put(String s)
• return s

• An ordinary (synchronous) method with no
  arguments, returning a string
• An asynchronous method (hence returning no
  result), with a string argument
• Joined together in a chord

46
A simple buffer

• class Buffer
• async put(String s)
• String get() put(String s)
• return s

• Calls to put() return immediately (but are
  internally queued if theres no waiting get()).
• Calls to get() block until/unless theres a
  matching put()
• When theres a match the body runs, returning the
  argument of the put() to the caller of get().
• Exactly which pairs of calls are matched up is
  unspecified.

47
A simple buffer

• class Buffer
• async put(String s)
• String get() put(String s)
• return s

• Does example this involve spawning any threads?
• No. Though the calls will usually come from
  different pre-existing threads.
• So is it thread-safe? You dont seem to have
  locked anything
• Yes. The chord compiles into code which uses
  locks. (And that doesnt mean everything is
  synchronized on the object.)
• Which method gets the returned result?
• The synchronous one. And there can be at most one
  of those in a chord.

48
Reader/Writer

• using threads and mutexes in Modula 3
• An introduction to programming with threads.
  Andrew D. Birrell, January 1989.

49
Reader/Writer in five chords

• public class ReaderWriter
• async Idle()
• async S(int n)
• public void Exclusive() Idle()
• public void ReleaseExclusive() Idle()
• public void Shared() Idle() S(1)
• S(int n) S(n1)
• public void ReleaseShared() S(int n)
• if (n 1) Idle() else S(n-1)
• public ReaderWriter() Idle()
• A single private message represents the state
• none ?? Idle() ?? S(1) ?? S(2) ?? S(3)

50
Asynchronous requests and responses

• Service exposes an async method which takes
  parameters and somewhere to put the result
• a buffer, or a channel, or
• a delegate

public delegate async IntCB(int v) public
class Service public async request(String arg,
IntCB callback) int result // do something
interesting callback(result)
51
Asynchronous requests and responses - Join

• class Join2
• async first(int r)
• async second(int r)
• void wait(out int i, out int j)
• first(int r1)
• second(int r2)
• i r1 j r2 return
• // client code
• int i,j
• Join2 x new Join2()
• service1.request(arg1, new IntCB(x.first))
• service2.request(arg2, new IntCB(x.second))
• // do something useful
• // now wait until both results have come back
• x.wait(out i,out j)
• // do something with i and j

52
Asynchronous requests and responses - Select

• class Select
• async reply(int r)
• int wait()
• reply(int r)
• return r
• // client code
• int i
• Select x new Select()
• service1.request(arg1, new IntCB(x.reply))
• service2.request(arg2, new IntCB(x.reply))
• // do something useful
• // now wait until one result has come back
• i x.wait()
• // do something with i

53
Active Objects
public abstract class ActiveObject protected
bool done abstract protected void
processmessage() public ActiveObject ()
done false mainloop() async
mainloop() while (!done) processmessage()

54
continued
class Stock ActiveObject public async
bid(BidOffer b) public async register(Client
w) override protected void processmessage()
bid(BidOffer thebid) // process bid
messages register(Client who) //
process registration requests
55
Why only one synchronous method in a chord?

• JoCaml allows multiple synchronous methods to be
  joined, as in the following rendezvous
• But in which thread does the body run? In C,
  thread identity is very observable, since
  threads are the holders of particular re-entrant
  locks. So we rule this out in the interests of
  keeping commutative.

int f(int x) int g(int y) return y to f
return x to g
56
The problem with inheritance
class C virtual void f() virtual async g()
virtual void f() virtual async h()
class D C override async g()

• Weve half overridden f
• Too easy to create deadlock or async leakage

void m(C x) x.g() x.f() m(new D())
57
The inheritance restriction

• Two methods are co-declared if they appear
  together in a chord declaration.

Whenever a method is overridden,every
co-declared method must also be overridden.

• Hence, the compiler rejects patterns such as
• public virtual void f() private async g()
• In general, inheritance and concurrency do not
  mix well.Our restriction is simple it could be
  made less restrictive.

58
Types etc.

• async is a subtype of void
• Allow covariant return types on those two
• An async method may override a void one
• A void delegate may be created from an async
  method
• An async method may implement a void method in an
  interface
• async methods are given the OneWay attribute,
  so remote calls are non-blocking

59
Predictable Demo Dining Philosophers
waiting to eat
eating
waiting to eat
thinking
eating
60
Code extract

• class Room
• async hasspaces(int n)
• async isfull()
• public Room (int size) hasspaces(size)
• public void enter()
• hasspaces(int n)
• if (n gt 1) hasspaces(n-1)
• else isfull()
• public void leave()
• hasspaces(int n)
• hasspaces(n1)
• isfull()
• hasspaces(1)

61
Dining philosophers demo
62
Summary

• Cw extends C with first class support for data
  access and asynchronous concurrency
• These are major issues for enterprise
  developers today, and becoming more pressing
• Concurrency model
• Model good for both local and distributed
  settings
• Efficiently compiled to queues and automata
• Based on join-calculus, but tweaked for
  compatibility with objects
• Data extensions
• Model good for both relational (SQL) and
  semistructured (XML) settings
• Compile to either in-memory operations or
  external queries
• Based on ideas from functional programming, but
  tweaked for compatibility with objects and XPath

63
Compiler release

• We plan to release a binary of the compiler Real
  Soon Now
• Download from http//research.microsoft.com/Comega
  
• No productization plans, this will be a free
  download
• Please play with it, and tell us what you think!