Distributed Computing with Python

1
Distributed Computing with Python

• Carlos Varela
• Rensselaer Polytechnic Institute

2
Distributed Computing with Python

• Using standard TCP/IP (or UDP) sockets
• Using high-level libraries for remote method
  invocation
• XML-RPC
• Pyro (PYthon Remote Objects)

3
Networking with Python Sockets

• Resembles C socket API
• Can use UDP (datagrams) or TCP (streamed
  connections)
• Does not require non-standard libraries
• Low-level, yet relatively easy to use and
  portable
• Enables more efficient communication and is
  programming language independent.
• New features require protocol extensions

4
Sockets An Echo Server
from socket import def main() s
socket(AF_INET, SOCK_STREAM) s.bind((,
50007)) port number s.listen(1) conn,
(remotehost, remoteport) s.accept() print
'connected by', remotehost, remoteport while
1 data conn.recv(1024) buffer
size if not data break
conn.send(data) conn.close() main()
5
Sockets An Echo Client
from socket import def main() s
socket(AF_INET, SOCK_STREAM)
s.connect((localhost, 50007)) port number
s.send(Hello World) data s.recv(1024)
buffer size s.close() print
data main()
6
Remote Procedure Calls with Python XML-RPC library

• XML-RPC is programming-language/O.S. independent
• Uses XML as message format for parameters and
  return values, e.g.
• ltparamsgt
• ltparamgt
• ltvaluegtltstringgtHello Worldlt/stringgtlt/val
  uegt
• lt/paramgt
• lt/paramsgt
• Uses HTTP as the transport protocol.
• Enabling technology for building web services.
• Example requires downloading xmlrpcserver.py
  library
• http//svn.python.org/view/python/trunk/Demo/xmlrp
  c/

7
XML-RPC An Echo Server
import SocketServer import xmlrpcserver import
xmlrpclib class EchoReqHandler(xmlrpcserver.Reques
tHandler) def call(self, method, params)
print "Dispatching ", method, params
try server_method getattr(self,
method) except raise
AttributeError, No method s" method
return server_method(params) def echo(self,
value) return xmlrpclib.dumps(value) ser
ver SocketServer.TCPServer(('', 8000),
EchoReqHandler) server.serve_forever()
8
XML-RPC An Echo Client
import xmlrpclib echosvr xmlrpclib.Server("htt
p//localhost8000") print echosvr.echo("Hello
World") print echosvr.echo(10)
9
Distributed Computing with Python Remote Objects
(PYRO)

• Resembles Java RMI Server Daemon, Client Proxy
• To find remote objects
• By name, e.g., PYRONAME//echo, requires name
  server
• By location, e.g., PYROLOC//localhost7766/echo
  
• By URI, e.g., PYRO//127.0.0.17766/7f00000105121
  27841288b1892e90a71
• Method arguments, return values, and exceptions
  marshalled (pickled) over the network
• Example code requires downloading and installing
  Pyro at
• http//pyro.sourceforge.net/

10
Pyro An Echo Server
import Pyro.core class Echo(Pyro.core.ObjBase) d
ef __init__(self) Pyro.core.ObjBase.__init__(se
lf) def echo(self, string) return
string Pyro.core.initServer() daemonPyro.core.D
aemon() uridaemon.connect(Echo(),"echo-object") p
rint "The object's uri is",uri daemon.requestLoo
p()
11
Pyro An Echo Client
import Pyro.core echoProxy Pyro.core.getProxyFo
rURI( "PYROLOC//localhost7
766/echo-object") print echoProxy.echo("Hello
World")
12
Python Remote Objects (PYRO)

• Python programming language-dependent
• Event server for publish-subscribe communication
• Mobile code, but BEWARE No security guarantees!

13
Exercises

• Compare the round trip time of the Echo
  application using
• TCP/IP
• XML-RPC
• Pyro
• Use the XML-RPC client in Python to connect to an
  existing web service (e.g., Google)
• http//www.xmlrpc.com/googleGateway
• Modify the echo example in Pyro to use the name
  service. What are alternative ways to get to the
  name server?
• Does Pyro support distributed garbage collection?