PIPER: PeertoPeer Distribution of Scientific Applications and DataFlow

1
PIPER Peer-to-Peer Distribution of Scientific
Applications and Data-Flow
J.W. Bizzaro Bioinformatics.org University of
Massachusetts Lowell
2
PROBLEMS

• Command-line (CL) programs have often been
  simpler to develop, and thousands of CL programs
  have been made for UNIX environments.
• The CL has only simple data-flow control.
• Work-path on the desktop and Web cannot be seen
  or pre-specified.
• CL programs are not made to work across a
  network. UNIX pipes "" aren't either.
• The CL has no GUI counterpart not easy to use.

3
SOLUTIONS

• Everything is a self-contained object ("node").
• Nodes input and/or output data-flow.
• Nodes can be linked to define paths of data-flow,
  procedural steps, and relationships "networks".
• Nodes can control data-flow.
• Every node has an Internet location.
• Nodes and links are only represented locally, if
  possible, and in simple user interfaces.

4
PROBLEMS

• Data-flow on the desktop and Web is managed
  manually.
• Without the CL, a scientist spends much of
  his/her time shuttling data.
• The desktop and Web do not indicate where one is
  working in the work-path.
• Web pages are static, requiring flipping.

5
SOLUTIONS

• Data-flow is automated.
• Data are "pulled" into nodes (nodes are
  sequentially activated in the direction opposite
  of data-flow).
• Windows can remain with nodes, indicating
  location in work-path.
• User interfaces are updated dynamically.

6
EXAMPLE
7
MORE SOLUTIONS

• Networks of nodes can be nested ("compiled") into
  a single node.
• The Internet-distribution of nodes/networks lets
  Piper be used as a "collaboratory" (e.g., The
  Open Lab).
• Nodes can be exchanged/shared.
• Data can be exchanged in more complex ways than
  with e-mail.
• Compiled networks provide the appearance of a
  single application, even if the components are
  Internet-distributed.
• GUI's can be combined.

8
EXAMPLE
9
MORE SOLUTIONS

• Piper is general-purpose in its design.
• More flexible in its use as a bio-cheminformatics
  application
• Standards don't exist!
• Standards will change!
• More widely used
• Other fields
• More widely contributed to
• Piper is Open Source software

10
STRUCTURE
User Interfaces Layer (UIL)

• Build-Time

• Definitions Layer (DL)

Brokering Layer (BL)

• Run-Time

• Processing Layer (PL)

11
STRUCTURE

• Build-Time Subsystem (BTS)
• Akin to a development environment generates an
  XML "script"
• Uses the "model-view-controller" (MVC) paradigm
• Permits the simulcasting of user interfaces
• User Interfaces Layer (UIL)
• Location of the "view" and "controller"
• Definitions Layer (DL)
• Location of the "model", described in XML

12
STRUCTURE

• Run-Time Subsystem (RTS)
• Akin to an interpreter follows XML "script"
• Pipes type-less data between peers
• Brokering Layer (BL)
• Control of data piping between processors
• Processing Layer (PL)
• Contains "processors" which operate on data
• Can be nodes and can contain nodes (e.g.,
  Overflow)

13
STRUCTURE
14
STRUCTURE

• Instances of Piper are connected in a
  Peer-to-Peer (P2P) or decentralized network.

15
PROTOCOLS LANGUAGES

• CORBA
• Layer-to-layer communication
• SECIOP
• Peer-to-peer communication
• XML
• Description of nodes, parameters, locations,
  networks, interfaces and meta data
• XML descriptions are passed between layers and
  peers (green arrows).
• XML is the "scripting language" of Piper.

16
MORE SOLUTIONS

• Linking of CL programs makes Pied/Piper an
  Internet-distributed, graphical scripting
  language. Thousands of nodes are already
  available!
• Linking of low-level objects makes Pied/Piper an
  Internet-distributed, graphical programming
  language.
• CL options are also nodes, making such programs
  easy to port and the options easy to change.

17
EXAMPLE
povray -I def.ini povray.pov
18
STANDARDS

• PiperNet Standards Organization
• Piper is a reference implementation of the
  PiperNet standards.
• Different components can take the place of Piper
  components.
• Additional components and layers can be added
  (e.g., intelligent agent system).
• XML descriptions (nodes, parameters, locations,
  networks, interfaces and meta data) need to be
  standardized.

19
DEVELOPERS

• J.W. Bizzaro (Massachusetts)
• Brad Chapman (Georgia)
• Dominic Letourneau (Quebec)
• Deanne Taylor (Michigan)
• Jean-Marc Valin (Quebec)
• Gary Van Domselaar (Massachusetts)
• Jarl Van Katwijk (Netherlands)

20
bioinformatics.org/piper