Kepler: a scientific workflow management system - PowerPoint PPT Presentation

About This Presentation
Title:

Kepler: a scientific workflow management system

Description:

Current downloadable version (from www.kepler-project.org) is alpha 9; ... Parameter tuning. Set breakpoint. How to compose a workflow? Select proper components ... – PowerPoint PPT presentation

Number of Views:139
Avg rating:3.0/5.0
Slides: 21
Provided by: zhim
Category:

less

Transcript and Presenter's Notes

Title: Kepler: a scientific workflow management system


1
Kepler a scientific workflow management system
  • Zhiming Zhao
  • zhiming_at_science.uva.nl
  • VL-e SP2.5, University of Amsterdam

2
Outline
  • About Kepler
  • Kepler in scientific experiments from a
    scientists point of view
  • Kepler in VL-e
  • Discussion

3
About Kepler
  • Kepler is a Scientific Workflow Management System
  • An open source cross-project collaboration. It is
    led by San Diego Supercomputer Center (SDSC),
    with participation from the SEEK ITR, DOE SciDAC
    SDM, and others.
  • Based on the Ptolemy II system.
  • State
  • Still active
  • Current downloadable version (from
    www.kepler-project.org) is alpha 9
  • A beta release has been announced, but is not
    available online yet.

4
Basic concepts workflow modeling
  • Actors basic workflow components
  • It can be atomic
  • A set of ports (for input and output)
  • A set of parameters (for configuration)
  • A fire() function the main functionality
  • Or composite
  • Contain a set of actors (atomic or composite)
  • And one director for interpreting the runtime
    behavior
  • Directors
  • For interpreting runtime behavior between actors
  • Relation links
  • Describing the port relations between actors
  • A workflow, at an abstract level, is a composite
    actor

5
What does Kepler offer?
  • A visual interface
  • For composing a workflow
  • For annotating components and flows
  • A set of components
  • Actors
  • Directors
  • Relation links
  • Ports
  • Execution control facilitates
  • Run/Pause/Stop
  • Parameter tuning
  • Set breakpoint

6
How to compose a workflow?
  1. Select proper components
  2. Develop necessary new components
  3. Describe flow logic
  4. Save/Open a flow description

7
How to compose a workflow (Cont.)
  • Select proper actors and directors
  • Browse the tree structure of a component library
  • Ontology based search (not mature)
  • Check the documentation and source code of a
    component through GUI
  • Check the functionality of an actor Port,
    parameter, pre-fire(), fire(), and post-fire().
  • Check the functionality of a director.
  • Develop new components, if needed
  • Describe flow logic
  • Save/Open a flow description

8
How to compose a workflow (Cont.)
  • Select proper components
  • Develop necessary new actors
  • Wrap a legacy code (as a command line)
  • Using CommandExecLine Actor
  • Specify location of the command, output file and
    location, and Environment variables
  • Wrap R scripts
  • Using RExpression Actor
  • Provide the script, working directory and Pixels
    for output
  • Needs a R engine

9
How to compose a workflow (Cont.)
  • Select proper components
  • Develop necessary new components
  • Import existing web services
  • Using Web service Actor
  • Provide WSDL URL, method name, authentication,
    and others
  • Only one method can be imported per actor
  • Interfacing Grid resources
  • Authentication
  • Globus Job
  • Globus Proxy
  • Grid FTP
  • Parameterized Globus Job
  • Run Job Grid Client
  • Update Grid FTP

10
How to compose a workflow (Cont.)
  • Select proper components
  • Develop necessary new components
  • Write java code and add a new actor
  • write the java code, according to the Kepler
    actor convention such as definition of ports,
    parameters and (pre/post) fire functions
  • Write make a actor.xml file describe the
    functionality and class location, component ID,
    ontology, and a MANIFEST.MF file
  • Put the code in kepler/src, and compile it
  • After compiling the code, you have to ant
    buildkarlib
  • And generate new kepler.jar file, you can view
    the source of the new actor
  • Remove .kepler and .ptolemy from the users home
    directory, before re-starting kepler

11
How to compose a workflow (Cont.)
  • Select proper components
  • Develop necessary new components
  • Describe flow logics
  • Between actors port/port connection.
  • A port is for input or output
  • In a workflow
  • A single port can only be connected by one
    relation link
  • A multi-ports can be connected by more than one
    links
  • Single ports can linked as more to more, such
    as fan-in or fan-out by using a relation
    component. Fan-out is only supported by few
    directors
  • Choose execution mode
  • Select an director
  • Continuous Time, Discrete Event, Process Net and
    Static Data Flow
  • There are more directors in Ptolemy
  • Customize it
  • Set parameters

12
How to compose a workflow (Cont.)
  • Select proper components
  • Develop new components, if needed
  • Describe flow logics
  • Save/Open a flow description
  • Modeling Markup Language (MoML)
  • in XML dialect

13
An example
  • Add a helloworld actor
  • Make three web service actors
  • Import a web service echoString
  • Connect them to output

14
Kepler at Runtime
  1. Human in the loop
  2. Scheduling mechanisms

15
Kepler at Runtime, Cont.
  • Human in the loop
  • Run a workflow in the same environment where the
    user compose the workflow.
  • Set break point in an actor
  • Modify parameters of Actors and directors
  • Scheduling mechanisms

16
Kepler at Runtime, Cont.
  • Human in the loop
  • Scheduling mechanisms
  • Parameter sweeping not directly, but can do it
    via a Nimrod actor
  • Job farming not directly support

17
Kepler in VL-e
  • At application SPs
  • In IBU case
  • A starting point for the new design of the VL-e
    framework
  • Virgil as the user interface
  • Developing a RTS (in VLAM) director

18
Discussion
  • Outstanding functionality Workflow modeling
  • Concept of Director (from Ptolemy) explicitly
    modeling the execution semantics
  • Different levels of encapsulation
  • a complex actora director actors (or complex
    actors),
  • a workflow a complex actor
  • Other functionality is provided in the similar
    way as the other systems, e.g., Taverna, Triana
    and VLAM
  • GUI drag and draw
  • Runtime parameter configuring
  • Grid access, Web service support, Parameter
    sweeping
  • Weak points
  • Triana provides better way to outsource web
    services
  • It is still in Alpha release

19
Cont.
  • Interoperability with a different workflow
    system
  • At the engine level
  • Wrap the workflow engine as a kepler actor.
  • Do not change the components in that system
  • At the actor level
  • Or wrap components of the workflow system as
    actors,
  • Optional (wrap the engine as a kepler director)
  • and use the Kepler engine to control them.

20
References
  • Kepler website http//kepler-project.org/
Write a Comment
User Comments (0)
About PowerShow.com