Scientific workflow: research - PowerPoint PPT Presentation

About This Presentation
Title:

Scientific workflow: research

Description:

Bob Hertzberger(University of Amsterdam, the Netherlands) ... WS and Semantic issues, Dr. Steve Ross-Talbot (CEO, and a co-founder, of Pi4 Technologies) ... – PowerPoint PPT presentation

Number of Views:45
Avg rating:3.0/5.0
Slides: 26
Provided by: zhim
Category:

less

Transcript and Presenter's Notes

Title: Scientific workflow: research


1
Scientific workflow research recent activities
  • Dr Zhiming Zhao
  • SNE_at_UvA,SP2.5_at_VL-e

2
Outline
  • Research theme
  • International cooperation
  • Ongoing work
  • Summary

3
Scientific workflow in e-Science

4
Scientific Workflows in e-Science
Experiment processes
  • A SWMS is able to
  • Automate experiment routines
  • Rapid prototype experimental computing systems
  • Hide integration details between resources
  • Manage experiment lifecycle

workflows for administration, e.g., AAA, and
other issues.
Abstract workflows
Executable (concrete workflows)
5
Inside a Scientific Workflow Management System
  • In our view, a SWMS at least implements
  • A model for describing workflows
  • An engine for executing/managing workflows
  • Different levels of support for a user to
    compose, execute and control a workflow.

Workflow (based on certain model)
Composition
A SWMS
User support
Engine level control
Engine
Resource level control
resources
6
Diversity in SWMS
  • Taverna
  • Web services based language Scufl
  • FreeFluo engine
  • Graphical viz of workflow
  • Triana
  • Components
  • Task graph
  • Data/control flow
  • Kepler
  • Actor,director
  • MoML
  • Execution models
  • Pegasus
  • Based on DAGMan
  • VDL
  • DAG

  • DAGMan
  • Computing tasks
  • DAG

7
Research context
  • Different levels of abstraction
  • Workflow services
  • Short term
  • Long term a generic and effective workflow
    management service

8
Mission
  • Effectively reuse existing workflow managements
    systems, and provide a generic e-Science
    framework for different application domains.
  • A generic framework can
  • Improve the reuse of workflow components and the
    workflows for different experiments
  • Reduce the learning cost for different systems
  • Allow application users to work on a consistent
    environment when underlying infrastructure
    changed

9
A workflow bus paradigm
Workflow
Sub workflow 1
Sub workflow 2
Sub workflow 3
Triana
Taverna
Kepler
Workflow bus
A workflow bus is a special workflow system for
executing meta workflows, in which sub workflows
will be executed by different engines.
10
Applications of workflow bus
  • Use case 1
  • A user has workflow in Taverna
  • Some functionality is missing in Taverna but can
    be provided by Triana
  • He can develop the workflow in two systems, and
    run it via the workflow bus
  • Use case 2
  • A user wants to execute a Taverna or Triana
    workflow in multiple instances with different
    input data

11
A JADE/Ptolemy based prototype
Scenario Mnger
Scenario Mnger
Scenario Mnger
Study Mnger
Ptolemy
Actor
Actor
Director
Actor
User interface
12
Workflow bus
  • See details in Rapid prototyping talk (a paper to
    be submitted to IEEE e-Science)

13
Activities in international community
  • Intl workshop on workflow systems in e-Science,
    in the context of ICCS 2006 in Reading (Zhiming
    Adam)
  • Industrial workflow standards and scientific
    workflows in e-Science, in the context of IEEE
    e-Science conference, in Amsterdam (Adam
    Zhiming)

14
Intl workshop on workflow systems in e-Science
(WSES06), in ICCS 06, Reading, UK(Zhao, Belloum)
  • Program committee
  • Marian Bubak (AGH University of Science and
    Technology, Krakow, Poland).
  • Rajkumar Buyya (The University of Melbourne,
    Australia).
  • Ewa Deelman (University of Southern California,
    USA).
  • Thomas Fahringer (University of Innsbruck,
    Austria).
  • Bob Hertzberger(University of Amsterdam, the
    Netherlands).
  • Minglu Li (Shanghai Jiaotong University, China).
  • Ling Liu (Georgia Institute of Technology, USA).
  • Peter Rice (European Bioinformatics Institute,
    UK).
  • Ian Taylor (Cardiff University, UK).
  • Zhiwei Xu (Chinese Academy of Sciences, China).
  • Scope
  • The WSES workshop focuses on practical aspects of
    scientific workflow management systems design,
    implementation, applications in all fields of
    computational science, interoperability among
    workflows and the e-Science infrastructure, e.g.,
    knowledge framework, for workflow management. 
    The workshop aims to provide a forum for
    researchers and developers in the field of
    e-Science to exchange the latest experience and
    research ideas on scientific workflow management
    and e-Science.
  • Paper
  • 27 submissions, 17 accepted (8 regular, 9 short),
    three sessions.
  • Audience
  • 25 audience

15
Session 1 Workflow applications(Zhiming Zhao)
  • Altintas presented the first two papers she
    discussed how Kepler was used in integrating GIS
    packages for geospatial modelling, and in
    coupling distributed computing processes and a
    GEON portal. She demonstrated the flexibility of
    using Kepler in wrapping command line based
    software resources and in controlling backend
    computing processes.
  • Paventhan presented the third paper he discussed
    the development and implementation of a wind
    tunnel grid system workflow using .NET-based CoG
    Toolkit and Globus grid services.
  • Afterwards, three short papers were presented.
    Navas-Delgado presented how reusable services in
    a workflow system were used to facilitate the
    rapid prototyping of scientific experiments,
    Czekierda discussed workflow issues in a
    distributed scientific experiment management
    environment called Virtual Laboratory, and
    Kaczmarek discussed work on integrating
    compute-intensive tasks into scientific workflow
    in BessyCluster.

16
Session 2 Workflow system architecture(Adam
Belloum)
  • The presenter of the first paper was absent it
    is about development of a Java based workflow
    engine.
  • The second and third papers reported research
    conducted in the project of ICENI II. Colling
    discussed how the high level services ICENI II
    environment added on-top of existing Grid
    architectures for supporting workflows involves
    different experiment instruments. McGough focused
    on the workflow deployment issues between
    different levels of abstraction in ICENI II.
  • Harrson presented a regular paper on handling
    data in scientific workflows using a light weight
    service called Styx.
  • Lee presented a short paper on using agent
    technology in developing workflow middleware and
    in coordinating workflows and a Grid portal.

17
Session 3 System development(Ilkay Altinas )
  • The first paper was presented by Hluchy he
    discussed tools developed in the project of
    K-WfGrid for supporting semantic level workflow
    composition.
  • Zuo and Merelli presented two short papers on
    optimising Grid computing processes via net
    solver, and on enacting workflows in an e-Science
    environment.

18
BOF discussion(Ilkay Altinas, Zhiming Zhao, Adam
Belloum )
  • Scientific workflows and Grid infrastructure
  • Utilization of computing resources in scientific
    workflows
  • Virtual Organizations, e.g., AAA issues
  • Industrial standards
  • Web services and data intensive applications
  • Workflow languages, BPEL and BPML, in scientific
    computing
  • Software engineering in developing scientific
    workflows and systems
  • Agent technologies in workflow systems
  • Engineering disciplines in developing workflow
    systems
  • Utilizing unstable academic workflow systems in
    e-Science applications
  • Scientific workflow systems usage and different
    levels of user support
  • Automatic flow composition
  • Dynamic workflows and human in the loop computing
  • Data provenance and analysis
  • Generic e-Science framework and knowledge
    transfer for different application domains
  • Knowledge infrastructure in scientific workflows
  • Interoperability among workflows and workflow
    systems

19
NOTES -1
  • Good end-user interaction
  • Who are the users? How do we break the ice?
  • Iterative development?
  • Create multi-disciplinary teams
  • Semantics and knowledge-base aid
  • How should I start if theres no ontology?
  • What is the point scientific workflows and
    business workflows split?
  • How about dynamic and adaptive workflows?
  • Not everything can be modeled as a service
  • No need to standardize the computation model, but
    standards are needed to interoperate
  • Actors (processors) in the workflow system should
    be designed free of technologies

20
NOTES-2
  • Topics of interest for the next workshop
  • Applications
  • Complete experiment lifecycle
  • Fault tolerance in execution
  • Enactment models
  • Process definition tools
  • Lessons learned
  • Methodologies for workflow construction

21
Follow up
  • A special issue in Scientific Programming
    journal
  • the workshop on Workflows in Support of
    Large-Scale Science 2006, and the 1st
    International Workshop on workflow systems in
    e-Science
  • In Scientific Programming journal
  • Target at the last issue of this year, or the
    first one in 2007.
  • A CFP is going to announced today. Limited to
    authors of these two workshops.
  • Workflow systems in e-Science 2007

22
Cont.
  • Industrial workflow standards and scientific
    workflows in e-Science, in the context of IEEE
    e-Science conference, in Amsterdam (Adam
    Zhiming)
  • Pegasus, Dr. Ewa Deelman (Department of Computer
    Science University of South California)
  • BPEL, Dr. Dieter König (IBM Research Germany
    Development Laboratory)
  • Kepler, Dr. Bertram Ludäscher (Department of
    Computer Science University of California, Davis)
  • Taverna, Prof. Peter Rice (European
    Bioinformatics Institute)
  • WS and Semantic issues, Dr. Steve Ross-Talbot
    (CEO, and a co-founder, of Pi4 Technologies)
  • Triana, Dr. Ian J. Taylor (Department of Computer
    Science Cardiff University)

23
Ongoing research
  • Web service in data intensive applications
  • Execution models for Grid workflows
  • Ptolemy and kepler
  • Workflow bus

24
Summary
  • Scientific workflow management is an important
    service in e-Science and crosses different Grid
    and e-Science layers
  • Re-use existing work and join international
    collaboration is important

25
  • Acknowledgement
  • Adam and all other members in SP2.5
  • Referneces
  • Z. Zhao A. Belloum H. Yakali P.M.A. Sloot and
    L.O. Hertzberger Dynamic Workflow in a Grid
    Enabled Problem Solving Environment, in
    Proceedings of the 5th International Conference
    on Computer and Information Technology (CIT2005),
    pp. 339-345 . IEEE Computer Society Press,
    Shanghai, China, September 2005.
  • Z. Zhao A. Belloum A. Wibisono F. Terpstra
    P.T. de Boer P.M.A. Sloot and L.O. Hertzberger
    Scientific workflow management between
    generality and applicability, in Proceedings of
    the International Workshop on Grid and
    Peer-to-Peer based Workflows in conjunction with
    the 5th International Conference on Quality
    Software, pp. 357-364. IEEE Computer Society
    Press, Melbourne, Australia , September 19th-21st
    2005.
  • Z. Zhao A. Belloum P.M.A. Sloot and L.O.
    Hertzberger Agent Technology and Generic
    Workflow Management in an e-Science Environment,
    in Hai Zhuge and G.C. Fox, editors, Grid and
    Cooperative Computing - GCC 2005 4th
    International Conference, Beijing, China, in
    series Lecture Notes in Computer Science, vol.
    3795, pp. 480-485. Springer, November 2005. ISBN
    3-540-30510-6. (DOI 10.1007/11590354_61)
  • Z. Zhao A. Belloum P.M.A. Sloot and L.O.
    Hertzberger Agent technology and scientific
    workflow management in an e-Science environment,
    in Proceedings of the 17th IEEE International
    conference on Tools with Artificial Intelligence
    (ICTAI05), pp. 19-23. IEEE Computer Society
    Press, Hongkong, China, November 14th-16th 2005.
Write a Comment
User Comments (0)
About PowerShow.com