Efrat Jaeger SDSC

1
Scientific Workflows GEON

• Efrat Jaeger SDSC
• Bertram Ludäscher UC DAVIS
• Krishna Sinha Virginia Tech
• Ashraf Memon SDSC
• Ghulam Memon SDSC
• Ilkay Altintas SDSC
• Kai Lin SDSC
• many others esp. KEPLER community

2
Scientific Workflows Pre-Cyberinfrastructure

• Data Federation Grid Plumbing
• access, move, replicate, query data (Data-Grid)
• authenticate SRB Sget/Sput OPeNDAP,
  Antelope/ORBs
• schedule, launch, monitor jobs (Compute-Grid)
• Globus, Condor, Nimrod, APST,
• Data Integration
• Conceptual querying integration, structure
  semantics, e.g. mediation w/ SQL, XQuery OWL
  (Semantics-enabled Mediator)
• Data Analysis, Mining, Knowledge Discovery
• manual/textbook (e.g. ternary diagrams), Excel,
  R, simulations,
• Visualization
• 3-D (volume), 4-D (spatio-temporal), n-D
  (conceptual views)

• one-of-a-kind custom apps., detached (island)
  solutions
• workflows are hard to reproduce, maintain
• no/little workflow design, automation, reuse,
  documentation
• need for an integrated scientific workflow
  environment

3
(No Transcript)
4
Analysis Workflow in KEPLER

• Scientific Workflow (SWF) design
• SWF automation
• Exploration discovery mode (change parameters,
  data sets, etc. and rerun)
• SWF reuse, documentation, reproducibility

5
Some KEPLER Components (Actors)
6
KEPLER Team Work GEON Dataset Generation
Registration
Makefile gt ant run
SQL database access (JDBC)
Matt,Chad, Dan et al. (SEEK)
Efrat (GEON)
Ilkay (SDM)
Yang (Ptolemy)
Xiaowen (SDM)
Edward et al.(Ptolemy)
7
KEPLER an open source, cross-project
collaboration

• Ilkay Altintas SDM, Resurgence, NLADR,
• Kim Baldridge Resurgence, NMI
• Chad Berkley SEEK
• Shawn Bowers SEEK
• Terence Critchlow SDM
• Tobin Fricke ROADNet
• Jeffrey Grethe BIRN
• Christopher H. Brooks Ptolemy II
• Zhengang Cheng SDM
• Dan Higgins SEEK
• Efrat Jaeger GEON
• Matt Jones SEEK
• Werner Krebs, EOL
• Edward A. Lee Ptolemy II
• Kai Lin GEON
• Bertram Ludaescher GEON, SDM, SEEK, BIRN, ROADNet
• Mark Miller EOL
• Steve Mock NMI
• Steve Neuendorffer Ptolemy II

Ptolemy II
www.kepler-project.org
Your Logos Names HERE!!!
8
(No Transcript)
9
Demonstration by Efrat Jaeger
10
Q A
11
KEPLER An Open Collaboration

• Initiated by members from NSF/ITR SEEK and DOE
  SDM/SPA now several other projects (GEON,
  Ptolemy II, EOL, Resurgence/NMI, )
• Open Source (BSD-style license)
• Intensive Communications
• Web-archived mailing lists
• IRC (!)
• Meetings, Hackathons
• Co-development
• via shared CVS repository
• joining as a new co-developer (currently)
• get a CVS account (read-only)
• local development contribution via existing
  KEPLER member
• be voted in as a member/co-developer

12
Scientific Workflow (SWF) Design

• Support SWF design reuse, via
• Structural data types
• Semantic types
• Associations (constraints) between them
• Type checking, inference, propagation
• ?Separation of concerns
• structure, semantics, WF orchestration, etc.

13
Related Publications

• Scientific Workflows
• Scientific Workflow Management and the Kepler
  System, B. Ludäscher, I. Altintas, C. Berkley, D.
  Higgins, E. Jaeger-Frank, M. Jones, E. Lee, J.
  Tao, Y. Zhao, Concurrency and Computation
  Practice Experience, Special Issue on
  Scientific Workflows, to appear, 2005.
• A Framework for the Design and Reuse of Grid
  Workflows, Ilkay Altintas, Adam Birnbaum, Kim
  Baldridge, Wibke Sudholt, Mark Miller, Celine
  Amoreira, Yohann Potier, and Bertram Ludaescher,
  Intl. Workshop on Scientific Applications on Grid
  Computing (SAG'04), LNCS 3458, Springer, 2005
• Kepler An Extensible System for Design and
  Execution of Scientific Workflows, I. Altintas,
  C. Berkley, E. Jaeger, M. Jones, B. Ludäscher, S.
  Mock, 16th International Conference on Scientific
  and Statistical Database Management (SSDBM'04),
  21-23 June 2004, Santorini Island, Greece.
• Kepler Towards a Grid-Enabled System for
  Scientific Workflows, Ilkay Altintas, Chad
  Berkley, Efrat Jaeger, Matthew Jones, Bertram
  Ludäscher, Steve Mock, Workflow in Grid Systems
  (GGF10), Berlin, March 9th, 2004.
• An Ontology-Driven Framework for Data
  Transformation in Scientific Workflows, S. Bowers
  and B. Ludäscher, Intl. Workshop on Data
  Integration in the Life Sciences (DILS'04), March
  25-26, 2004 Leipzig, Germany, LNCS 2994.
• A Web Service Composition and Deployment
  Framework for Scientific Workflows, I. Altintas,
  E. Jaeger, K. Lin, B. Ludaescher, A. Memon, In
  the 2nd Intl. Conference on Web Services (ICWS),
  San Diego, California, July 2004.

14
Data Integration
Eco Grid
Knowledge Representation
Data Federation
Process Integration (Scientific Workflows)
Source B. Ludaescher, UC DAVIS ECS-289
Scientific Data Management WQ05