CAST : CMS Analysis and Specification Tool

1
CAST CMS Analysis and Specification Tool

• Praveen Vutukuru (vpraveen_at_cs.umn.edu)
• Greg Graham (ggraham_at_fnal.gov)
• Jaideep Srivastava (srivasta_at_cs.umn.edu)

2
Ethos of MCRunjob

• Applications in complex production processing
  environments often need to be tamed
• Hundreds of input parameters during MC Production
• Heterogeneous runtime environments
• Complex multi-application workflows
• Dependencies and relationships among the metadata
  often modeled inside of obscure shell scripts
• MCRunjob captures such specialized knowledge and
  makes it available to non-expert users
• Metadata and schema oriented descriptions
• Tracks dependencies among metadata
• Tracks synonyms between groups of metadata,
  versions
• Organization of user registered functions that do
  the actual work
• Framework driven organization of tasks

3
MCRunjob

• Is a powerful grid workflow manager used to
  manage generation of large numbers of production
  processing jobs in high energy Physics
• It is used to manage large Monte Carlo production
  processing
• MCRunjob converts core metadata into jobs
  submittable in a variety of environments
• MCRunjob features a modular design which allows
  for easy expansion to new job description
  languages or new application level tasks

4
Architecture of MCRunjob

• There are three major components of McRunjob
• Configurator
• container for schema describing some well defined
  application input, task, and external interfaces
  to DB or grid services
• implements framework interfaces
• registers functions to handle framework calls,
  extends own interface, extends schema, defines
  rules and dependencies to construct values for
  parameters
• Linker
• a container for Configurators, checks
  dependencies, enables inter-configurator
  communication
• a container for script objects generated by
  Configurators
• runs the framework
• ScriptGen
• A class for Configurators that adds methods for
  script object generation and framework method
  delegation

5
McRunJob WorkFlow
6
Macro Script Language

• The Linker has a facility to read macro scripts
  and parse lines one by one
• Functions available include
• Attaching and naming Configurators, setting
  parameter values, adding schema values, defining
  synonyms, executing the framework or selected
  framework calls, executing selected methods,
  exception handling, executing other scripts
• Procedural constructs supported for handling
  multiple jobs
• Parsing is done by Configurators themselves
• Users can extend the macro script interface by
  registering their own parser functions to the
  Configurators

7
McRunjob Example from CMS
8
Macro Script Language

• The MCRunjob macros are intended to provide user
  interface to the Configurator and Linker APIs.
• Macros can also serve as a first order
  provenance information for the described or
  constructed workflow

9
Commands in Macro Scripts

• Example Commands
• cfg cfgIdentifier cmd
• define keyname expression
• synonym key cfgIdentifiernewkey
• Drawbacks of the macro language
• User must learn several cryptic commands and
  syntax to write a macro script
• User must remember all the commands

10
CAST

• CAST is a graphical editor for creating MCRunjob
  macro scripts
• Salient CAST features include
• Interactive, graphical editor, making it
  intuitive to use and easy to learn
• A built-in library of various MCRunjob features,
  including context files, configuration files,
  creating dependencies, metadata relationships,
  framework calls, ontologies, etc. these are
  represented as graphical elements
• Automatic generation of MCRunjob script
• Benefits of CAST
• Easy to learn, intuitive to use
• No need to remember cryptic MCRunjob commands
• Graphical programming paradigm makes dependencies
  explicit/visual, leading to better quality
  programs
• Overall, enhanced productivity reduced effort

11
Demo
srivastava Need to have a couple of slides after
this showing some screenshots to illustrate the
points made above about the salient features of
MCRunjob