1
Bookshelf.exe Executable Extensions of the
Bookshelf

• Marius Eriksen and Igor Markov
• University of Michigan, EECS

2
Outline

• Making the bookshelf more dynamic
• Implementations data algorithm evaluations
• Potentially more than that
• Related efforts
• SatEx, PUNCH, NEOS, OmniFlow
• Proposed solution Bookshelf.exe
• Services offered
• online execution, ASP, sharing of results, flow
  scripting,
• Interfaces
• Support for optimization-specific concepts
• Power versus ease-of-use and modularity
• Architecture and implementation details

3
Making the Bookshelf More Dynamic

• Existing bookshelf
• A static collection of data and implementations
• Manual addition of material
• No automatic evaluations, reporting of results
• Those limitations are not fundamental
• Artifacts of the current implementation
• Proposal for improvement Bookshelf.exe

4
Related Efforts

• SatEx http//www.lri.fr/simon/satex/satex.php3
• Specialized in satisfiability problems
• PUNCH http//punch.ece.purdue.edu
• Very broad selection of software
• From StarOffice and gnulot to Capo and Feng Shui
• Local to Purdue Univ.
• NEOS http//www-neos.mcs.anl.gov/neos
• Open-source, distributed architecture
• Used primarily for linear and non-linear
  optimization
• OmniFlow DAC 2001, Brglez and Lavana
• http//www.cbl.ncsu.edu/OpenProjects/OmniFlow
• Distributed Collaborative Design Framework for
  VLSI
• GUI-based flow control, chaining of design tools

5
SatEx

• Continual evaluation and ranking of submitted
  implementations
• Results produced and posted on the Web
  automatically
• Intuitive interface
• Popular
• 93,707 hits since March, 2000
• 23 SAT provers, 32,610 executions
• Limited scalability
• Runs on one workstation
• 529 days, 17 hours, 53 minutes and 20 seconds of
  CPU time
• Specialized to one application

6
PUNCH

• Very general execution framework
• Everything from domain-specific algorithm
  implementations to GUI-based office applications
• Custom-designed filesystem (restricts to Purdue
  hosts)
• 241,458 executions in 5 years
• 8,152 on VLSI CAD related applications
• 20 publications
• Only maintainer can add executables
• No support for evaluation and chaining/flows
• No stats for results of runs (cf. top 20 on
  SatEx )
• No MIME-like data types (even IE/netscape need
  them!)
• Difficult to use when multiple tools are involved

7
NEOS

• Open-source, distributed framework
• Wide use, a solid library of implementations
• Adding new implementations requires maintainer
  intervention (but less than on PUNCH)
• Each new implementation must come with a host
• Distributed maintenance
• Loose data typing
• No type system for data and implementations
• Compare to MIME

8
NEOS (what can be improved)

• Independent eval. and verification of results
• E.g., PUNCH offers a WL-eval. From the bookshelf
• Real-time on-line reporting of results stats
• Provisions for chaining multiple implementations
• Scripts to control the execution and evaluation
  flows
• Provisions for pairing execs with benchmarks
• Goal SatEx-like evaluation,but for multiple
  data types

9
OmniFlow

• Context collaborative VLSI Design
• collaboration in terms of computational resources
• no provisions for sharing results on the Web
• Distributed over multiple hosts
• Provides GUI-based flow control
• supports chaining of design tools
• several hard-coded conditions for flow control
• no support for execution conditional on results
• no scripting language limited by GUI
• Cannot dynamically add hosts

10
Bookshelf.exe

• Best of SatEx, PUNCH, NEOS and OmniFlow
• Reporting style similar to SatEx ( alternatives)
• The versatility of PUNCH
• Scalability and distributed nature of NEOS (or
  better)
• Flow control as in OmniFlow or better
• New features
• MIME-like data types and optimization-specific
  concepts
• Automatic submission of binaries and source code
• Chaining of implementations scripting for flow
  control
• Support for use models with proprietary data or
  code

11
Bookshelf.exe (2)

• Scalability
• Computation is distributed (unlike in SatEx)
• Maintenance is automated (unlike in NEOS)
• Support for multiple use models
• adapts to users
• Multiple levels of expertise
• Multiple levels of commitment
• Sharing of public data
• Hiding/protection of proprietary data
• Screen-saver mode, similar to SETI_at_Home,
  Entropia, etc

12
Bookshelf.exe (3)

• Provides basic algorithm evaluation
  infrastructure to research groups
• Avoid large maintenance costs
• Just as usable for larger groups

13
End-user Interface

• Web site
• Using CGI/PHP for interactive content and
  database interaction
• Submission through the web site (email is ok)
• Dynamic addition of computation hosts
• Stores data locally or uploads on demand
• Automatic Web-based reports and statistics

14
End-user Interface (2)

• User accounts (including anonymous)
• Every user has private resource-limited
  workspace
• Job submissions must be from accounts
• Results of jobs stored on accounts
• Anonymous jobs possible, but can use fewer
  resources
• users are encouraged to register
• Many policy questions (no roadblocks seen)
• e.g., can different owners chain their jobs?
• Transparent error diagnostics
• Greatly improves learning curve

15
End-user Interface (3)

• Result notification (similar to NEOS)
• E-Mail
• Web reports
• Script composer for making run flows without
  needing to know a scripting language
• less flexibility, but better learning curve
• GUI implemented as form that POSTs scripts

16
Script Composer

• HTML Form interface basic flow with fundamental
  predicates and functions
• All API functions
• E.g. run algorithm optimizer 2 and store the
  results
• Several common Perl functions
• Conditional statements
• E.g. verify if quality is more than 60
• Iteration support
• E.g. iterate until quality is 100 or the number
  of iterations is 100, run randomized solver 3
  timers, report the best result
• Start with one step add an arbitrary number
  of steps

17
Optimization-specific Concepts in Bookhelf.exe

• Type system (similar to MIME) for all types of
  submissions
• Rules for matching submissions of different types
• E.g., match a placer with a LEF/DEF benchmark
• Types include
• Algorithm implementations
• Deterministic and randomized optimizers, etc.
• Input data, results
• Common benchmarks

18
Architecture Overview
19
Component Types

• Where relevant, components also have types
• Data
• Data format
• E.g. Cadence LEF/DEF, DIMACS CNF
• Implementations
• Architecture (solaris, linux, source code,
  etc.)
• Class (optimizer, randomized solver, etc.)

20
Component Descriptors

• Unique id, name and description for each
  component
• Owner field
• Component history
• E.g. usage of a certain implementation, piece of
  data or benchmark
• Component specific fields
• Associated expert for implementation, incremental
  diff for data
• Component version
• Component changelog

21
Data Controller

• Backing store
• Permanent storage of all involved components
• Backed by database and file system
• Data, results suited for database
• Implementations suited for file system storage
• Components may be deferred to local storage
• Proprietary, non-disclosed components on a host
  owned by the proprietor of the component
• All data stored with complimentary descriptor

22
Job Controller

• Responsible for communicating with and
  controlling all agents (and thus jobs and
  implementations)
• Initiates jobs
• Delivers user instructions
• Handles data retrieval and storage
• Interfaces with data controller
• Keeps track of node load, etc. for proper
  distribution

23
Host Job Controller

• Resides on each node in the distributed network
• Basic token of presence of DS-OSS
• Reports statistics to job controller
• Load, memory usage
• Signals when ready to take jobs
• Launches agents as necessary
• Performs self upgrades
• New software revisions, bugfixes

24
The Agent

• Responsible for running job on node
• Provides environment for script to run in
• API
• Script sandbox/jail
• For security reasons
• Provides common callbacks if not provided by
  scripts
• Data storage/retrieval

25
The Agent (2)

• Input/Output of data from script
• Job notification and control
• Start, stop, termination
• Maintains local demands
• Resource usage
• Stop on local demand

26
Scripts

• In Perl
• Interfaces directly with an algorithm, or expert
  handling it
• API support
• Full Perl library
• Job control functionality
• Data storage and retrieval

27
Scripts (2)

• Checks for conflicting input data types and
  algorithm input type
• Generates appropriate, understandable errors
• A number of automatic tasks
• Pass thru data storage (e.g. results of a single
  run)
• Can provide all steps automatically, leaving the
  user to do exactly what he/she wants

28
The Expert

• Wrappers for proprietary implementation
  interfaces
• Knows how to deal with a particular interface
• E.g. command line options, configuration files
• Translates from well known parameters (specified
  from the API) to proprietary interface

29
In Conclusion

• DSS-OS provides features that are missing in
  current systems
• Existing systems are limited by their
  architecture and thus cannot provide these needed
  features.
• Provides an infrastructure that is scalable
• Computationally scalable
• Scalable to proprietary vendors

30
In Conclusion (2)

• Scripting support allows for flexible execution
  flows
• Conditional flows, iterative flows
• Provides an established infrastructure for
  smaller research groups
• Enjoying advantages that only large proprietary,
  high-maintenance systems previously could provide.