Grids in EDA Software Development

1
Grids in EDA Software Development
gridMatrix Technology
Tom Grotton Cadence Design Systems,
Inc. Director, IT Server Farm Initiative
2
Corporate Needs

• Need more computing capacity
• Products are becoming more complex and taking
  additional cycles to build and test.
• Products need to be supported on multiple
  platforms (OS and version)
• Process and machine utilization measurements are
  inadequate to effectively plan and manage capital
  equipment requirements. We continue to add more
  and more servers to meet this unplanned demand.
• Need a more reliable hardware, storage and
  network
• Nightly runs of product builds and tests fail too
  often due to machine or infrastructure problems
• To decrease the dependency on individual hardware
  resources
• Need more cost-effective IT contribution
• System administration costs are ever increasing
  as more and more hardware is added. According to
  Gartner and based on our experience, TCO costs
  are being primarily driven by support costs and
  not capital. For every 20 cents spent on
  equipment, 80 cents is spent supporting it.

3
How We Address these Needs

• Manage Cost (Total Cost of Ownership)
• Agility (Adaptive IT infrastructure)
• Increase Quality (Product and Service)
• Virtualization (Computing, Networking, Storage,
  Security, File systems, Monitoring, and Common
  Process Architecture)
• Federation (Brokering, Provisioning, Resource
  Sharing, Globalization)
• Automation (Process, Workflow and Ease-of-use)

4
Capitalizing On Our Expertise

• The key point of coordination, information
  exchange and collaboration for those involved in
  large-scale grid projects in the US, Europe,
  Canada and Asia-Pacific is the Global Grid Forum
  (GGF). GGFs platinum and silver sponsors are
  Compaq, HP, IBM, Sun, Microsoft, Platform
  Computing, AVAKI and Entropia.
• As a member of the GGF, Cadence chairs and leads
  the
• GGF Common Use Case Model Architecture Working
  Group, formerly known as NPi
• This Working Group is responsible for defining a
  particular lightweight, high-level architecture
  for distributed computing management. Creating an
  overall Reference Model for advanced distributed
  computing management
• GGF Job Submission Description Language working
  group
• This Working Group is to define a standard Job
  Submission Description Language (JSDL) for
  describing a computational 'job' and its required
  execution environment for submission to a Grid.
  This language would be used to construct a
  document that would encapsulate all of the
  information needed by a Distributed Resource
  Management (DRM) system or a job submission
  system on a Grid, such as a Grid scheduler, to
  place a job in its required environment for
  execution.
• GGF Grid Policy Architecture working group
• This Working Group is developing the requirements
  and architecture for interoperable policy
  management, and for how they are described,
  evaluated, stored, managed, distributed, and
  enforced. Since policies may also be associated
  with instrumentation to provide feedback into
  evaluating the effectiveness of the policies, the
  requirements and architecture will also provide a
  framework for associating the appropriate metrics
  and instrumentation with the policies.
• Is a seated member of the GGF DRMAA (Distributed
  Resource Management Application Architecture)
  working group.
• Reference The 451 Group Grids 2004

5
Grids

• Definition Grid computing refers to a network
  architecture designed for large-scale dynamic
  sharing computing resources. A grid works by
  taking the responsibility for input/output
  requests for storage, memory, processing, and/or
  communications resources away from individual
  machines and instead moves that responsibility to
  a network grid that searches for available
  resources to handle resource requests.
• Usage A growing number of companies are
  considering deploying Grid Technology or have
  deployed it. These companies are finding the
  Total-Cost-of-Ownership to deployment, management
  and support Grid Technology to be overly costly.
• EDA is the slowest industry to adopt massive
  parallel processing grids.
• Partly because of the industry itself and partly
  because of the nature of the products.
• Industries that are using Grid Technology are
  Life Science, Engineering, Commerce, Financial,
  Electronic Design Automation, Digital content
  creation, Testing, Server/Storage, Batch
  processing, Decision support/data mining.

6
Grid Industry

• 2005 grid Hardware and Software solutions 18
  billion
• 2005 grid Software solutions 4 billion
• There are several middleware DRM (Distributed
  Resource Management) companies Platform, Sun,
  IBM, etc.
• There is an increase in next-generation DRM
  products that are no-cost.
• There are also products that have their DRM
  layer, GridIron, DataSynapse.
• Enter Utility Computing - With the emergence of
  this universal computing grid, there will be a
  need for a new breed of utility provider.
  Although companies and individuals will retain
  some local computing capabilities, there will be
  a growing opportunity for trusted neutral parties
  to operate and manage shared resources within the
  grid. (ASPnews 9/02)
• Projected growth of the software grid technology
  market is 25 until 2007, then it jumps up
  considerably due to the larger number of
  industries that will start to use grid
  technology. EDA is one of these.

Data sources http//www.gridpartners.com,
http//www.idc.com, http//www.econstrat.org/,
http//gridcomputingplanet.com,
http//www.bloor-research.com/,
http//www.gridforum.org/
7
Grid Technology Evolution

• Compute Grids share excess PC compute cycles to
  provide a high-performance, low-cost computing
  environment. Their application is scientific
  research and engineering.
• Example, SETI_at_Home.
• Market leaders are Entropia, Sun, Platform, and
  United Devices.
• The EDA industry deploys this kind of grid, but
  in a UNIX environment and not across the
  internet.
• Information Grids are distributed architecture
  designed for large-scale dynamic sharing of
  commercial and technical applications, data, and
  compute power within the enterprise or across
  multiple external organizations. They are used
  in specialize industries, such as,
  pharmaceutical, biotech, medical, financial
  services, oil and gas exploration.
• Example, IBM/Upenn Mammography research project.
• Market leaders are AVAKI, Platform, IBM, Sun, and
  HP.

Data source The Grid Report, Bloor Research,
November 2002
8
Grid Technology Evolution

• Service Grids make use of the intersection of
  grid and web services technology concepts. As
  such, the service grid provides the underlying
  architecture for Utility Computing model. Sun
  describes a Service Grid as collections of
  services brought together from across the
  network. Service Grid are composed of services
  that you need at a particular time mail, stock
  feeds, word-processing, or flight information,
  etc. The idea of the service grid is that each
  piece comes from a source that specializes in an
  external function.
• Examples, Sun ONE and HPs Utility Data Center
  OGSA.
• Market leaders are HP, IBM and Sun.
• Intelligent Grids are self-managing utility
  architectures that go beyond the bounds of
  department and enterprise massive extended
  enterprises encompassing partners, suppliers,
  and customers. Currently, intelligent grids use
  homogeneous resources.
• Examples of intelligent grids are IBMs eLiza
  IBM Workload Manager, HPs Planetary Computing
  project, and Suns Management Center, N1.
• Market leaders are IBM, Sun, and HP.

Data source The Grid Report, Bloor Research,
November 2002
9
Grid Technology What to do?

• Utility Grids are to some extent conceptual to
  the grid industry. They use the power utility
  model, where there are main generation grids and
  peak request or overflow local grids. Utility
  grids takes the position that computing resources
  and the DRM layer are commodities, because these
  resources can be provisioned on demand. Utility
  grids are capable of managing higher-level
  functions, such as, security, data access and
  transformation in a massively parallel
  environment across the enterprise.
• According to several industry sources, Compute,
  Information, and Service grids will likely
  converge into Intelligent grids in the future.
  The fifth grid type Utility Grids or Corporate
  Utility Computing Grids are evolving and will
  likely replace the Service Grid and Intelligent
  Grid in time. This grid is designed to make
  heterogeneous computing power and heterogeneous
  resources available on-demand.

10
What does the future look like? Utility Grids
11
Cadence Grid

• Inside Cadence we are deploying Grid Technology
  today! So far, it has resulted in an average of
  67 reduction in process job run-time for RD and
  Services business units worldwide.
• We have around 3000 CPUs in our grid. Next year
  we are looking at doubling this!
• RD and Services business units require IT
  provide grid control, monitoring, statusing,
  error/alert reporting, performance reporting and
  job optimizing for massively parallel processes
  across hundreds and thousands of heterogeneous
  computer resources, ranging from Microsoft
  Windows, Linux and UNIX (AIX, HP-UX, Solaris).
• Cadence has developed and deployed patented grid
  technology.
• As most of you have done or are trying to do, we
  have developed an abstraction layer that performs
  Grid Automated Process Control allowing a
  Developer, CM, PV, Flow, Support engineer to
  interface with complex job definitions in a
  natural manner. Today, LSF is the Distributed
  Resource Management (DRM) layer and networked to
  this are Data Grids, which make computer
  resources, networking and storage virtually
  invisible.
• Results - Significant reduction of
  Total-Cost-of-Ownership by reducing cost of
  adoption and deployment, job run-time, costs in
  support, capital equipment, personnel and the
  technical knowledge.

12
Yesterdays Process
Harness Scripts (fire up DRM - bsub)
Logs Email Reporting (Some Web Reporting)
Run Job or Tool
Hardware (Servers/Workstations)
13
Todays Process
This is the only interface that is seen!
Engineer submits a grid-enabled job
Intelligent Job Controlincluding web-based
monitoring and job control
DRM
Hardware (Servers/Workstations)
Resource Monitoring Alert Handling(DRM, system
and system configuration)
14
What is Needed
Standard means to grid-enable EDA Toolby thread
Engineer submits a grid-enabled job
Intelligent Job Controlincluding web-based
monitoring and job control
DRM
Hardware (Servers/Workstations)
Resource Monitoring Alert Handling(DRM, system
and system configuration)
15
Where are we?
Intelligent Grid Technology
Dependency Management
Site Grid Health
Site
Workspace Management
Heterogeneous Resources
Heterogeneous Process
Natural Language
Compute Grid Health(Commission Platform)
Local
Job Definition(Process Control)
Process
Multi-Site/Cluster
Job Control
Job Priorities
Job Execution
Platform Definition
Utility Grid
Workspace Management(RCS, ClearCase, CVS, VNC,
Extendable)
Alert Handling(GRS)
Dynamic Web Reporting(Customizable)
Self-Healing
Process Metrics
Job Statistics
Metrics
Job Monitoring
Queue Definition Optimization
Host Groups and Host Partitions
Workspaces
Pool Configuration
DRM Grid
Job Distribution
Res. Control
Commission and Monitoring
16
Site Grid Status
17
Horizons
Cycle-times reduced 67
Examples - 4 Hrs to 17 Mins - 13 Hrs to 2 Hrs -
6 Hrs to 30 Mins- 24 Hrs to 3.5 Hrs

• Corporate Grid (2005)
• Multiple site grids sharing resources and data
  intelligently across the WAN.
• gt50 utilization

Site Grid (2003) Combines compute farms to
facilitate the sharing of capital resources
across multiple departments, projects, and
groups Increase server utilization from gt30
Compute Farms (2002) Enables shared system usage
across departmentsIncrease Utilization from gt11
18
QA
19
(No Transcript)