ATLAS Offline Computing: MONARC Simulations Job Scheduler

1
ATLAS Offline ComputingMONARC Simulations Job
Scheduler

• Presenter Mr. Kevan Victor
• Advisors Alois Putzer Ph. D
• Iosif Legrand Ph. D
• UM-CERN-REU 2002

2
Overview

• Project Goals
• Method of Approach
• Results
• Neural Networks
• Future Plans/Proposals
• Acknowledgements

3
Goals

• Use the MONARC (Models of Networked Analysis at
  Regional Centres for LHC Experiments)Simulation
  tool to try to determine the most efficient
  manner for Job Scheduling in the final ATLAS
  offline computing system.

4
Method of Approach

• Modify Job Scheduler Javatm code by hand
• Try to implement a Neural Network to automate
  certain processes

5
Background Info

• Large amounts of data will be produced by ATLAS
  (Peta-Bytes per year)
• Data Path Raw/SIM -gt ESD -gt AOD -gt DPD
• MONARC simulations are used to get an idea of how
  the data will be processed, stored and analyzed
• Job Scheduling is a very important part of the
  final systems middleware

6
Proposed Data Access Diagram
RC
RC
RC
Middleware
PGA
PGA
PGA
7
Parameters Modified

• Factors dictating job exportation
• Mean Efficiency (Total Efficiency/Jobs Done)
• Waiting Queue
• Job Ratio (Waiting Jobs/Active Jobs)
• Load Submitted for Analysis

8
Method of Modification

• Local and External CPU Loads were compared with
  the following factors -
• - Mean Job efficiency
• - Length of Job Queue
• - Ratio of Job Queue to Active Jobs
• Config. files were modified to change loads for
  analysis
• Based on the idea that Jobs should be exported
  to other regional centers only when when the
  current center is approaching its max. value.
• The current scheduler uses static values to
  determine when to do this.

9
Important Considerations

• Overall Processing Time
• Job Efficiency (time for a single job to be
  processed)
• Bandwidth use
• CPU Loading

10
Results (No change to Scheduler)
11
Results (Modified Scheduler)
12
Results
13
Results
14
Results

• Most parameter changes displayed little effect on
  the estimated analysis time
• Some changes increased the estimated analysis
  time
• Surprise result

15
(No Transcript)
16
Neural Networks
17
What is Neural Net (a.k.a. ANN Artificial
Neural Network)?

• A network patterned after the human brain
  (neurons/parallel processing)
• A pattern recognition tool
• A good forecasting tool

18
Basic Components of a Neural Net

• Simple Processing Elements (neurons)
• Layers (input, output, hidden)
• Weights

19
Types of Neural Networks

• Supervised (the networks are trained manually)
  e.g. Back Propagated Networks, Radial Base
  Function Networks
• Unsupervised Networks (the networks train
  themselves/recognize patterns in their inputs)

20
Real v Modeled Neuron
21
A model of a Network with hidden layers
22
Applications of Neural Nets

• Signature Analysis (first major use by banks)
• Voice recognition (e.g. home appliances, cars)
• Monitoring (e.g. Aircraft engines)
• Physics Analysis Tool

23
Future Plans/Proposals

• Find or Create a suitable Neural Network
• Implement it in order to replace Job Scheduler
  manual modifications
• Institute a Scheduler into the waiting queue for
  constant re-evaluation after all jobs have been
  submitted.
• Test and compare results

24
Acknowledgements

• UM-CERN-REU
• Alois Putzer
• Iosif Legrand
• Jean Krisch
• Homer Neal
• Tom Dershem
• Audience

25
References

• http//www.cd.stir.ac.uk/lss/NNIntro/InvSlides.ht
  ml
• http//www.emsl.pnl.gov2080/proj/neuron/neural/wh
  at.html
• http//www.willamette.edu/gorr/classes/cs449/brai
  n.html
• http//vv.carleton.ca/neil/neural/neuron-a.html
• Resources and Cost Sharing for the ATLAS Offline
  Computing (prepared by r. Jones et al.)