OurGrid: An approach to easily assemble grids with equitable resource sharing

1
OurGrid An approach to easily assemble grids
with equitable resource sharing

• Nazareno Andrade
• Wilfred Cirne
• Francisco Brasiliero
• Paulo Roisenberg

2
Talk Outline

• Introduction
• Assembling a Grid
• Bag-of-tasks Applications
• OurGrid Architecture
• Network of favors
• Resource Sharing Protocol
• Evaluation and Results
• Conclusions and Future Work

3
Introduction

• Access to resources on a Grid
• Via personal requests
• Negotiate with system administrator, obtain
  permissions and priorities.
• What if it crosses institutional boundaries?
• Offer and demand problem
• Past approaches based on grid economy Need
  well-deployed technologies for electronic
  currency and banking.

4
Grid assembling problems

• Resource users perspective
• Large, heterogeneous resource sets and dynamic
  users. (Mutually untrusted and unknown).
• Difficult for ordinary user to obtain access.
• Resource providers perspective
• Access to as many resources, fairness.
• Static constraints and guarantees.
• Neither flexible nor scalable.

5
OurGrid Motivation and Goals

• Research efforts in the dynamic access gaining to
  resources does not exist.
• Demand for understanding grid usage requirements
  and patterns in real settings.
• To provide an open, extensible infrastructure.
• Suitable for running a set of grid applications.
• Users willing to share resources in order to
  obtain access to the grid.
• To gather valuable information (workloads etc.)
  about needs and habits of grid users.
• Provide better guidance to future efforts.

6
OurGrid - approach

• Assumptions
• At least two peers willing to share their
  resources to obtain access to other resources.
• ? Exchange-based model.
• Applications executed without QoS guarantees.
• ? No negotiations, agreements.
• Promote equity with minimum QoS guarantees
  cannot guarantee equity, only a best-effort
  strategy.
• Users who do not own any resources cannot be part
  of grid
• Advance reservation of resources not possible
  without QoS guarantees.

7
Bag-of-Tasks (BoT) Applications

• Parallel applications composed of a set of
  loosely coupled independent tasks.
• Tasks require no communication among them during
  execution.
• E.g. Computational biology, parameter sweep,
  simulations
• Can be successfully executed without QoS
  guarantees.
• Users work cycle
• Plan details ? Run application ? Examine
  results

8
BoT applications contd

• Owner of resource has priority over foreign user.
• Applications may pose constraints on the
  resources it needs.

9
OurGrid architecture

• Peer-to-peer network of resources.
• All resources shared, respecting providers
  policies.
• User accesses grid through services provided by a
  peer.
• Peer acts as a grid broker to its users.
• Uses lower-level protocols
• Peer discovery
• Application-level routing

10
OurGrid Architecture contd

• A peer is both a consumer and a provider of
  resources.
• Clients are software used by the users.
• At least an application scheduler
• MyGrid, AppLeS etc.
• Resources can be of any granularity, but peers
  manage access to whole sites.
• Number of peers diminishes. (Improves performance
  of searches)
• Systems topology is closer to its network
  infrastructure topology. (Alleviates traffic
  problems)

11
Network of Favors

• Model of resource sharing.
• Favor Allocating a resource to a requesting
  consumer.
• Consumer becomes indebted to the owner of the
  consumed resources.
• Need to reciprocate favors when solicited.
• As debt grows, its gets less prioritized.
• Peer p keeps track of local balance for each
  known peer p, based on past interactions.

p
p
p ? X
Resources Y
p ? X - Y
Resources Z
p ? X Y Z
12
Network of favors contd

• Each peer can maintain ranking of all known
  peers.
• Ranking is updated on each provided or consumed
  favor.
• Quantifications of each favors value (MIPS) done
  independently.
• Prioritizing serves only to solve any conflicting
  situations.
• Free-rider peers may choose not to reciprocate
  favors. They get less prioritized.
• Totally decentralized system.

13
Resource Sharing Protocol

• Used by peers to gain access to, consume and
  provide resources.
• Three participants in the protocol
• Client Manages to access the grid and runs
  application tasks on them.
• Consumer Part of a peer that receives requests
  from clients to find resources
• Provider Part of a peer that manages the
  resources shared and provides them to consumers.

14
Resource Sharing Protocol contd
15
Resource Sharing Protocol contd
16
Evaluation

• Simulation using SimJava simulation toolkit.
• Analysis based on simplified version OurGame
  (captures key features of OurGrid)
• Key Objectives
• System-wide behavior of network of favors model
• Study how system deals with conflicting requests
• Grouping resource consumption into turns.
• In a turn, each peer is either a
• Consumer tries to consume all available
  resources
• Provider tries to allocate all resources it owns
  to the current turn consumers.

17
Evaluation OurGame

• Set of N peers P p1,p2,,pN
• Each peer pk owns rk resources.
• Resources are identical but differ in number
• A peer is a six-tuple
• id, r, state, ranking, ?, allocationStrategy
• Ranking is a list of pairs (peer_id, balance)
• ? Probability of peer being a provider in a
  given turn.
• AllocationStrategy Peers resource allocation
  behavior
• AllForOneAllocationStrategy
• ProportionallyForAllAllocationStrategy

18
Evaluation - Scenarios

• N 10, 100, 1000.
• AllocationStrategy (100, 0), (0, 100), (25,
  75), (50, 50), (75, 25).
• ? 0.25, 0.50, 0.75 (for all peers)
• Another scenario, where each peer has a ? given
  by a uniform distribution in 00.99.
• r All peers own an amount of resources in a
  uniform distribution in 1050.
• All combinations totally yielded 60 simulation
  scenarios.

19
Metrics

• Suppose over t turns,
• gk ? resources gained by pk from the grid.
• dk ? resources donated by pk to the grid.
• lk ? local resources consumed by pk
• Favor Ratio FRk (after t turns) gk /dk
• Used to gauge equity. In situations of resource
  contention, FRk 1 ? equity.
• Resource Gain RGk (after t turns) (lk gk) / lk
  
• Speed-up delivered by the grid.
• Used to gauge prioritization How much this peer
  was prioritized by the other peers.
• Greater (donation consumption) ? higher RGk

20
Metrics contd

• Suppose further that over t turns
• ik ? idle resources when pk was provider
• ?k ? probability that pk was provider in a given
  turn
• Rk ? total amount of resources pk had
• Rk t . rk Rk lk dk ik
• lk (1 - ?k) . Rk
• RGk 1 ?k . FRk - ik . FRk
• (1 - ?k) (1 - ?k) . t . Rk
• Resource Conservation law
• ? Rk ? gk ? lk ? ik

21
Results (All peers have same ?)

• FRk always converged to 1.
• RGk converged to different values depending on
  the scenarios parameters.

22
Results (All peers have same ?)
23
Results (All peers have same ?)

• When ik 0, FRk 1, RGk ? ?k
• When ik gt 0,
• FRk takes more turns to converge Each peer
  takes longer to rank other peers, as there were
  turns with no resource consumption.
• RGk converges to a smaller value By RCL.
• Allocation strategy does not affect the behavior
  of metrics.
• As number of peers increases, number of turns
  needed for metrics to converge increases.

24
Results (?k uniform distribution)
25
Results (?k uniform distribution)
26
Results (?k uniform distribution)
27
Conclusions and Future Work

• OurGrid aims to allow users of BoT applications
  to easily gain access to resources, dynamically
  forming a grid.
• Based on network of favors
• Completely decentralized
• Simple design, no QoS guarantees
• Simulations show that this approach is promising
• Simulating real grid user workloads on peers
• Studying the impact of malicious peers
• Actual implementation of OurGrid !!!

28

• QUESTIONS ?