Lofar Information System on GRID

1
Lofar Information System on GRID

• A.N.Belikov

2
Lofar Long Term Archive
Requirements to data storage
Prototypes EGEE Astro-WISE
Tiers
Astro-WISE adaptation
Problem optimal data placement
3
Lofar Science
4
Lofar Network
10 Gbps optical single data processing center
for raw data (CIT, Groningen)
5
Lofar Data Processing
Blue Gene/L linux cluster
centralized data processing for raw
data long-term data storage
6
Lofar LTA requirements
7
Lofar LTA requirements
20 PB for 5 years decentralized data
storage decentralized data reprocessing internatio
nal partners
8
LHC EGEE

• 15 PB data per year
• grid computing

EGEE 250 sites, gt45000 CPU
www.eu-egee.org EGEE-III
9
EGEE Tiers approach
10
Astro-Wise grid
11
Astro-WISE dataservers (1)
12
Astro-WISE dataservers (2)
Astro-Wise
LoWise
dataserver
data retrieved by URI
0.5 TB 7 sec 3.5 TB 20 sec 1 PB 8 h
13
LoWISE URI storage
14
LoWISE Tiers

• Astro-WISE-like interfaces
• user access (from PM to anonymous)
• Tier 1 as core element

15
LoWISE data storage
distributed file systems GPFS (IBM)
Lustre (Sun)
dCAche CASTOR
16
Lofar-WISE Data Storage
initial configuration
distributed FS EGEE
Oracle for metadata
17
Problem users
users observer, project manager privileged
users operators
18
Problem CE SE

• geographical location
• file transfer
• user access
• research group driven

19
Problem Access policy
20
Problem Optimal Data Placement

• priorities groups
• pool of queries
• TCF
• monitor system

21
Task

• create and implement a system for an optimal data
  placement