GGF Applications Working Group

1
GGF Applications Working Group

• Present co-Chairs
• Ed Seidel, Max-Planck-Institut für
  Gravitationsphysik ( NCSA)
• Tom Hinke, NASA
• Former co-Chairs
• Piyush Mehrotra
• Bob Hood
• Fran Berman
• Goals at GGF1
• Reinvigorate the group
• Define our mission
• Organize ourselves for the future
• Make ourselves a very active group, centrally
  important for the Grid!

2
What Happened to the old AWG?

• Charter provide critical info to grid
  developers
• Results 4 usage scenarios
• Problems
• Not enough real users
• Apps scenarios too abstract to provide real
  requirements
• Frustration decrease in group activity
• eGrid application group expressed interest in
  mergingpreliminary discussions held at GF-5
• Result may be a reinvigorated group, based on
• Advocating grid usage to applications groups
• Advocating needs of applications developers to
  Grid Forum
• Redefinition process is ongoing participation
  welcome

3
Basic Issue Where are the applications?

• Ratio of Grid Infrastructure developer/apps
  infinity!
• Where are the applications groups?
• Applications groups should be Driving the Grid
  Development
• This is starting, but we need to accelerate it...
• Even worse, where are the application users?
• They just want their work to get done
• They know very little about the computing
  environment, and dont want to know, but we need
  them!
• There is a HUGE gap between grid developers and
  the users!!!
• Expertise...
• Ideas on what to do with a Grid...
• Examples Globus installed all over US, but

4
Need more active Application Working Group

• We can (and must) encourage application
  developers and users to get involved, and develop
  the two-way advocacy
• Egrid began in this direction
• We have to show apps folks what they can do
• Algorithms, Grid computing scenarios use
  imagination! Hear Gab tomorrow
• Develop working examples
• That show them what they could do...
• that they can use
• We have to make it easy!!!
• Imperatives it is our responsibility to
• Build a community of applications developers and
  users...
• Communicate with all other working groups to make
  sure the right things are done
• Make Grid usable by the scientists
• Collect, provide information
• Get Portals Running, work with Centers
• Build Grid Application Development Toolkit...

5
Egrid

• Begun April, 2000
• Aimed at developing and deploying Grid
  technologies in Europe
• Application Driven from the start
• Gives focal point for many projects
• Most sites installed Globus, etc.
• Example devel. Projects
• Cactus as primary driver app
• Portals
• Schedulers, Info servers, perf. Monitor, etc.
• Beginning experiments single distributed
• Cactus run with portal tests everything!
• Want to keep it up as long term testbed
• Developed very fast, very excited
• and focused group!

6
How to move forward?

• Must get more application groups involved!!!
  (Thats why we are building the grid, isnt it?)
• Focus more efforts on
• Making the Grid Usable, Used!...
• Make it easier (Portals, Center involvement,
  etc)
• Cultivate user groups/communities, get them on
  board.
• Standard Grid environments
• Get them synchronized! Make them ubiquitous! Make
  them more robust!
• Test them with real user base!
• Plenary talks at GGF on Applications using the
  Grid
• Deploy some smaller scale testbeds asap, find
  appropriate feedback mechanisms to developers,
  iterate the implementation and design
• Make sure these things have features the users
  actually want/need...
• Just remote job submission on single machine
  would be extremely helpful
• Help us develop Grid Application Development
  Toolkits
• Make it easy to build grid apps
• Get different apps communities involved

7
Now, lets get Organized

• Charter
• Secretary
• Web Pages, Web Master(in)
• White papers
• Get representatives at all other WGs
• Propose a mini Steering Committee
• Meet to discuss activities in other WGs
• Make sure appropriate connections are made
• Formalize this!
• Signing up active members
• Advise potential application groups give advice
• First Projects
• We want to DO things, bottom up, get experience
• Other groups may work to design complete
  solution, but we cant wait

8
Proposed Charter for our AWG

• Applications provide one of the fundamental
  motivations for developing Grids. Grid
  Applications include scientific and engineering
  simulations, data analysis and management, and
  other processes in some way taking place on the
  Grid. Hence, Applications, and the groups using
  or developing them, should be one of the primary
  driving forces behind the Grid's development.
• A lack of integration between Application
  developers or users, and Grid infrastructure
  developers, would lead to the Application
  communities being unaware of or unable to exploit
  Grid potentials, and to the Grid communities
  developing technologies which may not fulfill
  Application requirements.

9
Charter, cont.

• The Applications Working Group (AWG) aims to
  serve as a central hub of the GGF, closely
  interacting with virtually all other working
  groups.
• The AWG seeks to advocate the needs and
  requirements of Application Developers and Grid
  Users through representation and interaction with
  the other WGs of the GGF, and to provide back to
  the Application community information and
  guidelines for current effective Grid
  utilization. The AWG also seeks to advocate uses
  of the Grid to the larger Application community
  to bring in a wider cross-section of ideas and
  experiences.

10
Charter, cont

• To further these causes, the AWG will work to
• Develop a community of representative application
  developers and users who will be active in
  testing and providing feedback on Grid services
• Act as a forum to collect information on various
  Grid application testbed activities
• Relate ongoing Grid infrastructure efforts in
  other working groups with efforts in the
  Applications communities
• Encourage active development of Grid technologies
  needed for successful Grid Applications
• Bring these developing Grid technologies to
  growing Application and User communities
• Act as a resource for potential Grid users and
  communities, to aid them in understanding if and
  how they can effectively use the Grid services.

11
Charter, cont...

• The AWG will have representatives assigned to
  connect to other GGF working groups, will
  maintain an active set of web pages to act as a
  resource for the communities it serves, maintain
  mailing lists, promote Grid computing through
  workshops and tutorials (online and at meetings),
  encourage grant applications, develop and
  maintain documents describing Grid applications,
  best practices, developing application
  technologies, etc.

12
Our Organizational Structure

• Co-Chairs Ed and Tom
• Secretary
• Web Master(in) must collect, post data on
  everything below!
• Apps Liason Committee
• Representatives of major Grid Apps projects
• GriPhyn, DataGrid, EuroGrid, EU Projects, Cactus,
  Grads, ASC, EU Astrophysics Network, etc
• Industrial Partners
• Testbed Information Coordinator (Committee?)
• Can we set up a set of GGF testbeds?
• Work with sites to develop policy for accounts
• How do we best connect to the Centers??? Need to
  do this.

13
AWG Centrally Located, Overlapping
Collaborative Environments
User Services
Account Management
Applications (Bottom up!!)
Security
Advanced Programming Models
Grid Performance
Grid Computing Environments
Grid Information Service
Scheduling
Grid Data Access
14
Who will take responsibility for each group?

• Advanced Programming Environments TBN
• Grid Computing Environments TBN
• Scheduling TBN
• Grid Collaborative Environments TBN
• User Services TBN
• Security TBN
• Grid Performance TBN
• Grid Information Services TBN
• Grid Data Access TBN
• Account Management TBN
• Others?? TBN
• Action Items
• Each person examine all docs, look at them from
  apps perspective, meet and discuss next time
• Start our own doc on what is happening in GGF
  from apps perspective
• Report back to other groups next time! Give a
  talk there

15
Getting apps group more involved

• Recruit them (active and potential groups)
• Advise them
• Get apps plenary talks
• Assign people to get them
• Numerical Relativity
• Astrophysics (e.g., Mike Norman)
• NSF ASC Project
• EU Network
• DLR/Airbus/Boeing
• NASA, TASC
• Grads Project
• Dutch ASCI
• Action items get people lined up for GGF2
• First stage Find leading groups, get them
  involved here
• Second stage bring masses in when ready

16
Industrial Partners

• Get software, hardware vendors involved
• Compaq
• Sun
• IBM
• SGI
• Etc
• Action item Assign people who will follow up,
  report at GGF2

17
Tools Develop, Provide Examples for Community

• Portals
• Application Toolkits
• Networking
• Performance tools
• Etc
• Assign people to represent each area?
• Examples
• Application development toolkitGab to discuss
  tomorrow
• Cactus group, others doing this. We could
  provide appropriate links to testbeds so people
  with ideas have a place to try them out
• Testing for VMRs Cactus worm could be used with
  diagostics

18
Proposals

• Can help coordinate different project proposals
• Keep links to relevant, existing, projects
• Provide possibilities to leverage various
  projects for better effect
• Have a Rep for major projects in our AWG
• Action Item Who???

19
Schedule

• Today
• Give overview of new group, perspective on
  previous group
• Go into detailed discussion of charter, recruit
  active people, assign tasks, define working
  papers, discuss testbeds, etc..
• Tomorrow Get started on our mission
• Testbed discussion
• Jason Leigh on Collaborative Environments
• Gabrielle Allen on Grid Applications, Scenarios
  and Toolkits
• Bernard Schutz on Gravitational Wave Data
  Analysis on the Grid
• Follow up Discussion on the meeting,
  responsibilities for the future

20
Keeping track of the people

• Mail lists
• Coming back

21
Testbeds

• What is a Testbed?
• Set of sites that agree to install things, make
  them work
• VMR (production environment)
• Research Environments
• Why are they so important?
• Simple Everything has to work for Grid to be
  useful and used!
• Community Building brings all groups together
• (Somewhat) Controlled Environment for testing and
  Dev.
• Two levels
• More production oriented
• More research oriented
• What is our Role?
• Encourage Development and Use of Testbeds
• Force communities to get to know each other
• Testing is HARD
• Develop testbed accounts, procedures for getting
  them

22
Two testbed examples

• Cactus Virtual Machine Room/Portal
• Portal allows access for users to all machines,
  queues, etc, without knowing local passwords,
  batch systems, file systems, OS, etc...
• Version of Alliance VMR with European sites
  (NCSA, ANL, UNM, AEI, ZIB)
• Same setup deployed on Egrid and Alliance VMR
• Cactus Worm example of Grid computing
• Simulation starts, queries MDS, finds available
  resources
• Migrates itself to next site
• Uses some logic to choose next resource
• Starts up remote simulation
• Transfers memory contents to remote simulation,
  registers new location to Cactus GRIS, terminates
  previous simulation
• User monitors/controls using thorn httpd,
  streaming data
• Continues around Europe, and so on