A Review of the DSGs Research and Development

1
A Review of the DSGs Research and Development

• Mark Baker
• The Distributed Systems Group
• University of Portsmouth,
• http//dsg.port.ac.uk/mab/

2
Outline

• Introduction.
• DSG Activities
• OGSA Testbed,
• MPJ Express,
• GridRM,
• jGMA,
• Slogger,
• OGSA-DAI and SDSS,
• Sakai VRE Demonstrator,
• VOTech.
• Group Performance.
• SWOT.
• Summary.

3
Introduction

• Aims of the talk are
• In this talk I will review the research,
  development and other activities that the DSG has
  been involved in over the past year.
• I will highlight both positive and negative
  aspects of our research and accomplishments.
• I will include a comparison of what we intended
  to do over the year, with what we actually did.
• I will conclude with an attempt to predict our
  activities over the coming year.

4
What does the DSG do?

• We are involved in the research and development
  of a variety of applications, libraries and tools
  for Grid and Cluster systems using Open Source
  software and COTS technologies.
• In particular using
• Java-based technologies,
• Messaging systems (local/wide-area),
• Parallel and distributed,
• SOA and P2P-based,
• Semantic and WS technologies.

5
The Distributed Systems Group

• The Last Years Activities

6
OGSA Testbed

• One year project Sep 2003 Oct 2004.
• With Daresbury, Manchester, Reading and
  Westminster.
• Extended for six months until April 2005 no
  extra funding.
• Aimed to install GTx at all sites and deploy apps
  across testbed.
• Hong (funded) and Mat worked on the effort.
• Regular Testbed meetings at Westminster.
• Fun with Starbug firewalls

7
OGSA Testbed

• From October 2004
• Liquid Crystal portal
• Move to GridSphere,
• Look at the hiding application server port ,
• Looked at a bunch of other related technologies.
• A report was created.
• Westminster March Workshop
• Successful.
• 50 attendees.
• Workshop report.
• Final Testbed report was
• submitted in June.
• No further feedback.

portals.dsg.port.ac.uk
8
MPJ Express

• Background
• Produce a reference Java messaging system MPJ.
• Aamir started working on the effort in Nov 2003.
• Working with Bryan Carpenter now at OMII in
  Southampton.
• MPJ layered architecture
• mpiJava API MPJ API later.
• Runtime infrastructure (not secure)
• Current status
• NIO device complete,
• Beta version released in September 2005,
• SCI (Guillermo) and Myrinet device being
  developed.
• Future
• Want to be able to program clusters of SMPs so
  looking at potential syntax/semantics for shared
  memory programming,
• Also interested in checking out the feature of
  emerging high productivity languages (DARPA
  funded) Chapel, Fortress and X.10.

9
Resource Monitoring

• GridRM is a unifying resource monitoring
  framework, capable of be used a number of diverse
  purposes including scheduling, performance,
  faults, and policing QoS or SLA.
• Progress
• GridRM framework is dormant, gridrm.org still up
  and running.
• There are plans to add it into the VOTech
  project.
• It has been written into European project that
  was submitted to the latest FP6 call SORMA.
• Sub-projects
• Last year UG working on Portal made some
  progress.
• Afrasyab Bashir schedulers broker!
• Note
• Most the other monitoring systems are moving
  towards the same approach that GridRM uses.

10
UISB

• The UISB project was developing technologies to
  interact and search Information Services data,
  e.g. UDDI, LDAP, Jini LUS.
• Status
• This research area was abandoned for a number of
  technical reasons most of which were related to
  using Eclipse as the development and deployment
  platform.
• RDF-based technologies have been moved into the
  Semantic Logging project see later slides.

11
jGMA

• Originally wanted a Java-based, lightweight!,
  easy to use, extensible version of GMA to bind
  together GridRM gateways.
• jGMA is an event-based messaging system with an
  integrated P2P-based registry
• Originally called JavaGMA, now jGMA, will
  probably be renamed again soon!
• Status
• Software redesigned and released again in
  September 2005
• Non-blocking API with a mediator with
  producer/consumer,
• Scalable virtual registry (in-memory, and
  persistent) and query API, virtualised using P2P
  technologies.
• Demos Web cam and SC one too
• Performance testing vs NaradaBrokering at the
  moment.

12
jGMA - Demos
13
jGMA

• Future Work
• Use jGMA in real distributed applications
• Distributed games,
• MyGrid steerable application.
• Other uses
• Infrastructure to provide Virtual Portal
  infrastructure,
• VOTech project bind client, broker and back-end
  submission systems together.

14
Semantic Logging - Slogger

• Slogger is a RDF-based system for unifying and
  annotating log data for a more complete
  analysis of distributed systems
• It purpose is to help identify performance
  bottlenecks, and application problems.
• Son/daughter of UISB uses technologies and
  experiences gained.
• Overall aim is input a variety of logs into
  Slogger, store them together in a RDF data store,
  and then explore (search through) the unified
  data and display via a flexible graphical
  interface.
• Software from UISB been refactored into servlets
  and portlets for use via a Web Portal.
• At last weve dropped Eclipse!
• We will explore a number of research issues
  related to the Semantic Web RDF and ontologies.
• Joining and searching across/through RDF models.

15
OGSA-DAI

• Initiated within the OGSA Testbed project.
• Working with Southampton on GITS for OGSA-based
  systems
• Perl scripts to examine and show the status of
  Grid services.
• Interested in OGSA-DAI as a pervasive technology
  for integrating relational databases.
• Eventually built a prototype system that pushes
  GITS data into a database that can then be
  searched and manipulated using an OGSA-DAI Grid
  Service and viewed via a Web browser
• Helen was working on this project.

16
OGSA-DAI
17
SDSS
18
SDSS Mirror

• Collaborating with Bob Nichol and other US SDSS
  people to set up and run a SDSS mirror at
  Portsmouth.
• Aims
• Host SDSS mirror (MS based at moment)
• DR3 (max 5 Tbytes with Raid 10).
• Migrate to Oracle and Non-MS technologies.
• Move to DR4 (9 Tbytes) and maybe DR5 (15
  Tbytes) next year.
• Research Issues
• Possibly distributed data sources using OGSA-DAI,
• Optimised queries,
• Possible develop tools to help manage database.

19
Sakai VRE Demonstrator

• Consortium of Daresbury, Lancaster, Oxford and
  Portsmouth.
• Funded by JISC one of a host of projects
  looking at Portals and related technologies for
  support of teaching and research in the UK.
• Project started on July 1st 2005 and should run
  for two years.
• Overall aim is to develop tools and services for
  the Sakai portal
• Use JSR-168 and WSRP standards,
• Portsmouth is developing Semantic and P2P tools.

20
Sakai VRE Demonstrator
21
Sakai VRE Demonstrator

• Status
• NaradaBrokering-based collaborative portlets
  being developed should be complete in next
  couple of week.
• Tool that uses Semantic technologies (using Jena
  and Joeski) to update user preferences
  prototype completed.
• Near Future
• Virtual Portal infrastructure using jGMA as
  messaging system
• Share information between portals instances
• Preferences, files, and other tools and services.
• Semantic tools
• Finish user preferences tools,
• Look at how to share and integrate exiting portal
  information in a RDF-based model.
• Maybe use RDF to store other unstructured data.

22
Sakai VRE Demonstrator
23
The European Virtual Observatory (Euro-VO)
initiative

• The VOTech project is a 6.6M Design Study under
  EU FP6 which aims at completing all technical
  preparatory work necessary for the construction
  of the European Virtual Observatory (Euro-VO).
• The is that all the world's data should feel like
  it sits on the astronomer's desk top, analysable
  with a user selected workbench of tools, made
  available through a standard interface!
• ICG/DSG project our part seems to be concerned
  with integrating, deploying and testing a range
  of technologies that allows tasks be seemlessly
  and transparently executed over the wide area.
• Garry will say more about this in his semester
  later in the year.

24
Performance Metrics

• Need to look at what are the measures of success
  for the group to be able to assess the potential
  of the DSG.
• The RAE uses
• PhD completions,
• Income,
• Papers,
• Esteem/activities.
• The question, usual is, are these good metrics?

25
Growth of DSG
26
DSG Papers (per year)
X - Conferences workshop
X - Int. Journal
5
X Books
4
Publications
2
1
98
99
00
01
02
03
05
06
04
Year
27
Funding proposals

• 2002 wrote/contributed to 7 proposals
• 2 funded!
• 2003 - wrote/contributed to 8 proposals
• 2 funded!
• 2004 - wrote/contributed to 5 proposals
• 1 funded - JISC JCSR VRE two years, with
  Daresbury, Lancaster, Oxford.
• 2005
• 1 written so far SORMA
• Other with Warwick planned.

28
DSG Income
EPSRC 45K
50
Myri
SORMA - 105K
CLRC 14K
40
DERA 36K
Money (k)
DERA 36K
20
IBM 14K
JISC JCSR 20K
JISC VRE 30K
NHSE 5K
10
TOPIC 10K
TOPIC 10K
CLRC 10K
TOPIC 10K
CLRC 4K
CLRC 4K
CLRC 4K
98
01
99
00
02
03
04
05
06
Year
29
DSG Membership

• MAB staff.
• Garry
• Submitted and defended PhD in November 2004.
• Now an RA working on AstroGrid from Feb 2005.
• Hong
• Submitted and defended PhD Feb 2004,
• Left us in April 05 to work at ORNL.
• Bryan Carpenter (OMII, Soton, regular and valued
  visitor).
• Rich - Faculty PhD bursary (0 years to go) needs
  to complete and write up over as soon as he can.
• Mat - DTA PhD bursary (1 year to go).
• Aamir - DSG PhD bursary (1 year to go).
• Helen CSSE bursary (1 year to go).
• Raz working on Sakai VRE (registered Oct 05).
• Kashif part time (registered Oct 05).

30
Esteem (reflects on the group)

• TCSC (co-chair) and TCPP (exeCom).
• Visiting chair Westminster/Reading.
• Steering Committee of ICCC/CCGrid/Grid.
• DSO section editor work done by Garry and Rich.
• Invited talks Cork/Israel/China/Daresbury.
• Grid Book published April 2005 sold 700
  copies,
• Reviewing (books, journals, conferences).
• Membership of conference PCs.
• Organisation of events (ICCC/CCGrid/Grid).
• WiP session, and NeSC/Westminster workshops.
• Evaluator for FP6 Nov 2004.
• Web site has high ranking and lots of hits
• Month Unique visitors Hits Bandwidth
• Sep 2005 4784 64274 3.15 GB

31
DSG Presence
32
DSG Activities
33
DSG Metrics

• PhD completions
• Hong February 2004.
• Garry November 2004.
• Rich to finish in early 2006.
• Papers need to produce at least one conference
  paper per year and one journal paper per PhD
  completion.
• Income OK, hopeful of success over coming year.
• Esteem group is well know in the US and has
  established a reputation in the UK and EU.

34
SWOT Analysis

• Strengths
• National recognition (Testbed, AHM, SC demos),
• Good relationships with Daresbury, Lancaster,
  Oxford, Westminster, and Reading initiated by
  the OGSA Testbed.
• Grown group summer studentships, PhDs, and
  visitors.
• Semi-autonomous (still a good and bad aspect).
• Group has extensive and complimentary skills,
• Interaction with the ICG is good, and growing
  these links especially now Bobs around.
• Knowledge in depth of Java technologies, gaining
  greater experience with WS, and Semantic Web
  technologies,
• Profile via TCSC, DSO, and DSG Web presence,
• Undertaking synergetic projects which leads to
  good group focus.

35
SWOT Analysis

• Weaknesses
• UoP CS research has a very low-profile, making it
  hard to get funds.
• Still no other permanent UoP staff.
• Still trying to get other UoP staff involved in
  DSG research.
• Administration and administration is still rather
  an effort, and not getting easier with emerging
  practices.
• VERY
• Ham-strung having to work through ISO this may
  be resolved soon!
• Still little industrial and/or commercial
  connections.
• Lost Hong very hard to replace!
• Semi autonomous justify our existence, not a
  very large umbrella.

36
SWOT Analysis

• Opportunities
• Seen as successful/useful by ICG and others,
• Releasing software, need good demos to show off
  our middleware.
• Interaction with Daresbury, Reading and
  Westminster,
• Semi-autonomous,
• Grid book has opened some doors,
• The OGSA Testbed helped our profile,
• Getting more involved in the JISC JCSR programme
  funding for development and deployment,
• With ICG we have the best research culture in the
  UoP!
• IEEE CS TCSC/DSO activities

37
SWOT Analysis

• Threats
• FUNDING
• Involvement of RAs and more senior staff.
• Semi-independence via the ICG!!
• Losing limited income to faculty,
• Getting bogged down in teaching and
  administration a large overhead is associated.
• Not retaining good staff! losing staff!
• PhD students undertaking to much teaching and
  support efforts.

38
SWOT Conclusions.

• Compared to last years SWOT!
• Some fears regarding threats and weakness have
  happened
• Lost Hong, no further permanent staff working
  with us.
• Teaching and university processes are sinks of
  time and effort!
• Our strengths and opportunities help
• Our reputation, skills and presence is helping us
  a lot,
• We are getting funding, and hope for more soon,
• New students has helped the group,
• Release of software is significant,
• Bryan Carpenters regular visits.
• Our perceived strengths and opportunities STILL
  out-weigh the weaknesses and threats, but we need
  to be careful!
• Our biggest threats and weakness are on the still
  on funding side and staffing.
• Strengths are focussed on applied research,
  useful skills, good interactions, and lots of
  enthusiasm!

39
Summary

• I think we have had a good year better than
  last
• We have
• New PhD students,
• Further new funding,
• Released several software offerings,
• Further positive interaction with ICG,
• Maintaining productive research culture,
• Widening our experience and collaborations.
• However
• Need to search for further funding,
• Lots of doing, probably not enough papers again.
• Congrats to Garry who was awarded his PhD in
  November 2004.

40
Five Years Hence (2002 view)

• Group size of ten plus permanent staff, post
  docs and post grads.
• Coherent and strongly focused research.
• Independent group working in a strong research
  culture.
• Broken down the external view of research at UoP.
  
• Internationally recognised in a number of related
  fields
• Wide-area resource monitoring,
• Java micro-kernel work,
• Java-based Grid technologies.
• Deployed and regularly downloaded software.
• Papers 30 conference and journal papers per
  year.
• Invites to collaborate/talk at international
  events/institutions.

41
A Competition

• First to finish and be awarded their PhD in 2006
  the prize is