Peer to Peer Search Engine

1
Peer to Peer Search Engine Programming
Project CS5204 Operating Systems Greg
Grothaus Dan Moisa
2
Outline

• Architecture
• Query Services
• Rendezvous point
• Client role peers
• System Issues
• References

3
Architecture Query Services

• In a general sense a Query Service is a peer in
  the system whose role is to respond to queries of
  a specific type.
• For this project, a Query Service
• Is a registered peer.
• Supported a method for identifying queries it
  could respond to.
• Utilized a set of web browser emulation tools to
  wrap around a remote product database for
  responding to queries in a uniform interface.

4
Architecture Query Services
5
Architecture Query Services

• Each query service consists of
• A network layer comprised of NetServer and the
  OAA communication classes
• A wrapper layer specific to the resource being
  wrapped including
• WrapperServer A class that handles the
  query/response protocols
• WrapperHelper batch commands resembling browser
  actions for manipulating a remote site.
• WrapperBase Generic tools providing higher level
  command interface for accessing common web
  browsing functions (clicking, filling out forms,
  etc.) by providing HTTP connections, content
  acquisition, HTML Parsing, and State Information
  transparently.

6
Architecture Query Services

• 9 query service wrappers were implemented
  allowing for different types of media searches
  from the following web sites

7
Architecture Rendezvous Point

• Peers in a sparse network cannot effectively
  locate another networked peer by chance.
• To solve this problem, a Rendezvous Point must be
  used to allow for peer discovery of other peers.
• In a large peer network, multiple bridged
  rendezvous points will exist to make the system
  more scalable and tolerant of rendezvous failure.
• Note that actual peer communication occurs in a
  peer to peer fashion after initial discovery has
  taken place at a rendezvous point.

8
Architecture Rendezvous Point

• In this project, Open Agent Architecture (OAA)
  was used as a rendezvous point.
• As used, OAA allows for registration of peers and
  advanced discovery of peers through predicate
  statements.
• In this respect, the rendezvous point itself
  implemented with OAA is a hybrid system that most
  resembles an event system.

9
Architecture Rendezvous Point
10
Architecture Client Role Peers

• A Peer in the Search Engine Network wishing to
  perform a query is acting the role of a Client
  Role Peer.
• The client role peers query consists of two
  major components
• The query itself.
• A query specification made from a discrete query
  space or from predicate statements.

11
System Issues

• Range of Roles
• Query Services are generally inflexibly tied to
  specific peers in the network due to the nature
  of the service or data. It is often difficult or
  impossible for another peer to take on the same
  query service role.
• Client Role Peers can be any peer in the network,
  including peers serving as query services.
• Since peers can take on multiple roles, it is
  important that there is a consistant
  communications layer regardless of the role that
  the peer is currently performing.

12
System Issues
13
System Issues

• Fault Tolerance
• In a P2P system, the peers are loosely coupled.
  As such, there should be no expectation of
  continuity of any peers service.
• No peer in a client role should expect a query
  service to respond. A 10 second timeout value is
  used in this case to handle unresponsive query
  services.

14
System Issues

• Fault Tolerance
• The rendezvous point should handle the failure of
  a peer.
• Since the rendezvous point only works on a
  request/response mechanism, it will never
  actively attempt to communicate with another peer
  that may no longer be accessible.

15
System Issues

• Fault Tolerance
• The rendezvous point itself can be a critical
  failure point.
• On a large scale, each peer would be aware of
  multiple rendezvous points to increase redundancy
  and decrease the effect of a rendezvous point
  failure. Rendezvous points themselves would be
  bridged to communicate information to peers
  connecting to distinct points.

16
References

• JXTA v2.0 Protocols Specification. Sun
  Microsystems, Inc. 2003.
• Open Agent Architecture Technical White
• Paper. SRI International.
• Waterhouse, Steve. JXTA Search Distributed
  Search for Distributed Networks. Sun
  Microsystems, Inc. May, 2001