Performance Evaluation of JXTA Communication Layers

1
Performance Evaluation of JXTA- Communication
Layers

• Mathieu Jan
• PARIS Research Group

Paris, November 2004
2
Outline

• Overview of JXTA communications layers
• Experimental setup and benchmarks used
• Evaluation of JXTA-J2SE
• Fast-Ethernet
• Myrinet
• Evaluation of JXTA-C
• Fast-Ethernet
• Myrinet

3
Motivations performance of JXTA?

• Papers from Emir Halepovic
• Pipe service only
• No in-depth explanations about the results
• No precise details about the experimental setup
• Mainly JXTA 1.x
• Complex topologies (relays, etc)
• Not clear what is benchmarked
• Results from the P3 project
• XtremWeb over JXTA

4
(No Transcript)
5
(No Transcript)
6
Project JuxTest

• A set of benchmarks for JXTA-J2SE 2.x
• Focuses of JXTA-J2SE Communications layers
• Uses JXTA Distributed Framework project (JDF)
• Co-written with David A. Noblet
• Undergraduate at the University of New Hampshire
• Source code soon available! (22/11)
• http//jdf.jxta.org/
• Results already on-line

7
JXTA-J2SE communications layers
8
JXTA-J2SE communication layers

• JXTA Socket
• Java Socket interface
• Data-stream interface
• Reliability
• Pipe service
• Pipe ID
• Unreliable
• Endpoint service
• Peer ID
• Unreliable

9
Experimental setup

• Nodes
• 2.4GHz Intel P4, 1GB of RAM, linux 2.4.22
• Networks
• Fast-Ethernet (MTU of 1500)
• Myrinet (GM 2.0.11, MTU of 9000)
• JVMs
• Sun 1.4.2_01-b06 (-server -Xms256M -Xmx256M)
• IBM 1.4.1 (-Xms256M -Xmx256M)
• JXTA-J2SE peers via JDF
• 1 rdv peer and 2 edge peers
• Benchmarks used
• RTT and protocol efficiency

10
Performance Evaluation of JXTA-J2SE on
Fast-Ethernet
11
Raw bandwidth results of JXTA Socket
Java socket 11.22 MB/s JXTA 2.3 9.72
MB/s JXTA 2.2.1 9.48 MB/s
12
Raw latency results of JXTA Socket
Java socket lt 0.1 ms JXTA 2.3 3 ms JXTA
2.2.1 4 ms
13
Protocol efficiency of JXTA Socket

• 4 message elements in a JXTA message
• ACK_NUMBER
• Message payload reliability
• EndpointRouterMsg (XML doc)
• Used by the endpoint router service
• 565 bytes! Not even needed!
• EndpointSourceAddress
• TCP address
• EndpointDestinationAddress
• TCP address EndpointService (Net group)
  Endpoint Router
• 1 byte 909 bytes sent on the wires!

14
Tunning the output buffer size of JXTA Socket
output buffer size size of a pipe message
Java socket 11.22 MB/s JXTA 2.2.1 (512KB)
11.12 MB/s
15
Raw bandwidth results of unicast pipe
Java socket 11.22 MB/s JXTA 2.3 9.59
MB/s JXTA 2.2.1 10.74 MB/s
16
Raw latency result of unicast pipe
Java socket lt 0.1 ms JXTA 2.3 2 ms JXTA
2.2.1 35 ms
17
Protocol efficiency of unicast pipe

• 4 message elements in a JXTA message
• PAYLOAD
• EndpointRouterMsg (XML doc)
• EndpointSourceAddress
• EndpointDestinationAddress
• 1 byte 877 bytes sent on the wires!

18
JXTA Socket vs unicast pipe (bandwidth)
JXTA Socket 9.72 MB/s JXTA unicast pipe 9.59
MB/s
19
JXTA Socket vs unicast pipe (latency)
JXTA Socket 3 ms JXTA unicast pipe 2 ms
20
Removing limit on message size (unicast pipe)
Java socket 11.22 MB/s JXTA 2.2.1 11.14 MB/s
21
IBM JVM vs Sun JVM on unicast pipe
JXTA 2.3 IBM 1.2 ms JXTA 2.3 SUN 1.8 ms
22
Raw bandwidth results of endpoint service
Java socket 11.22 MB/s JXTA 2.3 10.47
MB/s JXTA 2.2.1 11.01 MB/s
23
Raw latency results of endpoint service
Java socket lt 100 us JXTA 2.3 IBM 480
us JXTA 2.3 SUN 730 us
Issue 1228 2 TCP messages for 1 JXTA message!
24
Protocol efficiency of endpoint service

• 3 message elements in a JXTA message
• PAYLOAD
• EndpointSourceAddress
• EndpointDestinationAddress
• 1 byte only 300 bytes sent on the wires!
• If we can say only
• No EndpointRouterMsg message element
• Removed 565 bytes!
• Removed XML parsing

25
Cost of each layer (bandwidth)
26
Cost of each layer (latency)
27
Performance Evaluation of JXTA-J2SE on Myrinet
Or how to desperately reach the Gb/s!
28
Experimental setup

•  Ethernet emulation  feature of GM
• No OS-bypass mode -(
• Same benchmarks
• A different IP address when configuring JXTA
• Focus on bandwidth
• Removed limit on message size

29
JXTA Socket bandwidth results
Results are not very good!
Java socket 159.20 MB/s JXTA 2.3 14.70
MB/s JXTA 2.2.1 12.10 MB/s
30
Tuning the output buffer size of JXTA Socket
Above 80 MB/s
31
Unicast pipe bandwidth results
Above 1 Gb/s! -)
Java socket 159.20 MB/s JXTA 2.3 106.54
MB/s JXTA 2.2.1 136.78 MB/s
32
Endpoint service bandwidth results
Java socket 159.20 MB/s JXTA 2.3 112.32
MB/s JXTA 2.2.1 144.97 MB/s
33
Performance Evaluation of JXTA-C on Fast-Ethernet
34
Short overview of JXTA-C

• Needs Apache Portable Runtime (APR)
• Threads, Network, etc
• Newly added support for TCP communications
• PeerCom, Seoul (South Korea)
• Does not support rdv peers
• Need to use a J2SE rdv peer
• No JXTA Socket layer
• Only pipe service and endpoint service
• Several bug fixes
• Mostly EndpointRouterMsg
• TCP_NODELAY on
• Several TCP messages

35
Raw bandwidth results of unicast pipe
JXTA-J2SE 2.3 11.1 MB/s JXTA-J2SE 2.2.1 11.1
MB/s JXTA-C 11.1 MB/s
Similar results compared to JXTA-J2SE
36
Raw latency results of unicast pipe
JXTA-J2SE 2.3 1.8 ms JXTA-J2SE 2.2.1 35
ms JXTA-C 2 ms
37
Protocol efficiency of unicast pipe

• 4 message elements in a JXTA message
• PAYLOAD
• EndpointRouterMsg (XML doc)
• EndpointSourceAddress
• EndpointDestinationAddress
• 1 byte 834 bytes sent on the wires!
• JXTA-J2SE 877 bytes
• GatewayForward tag in EndpointRouter
• Waiting for Bernard Traversat answear
• - charset tag for each message element

38
Raw bandwidth results of endpoint service
JXTA-J2SE 2.3 11.15 MB/s JXTA-J2SE 2.2.1
11.19 MB/s JXTA-C 11.16 MB/s
39
Raw latency results of endpoint service
JXTA-J2SE 2.3 0.8 ms JXTA-J2SE 2.2.1 1
ms JXTA-C 0.8 ms
40
Protocol efficiency of endpoint service

• 3 message elements in a JXTA message
• PAYLOAD
• EndpointSourceAddress
• EndpointDestinationAddress
• 1 byte only 239 bytes sent on the wires!
• If we can say only
• JXTA-J2SE 300 bytes
• No EndpointRouterMsg
• Removing 595 bytes!
• Removed XML parsing

41
Unicast pipe vs endpoint service
42
Performance Evaluation of JXTA-C on Myrinet
Still using  Ethernet emulation  feature of GM
43
Unicast pipe bandwidth results
JXTA-J2SE 2.3 106 MB/s JXTA-J2SE 2.2.1 136
MB/s JXTA-C 107 MB/s
44
Endpoint service bandwidth results
JXTA-J2SE 2.3 112.32 MB/s JXTA-J2SE 2.2.1
144.97 MB/s JXTA-C 109 MB/s
45
JXTA-C on top of PadicoTM -)

• JXTA-C is working on top of PadicoTM! (17/11)
•  sed  modifications in apr
• Using marcel threads instead of pthread
• JXTA-C configured to use GDBM instead of SDBM
• Goal stop using  ethernet emulation  of GM
• But JXTA-C is not yet zero-copy -(
• Performance evaluation in progress

46
Conclusion

• Performance evaluation of the communication
  layers of JXTA- (J2SE and C)
• Fast-Ethernet
• Myrinet
• Much better results compared to other related
  work and explained results
• Known costs of JXTA
• JXTA-J2SE nearly saturate a Fast-Ethernet link
• JXTA-J2SE on Myrinet bandwidth greater than 1Gb/s
• JXTA-J2SE gt JXTA-C on Myrinet
• JXTA-C is not zero-copy

47
Ongoing work

• Performance evaluation of JXTA-C on top of
  PadicoTM
• JuxMem client in C for DIET
• Better knowledge of JXTA-C tricks -)
• Write papers
• GP2PC (1/12) on JXTA raw performance
• Fast-ethernet
• ? on JXTA tuned performance for grid computing
• Myrinet

48
Future work

• Benchmark secure and propagate pipes
• Update JuxTest results
• JXTA 2.3.1
• Unidirectional throughput test
• Plug-in for Ethereal
• An easier analysis of JXTA protocols