Building Scalable Web Services Using Apache JServ

1
Building Scalable Web Services Using Apache JServ

• Sunny Gleason
• COM S 717
• Tuesday, December 4, 2001

2
In This Lecture

• What is JServ?
• The Alternatives
• Java Servlet API
• Apache JServ / Tomcat
• Scalability
• Load Balancing
• Fault-Tolerance
• JServ Security

3
Introduction

• Running a web service has changed a lot since the
  early 1990s
• Originally static HTML, text, and images
• Still a great deal of HTML content
• Shift from static pages to dynamically generated
  content
• Database-driven content, WAP, XML, XSLT

4
What is JServ?

• JServ Server is a Java Servlet Engine(compliant
  with the Java Servlet API v2.0)
• Free software produced by the Apache Software
  Foundation
• Mod_jserv is a module for connecting JServ to
  Apache HTTP Server
• JServ engine has been replaced by Tomcat
• Mod_jserv has been replaced by mod_jk

5
HTTP Basics

• HyperText Transfer Protocol
• Built on top of TCP
• 2 Well-Known Methods
• GET
• POST
• Other Methods
• HEAD, PUT, DELETE, ...
• Stateless

6
HTTP GET

• Format
• GET url HTTP/1.1 crlf headers crlf crlf
• The url string contains the resource identifier
  i.e. /top.htm
• The headers contain optional information provided
  by the client to the server
• Query Data may follow a question mark in the URL
• i.e. /search.pl?querylinux

7
HTTP PUT

• Format
• PUT url HTTP/1.1 crlf headers crlf crlf
  form_data
• Form data not passed through URL
• Allows submission of data values which are larger
  than maximum URL length
• URL 2k on MS IE4.0 and above

8
HTTP Server Response

• HTTP 200 OK crlf headers crlf crlf content
• Headers include MIME-type, content length,
  content encoding
• Other responses 301 Redirect, 401 Authorization
  Required, 403 Access Forbidden, 500 Internal
  Server Error

9
Cookies

• Persistent Client-Side Information
• ltServer, Key, Value, Expiration Dategttuples
• Server sets cookie using Set-Cookie header
• All future requests to server (before expiration
  date) accompanied by cookie in header

10
Serving Dynamic Content

• We discuss 3 early models for dynamic content
• CGI
• Mod_perl
• Mod_php

11
The Alternatives CGI

• Common Gateway Interface
• Advantages
• Flexibility - run any program
• bash, perl, python, php
• Low process overhead when idle
• Disadvantages
• Reload interpreter upon every request
• Re-establish (costly) database connections
• Security concerns - passing parameters

12
The Alternatives Mod_Perl

• Apache module for Perl
• Memory-resident interpreter
• Precompiled scripts / Script cache
• Speed / Memory Tradeoff
• HTTP Processes maintain individual perl
  interpreters
• Allows persistent database connections, other
  persistent server state
• Consistency between HTTP processes was not always
  assured

13
The Alternatives Mod_Php

• Apache module for PHP(PHP Hypertext
  Preprocessor)
• Template-based language
• Code tags are embedded within HTML template
  files
• Similar to MS ASP
• Suitable where HTML to script code ratio is high
• Huge library of add-on modules
• Similar tradeoffs as mod_perl

14
The Alternatives Summary

• Should application logic be running on the web
  server?
• scalability
• fault-tolerance
• security
• Clearly, need something better for
  enterprise-scale applications

15
Apache JServ

• Separate Application Server from Web Server
• Clean up the architecture
• Improve Scalability
• Provide fault-tolerance
• Embrace Java Philosophy
• Write once, run anywhere
• Provide additional Servlet functionality
• Like user sessions

16
JServ Openness

• JServ is 100 Java Code
• Platform-Independent
• Runs on any compliant JVM (IBM, Sun, ...)
• JServ is built on top of TCP
• Part of the Apache Software Foundation
• Integrates nicely with Apache HTTP Server
• Ports available for Windows, BSD, Linux ...

17
JServ Security

• JServ/Apache can run on different hosts(also
  different users)
• JServ itself is comprised of many Zones
• A zone is a JVM which executes some number of
  Java Servlets
• JServ may be placed behind a firewall
• JServ offers ACL security by IP address
• Optional shared-key authentication
• Apache HTTP Server may integrate SSL for secure
  HTTP client-server interaction

18
JServ Load Balancing

• Level 0 1 - 1 Apache/JServ
• No load balancing, no redundancy
• Level 1 1 - n Apache/JServ
• Each JServ hosts different zones(load
  partitioning)
• Level 2 1 - mn Apache/JServ
• Each zone may be balanced among several JServs
• Level 3 p - mn Apache/JServ
• Multiple Apache Servers, multiple JServs

19
JServ Levels 0-1

• Level 0 allows smaller hosts to run entire
  application on a single machine
• Level 1 allows different hosts to serve
  different applications
• Typically difficult to plan/partition
  applications in this manner

20
JServ Level 2

• 1 - mn Apache/JServ
• Allows Apache to balance requests among several
  JServ servers hosting the same zones
• Apache configuration file specifies ratio of hits
  for each JServ
• Each HTTP process chooses server for each JServ
  zone, sends new requests to this target

21
JServ Level 3

• p - mn Apache/JServ
• Allows HTTP traffic to be load-balanced among
  several Apache servers
• Allows Servlet workload to be distributed among
  several JServ servers
• In order for the system to work, each Apache HTTP
  server must have identical JServ configuration
• (To preserve sessions, as well see later)

22
JServ Session Handling

• Once established, a session is bound to a
  particular JServ
• But, HTTP client accesses might be sprayed
  among many HTTP servers
• Allows HTTP Server fault-tolerance
• Identical mod_jserv configuration allows
  different Apache servers to route requests to
  the right JServ
• Mechanism requires client to maintain a cookie
  which contains JServ server ID

23
JServ Session Handling

• How does it work?
• Every time a request arrives for a balanced
  ServletMountPoint, mod_jserv chooses a JServ to
  handle the request
• mod_jserv adds a cookie trailer to the
  environment variables of the JServ request(i.e.
  JS3)
• JServ appends the cookie trailer to the end of
  the session cookie
• Upon subsequent requests, Apache examines cookie,
  and sends the request to the correct JServ

24
JServ Fault-Tolerance

• (Assume Level 3)
• No Single Point of Failure
• Apache can become overloaded and fail, but JServ
  servers continue to provide services (although
  SSL sessions lost)
• JServ redundancy allows applications to continue
  running even if multiple hosts fail (although
  application sessions will be lost)
• Since any Apache can route to any JServ, as long
  as one of each stay up, the system can work

25
JServ Fault-Tolerance

• How is the JServ fault tolerance implemented?
• Each Apache contains a memory-mapped file where
  it keeps JServ information
• Each Apache process has access to the file
• If a process does not receive a response from a
  JServ process, it marks it as DOWN in the file
• (Load is re-distributed fairly among the
  survivor JServs)
• A watchdog process pings the JServs
  intermittently, updates the JServ status in
  memory if the server is back online

26
JServ Fault-Tolerance

• Apache Fault-Tolerance Step 1
• 1. Web server requests www.jserv.com80
• 2. HTTP Load-balancing system routes request to
  111.222.244.103000
• 3. Apache server chooses a random JServ machine,
  say 192.168.0.518885
• 4. JServ machine responds to request with content
  of page, along with cookie with name
  JServSessionID and value xxxx-JS1

27
JServ Fault-Tolerance

• Apache Fault-Tolerance Step 2
• 1. Client requests another page from
  www.jserv.com80
• 2. HTTP Load-balancing system routes request to
  111.222.244.203000
• 3. Apache server recognizes session cookie, finds
  JS1 at end of the cookie
• 4. Apache looks up JS1 in JServ configuration,
  routes request to 192.168.0.518885

28
JServ Load-Balancing

• Step 1 JServ Load-Balancing
• 1. Client A requests a servlet (A1)
• 2. HTTP chooses target JServ (A1)
• 3. Client A cookie is set for JS1
• 4. Client B requests a servlet (B2)
• 5. HTTP chooses target JServ (B2)
• 6. Client B cookie is set for JS2

29
JServ Load-balancing

• Step 2 Session Handling
• 1. Client B requests a servlet (sends
  previously-set cookie)
• 2. HTTP server recognizes cookie
• 3. Request is routed to JServ2 (B2)

30
JServ Fault-Tolerance

• assume Jserv1 goes down
• Step 3 JServ Fault-Tolerance
• 1. Client A requests a servlet
• 2. HTTP Server recognizes the JS1 cookie
• 3. Request is passed to JServ1, resulting in
  timeout
• 4. HTTP marks JServ1 dead in shared memory
• 5. HTTP looks up another server for the servlet
  mount point, sends request to JServ3
• 6. If a new session is needed, a new one is
  created and the new cookie is set to JS3 (JS1
  erased)

31
JServ Fault-Tolerance

• Implementation Issues
• Denial of Service
• Failed requests must be re-distributed evenly!
• Otherwise, a single server will bear the brunt of
  the load, and probably crash
• Network Partitioning and Application-level Data
  Synchronization Issues
• Must still be anticipated by the app. designer
• Watchdog process
• For single-threaded watchdog, if timeout is t,
  time between crash and restoration could be ft,
  where f is the number of failed processes

32
JServ Manageability

• Shared JServ State allows HTTP process
  coordination
• Admins can mark JServs as shutdown in shared
  memory
• JServ processes can be brought down for
  maintenance
• Apache HTTP processes redirect requests among
  live servers
• Detailed availability information can be produced
  by logging contents of shared memory file

33
Tomcat New Features

• Enhanced security model
• Property files which specify access rights (open
  socket, write file, etc.)
• Allows different protection levels within the
  same JVM (i.e. Java 2 protection model)

34
Conclusion

• JServ provides
• Limited support for
• Load balancing
• Fault-tolerance
• External Security
• Good support for
• Internal Security
• N-tier application abstraction provides
  flexibility when needed, loopback option
  otherwise

35
The End

• Any questions/comments?
• Apache Web Server
• JServ / Tomcat Servlet Containers
• Scalability / Load-balancing
• Fault-tolerance
• Security

36
For Further Info

• Apache Jakarta Project
• http//jakarta.apache.org/
• http//jakarta.apache.org/tomcat/