EAS 309: EAServer 5.0 Performance Architecture

1
EAS 309 EAServer 5.0Performance Architecture
Evan Ireland Senior Staff Software
Engineer eireland_at_sybase.com August 15-19, 2004
2
The Enterprise. Unwired.
3
The Enterprise. Unwired.
Industry and Cross Platform Solutions
Unwire People
Unwire Information
Manage Information

• Adaptive Server Enterprise
• Adaptive Server Anywhere
• Sybase IQ
• Dynamic Archive
• Dynamic ODS
• Replication Server
• OpenSwitch
• Mirror Activator
• PowerDesigner
• Connectivity Options
• EAServer
• Industry Warehouse Studio

• Unwired Accelerator
• Unwired Orchestrator
• Unwired Toolkit
• Enterprise Portal
• Real Time Data Services

• SQL Anywhere Studio
• M-Business Anywhere
• Pylon Family (Mobile Email)
• Mobile Sales
• XcelleNet Frontline Solutions
• PocketBuilder
• PowerBuilder Family
• AvantGo

Sybase Workspace
4
EAServer 5.0 Performance Architecture

• Introduction
• SPECjAppServer Benchmarks
• Deployment Time Configuration
• Post-Deployment Configuration
• Flow Control
• Thread Monitors
• Lightweight Container (LWC)
• Entity Isolation Level
• Table Timestamp
• Object Cache
• Query Cache
• Just-in-Time Stored Procedures
• Prepared Statement Cloning
• Sybase Heap Manager
• Named Instance Pools
• Limiting Active Instances

5
Introduction

• Why do we care about benchmarks and performance?
• Our experience with EAServer has shown that in
  order to improve performance substantially, we
  must
• Rely more on code generation and less on
  generic code (kernel / runtime).
• Reduce memory utilization.
• Remove unnecessary code.
• Simplify complex algorithms.
• Utilize clever algorithms.
• This results in software that has a smaller
  footprint, has higher quality, is easier to
  maintain, and is easier to use.
• Why should you care about benchmarks?
• You care about the quality of infrastructure
  software used by your applications.
• You wish to see new deployment options for
  application servers in an increasingly mobile
  workplace.
• You just want to get on with your job, and dont
  have time to fight with complex application
  server deployment, configuration or management
  tools.
• You dont have 1,000,000 to spend on computer
  hardware ?

6
SPECjAppServer Benchmarks

• Standard Performance Evaluation
  Corporation (SPEC)
• is a non-profit corporation formed to establish,
  maintain and endorse a standardized set of
  relevant benchmarks that can be applied to the
  newest generation of high-performance computers.
• SPECjAppServer2001
• SPECjAppServer2002
• SPECjAppServer2004
• Sybase is an active participant in the SPEC OSG
  Java committee.

http//www.spec.org
7
SPECjAppServer2001

• http//www.spec.org/osg/jAppServer2001/index.html
• Similar to ECperf but with more stringent run
  rules(especially for data integrity).
• Includes coverage of
• Remote clients using RMI (IIOP) calling session
  beans.
• Stateful and stateless session beans calling
  entity beans.
• Entity beans with bean-managed persistence (BMP),
  or
• EJB 1.1 style entity beans with container-managed
  persistence.
• A little HTTP access with XML document exchange.

8
SPECjAppServer2001 Dual Node Results
http//www.spec.org/jAppServer2001/results/jAppSer
ver2001.html
9
Using Bubble Chart to Indicate CPU Speed
http//www.spec.org/jAppServer2001/results/jAppSer
ver2001.html
10
SPECjAppServer2002

• http//www.spec.org/osg/jAppServer2002/index.html
• Similar to SPECjAppServer2001 but with CMP entity
  beans changed from EJB 1.1 to EJB 2.0 style.
• Allowed vendors who didnt want to tune their EJB
  1.1 containers to participate.

11
SPECjAppServer2004

• http//www.spec.org/osg/jAppServer2004/index.html
• Much changed from earlier SPECjAppServer
  benchmarks.
• Adds coverage of
• The web container, including servlets and JSPs.
• EJB2.0 Container Managed Persistence.
• JMS and Message Driven Beans.
• Allows caching of ItemEnt (with timeout).

12
Deployment Time Configuration

• Uses sybase-easerver-config.xml.
• in META-INF folder inside Server Application .ear
  file.
• in META-INF folder inside EJB .jar file.
• In WEB-INF folder inside Web Application .war
  file.
• Can set properties of various Repository
  entitiesduring deployment.
• Servers.
• Thread Monitors.
• Connection Caches.
• Components.
• Packages.
• Servlets.
• JSPs that are referenced by a servlet element
  in web.xml.
• Web Applications.

13
sybase-easerver-config.xml

• http//www.spec.org/jAppServer2001/results/res2002
  q4/jAppServer2001-20021113-00010.zip
• Extract ZIP file and look in folder deploy/eas.
• A little inconvenient to work with as changes to
  the configuration require repackaging of the .ear
  file. ?
• Cant be used for non-J2EE components. ?

14
sybase-easerver-config.xml Example

• '-//Sybase, Inc.//DTD EAServer configuration
  1.0//EN'
• 'http//www.sybase.com/dtds/sybase-easerver-co
  nfig_1_0.dtd'
• value"com.sybase.jaguar.component"/
• /CustomerEnt"
• n" value"Customer Entity bean"/
• value"read_committed_with_cache"/
• ...

15
Post-Deployment Configuration

• EAServer Manager
• Formerly known as Jaguar Manager.
• New wizards for performance tuning. ?
• Tedious to repeat configuration changes after
  redeployment. ?
• jagtool
• Command-line configuration.
• Convenient for quick changes.
• jagant
• External XML configuration script.
• Highly recommended. ?
• Repository API
• JAGUAR/html/ir/Jaguar__Repository.html
• Used by EAServer Manager, PowerBuilder IDE,
  jagtool, jagant.

16
jagant

• A set of tasks for use with Apache Ant.
• http//ant.apache.org
• As with sybase-easerver-config.xml, can set
  properties of various Repository entities during
  deployment.
• Can also execute action tasks. ?
• Server refresh.
• Server restart.
• Any other ant task
• Changes to XML script do not require repackaging
  of deployment files. ?
• Can be used for configuration of all component
  types. ?

http//ant.apache.org/
17
jagant script Example

• default"configure"
• e.jaguar.management.jagtool.ant.SetPropsTask"/
• .component"/
• value"read_committed_with_cache"/
• ...

18
Flow Control

• Purpose to limit the number of threads
  concurrently executing requests received from
  remote clients.
• HTTP
• IIOP
• Why do this?
• To reduce CPU utilization (if desired).
• To reduce lock contention (if present).
• To reduce memory utilization (if desired).
• To reduce other resource utilization (if
  desired), e.g. database connections.
• Drawbacks ?
• All requests are considered equal you cant
  give more CPU or resources to one type of request
  as opposed to another type.
• Not guaranteed FIFO (first-in, first-out).

19
Flow Control - Continued

• jagant script fragment
• value"true"/
• value"true"/
• ads" value"500"/
• Experiment with maxexethreads property to find
  best value for your environment.

20
Thread Monitors

• Alternative to IIOP flow control.
• Available only for Java components. ?
• Implemented by extra code generated in skeleton.
• All requests are not considered equal. ?
• Components, or individual methods, can be
  assigned to named thread monitors.
• kOrderSes"
• value"MfgMonitor"/
• Different thread monitors can be assigned
  different maximum numbers of threads.
• erMonitor"
• value"20"/
• Queuing is approximately first-in, first-out. ?

21
Thread Monitors - Continued

• By default, monitors are not nested.
• Component CA has monitor MA.
• Component CB has monitor MB.
• Client calls CA.methodA(). Monitor MA is
  activated.
• CA.methodA() calls CB.methodB(). Monitor MB is
  not used since MA is already active.
• Nested monitors can be defined, but may cause
  deadlock.
• arefulWithMe"
• value"20"/
• valuetrue"/
• Other resources can behave like nested thread
  monitors ?
• e.g. connection caches with com.sybase.jaguar.conn
  cache.waittrue
• Avoid this if possible.
• Let non-nested thread monitors do all the queuing.

22
Thread Monitors Microbenchmark (1 CPU)
23
Thread Monitors SPECjAppServer2001

• Used primarily for session beans in Orders and
  Mfg domains.
• Allows predictable response times in the Orders
  domain while at the same time accommodating the
  periodic bursts of activity caused by the
  batch-processing Large Order Agent.
• Effectively limits JDBC connection usage, rather
  than placing limits directly on the connection
  cache.

24
Lightweight Container (LWC)

• Lightweight stubs for EJB inter-component calls,
  and supporting runtime classes. Extremely fast. ?
• These lightweight stubs dont know how to change
  transaction or security context. To preserve
  standard EJB semantics, they can only be used
  when the caller and callee are guaranteed to use
  the same transaction and security context (this
  is checked at run time).
• e.g. OK - session bean with TX attribute of
  Requires calls entity bean withTX attribute of
  Supports.
• e.g. NOT OK session bean with RunAs mode of
  Client calls session bean with RunAs mode of
  SpecifiedUser.
• Optimized parameter copying for co-located remote
  interface calls, preserves pass by value
  semantics.
• Recommended usage calls from session beans to
  entity beans.
• Set com.sybase.jaguar.component.lwctrue for
  entity beans.
• Set com.sybase.jaguar.server.lwctrue to enable.

25
Lightweight Container (LWC) - Continued

• LWC stubs do not pass calls through servers
  kernel.
• This is the primary reason that LWC calls are
  faster than non-LWC calls using a local
  interface.
• LWC stubs do not pass calls through component
  skeleton.
• Skeleton is the bridge between the kernel and the
  component. Since the kernel is not involved, the
  skeleton is not used.
• LWC stubs do not marshal parameters, except with
  remote interface calls when parameters cannot
  easily be cloned.
• Thread Monitor settings are ignored by LWC stubs.
• Use LWC stubs for entity beans.
• Use Thread Monitor settings on session beans.
• Recommendations
• Use with entity beans called from session beans
  (e.g. with SPECjAppServer).
• Use with local interfaces to avoid parameter
  copying.
• Ensure that session beans are the transaction
  initiators.

26
Lightweight Container Microbenchmark
27
Be Careful With Microbenchmarks!

• Not necessarily representative of real-world
  workloads.
• For example, with LWC microbenchmark
• Intercomponent calls only, no remote access, no
  marshalling.
• No significant end-user provided code.
• No database access.
• You may reach an incorrect conclusions if you try
  to extrapolate microbenchmark results to a
  scenario that involves other factors ?

28
Lightweight Container Future Possibilities

• Could extend to all inter-component calls.
• Not just those which can inherit transaction and
  security context from caller.
• Could extend to non-Java components.
• PB
• C
• .NET
• Could deliver ultra-high-performance container.
• Could deliver ultra-small-footprint container.
• Could embed container into
• Standalone client applications.
• Third-party application servers.
• If you are interested in seeing any of these,
  please let us know
• what usage scenarios you would find useful within
  your
• organization.

29
Entity Isolation Level

• Component Property com.sybase.jaguar.component.iso
  _level
• read_cache
• read_cache_verify_updates
• read_committed
• read_committed_with_cache
• read_committed_verify_updates_with_cache
• repeatable_read
• repeatable_read_with_cache
• serializable
• serializable_with_cache
• For use with CMP entity beans (used with
  SPECjAppServer2001).
• Different entities can use different isolation
  levels, within same transaction.
• Like using ASA/ASE select with holdlock (Oracle
  select for update) on only some of your queries
  in a 2-tier client/server application.
• Allows use of object cache while at the same time
  providing strong guarantee of data integrity. ?

30
Table Timestamp

• A timestamp (actually version counter) value that
  is incremented by a trigger whenever rows are
  inserted, deleted, or updated in the table
  referenced by the timestamp.
• One row in the timestamp table covers all rows in
  the referenced table(s). Suitable for read-mostly
  data.
• Multiple tables can share the same timestamp row.
• Allows one row select to be used at commit time
  to verify that no changes have occurred in the
  referenced table(s). ?
• Very efficient if multiple rows of the referenced
  tables have been used.
• Enabled via component property
• com.sybase.jaguar.component.timestampref_ts.ts
• Trigger creation can be automated via component
  property.
• com.sybase.jaguar.component.ts.triggerstrue
• Feature is unique to EAServer.

31
Object Cache

• Enabled for CMP entity beans if entity isolation
  level mentions cache in its name.
• With luck, avoids the need to run a database
  query when Home.findByPrimaryKey() or
  Bean.ejbLoad() methods are called. ?
• Data integrity is ensured by running commit-time
  verification queries. ?
• Best used with entity beans that are called by
  session beans with transaction retry enabled.
• com.sybase.jaguar.component.tx_retrytrue

32
Query Cache

• Enabled for CMP finder methods if finder mapQuery
  property has cache or cache verify suffix,
  e.g.
• iscountEnt"
• ...
• value"default cache verify"/
• With luck, avoids the need to run a database
  query when custom finder (other than
  findByPrimaryKey) is called. ?
• Data integrity is ensured by running commit-time
  verification queries. ?
• Verification occurs only if cache verify is
  used.
• Verification is only possible when entity uses
  table timestamp.
• Best used with entity beans that are called by
  session beans with transaction retry enabled.
• com.sybase.jaguar.component.tx_retrytrue

33
Object and Query Cache Future Possibilities

• Enhanced algorithms for Object Cache so that
  commit-time validation costs can be further
  reduced.
• Enhanced algorithms for Query Cache so that
  commit-time validation is available for
  parent-child relationships, and for data that is
  often updated.
• Object and Query Cache support at the JDBC API
  level.
• Would need some additional meta-data about
  required isolation levels.
• Object and Query Cache support for the PB
  DataStore.
• Such changes will allow significant portions of
  an applications data to be effectively and
  safely cached.
• If you are interested in seeing such
  enhancements, please let us
• know what usage scenarios you would find useful
  within your
• organization.

34
Just-in-Time Stored Procedures

• A feature of EAServers CMP/JDBC driver wrapper
  for jConnect. Can be used with Sybase ASA and
  ASE. Enable by setting connection cache to use
  the CMP driver wrapper.
• com.sybase.jaguar.conncache.conlibdllcom.sybase.e
  jb.cmp.SybaseDriver
• All insert, delete, update and verify statements
  that can be deferred until commit time will be
  batched.
• Maximum batch size is configurable. See
  documentation for details.
• See also documentation for com.sybase.jaguar.compo
  nent.defer.
• A custom stored procedure will be created for
  each distinct batch of statements encountered at
  run time (including selects). Very fast. ?
• The commit and begin tran statements are
  included in the stored procedure for a
  transactions final batch. ?
• Some transactions can execute as a single stored
  procedure call.
• The driver wrapper for Oracle has a similar
  effect, but uses Oracles prepared statement
  caching instead of stored procedures.
• com.sybase.jaguar.conncache.conlibdllcom.sybase.e
  jb.cmp.OracleDriver

35
Prepared Statement Cloning

• A patent-pending feature of jConnect.
• A memory-efficient alternative to prepared
  statement caching.
• Used by EAServers CMP/JDBC driver wrapper for
  jConnect.
• Can be used in customer code see jConnect
  documentation for com.sybase.jdbcx.SybConnection
  (copyCallableStatement, copyPreparedStatement).
• For prepared statement caching, S distinct cached
  statements for C distinct cached connections uses
  memory proportional to S C.
• Sybase ECperf submission used prepared statement
  caching.
• Performance was limited by available memory, not
  CPU or DBMS.
• For prepared statement cloning, S distinct cached
  statements for C distinct cached connections uses
  memory proportional to S.
• Sybase SPECjAppserver2001 submission used
  prepared statement cloning.
• Performance was better, limited by CPU, not
  memory or DBMS.

36
Sybase Heap Manager

• Actually two heap managers, near identical
  twins
• SybHeap API in EAServer (libjcc), also known as
  Sybase Heap Manager.
• PbHeap API in PowerBuilder VM, also known as
  PowerBuilder Heap Manager.
• A re-implementation of the original JagMemory
  API, intended to improve PB/EAS memory
  management.
• Goals (in order that they were requested and
  achieved).
• Reduce memory utilization of PB/EAS components.
• For Windows client applications, return freed
  memory to operating system.
• Allow effective management of DataWindow /
  DataStore with many rows.
• Allow effective management of very large blocks
  (e.g. 100Mb, result of GetFullSate on DataWindow
  / DataStore with many rows).
• Good multiprocessor performance.
• If you find PowerBuilder using significantly more
  memory or CPU to run your application than it
  used to, please open a case. ?

http//www.sybase.com/detail?id1027319
37
Memory Pools - Definition

• A memory pool API looks like
• MemoryPool allocPool()
• void freePool(pool)
• MemoryBlock allocFromPool(pool, size)
• void returnToPool(pool, block)
• Two key advantages over simple alloc/free model.
• freePool(pool) automatically frees all blocks
  allocated from the pool, without having to return
  all the blocks to the pool first.
• (Blocks can be returned to the pool explicitly
  though,when the pool is not to be freed until
  later).
• freePool(pool) might be able to return pools
  physical (and possibly virtual) memory to the
  operating system.

38
Memory Pools - PowerBuilder

• Each PB/EAS component instance has its own PBVM
  Session.
• Each PBVM Session uses a few memory pools.
• Each DataStore Instance uses a few memory
  pools.
• Thereby, a PB/EAS component instance using one or
  several DataStores will result in a handful of
  active memory pools.
• If a server has many PB component instances, it
  will have many active memory pools.
• Any per-memory-pool overheads will be
  particularly noticeable in terms of virtual
  memory utilization.
• Note PowerBuilder also uses the simple
  alloc/free APIs. Memory pools are used only in
  some cases.

39
SmartHeap

• An excellent third-party memory manager for
  C/C. ?
• Was used internally by the PowerBuilder VM until
  we replaced it with PbHeap (we needed a custom
  solution).
• SmartHeaps strategy for managing memory pools
  was not well suited to the large number of memory
  pools that result from having a large number of
  PB/EAS component instances.
• Specifically, the way that SmartHeap memory pools
  were used by PB/EAS components resulted in
  excessive virtual memory utilization. ?
• If you are in need of a C/C memory manager that
  is not tied to PB/EAS runtime libraries, we would
  still recommend SmartHeap.
• Just think twice (about your application design)
  if you plan to have many memory pools (e.g.
  thousands of pools).

http//www.microquill.com/smartheap/index.html
40
Instance Pooling - Microbenchmark
41
PowerBuilder with LWC - Simulation
42
Instance Pooling - Recommendations

• Use it. ?
• Especially for components with simple business
  methods.
• May give factor of 30 improvement in throughput
  for intercomponent calls.
• When the amount of work done in business methods
  is increased, the relative benefit of instance
  pooling is reduced.
• Still may give 10-20 improvement in throughput
  for intercomponent calls.
• Create DataStores in NVO constructor, not in each
  method.
• Note this will use more memory.
• Set DataObject in NVO constructor, not in each
  method.
• Note this will use more memory.
• May not be applicable if the DataObject changes
  from method call to method call, although you
  could always change DataObject only if it needs
  changing from its previous value (be optimistic).
• Do other slow stuff in NVO constructor. ?

43
Named Instance Pools

• Why use them?
• To avoid running out of memory.
• If you have many different components, with a big
  difference in usage levels from peak to non-peak
  times, the server may use a lot of memory due to
  underutilized pooled instances.
• Each component keeps its own instance pool.
• Many components results in many separate instance
  pools.
• A Named Instance Pool is a grouping of instance
  pools with a single LRU/MRU chain for eviction of
  least-recently-used entries.
• Might help to reduce the number of idle pooled
  instances.
• Might help to reduce the amount of memory used by
  the server.
• Helps to limit the number of pooled instances. ?
• Does not help to limit the number of active
  instances. ?

44
Limiting Active Instances

• Why do it?
• To avoid running out of memory, or to reduce
  memory utilization.
• To reduce lock contention.
• To reduce CPU utulization.
• Set maximum number of active instances and
  maximum wait time
• com.sybase.jaguar.component.objects123
• com.sybase.jaguar.component.maxwait60 (infinite
  by default, unit is seconds)
• Benefits
• Available for all component types.
• Somewhat like a thread monitor.
• Drawbacks
• Cant have multiple components managed by a
  single wait queue.
• Wait queue is not guaranteed first-in, first-out.
• Alternatives
• Flow Control
• Thread Monitors

45
Questions / Feedback

• Any questions?
• Lightweight Container Future Possibilities.
• Object and Query Cache Future Possibilities.