Jonas Bon

1
Clustering the JVM using AOP

• Jonas Bonér
• Terracotta, Inc.
• jonas_at_terracotta.org
• http//terracotta.org
• http//jonasboner.com

2

• Lets start with a demo
• since a picture says more than a thousand words

3
Agenda

• Overview of Terracotta
• Get started
• The inner workings of a clustered JVM
• Real-world examples
• QA

4
Agenda

• Overview of Terracotta
• Get started
• The inner workings of a clustered JVM
• Real-world examples
• QA

5
What is Terracotta?

• Network-Attached Memory
• Open Source Scalability and
• High-Availability for the JVM

6
What is Terracotta?

• Network-Attached Memory
• No API declarative configuration to selectively
  share object graphs across the cluster
• No serialization plain POJO clustering
• Fine-grained replication field-level, heap-level
  replication
• Cross-JVM coordination Java Memory Model
  maintained across the cluster e.g. cluster-wide
  wait/notify and synchronized
• Large virtual heaps that exceeds the capacity of
  a single JVM
• Distributed Method Invocations
• Runtime monitoring and control

7
Use-cases

• Relieving Database Overload
• Distributed Caching
• Hibernate Clustering
• HTTP Session Clustering
• Simplifying Application Architecture and
  Development
• Virtual Heap for Large Datasets
• Clustering OSS Frameworks (Spring, Struts,
  Lucene, Wicket, EHCache etc.)
• Master/Worker Managing Large Workloads
• POJO Clustering
• Messaging, Event-based Systems and
  Coordination-related Tasks

8
Terracotta approach

• Today's Reality
• Scale out is complex
• Requires custom Java code

• Our different approach
• Cluster the JVM
• Eliminate need for custom code

Scale-out
Custom Development
JMS
RMI
Serialization
Terracotta Server Clustering the JVM
9
The architecture of Terracotta

• Terracotta Server
• 100 Pure Java
• HA Active / Passive Pair
• Local JVM Client
• Transparent
• Pure Java Libraries
• Coordinator traffic cop
• Coordinates resource access
• Runtime optimizations
• Central Storage
• Maintains state across restarts

Scale-out
Terracotta Library
Terracotta Library
Terracotta Library
Terracotta Server (ACTIVE) Clustering the JVM
Terracotta Server Clustering the JVM
Terracotta Server (PASSIVE) Clustering the JVM
10
Network-Attached Memory

• Heap Level Replication
• Declarative
• No Serialization
• Fine Grained / Field Level
• GET_FIELD - PUT_FIELD
• Only Where Resident
• JVM Coordination
• Distributed Synchronized Block
• Distributed wait()/notify()
• Fine Grained Locking MONITOR_ENTRY -MONITOR_EXIT
  
• Large Virtual Heaps
• As large as available disk
• Dynamic paging

Scale-out
App Server
App Server
App Server
Web App
Web App
Web App
Shared Objects
Shared Objects
Shared Objects
JVM
JVM
JVM
TC Libraries
TC Libraries
TC Libraries
Terracotta Server Clustering the JVM
11
Agenda

• Overview of Terracotta
• Get started
• The inner workings of a clustered JVM
• Real-world examples
• QA

12
Hello World
13
Hello World - tutorial/HelloWorld.java
package tutorial import java.util. public cla
ss HelloWorld     private ListltStringgt hellos  n
ew ArrayListltStringgt()    public void sayHello()
        synchronized(hellos)           hellos.ad
d("Hello, World "  new Date())          for
(String hello  hellos)              System.out.p
rintln(hello)                         public 
static void main(String args)        new HelloW
orld().sayHello()    
14
Hello World - tc-config.xml
lt?xml version"1.0" encoding"UTF-8"?gt lttctc-conf
ig xmlnstc"http//www.terracotta.org/config"gt   
ltapplicationgt     ltdsogt       ltrootsgt         ltroo
tgtltfield-namegttutorial.HelloWorld.helloslt/field-na
megtlt/rootgt       lt/rootsgt       ltlocksgt         lta
utolockgt           ltmethod-expressiongt tutorial.H
elloWorld.(..)lt/method-expressiongt           ltlo
ck-levelgtwritelt/lock-levelgt         lt/autolockgt   
    lt/locksgt       ltinstrumented-classesgt         
ltincludegtltclass-expressiongttutorial..lt/class-expr
essiongtlt/includegt       lt/instrumented-classesgt   
  lt/dsogt   lt/applicationgt lt/tctc-configgt
15
Demo
16
Enhanced Hello WorldUsing java.util.concurrent
17
Coordination - HelloWorldConcurrent.java
package tutorial import java.util. import java
.util.concurrent.CyclicBarrier public class Hell
oWorldConcurrent     private ListltStringgt hellos 
 new ArrayListltStringgt()    private CyclicBarrie
r barrier    public void sayHello() throws Excep
tion        barrier  new CyclicBarrier(2)      
 barrier.await()             synchronized(hellos)
           hellos.add("Hello, World "  new Date(
))          for(String hello  hellos)          
    System.out.println(hello)                 
       public static void main(String args) th
rows Exception        new HelloWorldConcurrent().
sayHello()    
18
Coordination - tc-config-concurrent.xml
lt?xml version"1.0" encoding"UTF-8"?gt lttctc-conf
ig xmlnstc"http//www.terracotta.org/config"gt   
ltapplicationgt     ltdsogt       ltrootsgt         ltroo
tgtltfield-namegttutorial.HelloWorldConcurrent.hellos
lt/field-namegtlt/rootgt         ltrootgtltfield-namegttut
orial.HelloWorldConcurrent.barrierlt/field-namegtlt/r
ootgt       lt/rootsgt       ltlocksgt         ltautoloc
kgt           ltmethod-expressiongt tutorial.HelloWo
rldConcurrent.(..)lt/method-expressiongt          
 ltlock-levelgtwritelt/lock-levelgt         lt/autolock
gt       lt/locksgt       ltinstrumented-classesgt     
    ltincludegtltclass-expressiongttutorial..lt/class-
expressiongtlt/includegt       lt/instrumented-classes
gt     lt/dsogt   lt/applicationgt lt/tctc-configgt
19
Demo
20
Agenda

• Overview of Terracotta
• Get started
• The inner workings of a clustered JVM
• Real-world examples
• QA

21
Inner workings of a clustered JVM

• Maintain the semantics of the application (e.g
  Java Language Specification (JLS) and the Java
  Memory Model (JMM)) across the cluster
• Load-time bytecode instrumentation
• Hook in java.lang.ClassLoader and -javaagent
• Transparent to the user -- classes are woven
  on-the-fly
• Aspect-Oriented Programming techniques
• AspectWerkz aspects
• Custom bytecode instrumentation using ASM
• Declarative configuration (XML)
• Simplified pointcut pattern language (based on
  AspectWerkz)

22
What do we need to maintain across the cluster?

• State sharing
• Share data in the Java heap for the instances
  reachable from a shared top-level root
  reference
• Make sure each JVMs local heap are in sync (in
  regards to the clustered parts of the heap)
• Thread coordination
• Resource access and guarding
• Thread signaling

23
Sample application

• _at_Clustered public class Counter
• _at_Root public final static Counter instance
  new Counter()
• private final Object lock new Object()
• private volatile int counter 0
• public void increment()
• synchronized(lock) this.counter
  lock.notifyAll()
• public void waitFor(int expected)
• synchronized(lock)
• while(this.counter lt expected)
• try lock.wait() catch(InterruptedExce
  ption ex)

24
State sharing - field-level replication and
management

• We need to break down the shared object graph
  into literals
• int, long, float, String, boolean etc.
• Need to be able to detect changes to the object
  graph
• New object reference is added/removed
• Literals changes value
• Every reference has a shadow that is managing
  the reference
• Knows if the reference has been changed
• Can lazily page in and out the actual value
• Page in Reconcile changes done in another node
• Page out Null out the reference

25
Field-level replication
each instance has a shadow that is managing
access and modification
Grandparent
500 bytes

Parent2
Parent1
500 bytes each
Child2
Child1
Child3
Child3
16 bytes each
Total object graph size 1548 bytes
Terracotta replicates 16 bytes
26
Manage field access and modification

• Intercept the PUTFIELD and GETFIELD bytecode
  instructions
• Create two advice for managing get and set shared
  field
• before(TransparentAccess o, Object value)
• set( .) isClusteredTarget(o)
  args(value)
• // register changed field value in shadow
• before(TransparentAccess o)
• get( .) isClusteredTarget(o)
• // reconcile any pending changes

27
BCI Example original byte code

• public demo.sharededitor.controls.Dispatcher(demo.
  sharededitor.models.ObjectManager,
  demo.sharededitor.ui.Renderer)
• Code
• 0 aload_0
• 1 invokespecial 1 //Method
  javax/swing/event/MouseInputAdapter."ltinitgt"()V
• 4 aload_0
• 5 aload_2
• snip
• 33 aload_0
• 34 aload_1
• 35 putfield 6 //Field objmgrLdemo/sharededi
  tor/models/ObjectManager
• 38 aload_0
• 39 getfield 6 //Field objmgrLdemo/sharededi
  tor/models/ObjectManager
• 42 aload_0
• 43 getfield 2 //Field rendererLdemo/sharede
  ditor/ui/Renderer
• 46 invokevirtual 7 //Method
  demo/sharededitor/models/ObjectManager.addListener
  (Ldemo/sharededitor/events/IListListener)V
• 49 return

28
BCI Example instrumented byte code

• public demo.sharededitor.controls.Dispatcher(demo.
  sharededitor.models.ObjectManager,
  demo.sharededitor.ui.Renderer)
• Code
• 0 aload_0
• 1 invokespecial 99 //Method
  javax/swing/event/MouseInputAdapter."ltinitgt"()V
• 4 aload_0
• 5 aload_2
• snip
• 33 aload_0
• 34 aload_1
• 35 invokevirtual 117 //Method
  __tc_setobjmgr(Ldemo/sharededitor/models/ObjectMa
  nager)V
• 38 aload_0
• 39 invokevirtual 119 //Method
  __tc_getobjmgr()Ldemo/sharededitor/models/ObjectM
  anager
• 42 aload_0
• 43 getfield 101 //Field rendererLdemo/share
  deditor/ui/Renderer
• 46 invokevirtual 123 //Method
  demo/sharededitor/models/ObjectManager.addListener
  (Ldemo/sharededitor/events/IListListener)V
• 49 return

29
Thread coordination

• We need to maintain the semantics of the Java
  Memory Model (JMM) across multiple JVMs
• Maintain the semantics of
• synchronized(object) ..
• wait()/notify()/notifyAll()
• Flushing of changes
• Locking optimizations
• User defined
• Runtime optimized

30
Manage synchronized blocks

• Intercept MONITORENTER and MONITOREXIT bytecode
  instructions
• before(Object o)
• isClustered() lock() args(o)
• ClusterManager.lock(o) // lock o in cluster
• after(Object o) // after finally
• isClustered() unlock() args(o)
• ClusterManager.unlock(o) //unlock o in cluster
  flush changes

31
Manage thread coordination (wait/notify)

• Intercept the wait(), notify() and notifyAll()
  methods in java.lang.Object
• void around(Object o)
• isClusteredTarget(o) call(void
  Object.wait())
• ClusterManager.objectWait(o)
• void around(Object o)
• isClusteredTarget(o) call(void
  Object.notify())
• ClusterManager.objectNotify(o)
• // remaining methods omitted

32
Agenda

• Overview of Terracotta
• Get started
• The inner workings of a clustered JVM
• Real-world examples
• QA

33
HTTP Session Clusteringwithout Java Serialization
34
HTTP Session Clustering - Benefits

• Terracotta Sessions gives you
• Near-Linear Scale
• No java.io.Serializable
• Large Sessions - MBs
• Higher Throughput
• Supported Platforms
• Jetty,
• JBoss 4.x,
• Tomcat 5.0 5.5,
• WebLogic 8.1, WebLogic 9.2,
• WebSphere CE, Geronimo Alpha, WebSphere 6.1

35
HTTP Session Clustering - DummyCart.java
package demo.cart   import java.util.   public 
class DummyCart      private List items  new Arr
ayList()       public List getItems()          r
eturn Collections.unmodifiableList(items)       
    public void addItem(String name)          it
ems.add(name)            public void removeItem(
String name)          items.remove(name)     
36
HTTP Session Clustering - carts.jsp
lt_at_ page import"java.util.Iterator" gt lthtmlgt
ltjspuseBean id"cart" scope"session"
class"demo.cart.DummyCart" /gt lt String
submit request.getParameter("submit")
String item request.getParameter("item")
if (submit ! null item ! null)
if (submit.equals("add"))
cart.addItem(item) else if
(submit.equals("remove"))
cart.removeItem(item)
gt ltbodygt ltpgtYou have the
following items in your cartlt/pgt ltolgtlt
Iterator it cart.getItems().iterato
r() while (it.hasNext())
gtltligtlt out.print(it.next()) gtlt/ligt
lt gt lt/olgt
ltform methodget" action"ltrequest.getCont
extPath()gt/cart.jsp"gt ltpgtItem to
add or remove ltinput type"text" name"item"
/gtlt/pgt ltinput type"submit"
name"submit" value"add" /gt ltinput
type"submit" name"submit" value"remove" /gt
lt/formgt lt/bodygt lt/htmlgt
37
HTTP Session Clustering - tc-config.xml
lt?xml version"1.0" encoding"UTF-8"?gt lttctc-conf
ig xmlnstc"http//www.terracotta.org/config"gt   
ltapplicationgt     ltdsogt    ltweb-applicationsgt  
   ltweb-applicationgtcartlt/web-applicationgt    
   lt/web-applicationsgt       ltinstrumented-classes
gt         ltincludegt           ltclass-expressiongtde
mo.lt/class-expressiongt         lt/includegt       lt
/instrumented-classesgt     lt/dsogt   lt/applicationgt
lt/tctc-configgt
38
Demo
39
Cluster Spring andother OSS frameworks
40
Glimpse of supported integrations

• 'Supported' by Terracotta means
• Just include a Configuration Module (OSGi
  bundle) in your config to cluster your
  application (a one liner)
• Plugs in underneath without any setup
• Technically, Terracotta supports all integrations
  as long as it runs on the JVM (f.e. JRuby, PHP)

41
Example Configuration Module- EHCache clustering
lt?xml version"1.0" encoding"UTF-8"?gt lttctc-conf
ig xmlnstc"http//www.terracotta.org/config"gt   
ltclientsgt     ltmodulesgt       ltmodule name"cluste
red-ehcache-1.3" version"1.0.0"/gt     lt/modulesgt
  lt/clientsgt   ltapplicationgt     ltdsogt       ltinst
rumented-classesgt         ltincludegt           ltcla
ss-expressiongttutorial...lt/class-expressiongt    
     lt/includegt       lt/instrumented-classesgt     
lt/dsogt   lt/applicationgt lt/tctc-configgt
42
Example - Spring clustering
Terracotta config
Spring config
ltspringgt   ltapplication name"tc-jmx"gt     ltapplic
ation-contextsgt         ltapplication-contextgt     
    ltpathsgt           ltpathgt/applicationContext.x
mllt/pathgt         lt/pathsgt         ltbeansgt        
   ltbean name"clusteredCounter"/gt           ltbean
 name"clusteredHistory"/gt         lt/beansgt       
lt/application-contextgt     lt/application-contextsgt
         lt/applicationgt lt/springgt
ltbean id"localCounter"               class"demo.
jmx.Counter"/gt   ltbean id"clusteredCounter"      
  class"demo.jmx.Counter"/gt         ltbean id"loc
alHistory"               class"demo.jmx.HistoryQu
eue"/gt   ltbean id"clusteredHistory"        class
"demo.jmx.HistoryQueue"/gt

• Terracotta can declaratively cluster Spring
  beans (Singleton Session and Custom scoped)
  with zero code changes
• Can also cluster Spring ApplicationContext
  events, JMX State and Spring Web Flow

43
DemoCluster Spring Web Flows Web
Continuations (conversational/workflow state)
44
Agenda

• Overview of Terracotta
• Get started
• Real-world examples
• QA

45
Wrapping up
46
Wrapping up

• Terracotta is Network-Attached Memory for the JVM
• Turns Scalability and High-Availability into a
  deployment artifact
• Keep the simplicity of POJO-based development
  get Scale-Out with
  Simplicity
• Makes mission-critical applications simpler to
• Write
• Understand
• Test
• Maintain
• Endless possibilities for clustering and
  distributed programming these were just a
  few
• Be creative, use your imagination and have fun

47
Questions?http//terracotta.orgjonas_at_terracot
ta.org