Memory Management for Real-Time Java

1
Memory Management for Real-Time Java

• Wes Beebee and Martin Rinard
• Laboratory for Computer Science
• Massachusetts Institute of Technology
• Supported by DARPA Program Composition for
  Embedded Systems (PCES) Program

2
Goal Enable Use of Java for Real-Time and
Embedded Systems
3
Vision
Standard Java Applications
Real-Time Computation In Java
Downloaded Java Applets
Unified Language/Environment Facilitates
Interaction
4
Why Java?

• Type safe language, no memory corruption
• Reasonably modern approach
• Object oriented
• Garbage collected memory management
• Popular and supported
• Programmers available
• Tools available
• Libraries available

5
Implications and Issues

• Heterogeneous components with different needs and
  goals
• Real-time computation
• User interface
• Data management
• Issues
• Memory management
• Scheduling
• Event management and delivery
• Processor allocation

6
Why NOT Java

• Unpredictable memory usage
• Dynamic memory allocation
• Allocation hidden in extensive set of libraries
  and native methods
• Allocation hidden in exception model
• Unpredictable execution times
• Garbage collection
• No scheduling guarantees
• Thread scheduling
• Event delivery
• Complex libraries and native methods

7
Why NOT Java

• Impoverished set of abstractions
• Threads, mutex locks, signal and wait
• No good way express relationship between
• Events in system
• Corresponding pieces of computation
• No good way to express timing expectations
• Real-Time Java Approach
• Extend library
• Native methods for new mechanisms

8
Real-Time Java Standard

• Goal Augment Java to better support real-time
  systems
• Augment memory model to enable threads to avoid
  garbage collection pauses
• Augment thread scheduling model to add more
  control over task scheduling
• Augment synchronization model to include
  lightweight event delivery mechanism

9
Our View

• Real-time Java is a work in progress
• Many of extensions generate
• More complex programming model
• More possibilities for errors
• Our goal
• Isolate general principles/concepts we think will
  last
• Develop new program analyses and implementation
  mechanisms
• That help programmers use real-time extensions
  safely and effectively

10
Java Memory Models

• Java single garbage-collected heap
• Real-time Java multiple kinds of memories
• Garbage-collected heap memory
• Immortal memory (live for full computation)
• Scoped memories
  (live for specific subcomputations)
• Linear-time allocation (LTMemory)
• Variable-time allocation (VTMemory)

11
Problems/Issues with Memory Model

• Scoped memory issues
• Scoped memory reference checks
• Scoped memory sizes
• Avoiding garbage collection interaction issues
• No-heap real-time thread access checks
• Priority inversions caused by indirect
  interactions with garbage collector

12
Scoped Memory Overview
Standard Java Computation
13
Scoped Memory Overview
Objects in GC Heap
Standard Java Computation
14
Scoped Memory Overview
Objects in GC Heap
Standard Java Computation
15
Scoped Memory Overview
Objects in GC Heap
New Computation Typically new thread Maybe even
real-time thread
Standard Java Computation
16
Scoped Memory Overview
Objects in GC Heap
New Computation Typically new thread Maybe even
real-time thread
Standard Java Computation
Scoped Memory
New Thread Runs In Scoped Memory
17
Scoped Memory Overview
Objects in GC Heap
New Computation Typically new thread Maybe even
real-time thread
Standard Java Computation
Objects
Scoped Memory
Threads New Objects Allocated in Scoped Memory
18
Scoped Memory Overview
Objects in GC Heap
New Computation Typically new thread Maybe even
real-time thread
Standard Java Computation
Objects
Scoped Memory
Threads New Objects Allocated in Scoped Memory
19
Scoped Memory Overview
Objects in GC Heap
New Computation Typically new thread Maybe even
real-time thread
Standard Java Computation
Objects
Scoped Memory
Threads New Objects Allocated in Scoped Memory
20
Scoped Memory Overview
Objects in GC Heap
New Computation Typically new thread Maybe even
real-time thread
Standard Java Computation
Objects
Scoped Memory
Computation Terminates
21
Scoped Memory Overview
Objects in GC Heap
New Computation Typically new thread Maybe even
real-time thread
Standard Java Computation
Objects
Scoped Memory
Objects in Scoped Memory Deallocated as a
Unit without GC
22
Scoped Memory Motivation

• Dynamic memory allocation without GC
• Tie object lifetimes to computation lifetimes
• Eliminate need to dynamically trace out reachable
  objects
• Warning
• Example illustrates primary intended use
• Specification allows more behaviors
• Scoped memories shared by multiple threads
• Nested scoped memories
• Scoped memories entered multiple times

23
Safety Issue for Scoped MemoriesDangling
References

• Lifetimes of objects in scoped memory determined
  by lifetime of computation
• Must ensure that no reference goes from
  long-lived object to short-lived object

24
Nested Scoped Memories
Scoped Memory
Object
25
Referencing Constraints
Scoped Memory
Object
Referencing Down Scopes Is NOT OK
Referencing Up Scopes Is OK
26
Preventing Downward References

• Dynamic Reference Checks
• At every write of a reference into an object
  field or array element
• Check that written object is allocated in a scope
  with a lifetime at least as long as that of
  referred object
• If not, throw an exception
• Drawbacks
• Dynamic checking overhead
• New class of dynamic errors

27
Static Analysis

• Goal
• Eliminate need for dynamic checks by
• Statically checking that program does not violate
  referencing constraints
• Basic approach escape analysis

28
What Escape Analysis Provides
void compute(d,e)

• Control Flow Graph
• Nodes methods
• Edges invocation relationships

void multiplyAdd(a,b,c)

void multiply(m)
void add(u,v)
29
What Escape Analysis Provides
void compute(d,e)

• Control Flow Graph
• Nodes methods
• Edges invocation relationships

void multiplyAdd(a,b,c)

void multiply(m)
void add(u,v)
Allocation Site
30
What Escape Analysis Provides
void compute(d,e)

• Control Flow Graph
• Nodes methods
• Edges invocation relationships

void multiplyAdd(a,b,c)

void multiply(m)
void add(u,v)
Object Allocated Here Does Not Escape Computation
of multiplyAdd method
Allocation Site
31
Our Escape Analysis

• Interprocedural
• Analyzes interactions between methods
• Recaptures objects in callers of allocating
  methods
• Compositional
• Analyzes each method once
• Single analysis result that can be specialized
  for use in different calling contexts
• Suitable for multithreaded programs
• Analyzes interactions between threads
• Recaptures objects that do not escape a given
  multithreaded computation

32
Using Escape Analysis to Verify Correct Use of
Scoped Memories

• For each computation that runs in scoped memory
• Check that allocated objects do not escape

33
Implementation

• FLEX compiler infrastructure (www.flexc.lcs.mit.ed
  u)
• Full Java compiler
• Lots of utilities and packages
• Support for deep program analyses and
  transformations
• Implemented scoped memories and checks
• Implemented escape analysis
• Used results to eliminate checks
• In applications, eliminated all checks

34
Experimental Results
120
100
80
Scope Checks
Time (sec)
60
Application
40
20
0
Array
Array
Tree
Tree
Water
Water
Barnes
Barnes
(Heap)
(Scope)
(Heap)
(Scope)
(Heap)
(Scope)
(Heap)
(Scope)
Benchmarks
35
Scoped Memory Sizes

• Scoped memory creation and size
• MemoryArea ma new LTMemory(10000)
• create a new scoped memory with 10,000 bytes
• If try to allocate more than 10,000 bytes,
  implementation throws an exception
• Problems
• Java does not specify object sizes
• Size of given object may change during its
  lifetime in computation
• So how big to make scoped memory?

36
Modularity Problems
Objects in GC Heap
Scoped Memory Size Determined Here
Standard Java Computation
Objects
Scoped Memory
Required Size Determined by Behavior of Code in
this Computation
37
Modularity Problems
Objects in GC Heap
Scoped Memory Size Determined Here
Standard Java Computation
Objects
Scoped Memory

• If change program, size may need to change!
• Amount of allocated memory becomes part of
  interface!

38
More Issues

• Different executions may allocate different
  amounts of data
• Lots of hidden allocation in libraries
• Difficult to find out how much memory is really
  allocated
• If change implementation, may need to change
  scoped memory size in clients

39
Analysis Solution

• Analyze program to symbolically compute allocated
  memory sizes
• Input variables
• Object sizes
• Compiler knows object sizes, can conservatively
  generate scoped memory sizes

40
Interaction with Garbage Collector

• Standard Collector Assumptions
• Can interrupt computation at any point
• Can suspend for unbounded time
• Real-Time Java extension
• No-Heap Real-Time Threads
• Can Access
• Immortal memory
• Scoped memory
• Do not interact with GC heap AT ALL
• Can run asynchronously with GC

Immortal
Scoped
GC Heap
41
No-Heap Real-Time Thread Checks

• Dynamically check that no-heap real-time threads
  never access a location containing a reference
  into garbage-collected heap
• At every read, check to make sure result does not
  point into garbage-collected heap
• At every write, check to make sure not
  overwriting reference into GC heap
• If check fails, throw exception
• Drawbacks
• Dynamic checking overhead
• New class of dynamic errors

42
Implementation

• FLEX compiler infrastructure (www.flexc.lcs.mit.ed
  u)
• Implemented no-heap real-time threads
• Implemented access checks
• Measured performance with and without checks

43
Experimental Results
Scope Checks
Heap Checks
300
Application
250
200
150
Time (sec)
100
50
0
Array
Tree
Water
Benchmark
44
Program Analysis for Eliminating Checks

• Control-flow analysis to identify code that may
  execute in no-heap real-time thread
• Global value-flow analysis
• Tags each value that points to GC heap
• Identifies all locations into which these values
  may flow
• Combine results
• Look at all no-heap real-time thread code
• Check statically for access violations

45
Analysis Issues and Solutions

• Complicated, whole-program analysis
• Simple annotations can enable local analysis
• Annotate each reference with source
• Scoped memory
• Immortal memory
• Heap memory
• Annotation checker validates annotations
• Result
• Scalable analysis
• Eliminate checks, eliminate programming errors

46
Indirect Priority Inversions
Interaction Between Resource Sharing and Garbage
Collection
Lock Acquire
No-heap Thread
Garbage Collector
Standard Java Thread
Lock Acquire
47
Indirect Priority Inversions
Interaction Between Resource Sharing and Garbage
Collection
Lock Acquire
No-heap Thread
Blocks
Garbage Collector
Standard Java Thread
Lock Acquire
48
Indirect Priority Inversions
No-heap thread must wait for standard Java
thread to release lock Standard thread must
wait for GC to finish (heap inconsistent until
it finishes) No-heap thread must wait for GC!
Lock Acquire
No-heap Thread
Garbage Collector
Standard Java Thread
Lock Acquire
49
Using Non-Blocking Synchronization to Eliminate
Indirect Priority Inversions
Start Atomic Region
End Atomic Region
No-heap Thread
Does Not Block!
Garbage Collector
Abort, Retry
Standard Java Thread
Start Atomic Region
50
Implementation Status

• Non-blocking synchronization implemented for
  memory management primitives
• Useful when threads share scoped memory
• Uses non-blocking synchronization instructions
  from processor
• Software implementation underway for general
  atomic regions

51
Goal
Enable safe real-time code to interact
successfully with code that accesses GC data
Issues and Complications

• Dangling references for scoped memories
• Resource needs of computations
• Isolating computations from garbage collector
• Ensure threads with real-time constraints dont
  access garbage collected data
• Eliminate indirect interactions
• Our view
• Dynamic checks inadequate
• Statically verify correct use, eliminate checks

52
Broader View of Real-Time Java

• Java is best suited to express batch
  computations on objects
• Not so good for control in asynchronous,
  parallel, distributed, time-aware systems
• Inadequate for design/requirements
• Can be part of solution, but only a part

53
Multiple Perspectives

• Any system has many desired properties
• Data structure invariants
• Flow of data between different components
• Timing requirements for computations
• Each property is inherently partial
• Many properties are best expressed as
• Declarative constraints
• NOT translatable into implementation

54
Design Conformance
Object Referencing Relationships
Dataflow Interactions
Timing Constraints
Check that implementation conforms to design
properties
Implementation
55
Future

• Scheduling and event delivery
• More precise referencing relationship analysis
• Completely characterize aliasing behavior
• More flexible memory management algorithms that
  preserve predictability
• More flexible regions
• Immediate deallocation
• Less restricted memory access constraints for
  real-time threads
• Design conformance for more control-oriented
  properties

56
Summary

• Real-time Java code coexists and interacts with
  standard Java code
• New complications (overhead failure modes)
• Scoped memory checks
• Scoped memory sizes
• No-heap real-time threads
• Indirect priority inversions
• Attacked with program analysis
• Future scheduling and timing