Memory Management

1
Memory Management
Kathryn McKinley
2
Isnt GC a bit retro?
Languages without automated garbage collection
are getting out of fashion. The chance of running
into all kinds of memory problems is gradually
outweighing the performance penalty you have to
pay for garbage collection. Paul Jansen,
managing director of TIOBE Software, in Dr Dobbs,
April 2008
3
Course Logistics

• Syllabus
• Critical reading writing
• Presentations
• Discussion
• Critiques
• Schedule
• Volunteers for next week?

4
Outline

• Briefly introduce the challenges and key ideas in
  memory management
• Context modern VMs
• Explicit vs automatic
• Memory organization
• Allocation
• Garbage Identification
• Reclamation

5
Basic VM Structure
Program/Bytecode
Executing Program
Class Loader Verifier, etc.
Heap
Thread Scheduler
Interpreter /or Dynamic Compiler
Garbage Collector
6
Dynamic memory allocation and reclamation

• Heap contains dynamically allocated objects
• Object allocation malloc, new
• Deallocation
• Manual/explicit free, delete
• automatic garbage collection

7
Memory Management

• Objects/data in heap memory
• How does the runtime system efficiently create
  and recycle memory on behalf of the program?
• What makes this problem important?
• What makes this problem hard?
• Why are researchers still working on it?

8
Explicit memory managementchallenges

• More code to maintain
• Correctness
• Free an object too soon - core dump
• Free an object too late - waste space
• Never free - at best waste, at worst fail
• Efficiency can be very high
• Gives programmers control

9
Garbage collectionAutomatic memory management

• Reduces programmer burden
• Eliminates sources of errors
• which ones?
• Integral to modern object-oriented languages
• Java, C, PHP, JavaScript
• Mainstream
• Challenge performance

10
Why use Garbage Collection?

• Software engineering benefits
• Less user code compared to expilict memory
  management (MM)
• Less user code to get correct
• Protects against some classes of memory errors
• No free(), thus no premature free(), no double
  free(), or forgetting to free()
• Not perfect, memory can still leak
• Programmers still need to eliminate all pointers
  to objects the program no longer needs

11
Why use Garbage Collection?

• Performance space/time tradeoff
• Time proportional to dead objects (explicit mm,
  reference counting) or live objects (semi-space,
  mark-sweep)
• Throughput versus pause time
• Less frequent collection typically reduces total
  time but increases space requirements and pause
  times
• Hidden locality benefits?
• Layer of abstraction
• What else can the collector do when it visits all
  objects?

12
Key Issues

• For both
• Fast allocation
• Fast reclamation
• Low fragmentation (wasted space)
• How to organize the memory space
• Garbage Collection
• Discriminating live objects and garbage

13
What is Garbage?
14
Perfect live object detection

• Live object has a future use
• Prove that object is not live, and deallocate it
• Deallocate as soon as possible after last use

15
Estimating liveness in practice

• Approximate liveness by reachability from outside
  the heap
• An unreachable object cannot ever be used---it is
  garbage
• Once dead always dead!
• Find and preserve reachable objects
• Tracing or reference counting
• Recycle the space of garbage objects

16
How does the GC implement reachability?
17
How does the GC implement reachability?

• Tracing
• Counting

18
How does the GC find the pointers to trace or
count?

• Managed languages couple GC with safe pointers
• Programs may not access arbitrary addresses in
  memory
• Compiler can identify and provide the GC with all
  the pointers.enforcing
• Once garbage, always garbage
• Runtime system can move objects by updating
  pointers
• Unsafe languages can do non-moving GC by assuming
  anything that looks like a pointer is one.

19
Reachability with tracing

• Compiler produces a stack-map at GC safe-points
  and Type Information Blocks
• GC safe points new(), method entry, method exit,
  back-edges (thread switch points)
• Stack-map enumerate global variables, stack
  variables, live registers -- This code is hard to
  get right! Why?
• Type Information Blocks identify reference
  fields in objects

.... r0 obj PC -gt p.f obj
....
stack
globals
registers
heap
20
Reachability with tracing

• Compiler produces a stack-map at GC safe-points
  and Type Information Blocks
• Type Information Blocks identify reference
  fields in objects
• for each type i (class) in the program, a map

3
0
2
TIBi

.... r0 obj PC -gt p.f obj
....
stack
globals
registers
heap
21
Reachability with tracing

• Tracing collector (semispace, marksweep)
• Marks the objects reachable from the roots live,
  and then performs a transitive closure over them

mark

.... r0 obj PC -gt p.f obj
....
stack
globals
registers
heap
22
Reachability with tracing

• Tracing collector (semispace, marksweep)
• Marks the objects reachable from the roots live,
  and then performs a transitive closure over them

mark

.... r0 obj PC -gt p.f obj
....
stack
globals
registers
heap
23
Reachability with tracing

• Tracing collector (semispace, marksweep)
• Marks the objects reachable from the roots live,
  and then performs a transitive closure over them

mark

.... r0 obj PC -gt p.f obj
....
stack
globals
registers
heap
24
Reachability with tracing

• Tracing collector (semispace, marksweep)
• Marks the objects reachable from the roots live,
  and then performs a transitive closure over them
• All unmarked objects are dead, and can be
  reclaimed

mark

.... r0 obj PC -gt p.f obj
....
stack
globals
registers
heap
25
Reachability with tracing

• Tracing collector (semispace, marksweep)
• Marks the objects reachable from the roots live,
  and then performs a transitive closure over them
• All unmarked objects are dead, and can be
  reclaimed

sweep

.... r0 obj PC -gt p.f obj
....
stack
globals
registers
heap
26
Taxonomy of GC design choices

• Heap Organization
• Incrementality
• Composability
• Concurrency
• Parallelism
• Distribution

27
Heap organization basic algorithmic components

• Allocation

• Reclamation

Identification
Sweep-to-Free

Tracing (implicit)
Free List
Compact
Evacuate
Reference Counting (explicit)
Bump Allocation
28
One Big Heap?Incrementality

• Pause times
• it takes too long to trace the whole heap at once
• Throughput
• the heap contains lots of long lived objects, why
  collect them over and over again?
• Incremental collection
• divide up the heap into increments and collect
  one at a time.

to space
from space
to space
from space
Increment 1 Increment 2
29
Incremental Collection

• Ideally
• perfect pointer knowledge of live pointers
  between increments
• requires scanning whole heap, defeats the purpose

to space
from space
to space
from space
Increment 1 Increment 2
30
Incremental Collection

• Ideally
• perfect pointer knowledge of live pointers
  between increments
• requires scanning whole heap, defeats the purpose

to space
from space
to space
from space
Increment 1 Increment 2
31
Incremental Collection

• Ideally
• perfect pointer knowledge of live pointers
  between increments
• requires scanning whole heap, defeats the purpose

to space
from space
to space
from space
Increment 1 Increment 2
32
Incremental Collection

• Ideally
• perfect pointer knowledge of live pointers
  between increments
• requires scanning whole heap, defeats the purpose
• Mechanism Write barrier
• records pointers between increments when the
  mutator installs them, conservative approximation
  of reachability

to space
from space
to space
from space
Increment 1 Increment 2
33
Write barrier

• compiler inserts code that records pointers
  between increments when the mutator installs them
• // original program // compiler support for
  incremental collection
• p.f o if (incr(p) !
  incr(o)
• remembered set (incr(o)) U p.f
• p.f o

remset1 w
remset2 f,g
a b c d e f g
t u v w x y z
to space
from space
to space
from space
Increment 1 Increment 2
34
Write barrier

• Install new pointer d -gt v
• // original program // compiler support for
  incremental collection
• p.f o if (incr(p) !
  incr(o)
• remembered
  set (incr(o)) U p.f
• p.f o

remset1 w
remset2 f,g
a b c d e f g
t u v w x y z
to space
from space
to space
from space
Increment 1 Increment 2
35
Write barrier

• Install new pointer d -gt v, then update d-gt y
• // original program // compiler support for
  incremental collection
• p.f o if (incr(p) !
  incr(o)
• remembered set (incr(o)) p.f
• p.f o

remset1 w
remset2 f,g,d
a b c d e f g
t u v w x y z
to space
from space
to space
from space
Increment 1 Increment 2
36
Write barrier

• Install new pointer d -gt v, then update d-gt y
• // original program // compiler support for
  incremental collection
• p.f o if (incr(p) !
  incr(o)
• remembered set (incr(o)) p.f
• p.f o

remset1 w
remset2 f,g,d,d
a b c d e f g
t u v w x y z
to space
from space
to space
from space
Increment 1 Increment 2
37
Write barrier

• At collection time
• collector re-examines all entries in the remset
  for the increment, treating them like roots
• Collect Increment 2

remset1 w
remset2 f,g,d,d
a b c d e f g
t u v w x y z
to space
from space
to space
from space
Increment 1 Increment 2
38
Write barrier

• At collection time
• collector re-examines all entries in the remset
  for the increment, treating them like roots
• Collect Increment 2

remset1 w
remset2 f,g,d,d
a b c d e f g
t u v w x y z
to space
from space
to space
from space
Increment 1 Increment 2
39
Summary of the costs of incremental collection

• write barrier to catch pointer stores crossing
  boundaries
• remsets to store crossing pointers
• processing remembered sets at collection time
• excess retention

remset1 w
remset2 f,g,d,d
a b c d e f g
t u v w x y z
to space
from space
to space
from space
Increment 1 Increment 2
40
Taxonomy of Design Choices

• Incrementality
• Composability
• Concurrency
• Parallelism
• Distribution

41
GC Ideas

• Generational collection - Young objects die fast
• Older first - The longer you wait
• Garbage first
• Immix
• Reference counting - Deferred Ulterior
• Concurrent collection Steele, Djkistra et
  al.78
• at the same time as the mutator
• Parallel collection
• Real time
• make pause times tiny Metronome, Petrank et al.
• What else can you do when you visit all the
  objects?
• Memory utilization fragmentation
• Leaks

42

• Questions?