Concurrency Utilities in Practice

1
Concurrency Utilities in Practice
Using java.util.concurrent

• Tim Peierls
• Prior Artisans, LLC

2
Goals

• Review concurrency utilities
• Present several applications
• Measure and evaluate performance

3
Speakers
Tim Peierls is a member of the JSR 166 Expert
Group
4
Examples

• PseudoRandom
• Memoize
• SwingWorker
• BoundedBuffer
• WebCrawler

5
Concurrency Utilities

• Executors
• Executor
• ExecutorService
• ScheduledExecutorService
• Callable
• Future
• ScheduledFuture
• Delayed
• CompletionService
• ThreadPoolExecutor
• ScheduledThreadPoolExecutor
• AbstractExecutorService
• Executors
• FutureTask
• ExecutorCompletionService
• Queues
• BlockingQueue
• ConcurrentLinkedQueue

• Concurrent Collections
• ConcurrentMap
• ConcurrentHashMap
• CopyOnWriteArrayList,Set
• Synchronizers
• CountDownLatch
• Semaphore
• Exchanger
• CyclicBarrier
• Timing
• TimeUnit
• Locks
• Lock
• Condition
• ReadWriteLock
• AbstractQueuedSynchronizer

6
Caveats

• Access modifiers omitted unless relevant
• InterruptedException abbreviated as IE
• Most interrupt handling omitted for brevity, but
  very important in real life!
• Irrelevant methods, fields, and arguments elided
  with

7
PseudoRandom Example
Overview

• PseudoRandom similar to java.util.Random
• A simple but broken implementation
• Reimplement PseudoRandom in different ways
• Exercise the implementations
• Compare their performance

8
PseudoRandom Example
PseudoRandom similar to java.util.Random
interface PseudoRandom / Return
pseudorandom int in range 0, n) / int
nextInt(int n)
9
PseudoRandom Example
A simple but broken implementation
class NaivePseudoRandom implements PseudoRandom
public int nextInt(int n) int s
seed seed computeNext(seed) return s
n int computeNext(int s) return
int seed
10
PseudoRandom Example
Fixed using synchronized
class PseudoRandomUsingSynch
implements PseudoRandom public synchronized
int nextInt(int n) int s seed seed
computeNext(seed) return s n int
computeNext(int s) return int seed
11
PseudoRandom Example
Better fix using Lock
class PseudoRandomUsingLock
implements PseudoRandom public int
nextInt(int n) lock.lock() try
int s seed seed computeNext(seed)
return s n finally
lock.unlock() int computeNext(int s)
return int seed final Lock lock
new ReentrantLock(true)
12
Review Lock
Ensure consistency via exclusion

• interface Lock void lock() void
  lockInterruptibly() throws IE boolean
  tryLock() boolean tryLock(long timeout,
  TimeUnit unit)
  throws IE void unlock()
• class ReentrantLock implements Lock
  ReentrantLock(boolean fair)

13
PseudoRandom Example
Even better fix using AtomicInteger
class PseudoRandomUsingAtomic
implements PseudoRandom public int
nextInt(int n) for () int s
seed.get() int nexts computeNext(s)
if (seed.compareAndSet(s, nexts))
return s n int computeNext(int s)
return final AtomicInteger seed new
AtomicInteger()
14
Review AtomicInteger
Ensure consistency via atomic CAS

• class AtomicInteger extends Number
  implements Serializable
• AtomicInteger(int initialCount)
  boolean compareAndSet(int expect, int update)
  boolean weakCompareAndSet(int expect, int
  update)
• int get() int getAndSet(int newValue)
  int getAndAdd(int delta) int
  addAndGet(int delta) int
  getAndIncrement() int getAndDecrement()
  int incrementAndGet() int
  decrementAndGet()

15
PseudoRandom Example
Exercise the implementations

• DieRollTask repeatedly rolls a die
• Coordinate tasks with CountDownLatch
• Use Executor to run several DieRollTasks
• Time multiple trials of each implementation
• Test methodology disclaimer
• Results

16
PseudoRandom Example
Coordinate tasks with CountDownLatch
final int NTASKS // instances of
DieRollTask final int NROLLS // die rolls
per task instance final PseudoRandom rnd final
CountDownLatch startSignal
new CountDownLatch(1) final
CountDownLatch doneSignal
new CountDownLatch(NTASKS) class
DieRollTask implements Runnable public void
run() startSignal.await() for (int i
0 i doneSignal.countDown() / IE handling
omitted /
17
Review CountDownLatch
Coordinate thread activity

• Waiting threads are released when count becomes
  0
• class CountDownLatch CountDownLatch(int
  initialCount) void countDown()
  void await() throws IE void await(long
  timeout, TimeUnit unit)
  throws IE long getCount()

18
PseudoRandom Example
Use Executor to run several DieRollTasks
final Executor threadPool Executors.newFixedTh
readPool(NTASKS) long trial(final int NROLLS,
final PseudoRandom r) for (int
j 0 j w DieRollTask()) give tasks time to wait for
startSignal long start System.nanoTime()
startSignal.countDown() // release tasks
doneSignal.await() // wait til done
return System.nanoTime() start
19
Review Executor
Decouple task submission from execution

• Executor executes submitted tasks
• interface Executor void execute(Runnable
  task)
• Executors has static factory methods
• class Executors static ExecutorService
  // extends Executor newFixedThreadPool(int
  nThreads) many more

20
PseudoRandom Example
Time multiple trials of each implementation
interface PseudoRandomImpl PseudoRandom
newInstance() Collection
impls final int NTRIALS //
trials/implementation final int NROLLS //
die rolls/trial for (PseudoRandomImpl impl
impls) long nanos 0 PseudoRandom rnd
for (int k 0 k impl.newInstance() nanos trial(NROLLS,
rnd) long avg nanos / (NTRIALS
NROLLS) if (rnd ! null) report(rnd.getClass(),
avg)
21
PseudoRandom Example
Test methodology disclaimer

• One machine, one JVM, one platform
• Other things that could affect results
• Packing of fields on objects
• Cache locality of generated code
• Thresholds for native compilation
• Background user and system processes

22
PseudoRandom Example
Results on 3.2 GHz Pentium 4, -client (Windows)
23
Memoize Example
Implement "Memoizer"

• Memo Function A function that memorizes its
  previous results
• Optimization for recursive functions, etc.
• Invented by Prof. Donald Michie, Univ. of
  Edinburgh
• Goal Implement memoizer
• Function wrapper
• Provide concurrent access
• Compute each result at most once
• Materials ConcurrentHashMap, FutureTask

24
Memoize Example
Generic computation and example of use
interface Computable V compute(A arg)
throws Exception class Function
implements Computable
public BigInteger compute(String arg) //
after deep thought... return new
BigInteger("2") Function f new
Function() // f Memoize(f) f.compute("1
1") 2
25
Memoize Example
Caching the computed result
class Memoize1 implements Computable
final Map cache new HashMap()
final Computable c Memoize1(Computable, V c) this.c c public synchronized V
compute(A arg) throws... if
(!cache.containsKey(arg)) cache.put(arg,
c.compute(arg)) return cache.get(arg)
Computable f f new
Function() f new Memoize1BigInteger(f) f.compute("1 1") 2
26
Memoize Example
No synchronization, but computes may overlap
class Memoize2 implements Computable
final Map cache new
ConcurrentHashMap() final ComputableV c Memoize2(Computable c) this.c
c public V compute(A arg) throws Exception
if (!cache.containsKey(arg))
cache.put(arg, c.compute(arg)) return
cache.get(arg)
27
Memoize Example
Overlapping computes less likely
class Memoize3 implements Computable
final Map cache new
ConcurrentHashMap() public V
compute(final A arg) throws Exception if
(!cache.containsKey(arg)) Callable
eval new Callable() public V
call() throws Exception return
c.compute(arg)
FutureTask ft new FutureTask(eval)
cache.put(arg, ft) ft.run() return
cache.get(arg).get()
28
Review Callables and Futures
Representing asynchronous tasks

• Callable is invoked and returns a value
• interface Callable V call() throws
  Exception
• Future holds result of asynchronous computation
• interface Future V get() throws
  InterruptedException, ExecutionException V
  get(long timeout, TimeUnit unit) throws IE,
  void cancel(boolean mayInterrupt) boolean
  isCancelled() boolean isDone()
• FutureTask is a Runnable Future
• class FutureTask implements Future,
  Runnable   FutureTask(Callable c)  
  

29
Memoize Example
Exactly one compute() for each argument value
class Memoize implements Computable
final ConcurrentMap cache
new ConcurrentHashMap() final
Computable c Memoize(Computable c)
this.c c public V compute(final A arg)
throws Exception Future f
cache.get(arg) if (f null)
Callable eval new Callable()
public V call() throws Exception
return c.compute(arg)
FutureTask ft new FutureTask(eval)
f cache.putIfAbsent(arg, ft) if (f
null) f ft ft.run() return
f.get()
30
Inside Memoize
Exactly one compute() for each argument value
Future f cache.get(arg) if (f null)
Callable eval new Callable() public
V call() throws Exception return
c.compute(arg) FutureTask ft
new FutureTask(eval) f cache.putIfAbsent(a
rg, ft) if (f null) f ft
ft.run() return f.get()
31
SwingWorker Example
Reimplementation of SwingWorker

• SwingWorker performs GUI-related work in a new
  thread
• Created by Hans Muller
• Goal Reimplement using java.util.concurrent
• Provide more features with less code
• Materials FutureTask, Executor

32
SwingWorker Revisited
Perform GUI-related work in a new thread
abstract class SwingWorker   protected
abstract V construct()   protected void
finished()   public void start()   public V
get() SwingWorker sw new
SwingWorker()   protected String
construct()     Thread.sleep(5000)    
return "Done"     protected void finished()
    label.setText(get())   label.setText(
"Working...") sw.start()
33
SwingWorker Revisited
Working..
Done
34
SwingWorker Reimplemented
abstract class SwingWorker   FutureTask
task     new FutureTask(new Callable()
      public V call() throws Exception
        return construct()          )
      protected void done()        
EventQueue.invokeLater(new Runnable()          
public void run()            
finished()                )    
protected abstract V construct() throws
Exception   protected void finished   public
void start()     new Thread(task).start()    
 public V get() throws IE, ExecutionException
    return task.get()   
35
Inside SwingWorker
Calls construct, then invokes finished
FutureTask task   new FutureTask(new
Callable()     public V call() throws
Exception       return construct()      )
    protected void done()      
EventQueue.invokeLater(new Runnable()        
public void run()           finished()       
    )
36
SwingWorker Revisited
Implements Future and Runnable
abstract class SwingWorker implements
Future, Runnable public void run()
task.run()   public V get() throws IE,
ExecutionException    return task.get()    
public V get(long timeout, TimeUnit unit) throws
...     return task.get(timeout, unit) 
public boolean cancel(boolean mayInterruptIfRunnin
g) ... public boolean isCancelled() ...
public boolean isDone()...
37
SwingWorker Revisited
Pluggable Executor
abstract class SwingWorker ... static
final Executor EXECUTOR new Executor()
public void execute(Runnable command)    new
Thread(command).start()   private
Executor executor EXECUTOR public void
setExecutor(Executor e) ... public Executor
getExecutor() ... public void start()   
executor.execute(this)
38
SwingWorker Revisited
Executor choices

• Parallel execution in cached thread pool
• SwingWorker sw
• Executor e
• e Executors.newCachedThreadPool()
• e.execute(sw)
• Sequential execution on a single thread
• e Executors.newSingleThreadExecutor()
• e.execute(sw)

39
SwingWorker Example
Summary

• Concurrency utilities enable simple, powerful
  implementation
• cancellation
• get with timeout
• pluggable executor
• SwingWorker Future Callable Runnable
  Executor
• Note Also see http//foxtrot.sourceforge.net

40
BoundedBuffer Example
Overview

• BoundedBuffer interface
• Implement BoundedBuffer with Condition
• Exercise the implementation
• Compare with other implementations
• Alternative approach for single producer / single
  consumer settings using Exchanger

41
BoundedBuffer Example
BoundedBuffer interface
interface BoundedBuffer / Adds
element to buffer, blocking until buffer
not full / void put(E element)
throws InterruptedException / Removes
element from buffer, blocks until buffer not
empty / E take() throws InterruptedExceptio
n
42
BoundedBuffer Example
Implement BoundedBuffer with Condition
class BoundedBufferUsingCondition
implements BoundedBuffer final Lock lock
new ReentrantLock() final Condition notFull
lock.newCondition() final Condition notEmpty
lock.newCondition() E items int
putIndex, takeIndex, size public void put(E
element) throws IE lock.lock() try
while (size items.length)
notFull.await() itemsputIndex
element size if (putIndex
items.length) putIndex 0
notEmpty.signal() finally / IE
handling omitted / lock.unlock()
// to be continued
43
Review Condition
Multiple wait sets

• Condition analogous to Object.wait/notify/notifyA
  ll
• interface Condition void await() throws
  IE void await(long timeout, TimeUnit unit)
  throws IE
  void awaitUninterruptibly() void signal()
  void signalAll()
• Create Conditions with a Lock
• interface Lock Condition
  newCondition()

44
BoundedBuffer Example
Implement BoundedBuffer with Condition, contd
// continued public E take() throws
InterruptedException lock.lock() try
while (size 0) notEmpty.await()
E element itemstakeIndex --size if
(takeIndex items.length) takeIndex 0
notFull.signal() finally / IE
handling omitted / lock.unlock()

45
WebCrawler Example
Overview

• Application of producer/consumer pattern
• Implemented using thread pool
• Problem Implementation blocks on fetch
• Solution Pool runs each fetch as task
• Wrinkle Need concurrent collections
• Wrinkle Denial of service
• Resolution Schedule periodic fetches
• Refinements

46
WebCrawler Example
Application of producer/consumer pattern
WebCrawler webCrawler BlockingQueue rq
// result queue Future crawl
webCrawler.crawl(startUrl, rq) try while
(!foundEnough()) URL found
rq.poll(timeout, unit) if (found null)
break process(found) finally
crawl.cancel(true)
47
Review BlockingQueue
Blocking version of java.util.Queue

• interface BlockingQueue extends Queue
  boolean add(E e) boolean offer(E e) boolean
  offer(E e, long timeout, TimeUnit unit)
  throws IE E
  poll(long timeout, TimeUnit unit) throws IE
• void put(E e) throws IE E take() throws IE
• int remainingCapacity()
• int drainTo(Collection elements)
  int drainTo(Collection elements,
  int maxElements)

48
WebCrawler Example
Implemented using cached thread pool
class WebCrawler final ExecutorService pool
Executors.newCachedThreadPool()
Future crawl(URL startUrl, BlockingQueue
rq) return pool.submit(new
CrawlTask(startUrl, rq)) class CrawlTask
implements Runnable / Gets URL
contents and parses list of URL links
from them may block indefinitely. /
List fetchLinks(URL url) throws IOException

49
WebCrawler Example
Problem Implementation blocks on fetch
class CrawlTask implements Runnable public
void run() for (seen.add(url) !done() url
poll()) try for (URL link
fetchLinks(url)) add(link) if
(!rq.offer(url, timeout, unit)) return
catch (IOException e) // skip over
url, couldnt get its contents
void add(URL u) if (!seen.contains(u))
seen.add(u) pq.offer(u) boolean done()
return url null URL poll() return
pq.poll() URL url startUrl final
Set seen new HashSet() final
Queue pq new PriorityQueue(,
searchOrder)
50
WebCrawler Example
Solution Pool runs each fetch as task
public void run() for (seen.put(url, true)
!done() url poll()) pool.execute(new
Runnable() public void run()
try for (URL link fetchLinks(url))
add(link) if (!rq.offer(url, timeout,
unit)) done true return
catch (IOException e) / skip /
) boolean done() return done url
null volatile boolean done
false continued on next slide
51
WebCrawler Example
Wrinkle Need concurrent collections
final BlockingQueue pq new
PriorityBlockingQueue(CAP,
searchOrder) final ConcurrentMap
seen new ConcurrentHashMapBoolean() void add(URL url) if
(seen.putIfAbsent(url, true) null)
pq.put(url) URL poll() url
pq.poll(timeout, unit)
52
Review PriorityBlockingQueue
BlockingQueue with least element available first

• class PriorityBlockingQueue implements
  BlockingQueue
• PriorityBlockingQueue(int initialCapacity,
  Comparator comp)
  

53
Review ConcurrentMap
Atomic get-and-maybe-set methods for Map

• interface ConcurrentMap extends Map
• V putIfAbsent(K key, V value) V replace(K
  key, V value) boolean replace(K key, V
  oldValue, V newValue) boolean remove(K key, V
  value)

54
WebCrawler Example
Wrinkle Denial of service
class PeriodicHostTask implements Runnable
final String host final Queue hq
new ConcurrentLinkedQueue() public void
run() // called periodically URL url
hq.poll() if (url ! null) pq.put(url)
void add(URL url) hq.offer(url)
55
WebCrawler Example
Resolution Schedule periodic fetches
final ScheduledExecutorService sched
Executors.newScheduledThreadPool(1) final
ConcurrentMap hosts
new ConcurrentHashMap()
void add(URL url) if (seen.putIfAbsent(url,
true) ! null) return PeriodicHostTask
hostTask new PeriodicHostTask()
PeriodicHostTask existing
hosts.putIfAbsent(url.getHost(), hostTask) if
(existing null) existing hostTask
sched.scheduleWithFixedDelay( hostTask, 0,
1, TimeUnit.SECONDS) existing.add(url)
56
Review ScheduledExecutorService
Periodic and one-shot delayed tasks

• interface ScheduledExecutorService
  extends ExecutorService
• ScheduledFuture schedule(Callable c,
  long delay,
  TimeUnit unit)
• ScheduledFuture schedule(Runnable r, long
  delay, TimeUnit unit)
• ScheduledFuture scheduleAtFixedRate(Runnable
  r, long initDelay,
  long period, TimeUnit unit)
• ScheduledFuture scheduleWithFixedDelay(Runnab
  le r, long initDelay,
  long delay, TimeUnit unit)

57
Summary

• Dont use low level constructs when more
  appropriate high level ones exist, e.g.,
• Use Queues, BlockingQueues, or Exchangers in
  producer/consumer designs.
• Use Executors instead of
• new Thread(runnable).start()
• Measuring performance of concurrent programs is
  hard, but must be done
• Try simple examples first, but remember that they
  wont necessarily scale
• Be skeptical instrument as much as possible to
  confirm that your program is really working

58
For More Information

• API docs for java.util.concurrent
• In Tiger download or on Sun website
• Doug Lea's concurrency-interest mailing list
• http//gee.cs.oswego.edu/dl/concurrent/dist/docs/i
  ndex.html
• Concurrent Programming in Java
• The Last Word in Swing Threads
• http//java.sun.com/products/jfc/tsc/articles/thre
  ads/threads3.html
• More articles at
• http//java.sun.com/products/jfc/tsc/articles/
  

59
QA
59
60
Supplemental Material
61
BoundedBuffer Example
Comparing three implementations
62
BoundedBuffer Example
Compare hand-rolled to library implementation
63
BoundedBuffer Example
ArrayBlockingQueue faster than hand-rolled!

• May not be significant
• Crude testing methodology (e.g., no warmup)
• Different results obtained using server
• If it is significant, a possible explanation
• HotSpot not caching final fields across code
  blocks
• Library implementations have been tuned
• Moral Dont reinvent the wheel!

64
BoundedBuffer Example
Alternative for 1 producer / 1 consumer settings
final DataBuffer ebuf , fbuf final
Exchanger exch new
Exchanger() exec.execute(new Runnable() //
producer DataBuffer buf ebuf public void
run() while (buf ! null)
putIntoBuffer(buf) if (buf.isFull()) buf
exch.exchange(buf) / IE handling
omitted / ) exec.execute(new Runnable() //
consumer DataBuffer buf fbuf public void
run() while (buf ! null)
takeFromBuffer(buf) if (buf.isEmpty()) buf
exch.exchange(buf) / IE handling
omitted / )
65
Review Exchanger
Atomic exchange between threads
class Exchanger T exchange(T x) throws
IE T exchange(T x, long timeout, TimeUnit
unit)
throws IE
66
WebCrawler Example
Refinements

• Tune thread pools
• Remove HostTasks after poll failure
• Need ScheduledFuture to cancel tasks
• Better approach using asynchronous I/O
• Reduces context switching overhead
• Could use java.nio.channels.Selector wrapped as a
  CompletionService
• Fetch requests return immediately
• Separate task polls completion service for next
  ready list of links

67
Concurrency Utilities in Practice
Using java.util.concurrent

• Tim Peierls
• Prior Artisans, LLC