Concurrency in Java

1

• Concurrency in Java
• University of Miskolc
• Info Klub
• András Brunczel
• 03.03.2009

2
Contents

• Overview
• Thread Safety
• Sharing Objects
• Design Guidelines
• Java API
• Task Cancellation
• Liveness
• Testing
• Reference
• Brian Goetz Java Concurrency In Practice
• Addison-Wesley, 2006

3
Overview

• Thread thread of execution
• Basic unit of scheduling by the operating system
• Threads can execute simultaneously and
  asynchronously
• Share the same memory space within their owning
  process
• Benefits of multithreaded applications
• Resource utilization (I/O vs. CPU bounded
  operations)
• Fairness (multiple users or tasks)
• Simplicity of modeling single tasks (e.g. J2EE)
• Exploiting multiple processors
• Handling of asynchronous events
• Responsive user interfaces

4
Overview

• A correct multithreaded application is much more
  difficult to design and implement
• In old programming langauges multithreading was
  an advanced feature but Java offers a simple API
  ?programmers use it carelessly
• Risks of multiple threads
• Safety hazard (nothing bad ever happens)
• Liveness hazard (something good eventually
  happens)
• Performance hazard (good things happen quickly)
• Implicit use of multithreading
• Timers
• Servlets
• RMI
• Swing

5
Creating Threads in Java

• class MyRunnable implements Runnable
• public void run() System.out.println("1")
• new Thread(new MyRunnable()).start()
• new Thread(new Runnable()
• public void run()
• System.out.println("2")
• ).start()

6
Thread Safety - Example

• A servlet that counts the number of requests
• Whats wrong?
• _at_NotThreadSafe
• public class UnsafeCountingFactorizer implements
  Servlet
• private long count 0
• public long getCount() return count
• public void service(ServletRequest req,
  ServletResponse resp)
• BigInteger i extractFromRequest(req)
• BigInteger factors factor(i)
• count
• encodeIntoResponse(resp, factors)

7
Thread Safety Example (cont.)

• Lets correct it.
• Whats wrong now?
• _at_ThreadSafe
• public class SynchronizedFactorizer implements
  Servlet
• _at_GuardedBy(this)
• private long count 0
• public long synchronized getCount()
• return count
• public void synchronized service(ServletRequest
  req, ServletResponse resp)
• BigInteger i extractFromRequest(req)
• BigInteger factors factor(i)
• count
• encodeIntoResponse(resp, factors)

8
Thread Safety Example (cont. 2)

• Correct solution
• _at_ThreadSafe
• public class CountingFactorizer implements
  Servlet
• private AtomicLong count new AtomicLong(0)
• public long getCount() return count.get()
• public void service(ServletRequest req,
  ServletResponse resp)
• BigInteger i extractFromRequest(req)
• BigInteger factors factor(i)
• count.incrementAndGet()
• encodeIntoResponse(resp, factors)

9
Thread Safety - Definitions

• A code is thread-safe if it behaves correctly
  when accessed multiple threads regardless of
  scheduling and without additional synchronization
  of the calling code
• Two operations are atomic respect ot each other
  if while one is executing in one thread then the
  other has not started or it has already finished.
• An operation is atomic if it is atomic with
  respect to all operations, including itself, that
  operate on the same state
• Race condition result depends on timing (e.g.
  check-then-act operations, like initializing a
  singleton)

10
Thread Safety Definitions (cont.)

• Object state
• Mutable / Immutable objects
• Locks
• synchronized (lock)
• public synchronized doSomething() ? lock is
  the class instance
• Guarding a state with a lock
• Locks are reentrant if the lock is already held
  then the execution can enter a synchronized block
  guarded by the same lock

11
Thread Safety

• If multiple threads can access the same mutable
  state without snychronization, the program is
  broken. (access means read or write!)
• Ensuring thread safety
• Do not share the state (encapsulation)
• Make the state immutable (immutable objects are
  thread-safe)
• Use synchronization whenever accessing the state
  (mutable state variables must be guarded by
  exactly one lock)

12
Thread Safety

• Liveness and performance avoid holding locks
  during time-consuming operations
• _at_ThreadSafe public class CachedFactorizer
  implements Servlet
• _at_GuardedBy("this") private BigInteger
  lastNumber
• _at_GuardedBy("this") private BigInteger
  lastFactors
• public void service(ServletRequest req,
  ServletResponse resp)
• BigInteger i extractFromRequest(req)
• BigInteger factors null
• synchronized (this)
• hits
• if (i.equals(lastNumber))
• factors lastFactors.clone()
• if (factors null)
• factors factor(i)
• synchronized (this)
• lastNumber i

13
Sharing Objects - Publication

• Synchronization is not enough if the objects
  being modified during an atomic operation are
  visible by other threads
• Unsafe publication (publishing)
• class UnsafeStates
• private String states new String
• "AK", "AL" /.../
• public String getStates() return states
• Escaping reference to this during construction
• public class ThisEscape
• public int number
• public ThisEscape(EventSource source)
• source.registerListener(new EventListener()
• public void onEvent(Event e)
  doSomething(e)
• )
• number 10
• (Inner classes contain a hidden reference to the
  enclosing instance.)

14
Sharing Objects - Publication

• How to corrupt the previous example
• public class Concurrency
• public static void main(String args)
• new ThisEscape(new Concurrency().new
  MyEventSource())
• SafeListener.newInstance(new Concurrency().new
  MyEventSource())
• class MyEventSource implements EventSource
• public void registerListener(EventListener
  e)
• Field f e.getClass().getDeclaredField(
  "this0")
• Object enclosingObject f.get(e)
• System.out.println(
• enclosingObject.getClass().
• getField("number").get(enclosingObject))

15
Sharing Objects - Publication

• Avoiding the previous problem use a factory
  method and private constructor!
• public class SafeListener
• private final EventListener listener
• public int number
• private SafeListener()
• listener new EventListener()
• public void onEvent(Event e)
• doSomething(e)
• number 10
• public static SafeListener newInstance(EventSourc
  e source)
• SafeListener safe new SafeListener()
• source.registerListener(safe.listener)
• return safe

16
Sharing Objects - Publication

• Unsafe publication
• public class Holder
• private int n
• public Holder(int n) this.n n
• public void assertSanity()
• if (n ! n)
• throw new AssertionError("This statement is
  false.")
• public Holder holder
• public void initialize() holder new
  Holder(42)
• holder.assertSanity() can throw AssertionError
  when calling in other thread than the one that
  executes initialize()!
• Reason see later

17
Sharing Objects

• What is printed?
• public class NoVisibility
• private static boolean ready
• private static int number
• private static class ReaderThread extends Thread
  
• public void run()
• while (!ready)
• Thread.yield() System.out.println(number)
  
• public static void main(String args)
• new ReaderThread().start()
• number 42
• ready true

18
Sharing Objects Stale Data

• Without synchronization Java runtime can execute
  operation in weird order ?every time a variable
  is accessed it is possible to see a stale (not
  up-to-date) value
• 64-bit variables (double, long) even worse, the
  value can be not only stale but even corrupt
  (high 32 bits and low 32 bits are not consistent)
• How to avoid this?
• Synchronization
• Volatile variables (see next page)

19
Sharing Objects Volatile Variables

• Read of a volatile variable always returns the
  most recent write by any thread
• Operation on a volatile variable cannot be
  reordered with other memory operations
• Lighter-weight synchronization no locking during
  accessing volatile variables
• Locking gurantees visibility and atomicity
• Volatile variables gurantees only visibility
• ?Use with care!

20
Sharing Objects Volatile Variables

• Good example
• volatile boolean asleep
• ...
• while (!asleep)
• countSomeSheep()
• Bad example
• volatile int maxIndex
• volatile double maxValue
• double numbers
• for (int i 0 i ltnumbers.length i)
• if (numbersi gt maxValue)
• maxIndex i
• maxValue numbersi

21
Sharing Objects Confinement

• Thread confinement an object can be accessed
  only by one thread ?no synchronization is needed
• Example in Swing the visual component objects
  are not thread-safe but they can be accessed only
  by the event dispatch thread
• Stack confinement an object can be accessed only
  by local variables (local variables exist only in
  the executing thread, special type of thread
  confinement)
• ThreadLocal class implementation of thread
  confinement
• private ThreadLocalltConnectiongt connectionHolder
  
• new ThreadLocalltConnectiongt()
• public Connection initialValue()
• return DriverManager.getConnection(DB_URL)
• public Connection getConnection()
• return connectionHolder.get()

22
Sharing Objects Immutability

• An object is immutable if
• Its state cannot be modified after construction
• All its fields are final
• It is properly constructed (the this reference
  does not escape during construction)
• Immutable objects are thread-safe.
• Use final keyword if possible even if the whole
  object is mutable

23
Sharing Objects Immutability

• Using volatile reference to the immutable
  OneValueCache object
• public class VolatileCachedFactorizer implements
  Servlet
• private volatile OneValueCache cache new
• OneValueCache(null, null)
• public void service(ServletRequest req,
  ServletResponse resp)
• BigInteger i extractFromRequest(req)
• BigInteger factors cache.getFactors(i)
• if (factors null)
• factors factor(i)
• cache new OneValueCache(i, factors)
• encodeIntoResponse(resp, factors)

24
Sharing Objects - Summary

• How to publish an object safely?
• Initialize an object in a static initializer
• Store the reference to the object as volatile or
  AtomicReference
• Store the reference to the object in a final
  field of a properly constructed object
• Store the reference in a field guarded by a lock

25
Design Guidelines

• Steps of thread-safe design
• Identify the state variables of the object
• Identify invariants on these state variables
• Establish a policy for managing concurrent access
  to the state
• Document to the users of the class whether it is
  thread-safe or not
• Document to the maintainer of the class what
  synchronization policy is used
• Annotations can help
• Sample annotations
• _at_GuardedBy(this)
• _at_ThreadSafe

26
Java API Synchronized collections

• Synchronized collection classes (since 1.2)
• java.util.Vector
• java.util.Hashtable
• java.util.Collections.synchronizedXXX()
• Iterating on synchronized collections is
  fail-fast, i.e. next() or hasNext() throws
  ConcurrentModificationException if the collection
  was modified during the iteration
• Thus client side locking is required in some
  cases
• Iteration
• Check-then-act type of operations
• The object itself can be used as the lock (it is
  documented only in Java 5.0). E.g.
• public static Object getLast(Vector list)
• synchronized (list)
• int lastIndex list.size() - 1
• return list.get(lastIndex)

27
Java API More about iteration

• Following operations can also throw
  ConcurrentModificationException
• For-each loop (it is implemented with an
  iterator)
• toString() methods (use iterators)
• containsAll()
• removeAll()
• etc
• Locking the whole iteration might have poor
  performance or can lead to deadlock
• Alternative solution cloning the collection and
  iterate on the clone

28
Java API Concurrent collections

• java.util.concurrent (since 5.0)
• ConcurrentHashMap
• ConcurrentLinkedQueue
• CopyOnWriteArrayList
• Finer-grained locking mechanism lock striping.
  E.g
• public Object get(Object key)
• int hash hash(key)
• synchronized (lockshash N_LOCKS)
• for (Node m bucketshash m!null mm.next)
• if (m.key.equals(key))
• return m.value
• Replacing synchronized collections with them can
  result big scalability improvement
• ConcurrentHashMap implements ConcurrentMap that
  adds some check-then-act type operations to Map
  (putIfAbsent, ...). Client side locking could not
  be used here to implement these operations.

29
Java API Blocking Queues

• java.util.concurrent.BlockingQueue
• put() blocks until queue is full (reached its
  specified capacity)
• take() blocks unitl the queue is empty
• Supports the implementation of the
  producer-consumer pattern
• The producer-consumer pattern can be used for
  resource management in multithreaded application
• Dequeue and BlockingDequeue
• Double-ended queue
• In producer-consumer pattern it can be used for
  work stealing

30
Java API Synchronizers

• java.util.concurrent.CountDownLatch execution
  continues if a certain number of activities have
  been completed
• One-time used object
• Can be used in multithreaded performance
  measurements
• public class TestHarness
• public long timeTasks(int nThreads, final
• Runnable task) throws InterruptedException
• final CountDownLatch startGate
  new CountDownLatch(1)
• final CountDownLatch endGate new
  CountDownLatch(nThreads)
• for (int i 0 i lt nThreads i)
• Thread t new Thread()
• public void run()
• try
• startGate.await()
• try task.run()
• finally endGate.countDown()
• catch (InterruptedException ignored)
  
• t.start()

31
Java API Synchronizers

• java.util.concurrent.CallableltVgt - like the
  Runnable interface but it has a return value
• java.util.concurrent.FutureTask
• initialized with a Callable
• Started by encapsulating to a thread
• new Thread(futureTask).start()
• FutureTask.get() blocks until the result is
  available and then returns the result
• If interrupted then throws InterruptedException
• In case the computation throws exception then it
  throws ExecutionException (wraps Error,
  RuntimeException and Exception)

32
Java API Synchronizers

• java.util.concurrent.Semaphore
• Controls the number of activities that can access
  a certain resource at the same time
• Can be used to implement resource pools
• Maximum number of permits is specified in the
  constructor
• Semaphore.acquire() acquires one permit
• Semaphore.release() releases a permit
• Binary semaphore number of permits 1 (same
  functionality as the synchronized blocks)

33
Java API Synchronizers

• java.util.concurrent.CyclicBarrier - can be used
  as a rendezvous point, threads can continue at
  the same time, when all have reached this point
• Multi-use obejct any number of rendezvous
  points can follow each other
• Optionally a certain action can be performed at
  the rendezvous points specified as a Runnable
  at the constructor

34
Java API Example cache implementation

• public class Memoizer ltA, Vgt implements
  ComputableltA, Vgt
• private final ConcurrentMapltA, FutureltVgtgt
  cache new ConcurrentHashMapltA, FutureltVgtgt()
• private final ComputableltA, Vgt c
• public Memoizer(ComputableltA, Vgt c) this.c
  c
• public V compute(final A arg) throws
  InterruptedException
• while (true)
• FutureltVgt f cache.get(arg)
• if (f null)
• CallableltVgt eval new
  CallableltVgt()
• public V call() throws
  InterruptedException return c.compute(arg)
• FutureTaskltVgt ft new
  FutureTaskltVgt(eval)
• f cache.putIfAbsent(arg, ft)
• if (f null) f ft
  ft.run()
• try return f.get()
• catch (CancellationException e)
  cache.remove(arg, f)
• catch (ExecutionException e)

35
Java API Executor Framework

• Executor how to assign tasks to threads
• Can be created by using java.util.concurrent.Execu
  tors.
• newFixedThreadPool()
• newCachedThreadPool()
• newSingleThreadExecutor()
• newScheduledThreadPool()
• Always when you see new Thread(runnable).start()
  consider using an Executor instead, since it is
  more flexible
• Executors support shutdown

36
Task Cancellation

• Java does not provide any mechanism to safely
  cancel tasks
• Thread.interrupt() tells the thread to cancel
  its operation in a suitable time
• Inside the interrupted thread InterruptedException
  is thrown when calling interruptible blocking
  methods like Thread.sleep(), BlockingQueue.put()
• Two choices to handle the InterruptedException
• Rethrow it (you make your method interruptable,
  too)
• Propage it Thread.currentThread.interrupt()
• Do not swallow it, unless your class extends
  Thread!
• Do not interrupt a thread if you do not know its
  interruption policy

37
Task Cancellation (cont.)

• How to cancel?
• Cancellation flag is checked in a loop
• Use the Future interface , and its cancel method
• Use the Executor framework shutdown mechanism
• Poison pills in case of producer-consumer pattern

38
Liveness - deadlock

• Liveness the program eventually does something
  good
• Deadlock the threas-resource dependency graph
  is cyclical
• Lock-ordering deadlock. E.g dining philosophers
• Avoiding lock-ordering deadlock locks are
  acquired in a fixed global order.
• Calling an alien method with a lock held is risky
• Techniques to avoid deadlocks
• use open calls (no lock held during method calls)
• Use only one lock in the whole program
• Use timed lock (java.util.concurrent.locks
  package)

39
Liveness Other Liveness Hazards

• Starvation
• a thread can make no progress
• Thread piorities might help but use them
  carefully
• Thread piorities are handled in OS specific way
• It can cause other liveness problems
• Poor responsiveness
• CPU-intensive background tasks (altering thread
  priorities might help here)
• Poor lock management
• Livelock
• Threads are not blocked but still make no
  progress
• E.g. after exception during message processing
  the message is put back to the head of the queue
• E.g. package collision in ethernet network.
  Solution random retry delay

40
Testing

• Concurrent programs are nondeterministic, thus
  testing is more difficult
• Problem might occur rarely or only in certain
  circumstances
• Three types of tests
• Safety
• Liveness
• Performance (throughput, response time,
  scalability)

41
Testing - Advices

• The test code can introduce timing and
  synchronization artifacts that might hide bugs
• If possible avoid synchronization in test code
• Always start with testing correctness in single
  threaded environment
• If the test uses several threads, use e.g.
  CyclicBarrier to reach real concurrency
• Run the test in multiprocessor environment but
  there should be more active threads than CPUs
• Use a static code analyzer tool to find typical
  concurrency related bugs (e.g. http//findbugs.sou
  rceforge.net)

42
Testing Test Blocking

• How to test that a method blocks when necessary?
• void testTakeBlocksWhenEmpty()
• final SemaphoreBoundedBufferltIntegergt bb
  
• new SemaphoreBoundedBufferltIntegergt(10)
• Thread taker new Thread()
• public void run()
• try
• int unused bb.take()
• fail() // if we get here,
  it's an error
• catch (InterruptedException
  success)
• try
• taker.start()
• Thread.sleep(LOCKUP_DETECT_TIMEOUT)
• taker.interrupt()
• taker.join(LOCKUP_DETECT_TIMEOUT)
• assertFalse(taker.isAlive())

43
End

• Thanks for your attention!
• Questions?
• E-mail andras.brunczel_at_evosoft.com