Distributed Software Engineering

1
Concurrent access to Objects and variables
2
Concurrent access

• With multi-threaded programming, threads can
  access same data, controls need to be in place to
  ensure that their access has the desired effects
  and the data remains consistent.
• Problems expected
• Current thread get preempted halfway its
  execution for other thread to execute.
• While thread is preempted, other thread access
  same data (objects instance variables) and
  performs changes in data that was or was being
  manipulated by preempted thread.

3
Designing solutions

• Make operations atomic (to avoid race conditions
  two threads sending same message to same object
  at the same time and behavior of code depends on
  who wins)
• Do not allow a thread to be suspended in the
  middle of an operation while giving access to
  another thread to perform operations on same
  data.
• Warning these solutions create their own set of
  new problems.
• Hard news we need the solutions proposed but
  need to deal with the problems created by them.

4
Multi-threaded programming properties

• Safety nothing bad ever happens
• Safety failureUnintended behavior at run time
• Desired goal all object states are consistent.
• Liveness anything ever happens
• Liveness failure things just stop running
• Desired goal all threads have their chance to
  run to expected completion.
• These properties need to be balanced
• But better do nothing (liveness failure) that
  something that leads to a safety failure
• On the other hand there are many situations
  where is better to give an incorrect answer than
  not one at all.
• Warning some of the easiest thing to improve
  liveness property can destroy safety property and
  viceversa.
• Get them both right is the challenge in
  Multi-threaded programming.

5
Multi-threaded software properties

• On top of these two fundamental multi-threaded
  programming properties, we have the following two
  software eng. Properties
• Performance
• The extend to which activities execute soon and
  quickly.
• Reusability
• The utility of objects and classes across
  multiple contexts.

6
Programming gets harder

• Most multi-threaded safety matters cannot be
  checked automatically
• Must rely on the programmer and design being
  implemented
• Strong hint design up-front.
• Design vs. code ratio 10 1
• There are many methodologies to prove manually
  that designs are safe. We will cover some during
  this course.
• But most importantly safety properties relies
  strongly on the software engineering practices by
  the programmers.

7
Maintaining objects consistency

• Sometime hard to completely establish what is
  legal and meaningful for a class
• Better route establish conceptual-level
  invariants. (Example volume for water tank must
  be between zero and capacity)
• Objects consistency follows preservation of
  their invariants an object is consistent if all
  fields obey their invariants.
• To uncover invariants
• Clearly understand objects properties
• Constraints on objects representation
• Note safe objects may occasionally enter
  transiently inconsistent states, in the mist of
  methods but they never attempt to initialize new
  actions while inconsistent.

8
Importance of Objects invariants

• One reason for being more careful about
  invariants in multi-threaded programming it is
  much easier to break them than in sequential
  programming.
• Remainder the need of protection against
  inconsistency also arises in sequential
  programming
• Processing exceptions
• Making callbacks
• Self-calls between methods in a class.

9
Maintaining consistency Exclusion

• Guaranteeing atomicity
• Failures of maintaining atomicity
• Read/Write conflicts
• Write/Write conflicts
• Impossible to predict the consequences of actions
  when objects are inconsistent.

10
Liveness

• Goal every activity eventually progresses
  towards completion
• But an activity may fail (even transiently) for
  any number of possibly interrelated reason

11
Liveness failures

• Locking
• Waiting for a condition
• Input from another device
• CPU contention
• Failure due to exception, error or fault.
• But
• Temporal blocking of progress is expected, but
  this blocking time cannot be unbounded.

12
Potential permanent liveness failures

• Deadlock circular dependencies
• Thread A hold a lock for object X, and needs to
  acquire and lock object Y
• Thread B already locks object Y, and needs to
  acquire lock for object X.
• Missed signals thread started waiting after
  given notification to wake up was given.
• Nested monitor lockouts waiting thread holds
  lock needed by another thread attempting to wake
  it up.
• Livelock continuously retried action
  continuously fails.

13
Potential permanent liveness failures

• Starvation
• Resource exhaustion
• Distributed failure a remote server machine
  connected by a socket crashes or otherwise
  becomes inaccessible.

14
Performance

• Throughput number of ops/unit of time
• Latency time elapsed between issuing a message
  and its execution
• Capacity number of simultaneous activities
• Efficiency throughput/(comp resources)
• Scalability rate at which latency and throughput
  improve when resources are added to system.
• Degradation rate at which latency or throughput
  worsens as more clients, activities, operations
  are added without adding resources.

15
Efficiency trade-off

• Poorer efficiency for better latency, and
  scalability
• Locks
• Monitors
• Threads
• Context-switching
• Scheduling
• Locality
• Algorithmics
• All the above reduce efficiency due to overhead
  and contention that can slow down program.

16
Strategies for safety in objects

• Immutability
• Avoid state changes.
• Synchronization
• Dynamically ensuring exclusive access.
• Containment
• Structurally ensuring exclusive access.
• Variants
• Stateless methods
• Partial synchronization
• Managed ownership

17
Immutable Objects

• To avoid synchronization, do not change the state
  of the objects.
• Programs are easier to understand
• But, these programs are unable to handle user
  interactions, cooperating threads, etc.
• But we can at least search for selected
  immutability.

18
Simplest immutable objects

• Do not have state at all.
• Their methods are stateless they are given data
  through parameters. Such data is primitive type
  data or object reference which are not updated by
  the method.
• Class StatelessAdder
• public int add(int a, int b) return a b
• Methods are always safe and live

19
Simplest immutable objects

• This safety and liveness also hold for classes
  with only final fields.
• class ImmutableAdder
• public ImmutableAdder (int a) offset a
• public int addOffset(int b)
• return offset b
• private final int offset

20
Immutability in APIs

• ADTs implementing values colors, strings. These
  classes are called Value containers.
• Can create a numeric class where the methods
  never alter their fields but construct and return
  values as the result of the methods.
• Design different classes supporting different
  usages provide immutable and updateable
  versions String, StringBuffer.
• Protect objects via copying, when is rarely done
  and is cheap.

21
Immutability for sharing

• Immutability is a useful technique for sharing
  values and avoid costs associated from exclusion
  in multi-threaded programming. (MTP)
• Instances of many utility classes in MTP are
  intrinsically immutable and shared by many other
  objects.
• class Relay
• public Relay ( Server s) server s
• public void execute() server.execute()

22
Constructing immutable objects

• Fields must be final
• Do not allow fields to be accessed until
  construction is completed. (must lock)
• Field values must be copied (or can use
  references if values are immutable themselves)
• Constructors depending on the construction of
  other objects must relay on locking.

23
Immutability variant stateless methods

• A mutable object can provide services via
  strictly stateless methods, ie, have no bearing
  on the objects state.
• Stateless methods can be used as if they where
  methods of an immutable object.
• Stateless methods often make copies of arguments
  and results.
• Copying only works if it is to be used locally.
• Java does not provide pass-by-copy parameter
  passing mechanism, so it is left up to the
  programmer.

24
Basic concepts atomic operations (atomicity).

• Atomic operations can't be interrupted (divided)
• In java assigment to int, boolean is atomic
• But assignment to double or long is not atomic
• long x
• thread 1 may attempt to execute
• x 0x0123456789abcdef
• thread 2 may attempt to execute
• x 0
• possible results
• 0x0123456789abcdef
• 0x0123456700000000
• 0x0000000089abcdef
• 0x0000000000000000

25
64-bit assignment

• 64-bit assignment is effectively implemented as
• x.high 0x01234567
• x.low 0x89abcdef
• You can be preempted between the assignment
  operations, unless you lock.

26
Basic concepts synchronization

• Mechanism to provide locking or exclusion by
  other threads while a thread is executing some
  common code.
• Mechanisms to assure that multiple threads
• Start execution at the same time and run
  concurrently ("condition variables" or "events").
• Do not run simultaneously when accessing the same
  object ("monitors").
• Do not run simultaneously when accessing the same
  code ("critical sections").
• The synchronized keyword is essential in
  implementing synchronization (but is poorly
  designed.)
• No timeout, so deadlock detection is impossible.
• There is no way to determine if an object is
  already lock due to synchronization
• If an object attempts to lock an already lock
  object, it is placed in a list waiting to acquire
  the lock, but no thread can be removed from that
  list by client.

27
Example of need to exclude

• public class Even
• /
• ensure nVal() 2 0
• /
• public void next()
• n
• n
• public int nVal()
• return n
• private int n 0

Typical behavior This code might work well most
of the time, and possibly only rarely will
exhibit safety failure. When it does hard to
debug, as the inconsistency is usually manifested
in other unrelated objectstate.
Solution Declare next() synchronized. This
excludes access to this code by one thread when
other thread is executing it becomes atomic.
28
Locks and Objects

• Every object instance has ONE LOCK and only one.
• Scalar fields can only be locked via their
  enclosing object instance.
• Methods and blocks of code can be marked as
  synchronized fields are not.
• Arrays are objects as is Class object.
• Locking an array of objects does not
  automatically lock all its elements.
• There are NO constructs for simultaneously
  locking multiple objects in a single atomic op.

29
Lock ownership

• An objects lock is held (owned) by the thread
  executing synchronized code of the object.
• Thus, same thread can call other synchronized
  methods of the same object.

30
Block synchronization

• A block of code can be synchronized, but it takes
  an argument on which object to block. Very
  commonly the object is this
• synchronized(obj)
• //code to lock
• Semantics
• No other thread can execute block concurrently.
• No other synchronized method or block of code can
  be executed by any other thread on same object.
• Block synchronization can be done on any object
• Use it for protecting access to objects with no
  synchronized methods.
• For synchronizing a series of accesses to an
  object.

31
Synchronization and inheritance

• The synchronized keyword is not part of the
  method signature
• It is NOT a method modifier
• It is not inheritable
• Methods in interfaces cannot be specified
  synchronized
• Constructors can not be synchronized either, but
  block synchronization can be used within.
• Synchronized instance methods in subclasses
  employ the same lock as those in the superclass.
• Synchronizing an inner class method is
  independent of its outer class.
• Inner classes can use a block with the instance
  of the outer lock as the object to lock.

32
Acquiring and releasing lock

• Lock is acquired on entry to method or block.
• Lock is released on exit of synchronized code,
  including exit due to an exception.
• Locks are per-thread not per invocation of
  method.
• A thread hitting synchronized accesses such code
  if
• No thread has the lock on the object for such
  code. OR
• Same thread already possesses the lock of such
  object.
• Therefore, a thread possessing an objects lock
  can make recursive calls without being locked
  out.
• a synchronized method or block obeys these rules
  only with respect to other synchronized methods
  or blocks on the same object.
• Methods not synchronized still can execute at any
  time by any other thread even if synchronized
  code is executing.
• Any thread locked out due to a synchronized code
  will be place on hold in a queue
• When the objects lock is released, there is no
  guarantee on which thread will acquire the lock
  next. No fairness, but indeterminancy.

33
Static fields or methods and synchronization

• Object synchronization does not affect static
  class fields or methods
• Static class fields or methods are synchronized
  with the lock of the Class object for that class.
  
• When synchronizing a static method it acquires a
  different lock than the lock for any object of
  that class but all static methods will use same
  lock.
• A static block can be used against the lock of
  the object Class for that class.
• Example having class C
• synchronized (C.class) .

34
Fully synchronized objects

• Safest (but might not be the best) multi-threaded
  strategy is to restrict attention to fully
  synchronized objects
• All public methods are synchronized
• There are no public fields or other encapsulation
  violations
• All methods terminate
• All fields are initialized to a consistent state
  in the constructors
• State of object is consistent (obey invariants)
  at both the beginning and end of each method,
  even in the presence of exceptions.
• But this use costs and may lead to deadlocks.

35
Code execution exclusion via synchronization

• With synchronized code declaration, two or more
  threads are guarantee not to interfere with each
  other on the same object.
• If an object has synchronized methods and two
  threads invoke such methods simultaneously on the
  same object, synchronization guarantees exclusion
  but it does not guarantee order of execution.

36
Constructors and Synchronization

• Constructors are not synchronized because it is
  executed only when creating an object, which can
  only happen in one thread executing the creation.
• Constructors cannot be synchronized.

37
Synchronization

• Always lock during updates to objects.
• Always lock during access of possibly updated
  object fields.
• Never lock when invoking methods on other objects.

38
Basic concepts semaphores

• A semaphore is any object that two threads can
  use to synchronize with one another.

39
The mutex (mutual-exclusion semaphore)

• The mutex is the key to a lock
• Though it is sometimes called a lock.
• Ownership is the critical concept
• To cross a synchronized statement, a thread must
  have the key, otherwise it blocks (is suspended).
• Only one thread can have the key (own the mutex)
  at a time.
• Every Object contains an internal mutex
• Object mutex new Object()
• synchronized( mutex )
• // guarded code is here.

40
Monitors and airplane bathrooms

• A monitor is a body of code (not necessarily
  contiguous), access to which is guarded by a
  single mutex.
• Every object has its own monitor (and its own
  mutex).
• Think airplane bathroom
• Only one person (thread) can be in it at a time
• Locking the door acquires the associated mutex.
• You can't leave without unlocking the door.
• Other people must line up outside the door if
  somebody's in there.
• Acquisition is not necessarily FIFO order.

41
Synchronization with individual locks

• Enter the monitor by passing over the
  synchronized keyword I.e. acquiring an objects
  lock,
• Object has a synchronized method and a client
  invoked such method
• Client is executing a synchronized block on an
  object
• Entering the monitor does not restrict access to
  objects used inside the monitorit just prevents
  other threads from entering the monitor.
• long field
• Object lock new Object()
• synchronized(lock)
• field new_value

42
Method-level synchronization

• The monitor is associated with the object, not
  the code.
• Two threads can happily access the same
  synchronized code at the same time, provided that
  different objects receive the request.
• E.g. Two threads can enqueue to different queues
  at the same time, but they cannot simultaneously
  access the same queue
• Same as synchronized(this)
• class Queue
• public synchronized void enqueue(Object o)
  //
• public synchronized Object dequeue() //

43
He came in the Bathroom Window

• The Bathroom can have several doors
• class Bathroom_window
• private double guard_this
• public synchronized void ringo(double value)
  guard_this value
• public double george()// WRONG!
• return guard_this // needs synchronization

44
Constructors cant be synchronized, so always
have back doors

• class Unpredictable
• private final int x
• private final int y
• public Unpredictable(int init_x, int init_y)
  new Thread()
• public void run() System.out.println(x
  x
• y y)
• .start()
• x init_x
• y init_y
• Putting the thread-creation code at the bottom
  doesnt help (the optimizer might move it).

45
Locking the constructors back door

• class Predictable
• private final int x
• private final int y
• public Predictable(int init_x, int init_y)
• synchronized( this )
• new Thread()
• public void run()
• synchronized( Predictable.this)
  System.out.println(xx yy)
• .start()
• x init_x
• y init_y

46
Synchronization isnt cheap

• class Synch
• synchronized int locking ( int a, int b )
  return a b
• int not_locking ( int a, int b ) return a
  b
• static public void main(String arguments)
• Synch tester new Synch()
• double start new Date().get Time()
• for(long i 1000000 --i gt 0 )
• tester.locking(0,0)
• double end new Date().getTime()
• double locking_time end - start
• // repeat for not_locking

47
Synchronization isnt cheap

• java -verbosegc Synch
• Pass 0 Time lost 234 ms. 121.39 increase
• Pass 1 Time lost 139 ms. 149.29 increase
• Pass 2 Time lost 156 ms. 155.52 increase
• Pass 3 Time lost 157 ms. 155.87 increase
• Pass 4 Time lost 157 ms. 155.87 increase
• Pass 5 Time lost 155 ms. 154.96 increase
• Pass 6 Time lost 156 ms. 155.52 increase
• Pass 7 Time lost 3,891 ms. 1,484.70 increase
• Pass 8 Time lost 4,407 ms. 1,668.33 increase
• 200MHz Pentium, NT4/SP3, JDK 1.2.1, HotSpot
  1.0fcs, E
• Contention in last two passes (Java Hotspot
  cant use
• atomic-bit-test-and-set).

48
Synchronization isnt cheap

• BUT
• The cost of poor design is always higher than the
  cost of synchronization.
• Pick a fast algorithm.
• Overhead can be insignificant when the
  synchronized method is doing a time-consuming
  operation.
• But in OO systems, small synchronized methods
  often chain to small synchronized methods.

49
Avoiding synchronization

• Reentrant code doesnt need to be synchronized.
• Code that uses only local variables and arguments
  (no static variables, no fields in the class).
• Atomic operations do not need to be synchronized,
  but beware of reordering.
• Assignment to all non-64-bit things, including
  booleans and references are usually safe, but
  sequence not preserved.
• Must be declared volatile, but volatile might not
  work.
• Assignment to volatile doubles and floats should
  be atomic (but most JVMs dont do it).
• Code may be reordered, so assignment to several
  atomic variables must be synchronized.
• Sequence of volatile operations should be
  preserved, but usually isnt.

50
Avoiding synchronization

• Synchronize the smallest block possible to
  minimize the odds of contention.
• Method-level synchronization should be avoided in
  very-high-performance systems.
• Dont synchronize the methods of classes that are
  called only from one thread.
• Use Collection-style synchronization decorators
  when you need synchronized behavior.
• Collection c new ArrayList()
• c Collections.synchronizedCollection(c)

51
Avoiding synchronization

• Dont access synchronized methods from
  synchronized methods.
• Synchronize public methods. Dont synchronize
  private ones.
• Dont use protected.
• Avoid Vector and Hashtable in favor of Collection
  and Map derivatives.
• Dont use BufferedInputStream, BufferedOutputStrea
  m, BufferedReader, or BufferedWriter unless the
  stream is shared between multiple threads.
• You can use InputStreams read(byte)
• You can roll your own decorators.

52
Critical sections

• static final Object critical_section new
  Object()
• synchronized( critical_section )
• // only one thread at a time can execute this
  code
• A critical section is a body of code that only
  one thread can enter at a time.
• Do not confuse a critical section with a monitor.
• The monitor is associated with an object
• A critical section guards code
• The easiest way to create a critical section is
  by synchronizing on a static field

53
Critical sections can also synchronize onthe
class object

• class Flintstone
• public void fred()
• synchronized( Flintstone.class )
• // only one thread at a time can execute this
  code
• public static synchronized void wilma()
• // synchronizes on the same object as fred().

54
Class vs. instance variables

• All synchronized static methods synchronize on
  the same monitor.
• Think class variables vs. instance variables
• The class (static) variables and methods are
  effectively members of the Class object.
• The class (static) variables store the state of
  the class as a whole.
• The class (static) methods handle messages sent
  to the class as a whole.
• The instance (non-static) variables store the
  state of the individual objects.
• The instance (non-static) methods handle messages
  sent to the individual objects.

55
But remember the bathroom with multipledoors

• class Foo
• static long x 0
• synchronized static void set_x( long x )
• this.x x
• synchronized / not static / double get_x()
• return x
• Thread 1 Thread 2
• Foo o1 new Foo() Foo.set_x(-1)
• long x o1.get_x()
• Results are undefined. (There are two locks here,
  one on the class object and one on the instance.)

56
Lock the extra doors

• Access all static fields through synchronized
  static methods, even if the accessor is a method
  of the class that contains the field.
• class Okay
• private static long unsafe
• private static synchronized get() return
  unsafe
• private static synchronized set(long x) unsafe
  x
• public /not static/ void foo(long x)
• //...
• set(x)

57
Lock the extra doors

• Synchronize explicitly on the class object when
  accessing a static field from an instance method.
• class Okay
• private static long unsafe
• public void foo(long x) //...
• synchronized( Okay.class )
• unsafe x

58
Lock the extra doors

• Encapsulate all static fields in an inner class
  and provide exclusive access through synchronized
  methods of the inner class.
• class Okay
• private class Class_Variables
• private long unsafe
• public synchronized void do_something(long x)
• unsafe x //. . .
• static Class_Variables statics new
  Class_Variables()
• public foo(long x)
• statics.do_something( x )

59
Singletons (one-of-a-kind objects)

• Singletons often use critical sections for
  initialization.
• public final class Singleton
• staticnew JDK_11_unloading_bug_fix(Std.class)
  
• private static Singleton instance
• private Singleton() // prevent creation by new
• public synchronized static Singleton instance()
• if( instance null )
• instance new Singleton()
• return instance
• Singleton s Singleton.instance()

60
Synchronization and Collection API

• Methods in this collection are not synchronized
  by design to avoid overhead.
• There are several static methods in
  java.util.collections class that are used to wrap
  unsynchronized Collection instances with
  synchronized methods

61
Synchronization and Collection API

• static Collection synchronizedCollection(Collectio
  n c)
• Returns a synchronized (thread-safe)
  collection backed by the specified collection.
• static List synchronizedList(List list)
• Returns a synchronized (thread-safe) list
  backed by the specified list.
• static Map synchronizedMap(Map m)
• Returns a synchronized (thread-safe) map
  backed by the specified map.
• static Set synchronizedSet(Set s)
• Returns a synchronized (thread-safe) set
  backed by the specified set.
• static SortedMap synchronizedSortedMap(SortedMap
  m)
• Returns a synchronized (thread-safe) sorted
  map backed by the specified sorted map.
• static SortedSet synchronizedSortedSet(SortedSet
  s)
• Returns a synchronized (thread-safe) sorted
  set backed by the specified sorted set.

62
Thread deadlock

• Most common scenario
• Two threads are waiting for each other to finish
  doing something, so that locks are released.
• Nested monitor lockout
• (A one thread problem) The common situation
  occurs when you call a blocking function within a
  synchronized method, and the only way to release
  the block is to call another synchronized method.

63
Example

• public class BlackHole
• public synchronized void put( Object thing)
• queue.enqueue(thing)
• public synchronized void get ()
• return queue.dequeue()
• private Queue queue

64
Example

• This situation occurs anytime you have a blocking
  call ( such as a file read, a socket) inside a
  synchronized method.
• class BlackHole
• private InputStream input
• public blackHole()
• Input new Socket (www.cs.uno.edu,
  80).getInputStream()
• public synchronized int read()
• Return input.read()
• public synchronized void close()
• input.close()

65
Nested monitor blocks

• Do not make blocking calls in synchronized
  methods.
• Make sure there is a way to talk to the blocking
  object via another class or a non-synchronized
  method.

66
Deadlock on a Blocking I/O

• Once a read has been issued, it will block until
  data is read or there is an indication of end of
  file.
• If no data is coming in, read will block and
  there is no way to make it quit, short of
  aborting program.
• So, you must be very careful designing threads
  with I/O sequences of operations to execute.

67
Stopping threads

• stop() is deprecated for very good reasons.
• Use t.interrupt() to stop a thread.
• Your thread should test if is interruted.
• Class StopRight
• Private thread t new Thread()
• Public void run()
• Try
• While (!this.isInterrupted())
• // blocking call
• catch(InterruptedException e) //ignore
• Public stop() t.interrupt()

68
Exception and threads

• An uncaught exception in a thread will terminate
  the run() method, with a stack dump, but will not
  terminate the application.
• Not a good situation
• App user may not understand the stack dump.
• It may not be known that thread terminated
  abnormally.