CSCI0320 Introduction to Software Engineering

1
CSCI0320Introduction to Software Engineering
Lecture 5 Introduction to Threads
June 28, 2009
Lecture 5
Page 1
2
The Dark Ages
Memory
0
Kernel
User Program
CPU
Disk
PC
User Data
2n
June 28, 2009
Lecture 5
3
Time Sharing
Memory
0
Disk
Kernel
0
PC
User A Computer
PC
2a
PC
User B Computer
0
2b
0
User C Computer
2c
2n
June 28, 2009
Lecture 5
4
Multiple Processes
Memory
0
Disk
Kernel
0
PC
User A Program A
PC
2a
PC
Pipes
User A Program B
0
Sockets
2b
0
User B Program C
2c
2n
June 28, 2009
Lecture 5
5
Virtual Memory
Memory
0
Disk
Kernel
PC
PC
PC
Pipes
Sockets
2n
June 28, 2009
Lecture 5
6
Virtual Memory

• Memory could be shared
• Shared libraries read-only
• Writable shared memory
• Used for communications
• Let processes interact quickly
• Typically processes share writable
• Only a few pages (for communications)
• The rest is private
• What happens if processes share all
• Both initially and as they grow
• The result is multiple threads

June 28, 2009
Lecture 5
7
Multiple Threads
Program
0
User Program
PC
PC
PC
2n
June 28, 2009
Lecture 5
8
Uses of Threads

• Dump program
• Read from multiple disks
• Write to multiple tape drives
• Need to keep all the tape drives busy
• But tape writes, disk reads
• Take different lengths of time
• Not predictable
• Result is a very messy program
• But is trivial if you use threads

June 28, 2009
Lecture 5
9
Uses of Threads

• SOLAR
• Your code does physics computations
• Continuously
• Graphics needs to update 30 times/sec
• Independent of the computations
• User interface code needs to respond
• Immediately to mouse or keyboard
• Messy control to do all of these together
• Simple control to think of 3 threads
• Each doing its thing in a loop

June 28, 2009
Lecture 5
10
Uses of Threads

• GALAXY
• Your computations are more complex
• But most are independent of each other
• Can do multiple computations at once
• The machine has multiple processors
• But you have to accumulate forces
• Globally for each object
• Multiple threads doing computation
• Should speed up the computation
• Need to rearrange the oct tree as well

June 28, 2009
Lecture 5
11
Use of Threads

• Browsers (Firefox, IE, Safari)
• The browser is doing multiple things
• Downloading multiple URLs simultaneously
• Updating layouts
• Handling multiple page requests
• Ever notice how it hangs while waiting
• Even if what you want to do isnt what it is
  waiting for
• This is where multiple threads would help

June 28, 2009
Lecture 5
12
Uses of Threads

• Web Servers (Apache)
• What does a web server do
• Handle multiple simultaneous clients
• Each client is essentially independent
• Simplest design
• Process each client in a separate thread
• Start a thread for each request
• This doesnt work (WHY?)
• Pool of worker threads
• Queue of requests
• Workers take next request off the queue, process,
  repeat

June 28, 2009
Lecture 5
13
Computers Today

• Multiple CPUs in a single box
• Fred, Fred3, Internet lab back end
• Most servers, some workstations
• Multiple CPUs on a single chip
• Dual cores are standard today
• Quad cores are common, 6 are available
• Higher number of cores are coming
• How to take advantage of this
• Multiple virtual machines
• Multiple processes
• Multiple threads

June 28, 2009
Lecture 5
14
Why Not Threads

• Programming is more complex
• Considerably
• Different conceptual model
• Need to think differently about the program
• New problems are introduced
• Programs become nondeterministic
• Programs become unpredictable
• Much harder to debug and test
• Problems can be very subtle and temporal
• Care needed to get things right

June 28, 2009
Lecture 5
15
Threads in Java

• Inherit from class java.lang.Thread
• Implement public void run()
• Or pass in a Runnable
• Implementation notes
• Name all the threads (pass name to super)
• Need to start a thread as well
• Call start() on the thread
• Causes run method to be executed in the future
• Threads are resource intensive
• Require a OS thread
• Require a stack

June 28, 2009
Lecture 5
16
Thread Problems

• Race Conditions
• Coordination
• Deadlock
• Overhead

June 28, 2009
Lecture 5
17
Race Conditions

• Two threads competing for a resource
• Having a race so to speak
• Consider an event counter
• int nextCount()
• count count 1
• return count
• Whiteboard simulate
• One thread
• Four threads

June 28, 2009
Lecture 5
18
Race Conditions

• What does the counter do
• Does it always increase?
• Does it remain monotonic?
• This is a simple shared structure
• Think about what can happen with
• Lists, maps, queues, trees,

June 28, 2009
Lecture 5
19
Other Resources

• Memory (C/C)
• What if two threads want to allocate memory at
  the same time
• Need to coordinate what is free and what isnt
• Need to coordinate how to grow the program
• Files
• Two threads wanting to use the same file
• At the same time
• What gets read/written
• Display, Keyboard and Mouse

June 28, 2009
Lecture 5
20
Galaxy Races

• Resources are the objects
• Draw thread needs to access properties
• Compute thread needs to access set these
• Can the properties be inconsistent?
• Does it matter?
• Will it affect the computation?
• How will it affect the display?
• If the GUI could change mass, radius, etc?

June 28, 2009
Lecture 5
21
Avoiding Race Conditions

• Need to determine the winner
• In a consistent and fair manner
• So that everyone knows who won
• Typically done with critical regions
• Portions of code where we guarantee that only one
  thread executes at a time
• Mediator decides who gets to run
• Example
• Incrementing the counter

June 28, 2009
Lecture 5
22
Critical Regions

• Various techniques have been proposed
• Some suitable for hardware implementation
• Test and set
• Compare and swap
• Some built for software
• Mutex or Lock
• Mutual exclusion
• Operations lock and unlock
• Only one thread can hold a lock
• Others wait if they try to acquire it

June 28, 2009
Lecture 5
23
Critical Regions

• Semaphores
• Mutex with a counter attached
• Lock decrement the counter if gt 0, else wait
• Unlock increment the counter
• Useful when there are k copies of a resource
• Read/Write Locks
• Lock for read/lock for write/unlock
• Readers block writer
• Writers block everyone
• Multiple simultaneous readers allowed
• Useful for read-heavy workloads
• Upgrading a read lock is tricky!

June 28, 2009
Lecture 5
24
Monitors

• More complex critical region
• Essentially code-control of who is locked out
• Rather than none or counters or access type
• This is the Java language provides
• The others we will get back to with C/C
• They are also provided by Java libraries
• In java.util.concurrent
• Java Provides basic control operations
• Waiting for condition to hold
• Notifying others of change in condition

June 28, 2009
Lecture 5
25
Java Constructs

• Synchronized regions
• synchronized (object)
• Every object can act as a lock
• Each object is a different lock
• Only one thread can hold that lock at a time
• A thread acquires the lock to enter the body of
  the synchronized statement
• Simple use makes it a mutex based on object
• Synchronized methods
• synchronized keyword in front of method
• Equivalent to synchronized (this) for body
• What happens with a static method?

June 28, 2009
Lecture 5
26
Examples

• Counter
• synchronized nextCount()
• return count
• Data Structures
• List
• add()/remove() in synchronized regions
• Typically synchronize on the entire object
• Can you safely do lookups without synch?
• Can you iterate without synch
• Only if you copy the list first
• What is the difference between Vector
  ArrayList?
• Maps, Sets, and other collections
• Your oct-trees?

June 28, 2009
Lecture 5
27
Examples

• User interface code
• The user interface runs in the AWT thread
• Created by Java during the first swing/awt call
• Used for all the callbacks (so don't waste time!)
• Any changes to the UI either
• Have to be synchronized with the UI thread
• This can be difficult to do
• Or have to be done within the UI thread
• This is easier
• Actions typically done in response to UI
• SwingUtilities.invokeLater
• Javax.util.swing.Timer

June 28, 2009
Lecture 5
28
Examples

• File I/O
• Files are not inherently thread-safe
• A file should be used by a single thread
• Or operations must be synchronized
• Log files are problematic
• Can use synchronization
• If a debugging log, can allow messed up output
• POSIX specifies some guarantees, but libraries
  obfuscate details.
• Memory allocation
• Java new is thread safe
• C/C malloc/new is safe if you ask nicely

June 28, 2009
Lecture 5
29
Thread Coordination

• Threads need to interact with each other
• Galaxy
• Do the gravity computations in parallel
• Then do the updates given all the forces
• Must wait for all gravity computations to finish
• Then check for revising the oct-tree
• Must wait for all updates to finish
• Threads need to coordinate
• Can be done via synchronized regions
• But this is tricky
• There are more direct constructs.

June 28, 2009
Lecture 5
30
Fork/Join

• Think of operations on threads
• Fork create and start a new thread
• Join wait for a thread to exit
• Common operation
• Structured in some languages
• Fork set of threads
• Java is unstructured
• Thread.join() or Thread.join(timeout)
• Waits for a thread to finish
• Use in Java
• How would you join multiple threads?

June 28, 2009
Lecture 5
31
Wait-Notify

• A thread can wait for an event to occur
• Coordination without termination
• Wait for another thread to provide needed data
• Wait for user to push a button
• Wait for timer (frame rate)
• Java provides this in a limited way
• Event must be recognized by another thread
• Java Implementation
• Wait by calling wait() or wait(timeout)
• Notify by calling notify()
• Or notifyAll()

June 28, 2009
Lecture 5
32
Wait-Notify

• Implementation is slightly tricky
• Wait on an object, not a condition
• Wait must occur while synchronized on it
• Then object.wait() is called
• Notify must occur on the same object
• And also while synchronized by that object
• notify versus notifyAll
• Typical code pattern
• synchronized (object)
• while (not_ready)
• object.wait()

June 28, 2009
Lecture 5
33
Producer-Consumer

• Consumer
• Gets next item from queue
• Processes that item
• Code
• while (true)
• synchronized (queue)
• while (queue.isEmpty())
• try
• queue.wait()
• catch (InterruptedException e)
• next queue.remove()
• next.process()

June 28, 2009
Lecture 5
34
Producer-Consumer

• Producer
• Adds items to queue
• Code
• while (true)
• item ltget next itemgt
• synchronized (queue)
• queue.add(item)
• queue.notify()

June 28, 2009
Lecture 5
35
Dining Philosophers

• Five diners at table
• Need 2 forks to eat
• Only 5 on table
• Each picks up left
• Then the right
• Then eats
• Then puts both down
• What happens?
• How can you fix it?

June 28, 2009
Lecture 5
36
Deadlocks

• Synchronized regions introduce problems
• Thread 1 Thread 2
• Lock A
• Lock B
• Lock B
• ltwaitgt Lock A
• ltwaitgt ltwaitgt
• This is Deadlock
• Doesnt just affect locks
• Wait for I/O from another thread/process

June 28, 2009
Lecture 5
37
Avoiding Deadlocks

• Be careful. Keep code simple!
• If you require multiple locks, order them
• Lock the minimum amount needed
• Avoid synchronized methods
• Unless they are trivial
• Avoid calls inside locked regions
• Avoid nested locks
• Avoid doing I/O inside locked regions
• Avoid Thread.sleep()

June 28, 2009
Lecture 5
38
Other Problems

• Synchronization isnt free
• Actually quite expensive
• Can slow code down by 2x 10x
• Synchronization isnt obvious
• Determining what is or isnt shared is hard
• Things are often overlooked
• Concurrent modifications
• Iterators often overlooked
• Libraries vary
• May or may not be thread safe
• Poorly documented

June 28, 2009
Lecture 5
39
Using Threads

• Getting the most out of threads is tricky
• Figuring out where to use threads is hard
• Some things are obvious
• But most potential uses arent
• Bottom line
• Use threads where appropriate
• Use them carefully
• Minimize (mutable) data sharing

June 28, 2009
Lecture 5
40
Next Time

• Designing with threads

June 28, 2009
Lecture 5