MET TC670 B1 Computer Science Concepts in Telecommunication Systems

1
MET TC670 B1Computer Science Concepts in
Telecommunication Systems

• Fall 2003

2
Lecture 2, September 16, 2003

• About Programming Projects
• Processes
• Threads
• Scheduling Policies

3
Programming Projects

• Still required (20 of final grade)
• Format I will provide some incomplete programs
  you will read/finish them, write a document about
  it.
• Help A half of the next lecture will focus on
  computer programming (may help you
  understand/write program).
• Exception (not encouraged) You may write a term
  paper (I will assign several topics for
  choosing), but I will set a high bar.

4
Lecture 2, September 16, 2003

• About Programming Projects
• Processes
• Threads
• Scheduling Policies

5
The Process

• The process is the OSs abstraction for execution
• the unit of execution
• the unit of scheduling
• the dynamic (active) execution context
• compared with program static, just a bunch of
  bytes
• Process is often called a job, task, or
  sequential process
• a sequential process is a program in execution
• defines the instruction-at-a-time execution of a
  program

6
Process versus Program
Process
Program
A run of the binary code
Binary code, i.e., a list of Instructions (an
executable file)
Dynamic or active (it is running, at different
times different instructions are executed)
Static or passive (stored on disks and the system
may run it later)
7
An Example
8
Whats in a Process?

• A process consists of (at least)
• an address space
• the code for the running program
• the data for the running program
• an execution stack and stack pointer (SP)
• traces state of procedure calls made
• the program counter (PC), indicating the next
  instruction
• a set of general-purpose processor registers and
  their values
• a set of OS resources
• open files, network connections, sound channels,
  
• The process is a container for all of this state
• a process is named by a process ID (PID)

9
A processs address space
0xFFFFFFFF
stack (dynamic allocated mem)
SP
heap (dynamic allocated mem)
address space
static data (data segment)
code (text segment)
PC
0x00000000
10
Process states

• Each process has an execution state, which
  indicates what it is currently doing
• ready waiting to be assigned to CPU
• could run, but another process has the CPU
• running executing on the CPU
• is the process that currently controls the CPU
• pop quiz how many processes can be running
  simultaneously?
• waiting waiting for an event, e.g. I/O
• cannot make progress until event happens
• As a process executes, it moves from state to
  state
• LINUX run ps, STAT column shows current state
• which state is a process in most of the time?

11
Process state transitions
create
New
Ready
I/O done
Waiting
Un-schedule
schedule
kill
Terminated
Running
I/O, page fault, etc.

• What can cause schedule/unschedule transitions?

12
Process data structures

• How does the OS represent a process in the
  kernel?
• at any time, there are many processes, each in
  its own particular state
• the OS data structure that represents each is
  called the process control block (PCB)
• PCB contains all info about the process
• OS keeps all of a process hardware execution
  state in the PCB when the process isnt running
• PC
• SP
• registers
• when process is unscheduled, the state is
  transferred out of the hardware into the PCB

13
PCB

• The PCB is a data structure with many, many
  fields
• process ID (PID)
• execution state
• program counter, stack pointer, registers
• memory management info
• UNIX username of owner
• scheduling priority
• accounting info
• pointers into state queues
• In LINUX
• defined in task_struct (include/linux/sched.h)
• over 95 fields!!!

14
State queues

• The OS maintains a collection of queues that
  represent the state of all processes in the
  system
• typically one queue for each state
• e.g., ready, waiting,
• each PCB is queued onto a state queue according
  to its current state
• as a process changes state, its PCB is unlinked
  from from queue, and linked onto another

15
State queues
Ready queue header
netscape pcb
emacs pcb
ls pcb
head ptr
tail ptr
Wait queue header
cat pcb
netscape pcb
head ptr
tail ptr

• There may be many wait queues, one for each type
  of wait (particular device, timer, message, )

16
PCBs and State Queues

• PCBs are data structures
• dynamically allocated inside OS memory
• When a process is created
• OS allocates a PCB for it
• OS initializes PCB
• OS puts PCB on the correct queue
• As a process computes
• OS moves its PCB from queue to queue
• When a process is terminated
• OS de-allocates its PCB

17
Process creation

• One process can create another process
• creator is called the parent
• created process is called the child
• UNIX do ps, look for PPID field
• what creates the first process, and when?
• In some systems, parent defines or donates
  resources and privileges for its children
• UNIX child inherits parents userID field, etc.
• when child is created, parent may either wait for
  it to finish, or it may continue in parallel, or
  both!

18
LINUX process creation

• Process creation through fork() system call
• creates and initializes a new PCB
• creates a new address space
• initializes new address space with a copy of the
  entire contents of the address space of the
  parent
• initializes kernel resources of new process with
  resources of parent (e.g. open files)
• places new PCB on the ready queue
• The fork() system call returns twice
• once into the parent, and once into the child
• returns the childs PID to the parent
• returns 0 to the child

19
fork( )

• main()
• int child_pid fork()
• if (child_pid 0)
• printf(Child is d\n, child_pid)
• return 0
• else
• printf(My child is d\n, child_pid)
• return 0

20
Output

• anvil gcc -o testfork testfork.c
• anvil ./testfork
• My child is 1203
• Child is 0
• anvil ./testfork
• Child is 0
• My child is 486

21
Lecture 2, September 16, 2003

• About Programming Projects
• Processes
• Threads
• Scheduling Policies

22
Problems with Processes

• A process includes many things
• an address space (all the code and data pages)
• OS resources (e.g., open files) and accounting
  info
• hardware execution state (PC, SP, registers)
• Creating a new process is costly, because of all
  of the data structures that must be
  allocated/initialized
• Linux over 95 fields in task_struct
• on a 700 MHz pentium, forkexit 251
  microseconds, forkexec 1024 microseconds
• Inter-process communication is costly, since it
  must usually go through the OS
• overhead of system calls
• 0.46 microseconds on 700 MHz pentium

23
Example Parallel Processing

• Imagine a Web server, which forks off copies of
  itself to handle multiple simultaneous tasks
• or, imagine we have any parallel program on a
  multiprocessor
• To execute these, we need to
• create several processes that execute in parallel
• cause each to map to the same address space to
  share data
• have the OS schedule them in parallel
• multiprogramming or true parallel processing on
  an SMP
• This is really inefficient
• space PCB, page tables, etc.
• time create OS structures, fork and copy address
  space, etc.

24
Can we do better?

• Whats similar in these processes?
• they all share the same code and data (address
  space)
• they all share the same privileges
• they all share the same resources (files,
  sockets, etc.)
• Whats different?
• each has its own hardware execution state
• PC, registers, stack pointer, and stack
• Key idea
• separate the concept of
• a process (address space, etc.) from that of
• a minimal thread of control (execution state
  PC, etc.)
• this execution state is usually called a thread,
  or sometimes, a lightweight process

25
Threads and processes

• Most modern OSs (Mach, Chorus, NT, modern Unix)
  therefore support two entities
• the process, which defines the address space and
  general process attributes (such as open files,
  etc.)
• the thread, which defines a sequential execution
  stream within a process
• A thread is bound to a single process
• processes, however, can have multiple threads
  executing within them
• sharing data between threads is cheap all see
  same address space
• Threads become the unit of scheduling
• processes are just containers in which threads
  execute

26
Thread Design Space
older UNIXes
MS/DOS
one thread/process
address space
one thread/process
one process
many processes
thread
Java
Mach, NT, Chorus, Linux,
many threads/process
many threads/process
many processes
one process
27
A processs address space with threads
thread 1 stack
SP (T1)
0xFFFFFFFF
thread 2 stack
SP (T2)
SP
thread 3 stack
SP (T3)
address space
heap (dynamic allocated mem)
static data (data segment)
0x00000000
code (text segment)
PC (T1)
PC (T2)
PC
PC (T3)
28
Process/Thread Separation

• Separating threads and processes makes it easier
  to support multi-threaded applications
• creating concurrency does not require creating
  new processes
• Concurrency (multithreading) is useful for
• improving program structure (the Java argument)
• handling concurrent events (e.g., web servers)
• building parallel programs (e.g., raytracer)
• So, multithreading is useful even on a
  uniprocessor
• even though only one thread can run at a time

29
Kernel thread and user-level threads

• Who is responsible for creating/managing threads?
• Two answers, in general
• the OS (kernel threads)
• thread creation and management requires system
  calls
• the user-level process (user-level threads)
• a library linked into the program manages the
  threads
• Why is user-level thread management possible?
• threads share the same address space
• therefore the thread manager doesnt need to
  manipulate address spaces
• threads only differ in hardware contexts
  (roughly)
• PC, SP, registers
• these can be manipulated by the user-level
  process itself!

30
Kernel Threads

• OS now manages threads and processes
• all thread operations are implemented in the
  kernel
• OS schedules all of the threads in a system
• if one thread in a process blocks (e.g. on I/O),
  the OS knows about it, and can run other threads
  from that process
• possible to overlap I/O and computation inside a
  process
• Kernel threads are cheaper than processes
• less state to allocate and initialize
• But, they can still be too expensive
• thread operations are all system calls
• OS must perform all of the usual argument checks
• but want them to be as fast as a procedure call!
• must maintain kernel state for each thread
• can place limit on of simultaneous threads,
  typically 1000

31
User-Level Threads

• To make threads cheap and fast, they need to be
  implemented at the user level
• managed entirely by user-level library, e.g.
  libpthreads.a
• User-level threads are small and fast
• each thread is represented simply by a PC,
  registers, a stack, and a small thread control
  block (TBC)
• creating a thread, switching between threads, and
  synchronizing threads are done via procedure
  calls
• no kernel involvement is necessary!
• user-level thread operations can be 10-100x
  faster than kernel threads as a result

32
Performance Example

• On a 700MHz Pentium running Linux 2.2.16
• Processes
• fork/exit 251 ms
• Kernel threads
• pthread_create()/pthread_join() 94 ms
• User-level threads
• pthread_create()/pthread_join 4.5 ms

33
User-level Thread Limitations

• But, user-level threads arent perfect
• tradeoff, as with everything else
• User-level threads are invisible to the OS
• there is no integration with the OS
• As a result, the OS can make poor decisions
• scheduling a process with only idle threads
• blocking a process whose thread initiated I/O,
  even though the process has other threads that
  are ready to run
• unscheduling a process with a thread holding a
  lock
• Solving this requires coordination between the
  kernel and the user-level thread manager

34
User-level Thread Implementation

• A thread scheduler determines when a thread runs
• it uses queues to keep track of what threads are
  doing
• just like the OS and processes
• but, implemented at user-level as a library
• run queue threads currently running
• ready queue threads ready to run
• wait queue threads blocked for some reason
• maybe blocked on I/O, maybe blocked on a lock
• How can you prevent a thread from hogging the
  CPU?
• how did the OS handle this?

35
Thread context switch

• Very simple for user-level threads
• save context of currently running thread
• push machine state onto thread stack
• restore context of the next thread
• pop machine state from next threads stack
• return to caller as the new thread
• execution resumes at PC of next thread
• This is all done by assembly language
• it works at the level of the procedure calling
  convention
• thus, it cannot be implemented using procedure
  calls

36
Multithreaded Web server

• Common processing
• Open a socket (interface)
• Read URL request
• Parse it
• Find the file if it exists
• Open file and send to socket
• Close the file
• Close the socket
• Exit
• Similar code and processing, few private data
  structure

37
Lecture 2, September 16, 2003

• About Programming Projects
• Processes
• Threads
• Scheduling Policies

38
What is scheduling?

• In discussion processes, we talked about context
  switching between threads/process on the ready
  queue
• but, we glossed over the details of which process
  or thread is chosen next
• making this decision is called scheduling
• scheduling is policy
• context switching is mechanism
• Today, well look at
• the goals of scheduling
• well-known scheduling policies

39
Multiprogramming and Scheduling

• Multiprogramming increases resource utilization
  and job throughput by overlapping I/O and CPU
• today look at scheduling policies
• which process/thread to run, and for how long
• schedulable entities are usually called jobs
• processes, threads, people, phone calls,
• There are two time scales of scheduling the CPU
• long term determining the multiprogramming level
• how many jobs are loaded into primary memory
• act of loading in a new job (or loading one out)
  is swapping
• short-term which job to run next to result in
  good service
• happens frequently, want to minimize
  context-switch overhead
• good service could mean many things

40
Scheduling

• The scheduler is the module that moves jobs from
  queue to queue
• the scheduling algorithm determines which job(s)
  are chosen to run next, and which queues they
  should wait on
• the scheduler is typically run when
• a job switches from running to waiting
• when an interrupt occurs, especially a timer
  interrupt
• when a job is created or terminated
• There are two major classes of scheduling systems
• in preemptive systems, the scheduler can
  interrupt a job and force a context switch
• in non-preemptive systems, the scheduler waits
  for the running job to explicitly (voluntarily)
  block

41
Scheduling Goals

• Scheduling algorithms can have many different
  goals (which sometimes conflict)
• maximize system utilization
• maximize job throughput (jobs/s)
• minimize job turnaround time (Tfinish Tstart)
• minimize job waiting time (Avg(Twait) average
  time spent on wait queue)
• minimize response time (Avg(Tresp) average time
  spent on ready queue)
• Goals may depend on type of system
• batch system strive to maximize job throughput
  and minimize turnaround time
• interactive systems minimize response time of
  interactive jobs (such as editors or web browsers)

42
Scheduling Non-goals

• Schedulers typically try to prevent starvation
• starvation occurs when a process is prevented
  from making progress, because another process has
  a resource it needs
• A poor scheduling policy can cause starvation
• e.g., if a high-priority process always prevents
  a low-priority process from running on the CPU
• Synchronization can also cause starvation
• well see this in a future class
• roughly, if somebody else always gets a lock I
  need, I cant make progress

43
Policy 1 FCFS

• First-come first-served (FCFS)
• jobs are scheduled in the order that they arrive
• real-world scheduling of people in lines
  supermarket
• typically non-preemptive no switching at
  supermarket!
• jobs treated equally, no starvation
• except possibly for infinitely long jobs

A
B
Job arrivals
C
time
A
B
C
Scheduling
44
FCFS / multiple handlers

• One queue, multiple CPUs
• Example MacDonald, Telephone

A
B
Phone calls
C
D
E
time
Line1
A
E
Line2
B
C
D
45
Policy 2 SJF

• Problems with FCFS
• average response time and turnaround time can be
  large
• e.g., small jobs waiting behind long ones
• results in high turnaround time
• may lead to poor overlap of I/O and CPU
• Shortest job first (SJF)
• choose the job with the smallest expected CPU
  burst
• can prove that this has optimal min. average
  waiting time
• Can be preemptive or non-preemptive
• preemptive is called shortest remaining time
  first (SRTF)
• Sounds perfect!
• whats the problem here?

46
SJF example non-preemptive
A
B
Job arrivals
C
time
Scheduling
B
A
C
47
SJF example - preemptive
A
B
Job arrivals
C
time
A
Scheduling
B
C
A
A
Schedule B first
Back to A
Schedule C first
Continue A
48
SJF Problem

• Problem impossible to know size of future CPU
  burst
• from your theory class, equivalent to the halting
  problem
• can you make a reasonable guess?
• yes, for instance looking at past as predictor of
  future
• but, might lead to starvation in some cases!

49
Policy 3 Priority Scheduling

• Assign priorities to jobs
• choose job with highest priority to run next
• if tie, use another scheduling algorithm to break
  (e.g. FCFS)
• to implement SJF, priority expected length of
  CPU burst
• Abstractly modeled as multiple priority queues
• put ready job on queue associated with its
  priority
• Sound perfect!
• whats wrong with this?

50
Priority Example
Priority SJF (as tiebreaker)
A-low
B -high
Job arrivals
C-low
time
A-low
B -high
C-low
51
Priority Scheduling problem

• The problem starvation
• if there is an endless supply of high priority
  jobs, no low-priority job will ever run
• Solution age processes over time
• increase priority as a function of wait time
• decrease priority as a function of CPU time
• many ugly heuristics have been explored in this
  space

52
Policy 4 Round Robin

• Round Robin scheduling (RR)
• ready queue is treated as a circular FIFO queue
• each job is given a time slice, called a quantum
• job executes for duration of quantum, or until it
  blocks
• time-division multiplexing (time-slicing)
• great for timesharing
• no starvation
• can be preemptive or non-preemptive
• Sounds perfect!
• whats wrong with this?

53
RR Example
A
B
Job arrivals
C
C
time
Scheduling
54
RR problems

• Problems
• what do you set the quantum to be?
• no setting is correct
• if small, then context switch often, incurring
  high overhead
• if large, then response time drops
• treats all jobs equally
• if I run 100 copies of SETI_at_home, it degrades
  your service
• how can I fix this?

55
Combining algorithms

• Scheduling algorithms can be combined in practice
• have multiple queues
• pick a different algorithm for each queue
• and maybe, move processes between queues
• Example multi-level feedback queues (MLFQ)
• multiple queues representing different job types
• batch, interactive, system, CPU-bound, etc.
• queues have priorities
• schedule jobs within a queue using RR
• jobs move between queues based on execution
  history
• feedback switch from CPU-bound to interactive
  behavior
• Pop-quiz
• is MLFQ starvation-free?

56
A Hybrid Policy
Priority RR
A-low
Job arrivals
B -high
C-high
time
Scheduling
57
A Concrete Example

• 3 users download 3 large files (1MB,500KB,1MB)
• At time 0, 2, 3 seconds
• Receiving bandwidth 100KBps
• Server bandwidth 200KB

0 2 3
9
Only 1 job
2 jobs
3 parallel jobs (Job 2 finishes at time 9)
58
This Lecture

• Processes
• Threads
• Scheduling
• Reading Chapter 2, Section 1, 2, 5
• Homework assignment 1