Deferrable Functions in Linux

1
Deferrable Functions in Linux

• Shang Rong Tsai

2
Deferrable Functions in Linux

• Mechanisms supported for delaying the execution
  of not urgent functions, mostly in interrupt
  handlers
• Types of deferrable functions in Linux2.4
• Softirq
• Tasklet
• Bottom-half

3
Why Deferrable functions?

• In servicing an interrupt, some tasks to be done
  are time critical (e.g. save device data to
  memory buffer), some are not (e.g. awaken
  processes, start servicing another I/O request.
• In servicing an interrupt, we dont want to
  disable interrupt too long, So, the interrupt
  handler should enable interrupts as soon as
  possible, as to allow interrupts of other devices
  (or even this device) to be accepted and
  serviced. However, simply enabling interrupt in
  an interrupt handler would have the designers of
  the interrupt handlers to worry about the
  problems for possible shared data access

4
Disk driver
Network driver
Strategy routine
Strategy routine
Copy data .. Schedule deferrable_function
(function pointer B, arguments)
Copy data .. Schedule deferrable_function
(function pointer A, arguments)
Top half
Interrupt handler
Interrupt handler
deferrable_function A
Bottom half
deferrable_function B
Queue to keep the list of the deferrable jobs of
the same type to be done
deferrable_function B
deferrable_function A
5
General Operations for Deferrable functions

• Defining a new deferrable function
• Usually defined on kernel initialization
• Scheduling (activating) a deferrable function
• May be activated at any time in kernel, for
  example
• When executing an interrupt handler
• When make a disk I/O request, and the disk driver
  request queue is empty gt device
  plugging/unplugging
• Executing deferrable functions
• Kernel will begin to execute the deferrable
  functions at a proper time later
• Masking (Selectively disable) a deferrable
  function

6
Differences among softirq, tasklet, and bottom
half
7
Softirq

• Softirqs (even with the same type) can be run
  simultaneously on MP
• Basically, softirqs are re-entrant functions,
  access to their data would need to be
  synchronized with spin locks
• Max 32 softirqs are defined, In 2.4 only 4 are
  used.
• Softirqs are activated by __CPU_raise_softirq
  macro, it will set the ith (i is the index of the
  softirq) bit of the softirq_pending(cpu) (softirq
  pending means the softirq has been activated

8
Softirq used in Linux 2.4

• HI_SOFTIRQ
• For handling high priority tasklets and bottom
  halves
• NET_TX_SOFTIRQ
• Transmit network packets
• NET_RX_SOFTIRQ
• Receiving network packets
• TASKLET_SOFTIRQ
• For handling tasklets

9
Softirq data structure

• Structure representing softirq entries

Struct softirq_action void (action)(struct
softirq_action ) //function to run void
data //data passed to function Static struct
softirq_action softirq_vec32 // max 32 softirqs
index
Pointer to the softirq function
0
data
Index 0 has highest priority in execution
1
10
Tasklet descriptor
Tasklet descriptor
tasklet_vec
(one entry for one CPU)
state
state
count
count
tasklet function pointer
Data
Data

• State
• - 0
• - TASKLET_STATE_SCHED indicate this tasklet is
  already scheduled
• TASKLET_STATE_RUN indicate this tasklet is being
  run , (this flag is not
• needed in a single CPU system)

data A unsigned long, for casting to any data
structure used by the tasklet function
11
Tasklet in Linux 2.4

• Built on top of two softirqs (HI_SOFTIRQ and
  TASKLET_SOFTIRQ)
• Several tasklets may be associated with the same
  softirq
• Can be dynamically or statically created
  (softirqs are statically allocated at complie
  time)
• As softirqs, tasklets can not sleep. (Basically,
  code execute in interrupt context can not sleep,
  or call functions that would sleep)
• Two of the same tasklets never run concurrently
• If your tasklet share data with other tasklets or
  softirq, you need to use proper locking
• If the same tasklet is scheduled again, before it
  has had a chance to run, it still runs only once

12
Defining tasklets

• Static defining by one of the following MACROs
• DECLARE_TASKLET(name, func, data)
• name tasklets name
• func function to be executed
• data argument for the function
• DECLARE_TASKLET_DISABLED(name, func, data)
• Initially the tasklet is disabled
• Dynamically creating by
• struct tasklet_struct t //tasklet descriptor
• tasklet_init(t, tasklet_handler,argument)
• The tasklet handler would be
• void tasklet_handler(unsigned long data)

13
Scheduling Tasklets

• Scheduling a tasklet by
• Tasklet_schedule(my_tasklet)
• Disabling a tasklet (disable its execution)
• Tasklet_disable(my_tasklet)
• Enabling a tasklet (enable its execution)
• Tasklet_enable(my_tasklet)

14
Executing tasklets

• A kernel thread named ksoftirqd_CPUn calls
  do_softirq() which invokes the softirq function
  associated with TASKLET_SOFTIRQ (or HI_SOFTIRQ)
• The TASKLET_STATE_RUN is used to prevent the
  tasklets with the same type from simultaneous
  execution on multiple CPU

15
Bottom Half

• Bottom haves are high priority tasklets
• All bottom halves are globally serialized
• A global spin lock global_bh_lock is used to
  ensure the mutual exclusion
• Bottom halves are scheduled by invoking
• mark_bh()
• The execution of bottom halves will be via
  tasklet mechanism (and actually via softirq)

16
Bottom Half usage in Linux 2.4

• Timer_BH
• Serial_BH
• SCSI_BH
• IMMEDIATE_BH //for immediate task queue

17
Internal data structures about bottom half

• Linux uses an array bh_base to group all bottom
  havles together, each element is a pointer to
  each bottom half.
• Array bh_task_vec used to store 32 tasklet
  descriptors, one for each bottom half
• Before first time activation, a bottom half
  should be initialized using inti_bh(n, routine)
  which insert the address of routine into the nth
  entry of bh_base)

18
Task Queue in Linux 2.4

• Mechanism support for delaying execution in
  kernel, not necessarily in interrupt handler. gt
  device plugging/unplugging use it
• For queuing tasks (functions) to be done at later
  time
• Predefines task queues
• User defined task queues

19
Predined task queues

• scheduler queue
• Functions in this queue is executed in process
  context
• tg_timer
• Run by time tick
• tg_immediate
• Run either on return from a system call or when
  the scheduler is run.
• Run in interrupt context

20
Task queue data structures and operations

• Operations on task queues
• DECLARE_TASK_QUEUE(name) //declare a task queue
• Int queue_task(struct tg_struct task, task_queue
  list) // queue a task
• Void run_task_queue(task_queue(task_queue list)
  // call to execute tasks for user defined task
  queues
• Task queue element

Struct tg_struct struct tq_struct next
// linked list of active bottom halves int
sync void (routine)(void) // function
(the task) to call void data
//argument to function
21
Interrupt context

• Some restrictions on code executed in interrupt
  context
• No access to user space is allowed (interrupt is
  asynchronous, no process context is currently
  associating to it)
• The current pointer is not valid, can not use it
  to get the process specific data
• No sleeping, no call to blocking functions
• VAX supports interrupt stack in hardware for
  interrupt handling