Chapter 1: Kernel Overview

1
Chapter 1 Kernel Overview
2
Objectives

• Review development and history of UNIX and Linux.
• Clarify the nature and extent of the kernel
  within the context of the operating system
  proper.
• Identify operating system design goals and
  tradeoffs.
• Review established architectural approaches for
  implementing an operating system.
• Introduce the notion of the core Linux kernel.
• Understand the Linux approach to hardware
  (architecture) abstraction and independence.
• Explore the layout of the Linux source code tree.

3
History

• UNIX 1969 Thompson Ritchie ATT Bell Labs
• BSD 1978 Berkeley Software Distribution
• Commercial Vendors Sun, HP, IBM, SGI, DEC
• GNU 1984 Richard Stallman, FSF
• POSIX 1986 IEEE Portable Operating System unIX
• Minix 1987 Andy Tannenbaum
• SVR4 1989 ATT and Sun
• Linux 1991 Linus Torvalds Intel 386 (i386)
• Open Source GPL, LGPL, Cathedral and the Bazaar

4
Linux Features

• UNIX-like operating system
• aims at standards compliance
• all the features you would expect in a modern
  UNIX
• preemptive multitasking
• virtual memory (protected memory, paging)
• shared libraries
• demand loading, dynamic kernel modules
• shared copy-on-write executables
• TCP/IP networking
• other features
• SMP support, large memory, large files
• advanced networking, advanced filesystems
• efficient, stable, highly portable, supports most
  device hardware
• active development community, support,
  documentation, open source
• GUIs, applications

5
Whats a Kernel?

• aka executive, system monitor, nucleus
• controls and mediates access to hardware
• implements and supports fundamental abstractions
• processes, files, devices, users, net, etc.
• schedules fair sharing of system resources
• memory, cpu, disk, descriptors, etc.
• enforces security and protection
• responds to user requests for service (system
  calls)
• performs routine maintenance, system checks, etc.

6
Kernel Design Goals

• performance efficiency, speed
• utilize resources to capacity, low overhead, code
  size
• stability robustness, resilience
• uptime, graceful degradation
• capability features, flexibility, compatibility
• security, protection
• protect users from each other, secure system from
  bad guys
• portability
• clarity
• extensibility

7
Design Tradeoffs

• Butler Lampson choose any three design goals
• efficiency vs. protection
• more checks, more overhead
• clarity vs. compatibility
• ugly implementation of broken standards (e.g.
  signals)
• flexibility vs. security
• the more you can do, the more potential security
  holes!
• not all are antagonistic
• portability tends to enhance code clarity

8
Waterloo Diagrams
mm
ipc
fs
sched
vfs
sched
ipc
net
mm
net
Conceptual
Concrete
9
Stephen Tweedies Diagram
User Processes
Scheduler
Traps
VM
Syscalls
Process Manager
VFS
Socket Manager
Memory Allocator
Math Support
Network Protocols
File Systems
Char Devices
Block Devices
Packet Requestor
IO Requestor
Net Devices
10
Vahalias Diagram

• from Unix Internals The New Frontiers
• Uresh Vahalia / Prentice-Hall 1996

elf
a.out
coff
file
NFS
exec switch
device
FFS
virtual memory framework
vnode/vfs interface
s5fs
anonymous
core utilities
scheduler framework
block device switch
time-sharing
disk
STREAMS
real-time
tape
system
network
tty
11
Core Kernel
Applications
System Libraries (libc)
System Call Interface
I/O Related
Process Related
Scheduler
File Systems
Modules
Memory Management
Networking
IPC
Device Drivers
Architecture-Dependent Code
Hardware
12
Architectural Approaches

• monolithic
• layered
• modularized
• micro-kernel
• virtual machine

13
Isolating Hardware Dependencies

• architecture (cpu)
• dependent (/arch)
• independent (everything else)
• abstract dependencies behind functions and macros
• link appropriate version at compile-time
• device-dependencies isolated in device drivers
• provide general abstractions that map to reality
• e.g. three-level page tables
• tradeoff exploiting special hardware features

14
Source Tree Layout
/usr/src/linux
scripts
Documentation
ipc
kernel
init
net
arch
mm
lib
fs
drivers
include
802 appletalk atm ax25 bridge core decnet econet e
thernet ipv4 ipv6 ipx irda khttpd lapb
acorn atm block cdrom char dio fc4 i2c i2o ide iee
e1394 isdn macintosh misc net
adfs affs autofs autofs4 bfs code cramfs devfs dev
pts efs ext2 fat hfs hpfs
asm-alpha asm-arm asm-generic asm-i386 asm-ia64 as
m-m68k asm-mips asm-mips64 linux math-emu net pc
mcia scsi video
adfs affs autofs autofs4 bfs code cramfs devfs dev
pts efs ext2 fat hfs hpfs
alpha arm i386 ia64 m68k mips mips64 ppc s390 sh s
parc sparc64
15
linux/Documentation

• spotty but important collection of
  developer-generated documentation you need to
  read whats in here!
• recent effort to produce javadoc-style
  documentation from source header comments using
  OpenDoc
• an ambitious open-source kernel book effort has
  begun see kernelbook.sourceforge.net for details
• some especially interesting entries
• kernel-docs.txt (a bit out of date but good)
• filesystems/ (very extensive)
• networking/ (very extensive)
• kmod.txt
• oops-tracing.txt
• spinlocks.txt (the official story from Linus)

16
linux/arch

• subdirectories for each current port
• each contains kernel, lib, mm, boot and other
  directories whose contents override code stubs in
  architecture independent code
• lib contains highly-optimized common utility
  routines such as memcpy, checksums, etc.
• arch as of 2.4
• alpha, arm, i386, ia64, m68k, mips, mips64
• ppc, s390, sh, sparc, sparc64

17
linux/drivers

• largest amount of code in the kernel tree (1.5M)
• device, bus, platform and general directories
• drivers/char n_tty.c is the default line
  discipline
• drivers/block elevator.c, genhd.c, linear.c,
  ll_rw_blk.c, raidN.c
• drivers/net specific drivers and general
  routines Space.c and net_init.c
• drivers/scsi scsi_.c files are generic sd.c
  (disk), sr.c (CD-ROM), st.c (tape), sg.c
  (generic)
• general
• cdrom, ide, isdn, parport, pcmcia,
• pnp, sound, telephony, video
• buses fc4, i2c, nubus, pci, sbus, tc, usb
• platforms acorn, macintosh, s390, sgi

18
linux/fs

• contains
• virtual filesystem (VFS) framework
• subdirectories for actual filesystems
• vfs-related files
• exec.c, binfmt_.c - files for mapping new
  process images
• devices.c, blk_dev.c device registration, block
  device support
• super.c, filesystems.c
• inode.c, dcache.c, namei.c, buffer.c,
  file_table.c
• open.c, read_write.c, select.c, pipe.c, fifo.c
• fcntl.c, ioctl.c, locks.c, dquot.c, stat.c

19
linux/include

• include/asm-
• architecture-dependent include subdirectories
• include/linux
• header info needed both by the kernel and user
  apps
• usually linked to /usr/include/linux
• kernel-only portions guarded by ifdefs
• ifdef __KERNEL__
• / kernel stuff /
• endif
• other directories
• math-emu
• net
• pcmcia
• scsi
• video

20
linux/init

• just two files version.c, main.c
• version.c contains the version banner that
  prints at boot
• main.c architecture-independent boot code
• start_kernel is the primary entry point

21
linux/ipc

• System V IPC facilities
• if disabled at compile-time, util.c exports stubs
  that simply return ENOSYS
• one file for each facility
• sem.c semaphores
• shm.c shared memory
• msg.c message queues

22
linux/kernel

• the core kernel code
• sched.c the main kernel file
• scheduler, wait queues, timers, alarms, task
  queues
• process control
• fork.c, exec.c, signal.c, exit.c
• acct.c, capability.c, exec_domain.c
• kernel module support
• kmod.c, ksyms.c, module.c
• other operations
• time.c, resource.c, dma.c, softirq.c, itimer.c
• printk.c, info.c, panic.c, sysctl.c, sys.c

23
linux/lib

• kernel code cannot call standard C library
  routines
• files
• brlock.c Big Reader spinlocks
• cmdline.c kernel command line parsing routines
• errno.c global definition of errno
• inflate.c gunzip part of gzip.c used during
  boot
• string.c portable string code
• usually replaced by optimized, architecture-depend
  ent routines
• vsprintf.c libc replacement

24
linux/mm

• paging and swapping
• swap.c, swapfile.c (paging devices), swap_state.c
  (cache)
• vmscan.c paging policies, kwapd
• page_io.c low-level page transfer
• allocation and deallocation
• slab.c slab allocator
• page_alloc.c page-based allocator zone
  allocator
• vmalloc.c kernel virtual-memory allocator
• memory mapping
• memory.c paging, fault-handling, page table
  code
• filemap.c file mapping
• mmap.c, mremap.c, mlock.c, mprotect.c

25
linux/net

• changing too fast!
• i havent figured it out yet ?

26
linux/scripts

• scripts for
• menu-based kernel configuration
• kernel patching
• generating kernel documentation

27
Sizes (linux-2.4.0-test2)

• size directory entries files loc
• 90M /usr/src/linux/ 19 7645 2.6M
• 4.5M Documentation 97 380 na
• 16.5M arch 12 1685 466K
• 54M drivers 31 2256 1.5M
• 5.6M fs 70 489 150K
• 14.2M include 19 2262 285K
• 28K init 2 2 1K
• 120K ipc 6 6 4.5K
• 332K kernel 25 25 12K
• 80K lib 8 8 2K
• 356K mm 19 19 12K
• 5.8M net 33 453 162K
• 400K scripts 26 42 12K

28
Summary

• Linux is a modular, UNIX-like monolithic kernel
• Kernel is the heart of the OS that executes with
  special hardware permission (kernel mode)
• Core kernel provides framework, data
  structures, support for drivers, modules,
  subsystems
• Kernel designers must consider many competing
  goals
• Linux source tree mirrors kernel structure
• Architecture dependent source subtrees live in
  /arch
• main lives in /kernel/init.c
• lxr.linux.no is a nice web-based source browser