Advanced Operating Systems

1
Advanced Operating Systems
Lecture 1 Introduction

• University of Tehran
• Dept. of EE and Computer Engineering
• By
• Dr. Nasser Yazdani

2
Goal

• Introducing the course and policies
• Some reviews from OS
• References
• Review any undergraduate level OS book. We
  suggest Silbershutz , Galvin, .. Book
• Read On building Systems that will fail by
  Fernando J. CorBato

3
Outline

• Agenda
• Policy, Grading, reading materials, etc.
• Some review from OS
• Overview of course materials
• Some deep question in OS
• Some general guides (Taken mostly from Dr.
  Shenoy, Univ. of Massachusetts Amherst, website)

4
Agenda

• To well understand OS architecture and design.
• To establish a base for future Operating System
  research and development.
• To discuss latest developments in OS computer
  via class lectures and assigned readings.
• To find the current research issues in the field.
• Finally, prepare to perform some projects in OS
  which are essential in national development and
  grows.

5
Course Materials

• Course Web page
• visit regularly
• Htt//cec.ut.ac.ir/classpages/advanceOS
• Research papers
• On pdf/ps format on the Website
• Combination of classic and recent work.
• Textbooks It is mostly paper based but the
  following book is good for some parts
• Distributed Systems , Andrew Tanenbaum. Last
  edition
• Other good books
• Modern Operating Systems , Andrew Tanenbaum.
• Coulouris, et al., Distributed Systems Concepts
  and Design, 2nd ed., 004.36 C85d2
• Linux Kernel Development (2.6 kernel)

6
Grading

• Homework assignments, around 30
• Presentations
• Paper presentation, each paper around 1 hour.
  (For PhD student only)
• Final project presentation, around 20 minutes.
• Paper reviews.
• Homework
• Mostly kernel simulation and programming.
• Late penalties!
• Project around 30
• Exam, final around 40.

7
Policy

• The course is research oriented. Then, the main
  focus will be on paper reading and projects.
• We will focus more on essential OS concepts and
  problems and especially on Distributed Systems.
• There will be reading and work. Then, be
  prepared!.

8
Prerequisites

• We assume Programming experience in C
• Some familiarity with Unix and system calls
• Undergraduate Operating system course
• An undergraduate textbook OS class
• Familiar with concepts like Virtual Memory,
  processes, etc.
• But maybe never seen a real implementation, or
  written code to do things like manipulate page
  tables

9
Basic Question?

• What is OS?
• Why do we need OS?
• What does OS provide to us?
• OS is a facilitator and make life easier for user
• We will see more later?

10
Why Study OS?

• Operating systems are a maturing field
• High-performance servers are an OS issue
• Face many of the same issues as OSes
• Resource consumption is an OS issue
• Battery life, radio spectrum, etc.
• Security is an OS issue
• Hard to achieve security without a solid
  foundation
• New smart devices need new Oses
• Today, everything is embedded systems and OS
  an essential part of it.
• We need good knowledge on OS inside the country.

11
Operating System?

• Software that turns silicon into something
  useful
• Provides abstractions for applications
• Manages and hides details of hardware
• Accesses hardware through low/level interfaces
  unavailable to applications.
• Provides isolation/protection
• Prevents one process/user from clobbering another

12
Operating System?

• It is an extended machine
• Hides the messy details which must be performed
• Presents user with a virtual machine, easier to
  use
• It is a resource manager
• Each program gets time with the resource
• Each program gets space on the resource
• Real life example
• Government

13
History of OS (1)

• First generation 1945 1955
• vacuum tubes, plug boards (no OS)
• Second generation 1955 1965
• transistors, batch systems, IBM 360
• Third generation 1965 1980
• ICs and multiprogramming
• Fourth generation 1980 present
• personal computers, hand-held devices, sensors
• Software has profound though slower impact on
  computer architecture
• Modern architects cannot avoid paying attention
  to software and compilation issues

14
History of OS (1945-55)

• Early days, use of computer was very primitive,
  very difficult and inefficient (why?)
• Improvement assembly language, Fortran, Compiler
  and finally Batch Processing?

15
History of OS (1955-65)

• Structure of a typical JCL job 2nd generation
• Single user
• Programmer/User as the operator
• Secure, but inefficient use of expensive
  resources
• Low CPU utilization-slow mechanical I/O devices

16
Batch System

• What it does?
• Schedules task (Usually FIFO)
• Starts and Terminates Jobs
• Compile Programs binds with libarary
• Then,
• Make life easier for user (user friendly)
• Isolate user (Application) from Hardware
• Resource manager
• Problem?
• Slow Response time

17
History of Operating Systems (1965-80)

• Sharing System among user
• Multitasking and time sharing system
• Extend Resource management to memory, I/O, etc.
• Protect programs

18
The OS Zoo (1980-present)

• Mainframe operating systems
• Server operating systems
• Multiprocessor operating systems
• Personal computer operating systems
• Real-time operating systems
• Embedded operating systems
• Smart card operating systems

19
History of OS Change!
1980 2000 Factor
Speed CPU 1 MIPS 1000 MIPS 1000
Memory 500 ns 2 ns 250
Disk 18 ms 2 ms 9
Modem 300 bits/sec 56 Kbits/sec 200
Capacity Memory 64 Kbytes 128 Mbytes 2000
Disk 1 Mbytes 6 Gbytes 6000
Cost Per MIP 100K lt 1 100000
Other Address bits 8 64 8
Users/machine 10s lt1 .01
20
What Is an OS?
Provide environment for application and go
away It is like government (?).

• Services
• Abstraction
• Simplification
• Convenience
• Standardization

• Resources
• Allocation
• Protection
• Reclamation
• Virtualization

Makes computers simpler
21
What Is an OS?
Government

• Resources
• Allocation
• Protection
• Reclamation
• Virtualization

• Finite resources
• Competing demands
• Examples
• CPU
• Memory
• Disk
• Network

Limited budget,Land,Oil,Gas,
Linux or Windows?
Democrat or Republic?
22
What Is an OS?
Government

• Resources
• Allocation
• Protection
• Reclamation
• Virtualization

• You cant hurt me
• I cant hurt you
• Implies some degree of safety security

Law and order
23
What Is an OS?
Government

• Resources
• Allocation
• Protection
• Reclamation
• Virtualization

Income Tax

• The OS gives
• The OS takes away
• Voluntary at run time
• Implied at termination
• Involuntary Cooperative

24
What Is an OS?
Government

• Resources
• Allocation
• Protection
• Reclamation
• Virtualization

• illusion of infinite, private resources
• Memory versus disk
• Timeshared CPU
• More extreme cases possible ( exist)

Social security
25
What we will do?

• OS does a lot of things gt It is a huge program
• How does it affect Hardware?
• How to organize, structure it?
• What are important?
• Scalability
• Extensibility
• Manageability
• Robustness,
• Efficiency , Performance
• More important user-friendly
• Of course, cost

26
Covered Topics

• First, we look at some design principle? It is
  usually general.
• Effects on the Hardware
• New design approach, micro kernel
• Some basic challenges
• Threads
• Interprocess communication
• Scheduling
• Concurrency
• File system
• Virtual machine
• Multicore OS

27
Topics (Distributed)

• Distributed system Architecture
• Clould
• Gride
• Peer to peer
• Process communication
• RPC
• Process migration, agents
• Naming
• Clock time, synchronization
• Distributed file system
• Kernel support
• Transaction
• Etc.

28
Some Deep Questions

• How do we organize the OS effectively for
  development, evolution, performance, and
  security?
• How do we design a distributed OS that can be
  used on multiple machines?
• How do we use multi-processor machines
  effectively?

29
Some Questions on processes

• How do processes communicate and share states
  efficiently and securely on the same machine?
  Across multiple machines?
• How do we improve the computing process model?
• How do we achieve fairness, high throughput, and
  responsiveness at the same time?
• How do we reduce or avoid the cost of context
  switching?

30
Some Deep Questions

• How do systems achieve agreement across multiple
  machines?
• How do you represent the notion of time and the
  ordering of events across multiple machines?
• How do we coordinate machines to share memory?
• How can we simplify memory management as memory
  becomes abundant?

31
Some Deep Questions

• How do we make different file systems work
  together, even across machines?
• How do we provide consistency, availability, and
  reliability to copies of a file across multiple
  machines?
• How do we handle very large data sets?
• How do we coordinate the memory resources across
  machines to enhance performance?
• How do we handle new devices with new
  characteristics?

32
You Live in Interesting Times

• Processors speed used to double in 18 months, but
  we are reaching the upper bound (due to thermal
  problems) and need to go towards processor
  parallelism to increase the processing power
• Disk doubling every 12 months
• Global bandwidth every 6 month
• What will the future OS be?
• (If population doubles every year, or people can
  move twice faster every year, what does the
  government do?)

33
General guides (Broading)

• Main Point
• There aren't very many influential nerds, and
  there never will be. Why? They're too narrow.
  Real breakthroughs tend to come from people with
  breadth.
• One of the most important things you should do is
  to force yourself to stay broad.

34
General guides (Broading)

• Reasons to Stay Broad
• Breadth helps depth. The more you understand, the
  more you'll understand about each individual
  thing. Seeing how things in different areas are
  similar or different is a very good way of seeing
  what's important.
• Breakthroughs often occur when people can cross
  traditional boundaries compilers and
  architecture, graphics and VLSI, etc.
• Computers are tools they serve people. In order
  to build effective tools, you have to understand
  both the capabilities of computers, and the needs
  of the application areas where they'll be used.
• Technology is changing fast. Why is there a
  shortage of 25-year-old engineers and a surplus
  of 45-year-old ones? Companies encourage new
  graduates to get so narrow that they're instantly
  obsolete.

35
General guides (Broading)

• Solutions
• Continuing education. Try always to be learning
  in whatever you do. Don't let your education stop
  when you leave University.
• Explore new areas, both inside and outside
  Computer Science. Everything you learn will
  someday be helpful, no matter how unlikely it
  seems. English, art, hobbies, all things are
  helpful. Here's an example Steve Wozniak. After
  building the Apple 2, came back to Berkeley to
  get a masters. Around the same time, he and Jobs
  visited Xerox PARC, learned from the good ideas
  there, and built the Macintosh.

36
A computer system
37
Computer Hardware Review (1)
Monitor
Bus

• Components of a simple personal computer

38
Computer Hardware Review (2)

• (a) A three-stage pipeline
• (b) A superscalar CPU

39
Computer Hardware Review (3)

• Typical memory hierarchy
• numbers shown are rough approximations

40
Computer Hardware Review (4)

• Structure of a disk drive

41
Computer Hardware Review (5)

• One base-limit pair and two base-limit pairs

42
Computer Hardware Review (6)
(a)
(b)

• Steps in starting an I/O device and getting
  interrupt
• How the CPU is interrupted

43
Computer Hardware Review (7)

• Structure of a large Pentium system

44
Operating System Concepts (1)

• A process tree
• A created two child processes, B and C
• B created three child processes, D, E, and F

45
Processor (CPU) Management

• Goals
• Time sharing
• Multiple CPU allocations
• Issues
• Do not waste CPU resources
• Synchronization and mutual exclusion
• Fairness
• deadlock free

Analogy Video Games
46
OS Concepts (2)

• (a) A potential deadlock. (b) an actual deadlock.

47
Operating System Concepts (3)

• File system for a university department

48
Operating System Concepts (4)

• Before mounting,
• files on floppy are inaccessible
• After mounting floppy on b,
• files on floppy are part of file hierarchy

49
Operating System Concepts (5)

• Two processes connected by a pipe

50
Making a System Call

• There are 11 steps in making the system call
• read (fd, buffer, nbytes)

51
System Calls (1)

• A stripped down shell
• while (TRUE) / repeat forever /
• type_prompt( ) / display prompt /
• read_command (command, parameters) / input
  from terminal /
• if (fork() ! 0) / fork off child process
  /
• / Parent code /
• waitpid( -1, status, 0) / wait for
  child to exit /
• else
• / Child code /
• execve (command, parameters, 0) / execute
  command /

52
System Calls (2)

• Processes have three segments text, data, stack

53
System Calls (3)

• (a) Two directories before linking/usr/jim/memo
  to ast's directory
• (b) The same directories after linking

54
System Calls (4)

• (a) File system before the mount
• (b) File system after the mount

55
Operating System Structure (1)

• Simple structuring model for a monolithic system

56
Operating System Structure (2)

• Structure of the THE operating system

57
Operating System Structure (3)

• Structure of VM/370 with CMS
• CMS- Conversational Monitor System

58
Operating System Structure (4)

• The client-server model

59
Operating System Structure (5)

• The client-server model in a distributed system

60
Scheduling polling vs. interrupts

• Maintain peak performance under heavy load
• Interrupts model can lead to livelock
• Solution
• Use interrupts under low load (good latency)
• Use polling under heavy load (good throughput)
• Polling is typically more efficient than
  interrupts
• Fits naturally into asynchronous I/O model

61
Another Look Unix Onion
Applications
User and Kernel boundary
OS Service
Device Driver
Hardware
62
Other design issues

• Disk scheduling
• Elevator algorithm
• Memory management
• File system buffer cache
• Address spaces (VM management)
• Fault isolate different servers
• Efficient local communication?
• Efficient transfers between disk and networks
• Avoid copies

63
More than one processor

• Problem single machine may not scale to enough
  clients
• Solutions
• Multiprocessors
• Helps when CPU is bottleneck
• Server clusters
• Helps when bandwidth between server and backbone
  is high
• Distributed server clusters
• Helps when bandwidth between client and distant
  server is low

64
Bootstrapping

• Power up a computer
• Processor reset
• Set to known state
• Jump to ROM code
• Load in the boot loader from stable storage
• Jump to the boot loader
• Load the rest of the operating system
• Initialize and run

Boot loader
Boot loader
OS sector 1
OS sector 2
. . .
OS sector n
65
Design Tradeoffs

• All in the kernel (Windows)
• Pros efficient?
• Cons difficult to develop new services
• All at user level - Nemesis (Cambridge, UK)
  only device drivers exist, all other management
  was done at user level), TinyOS (UCB) runs on
  Modes/ Sensors
• Pros easy to develop new apps
• Cons protection
• Split between user and kernel (Unix/Linux)
• Kernel display driver and mouse driver
• User the rest

66
Next Lecture

• System design
• Read The Interaction of Architecture and
  Operation System Design, Thomas E. Anderson, et
  al.
• Lisp Good News, Bad news, How to Win Big,
  Richard P. Gabriel