TDC368 UNIX and Network Programming

1
TDC368UNIX and Network Programming

• Week 1
• Course Introduction
• Overview of UNIX Operating Systems Architecture
• UNIX process concepts

• Camelia Zlatea, PhD
• Email czlatea_at_cs.depaul.edu

2
Course Outline

• Overview of UNIX operating system architecture.
• UNIX process concepts. Process control primitives
  (fork, exec, exit, wait, waitpid).
• Filters and I/O Redirection. Interprocess
  Comunications (IPC) pipes, FIFO's.
• Asynchronous Events. Signals. Signal Functions
  (kill, system, sleep, raise, alarm, pause,
  abort).
• UNIX system calls C library functions, C
  classes, POSIX API's.
• UNIX File System files, catalogs (inodes
  types,size, permissions).
• UNIX File I/O unbuffered I/O functions (open
  ,close,seek,umask,read,write).
• Standard I/O buffering (fopen, fclose, fflush).
  File Sharing (dup,dup2,fcntl).

3
Course Outline (cont.)

• Interprocess Communication Shared Memories,
  Semaphores, POSIX Semaphores, lock files.
• Socket API (TCP, UDP).
• Design Methods for Client/Server Applications
• UNIX shells. C-shell commands, shell variables.
  C-Shell Programming.
• Managing UNIX projects (make). Compilation and
  Debugging.
• UNIX shells I/O redirection, filters(grep, sed),
  awk.

4
Objectives

• Upon completion of this course, students
• will understand the fundamental concepts of UNIX
  operating system and use them to create various
  multitasking applications.
• Will be familiar with UNIX programming
  environment, system call interface, ANSI C and
  ANSI/ISO C functions and classes
• Will be familiar with Client/Server methods and
  Socket APIs
• Will be proficient in UNIX shell programming

5
Prerequisites

• CSC 309
• Basic understanding of operating systems and
  programming techniques
• C/C will be used to illustrate the concepts
  presented in the lecture material and for the
  implementation of the programming assignments and
  the project.

6
Texts

• Dave Curry, UNIX Systems Programming for SVR4,
  O'Reilly Assoc., Sebastopol, CA, 1996.
• W. Richard Stevens, UNIX Network Programming -
  Interprocess Communication, Volume 2, Second
  Edition, Prentice Hall PTR, Upper Saddle River,
  NJ, 1999.
• W. Richard Stevens, Advanced Programming in the
  UNIX Environment, Addison-Wesley, Reading, MA,
  1992.
• John Shapley Gray, Interprocess Communications in
  Unix, The Nooks Crannies, 2nd ed. Prentice Hall
  PTR, NJ, 1998.
• HP-UX man pages
• http//docs.hp.com/hpux/onlinedocs/B2355-90682/B2
  355-90682.html (system calls)

7
Other References

• Daniel Gilly, Unix in a Nutshell, O'Reilly and
  Associates, Inc., Sebastopol, CA, 1992.
• K.A.Robbins and S. Robbins, Practical UNIX
  Programming - A Guide to Concurrency,
  Communication, and Multithreading , Prentice Hall
  PTR, NJ, 1996
• Ellie Quingley, UNIX Shells by Example, Prentice
  Hall PTR, NJ, 1997
• Prabhat K. Andleigh, UNIX System Architecture,
  Prentice-Hall, Englewood Cliffs, NJ, 1990.
• Andrew Oran and Steve Talbot, Managing Projects
  with make, O'Reilly and Associates, Inc.,
  Sebastopol, CA, 1991.
• Brian W. Kernighan and Dennis M. Ritchie, The C
  Programming Language, Second Edition,
  Prentice-Hall, Englewood Cliffs, NJ, 1989.

8
Class Work and Grading
9
Programming Environment

• All students will get an account on
  hawk.depaul.edu server (running HP-UX).
• Optional, you can use Linux system
• Project submissions via COL (dlweb.cti.depaul.edu
  )
• use Submit Assignments link
• Grading will be done on HP-UX or Linux
• An experimental report is required for each
  programming project
• The the report should include the program design
  description and the test cases and test
  scenarios for the application.

10
TDC368 Network Programming

• Instructor Camelia Zlatea, PhD
• Office Hours T 445-545PM 900-930PM
• Where Rm. 429, ext. x26149 (during office hours)
• or leave a message to setup a conference
  630-979-3621
• Email czlatea_at_cs.depaul.edu
• Web http//condor.depaul.edu/czlatea/TDC368/
  ()
• Note
• In addition, course materials will be available
  from dlweb.cti.depaul.edu

11
Class Communication

• COL (dlweb.cti.depaul.edu) will be used primarily
  for
• Links to course materials/assignments
• Grades
• Announcements
• Email/Communication
• Newsgroup
• Asking questions
• Send me an email at czlatea_at_cs.depaul.edu
  OR
• Use dlweb.cti.depaul.edu and post questions to
  class forum
• Anybody may reply (including your classmates)
• Everyone benefits from common issues

12
UNIX - Brief History

• 1969, Ken Thompson, Bell-Labs.
• UNIX OS written in Assembly on PDP-7 (DEC)
• 1973, K.ThompsonD.Richie
• UNIX re-written in C
• 1975, Bell-Labs. offered UNIX to educational
  institutions, at minimal cost

13
UNIX Features

• Portability
• Multi-process architecture (multitasking)
• Multi-user capability
• Ability to initiate asynchronous processes
• A hierarchical file system
• Device independent I/O operations
• User interface Shell selectable per user basis

14
UNIX Major Versions

• ATT UNIX version V
• 1983 /SV, 1986 /SVR3.0
• ATT and Sun Microsystems
• 1988 /SVR4.0 - tries to unify Berkeley UNIX
  other
• Berkeley UNIX, CSRG at U. of California
• BSD (Berkeley Software Distribution)

15
UNIX Standards

• System V Interface Definition (SVID), ATT
• Portable Operating System Interface for Computer
  Environments (POSIX), based on SVID, IEEE
• ANSI C, American National Standard Institute
• ANSI/ISO C Standard (draft)

16
UNIX Implementations

• Solaris - Sun Microsystems
• SunOS (later called Solaris), Solaris 2.x based
  on SVR4
• HP-UX, Hewlett-Packard, SVR2
• Linux, Linus Torvalds, free distribution,
  PC-based
• AIX, IBM, similar to SVR4
• IRIX, Silicon Graphics, SVR4

17
UNIX Implementations

• UNIX-like
• same facilities as SV, does not use entirely the
  same code
• Mach (CMU), kernel is based on OO model server
• Next (implementation inspired by Mach with MacOS
  UI)
• UNIX-alike
• portability is no longer supported
• attributes not compliant with SV or BSD
• OS/9(MicroWare, Motorola 680xx, RT kernel)
• QNX(Quantum Software, Intel, RT processing)

18
UNIX Architectural Overview
19
UNIX Features

• Portability
• Multi-process architecture (multitasking)
• Multi-user capability
• Ability to initiate asynchronous processes
• A hierarchical file system
• Device independent I/O operations
• User interface Shell selectable per user basis

20
Multi-process/Multi-user architecture

• Virtual Machine
• timesharing OS
• process, process quantum, process states
• Kernel, base OS
• manages all HW dependent functions
• users have no direct access to it
• System Calls Interface
• service routine performing user requests

21
UNIX Processes

• A process may create sub-processes
• fork()
• A process may terminate
• exit()
• A process may put itself to sleep temporarily
• sleep(20)
• pause()
• wait()
• Processes
• synchronization mechanisms
• communication mechanisms

22
UNIX Threads

• Multiple Processes - concurrency at OS level
• Multiple Threads - concurrency at process level
• thread flow of control in a process
• multiple threads (stream of instructions) are
  executed within the same process
• threads share code data (address space)
• threads have their own execution stack, PC,
  register set and states
• context switches are avoided
• efficient mapping on multi-processor machines

23
Some Unix Terms

• TRAP INSTRUCTION
• switches a system call from user to kernel mode
• handles error conditions
• synchronous execution
• Ex division by zero/overflow
• INTERRUPT
• used by peripherals to request services from the
  devices handles
• asynchronous execution
• SIGNALS software notification of an event
• example
• when typing ctrl-c an interrupt is generated
  for the keyboard driver, which notifies the
  process by sending a signal

24
UNIX Kernel - model
User Space
User Processes
System Call Interface (Library Routines)
Process Memory File System I/O Services Mgmt.
Mgmt.
Kernel Space
Scheduler Device Drivers I/O Buffers
Hardware
25
Apps./Utilities
System Calls
UNIX KERNEL
Hardware Interrupts
Hardware
26
UNIX Kernel - low level

• Process representation, scheduling, dispatching
• Memory allocation and de-allocation
• Interrupt handling
• Low level device control
• Disk Mgmt., data buffering
• Process synchronization and IPC

27
Kernel - services level

• Maps user-level requests with device driver
  actions
• A user system call is translated to a call of the
  kernel routine, providing that requested service
• Type of Services
• process creation and termination
• I/O services
• UNIX file system services
• terminal handling services

28
Kernel - services level

• User program
• system(ps -e)
• System calls
• actions
• fork()
• //process creation service
• execl(/bin/ps, ps -e)
• //code execution

29
System Call Interface level

• A user mode process is translated into a
  protected kernel mode process
• Now, program can call kernel routines

30
User Processes level

• User processes running
• shells
• Unix commands
• utilities
• application programs

31
UNIX Standards

• UNIX and POSIX API
• Library Functions
• ANSI C Standard
• ANSI/ISO C Standard

32
UNIX and POSIX API

• UNIX API - system calls
• UNIX API are called by
• C library functions and
• C standard classes
• Example iostream class
• Program confirms POSIX.1?
• define _POSIX_SOURCE
• or
• CC -D_POSIX_SOURCE .C

33
UNIX and POSIX API

• API set to perform
• determine system configuration and user
  information
• file management
• process creation and management
• inter-process communication
• network communication

34
UNIX and POSIX API
User Process (User Mode of Execution)
an API is invoked
API execution completed
UNIX APIs level
Kernel mode of execution
API executed in protected mode
35
UNIX and POSIX API

• Context Switch from user to kernel mode
• more overhead than library functions, for the
  same task
• I/O lib.functions are buffered

36
UNIX and POSIX API

• UNIX API - system calls
• UNIX API are called by
• C library functions and
• C standard classes
• Example iostream class
• Program confirms POSIX.1?
• define _POSIX_SOURCE
• or
• CC -D_POSIX_SOURCE .C

37
UNIX and POSIX API

• Development elements
• Headers
• ltunistd.hgt
• ltsysgt (/usr/include/sys directory)
• ltstdio.hgt
• function perror - system diagnostic message for
  an API execution failure
• Object Code
• libc.a and libc.so (on most systems)
• other special libraries, libsocket.a and
  libsocket.so

38
UNIX and POSIX API

• include ltstdio.hgt
• void perror(const char s)
• include lterrno.hgt
• int errno
• include ltstring.hgt
• char strerror(int errnum)

39
Library Functions

• Library of functions
• object code of the functions stored in archives
  (library files)
• run-time library routines
• UNIX archive ar utility manage library files.
• Displaying the table of contents of the standard
  library
• ar -t /usr/lib/libc.a

40
ANSI C Standard

• Standard C library functions
• memory management
• get system information
• get/set time information
• data conversion, processing
• Advantages
• portability
• maintainability
• cost effective for application design
• Disadvantages
• duration of execution
• weak support for time-critical apps.

41
Standard C library functions

• Header files directory /usr/include
• Header files
• - ltstdio.hgt - ltstdlib.hgt
• - ltstring.hgt
• - ltmemory.hgt - ltmalloc.hgt
• - lttime.hgt
• - ltassert.hgt - ltstdarg.hgt
• - ltgetopt.hgt - ltsetjmp.hgt
• Object code directory /usr/lib
• libc.a , libc.so

42
Standard C library functions

• ltstdio.hgt - declares FILE data type
• functions/macros -manage stream files
• open/close stream files fopen/fclose
• read/write block data fread/fwrite
• read/write lines fgets/fputs
• write formatted data fprintf
• positions/returns stream file locationfseek/ftell
  
• macros feof, ferror
• unnamed pipes popen/pclose

43
Example ls -lgrep tdc368

• include ltstdio.hgt
• int main()
• / execute the command /
• FILE cmd popen(ls -lgrep tdc368,r)
• if (!cmd)
• perror(popen) return 1
• char answer256
• / read grep process execution output /
• while(fgets(answer, sizeof(answer),cmd))
• fputs(answer, stdout) / echo /
• pclose(cmd)
• return 0

44
Standard C library functions

• ltstdlib.hgt - declares set of functions for data
  conversions, get/set env. variables, shell
  command execution, process termination
• atoi, atof, atol, strtod, strtol
• rand, srand
• getenv, putenv
• system
• exit

45
Example ls -lgrep tdc368

• include ltstdlib.hgt
• int main()
• / execute the command /
• if (system(ls -lgrep tdc368) -1)
• perror(system) return 1
• return 0

46
Example getenv/putenv

• include ltiostream.hgt
• include ltstdlib.hgt
• int main()
• char env getenv(REMOTEHOST)
• cout ltlt \REMOTEHOST\ isltltenvltlt\n
• if (putenv(CCc))
• cerltltputenv of CC failed\n)
• return 0

47
Example ltmalloc.hgt

• //C style
• char adr1 (char )malloc(4096)
• free(adr1)
• //C style , no use of malloc !
• char adr2 new char4096
• delete adr2

48
Standard C library functions

• lttime.hgt - declares set of functions for system
  clock query
• time - sec since 01/01/70 (UNIX birth)
• ctime - current local time
• localtime
• gmtime
• asctime
• mktime
• clock - (ANSI C) , microsec since the process
  first called clock

49
Example lttime.hgt

• // evaluate process execution time
• include ltiostream.hgt
• include lttime.hgt
• main()
• time_t tick CLOCKS_PER_SECOND
• clock_t start_time clock() // start timer
• / process code here /
• clock_t duration clock() - start_time
• cout ltltDurationltlt(duration/tick)ltltendl

50
Standard C library functions

• ltassert.hgt - declares a macro to assert
  conditions or invariants of the program
• easier program debugging
• compiler option -DNDEBUG take them out
• or
• ifndef NDEBUG
• define assert(p) if (!(p)) \
• fprintf(stderr,Assertion failed \
• file\s\,line d\n,\
• _FILE_, _LINE_) exit(1)
• endif

51
Example ltassert.hgt

• include ltfstream.hgt
• include ltstring.hgt
• include ltassert.hgt
• int main(int argc, char argv)
• assert(argc gt1) // should have 1 arg
• ifstream ifs(argv1)
• assert(ifs.good()) // should be opened OK
• char adr new charstrlen(argv1)1
• assert(adr) // should not be NULL

52
ANSI/ISO C Standard

• 1980, Bjarne Stroustrup, C
• 1994, ANSI/ISO C standard (draft)
• C compiler (ATT C V3.0)
• support
• C classes, derived classes
• virtual functions, operator overloading
• templates classes, functions
• exception handling
• iostream classes

53

• Compilation
• cc -o file file.c
• file
• Man Pages
• man cc
• man sys_call
• man shell_cmd

54
Processes

• Process - a program in execution
• process - active entity
• program - passive entity (binary file)
• Address Space - list of memory locations from
  where a process reads/writes (code/data/stack)
• Set of registers (PC, SP, ...)
• Process Table - linked list of structures
  associates w/ processes
• System Calls - interface between OS and User
  process

55
Process Control Block (process attributes)

• Process State
• new, ready, running, blocked, terminated
• Process Image Map Pointer
• Process ID
• assigned at process creation
• Program Counter (PC)
• address of next instruction to be executed
• CPU Registers (saved process context)
• List of Open File Descriptors
• I/O Devices Attached
• CPU Scheduling Info (priority)

56
Process Image Map
Process Table

Process Image
Proc. n
Proc. 1
Text/Code Segment
Data Segment
Stack Segment
Process Control Block
57
(No Transcript)
58
Example

• / Display Segment Address Information
• /
• include ltstdio.hgt
• extern int etext,edata,end
• void main(void)
• printf(etext 6X\t edata 6X \t end 6X \n,
  etext, edata, end)

59
Process States

• New - process created ( Ex fork() )
• Ready - process is waiting to be assigned to
  processor (inserted in ready queue)
• Running - instructions are being executed
• Blocked - wait for events to occur (inserted
  in queue) Ex wait() pause()
• Terminated - normal/abnormal
  termination (exit())

60
Process Model

New
created
Ready
wakeup
Quantum Expired
dispatch
Blocked/ Suspended
Running User Mode
sleep
exit
System Call Interrupt
Return
Running Kernel Mode
Terminated
Interrupt Interrupt return
61

• Context of a Process
• process state (defined by its code)
• value of u-area
• values of registers the process uses
• contents of user and kernel stacks
• is associated with process image map
• Context Switching
• system executes a process in the context of the
  process
• when the kernel decides to execute another
  process, it does context switching
• kernel saves enough information such that it can
  later switch back to the first process and
  resumes its execution
• Mode Switching
• moving from user to kernel mode
• kernel save information to return to user mode

62

• User mode
• processes in use mode can access their own
  instructions and data NOT kernel or other
  processs code or data
• Kernel mode
• process can access system(kernel) code and data
  and user addresses
• Kernel is part of each process
• Kernel executes on behalf of the process
• P1 P2 P3 P4

OS
HW
Kernel Mode User Mode
K
K
U
U
63
Context Switching
P2
P1
OS
Save state in PCB1
Reload state from PCB2
Save state in PCB2
Reload state from PCB1
64

Context Switching

• Switching the CPU to another process by saving
  the state of the old process (PCB) and load the
  state of the new process (PCB)
• Pure Overhead
• Performance Bottleneck
• Avoid Overhead of Context Switching by
  introducing new structures
• THREADS

65

Context Switching

• Multitasking
• Sequential Execution