File System and Input/Output Lecture 10: File Systems

1
File System and Input/OutputLecture 10 File
Systems

• ??????

2
Content

• Basic Functions of File Management
• Hierarchical Model of a File System
• File Directories
• Hierarchical Directory Organizations
• Operations on Directories
• Implementation of File Directories
• Basic File System
• File Descriptors
• Opening and Closing of Files
• Physical Organization Methods
• Contiguous Organization
• Linked Organization
• Indexed Organization
• Management of Free Storage Space
• Distributed File Systems
• Directory Structures and Sharing
• Semantics of File Sharing
• Implementing DFS

3
File System and Input/OutputLecture 10 File
Systems

• Basic Functions of File Management

4
Basic Functions of FS

• Present logical or abstract view of files and
  directories
• Hide complexity of hardware devices
• Facilitate efficient use of storage devices
• Optimize access, e.g., to disk
• Support sharing
• Provide protection

5
File System and Input/OutputLecture 10 File
Systems

• Hierarchical Model of a File System

6
Hierarchical Model of FS
7
Hierarchical Model of FS
8
The File System

• Presents the abstract or logical view of files
  to users
• Defines a set of operations for file/directory
  manipulation

File manipulation
9
The File System
Map logical name tounique Id, file descriptor
10
The File System

• Open/close files
• access right verification
• maintain open file tables (OFT)

11
The File System

• Map file data to logical blocks on the device
• Main memory buffer allocation/deallocation
• Keep track the mapping btw memory buffer
• and data blocks

12
File Names and Types

• File names and types
• Logical file organization
• Other file attributes
• Operations on files

• Valid name
• Number of characters
• Lower vs upper case
• Extension
• Tied to type of file
• Used by applications
• File type recorded in header
• Cannot be changed (even when extension changes)
• Basic types text, object, load file directory
• Application-specific types, e.g., .doc, .ps, .html

13
File Names and Types

• File names and types
• Logical file organization
• Other file attributes
• Operations on files

• Valid name
• Number of characters
• Lower vs upper case
• Extension
• Tied to type of file
• Used by applications
• File type recorded in header
• Cannot be changed (even when extension changes)
• Basic types text, object, load file directory
• Application-specific types, e.g., .doc, .ps, .html

14
Logical File Organization

• File names and types
• Logical file organization
• Other file attributes
• Operations on files

• Logical Records
• Fixed or variable-size records
• Keyed or unkeyed records

a) Fixed Length Record b) Variable Length
Record c) Fixed Length with Key d) Variable
Length with Key
15
Logical File Organization

• File names and types
• Logical file organization
• Other file attributes
• Operations on files

• Logical Records
• Fixed or variable-size records
• Keyed or unkeyed records

a) Fixed Length Record b) Variable Length
Record c) Fixed Length with Key d) Variable
Length with Key
16
Logical File Organization

• File names and types
• Logical file organization
• Other file attributes
• Operations on files

• Record Addressing
• Implicitly (sequential access)
• Explicitly by position or key (direct or random
  access)

a) Fixed Length Record b) Variable Length
Record c) Fixed Length with Key d) Variable
Length with Key
17
Logical File Organization

• File names and types
• Logical file organization
• Other file attributes
• Operations on files

• Memory Mapped File
• Making use of virtual memory system
• Read virtual memory instead of read(i,buf,n)
• Map file contents 0(n?1) to va(van?1)

CreateFileMapping
18
Other file attributes

• File names and types
• Logical file organization
• Other file attributes
• Operations on files

• Ownership
• File Size
• File Use
• Time of creation, last access, last modification
• File Disposition
• Permanent or Temporary
• Protection
• Who can access and what type of access
• Location

19
Operations on files

• File names and types
• Logical file organization
• Other file attributes
• Operations on files

• Create/Delete
• Create/delete file descriptor
• Modify directory
• Open/Close
• Open ? Setup open file table (OFT) buffer
• Close ? Reverse the effect of open
• Read/Write (sequential or direct)
• Modify file descriptor
• Transfer data between disk and buffer
• Seek/Rewind
• Modify open file table

20
File System and Input/OutputLecture 10 File
Systems

• File Directories

21
File Directories

• Every file directory is itself a file.
• It records the information about other files,
  including possibly other directories.

22
Hierarchical Directory Organizations

• Tree-Structured
• DAG-Structured

23
Tree-Structured Directories

• Simple search, insert, delete operations
• Sharing is asymmetric (only one parent, owner)

24
DAG-Structured Directories
25
DAG-Structured Directories
File F8 shared
\a\r\p
\b\p
\a\t\e
26
DAG-Structured Directories
Directory D6 shared
\z
\a\t
27
DAG-Structured Directories

• Sharing is symmetric, but
• What are semantics of delete?
• Any parent can remove file.
• Only last parent can remove it.Need reference
  count

28
DAG-Structured Directories

• Must prevent cycles

29
DAG-Structured Directories

• Must prevent cycles
• If cycles are allowed
• Search is difficult (infinite loops)
• Deletion needs garbage collection (reference
  count not enough)

30
DAG-Structured Directories
Deletion needs garbage collection (reference
count not enough)
How to do rd \a, i.e., remove directory D2 ?
?
?

1. Delete entry a of root.
2. Start from root to mark all reachable nodes.
3. Delete all unmarked nodes.

marked
Very inefficient
unmarked
31
Symbolic Links
Main directory is tree-structured. Sharing via
symbolic link ? allow cycles.
32
Symbolic Links
For read/write access Symbolic link is the
same as actual link For deletion Only
symbolic link is deleted
Root
33
Symbolic Links
For read/write access Symbolic link is the
same as actual link For deletion Only
symbolic link is deleted
Root
D1
a
z
b
D2
D3
c
n
a
p
l
r
t
D4
D5
D6
e
f
h
k
p
n
m
k
F1
F2
F3
F9
D7
q
b
c
F8
F4
F5
F6
F7
del \b\p
F10
F11
34
Symbolic Links
For read/write access Symbolic link is the
same as actual link For deletion Only
symbolic link is deleted
Root
D1
a
z
b
D2
D3
c
n
a
l
r
t
D4
D5
D6
e
f
h
k
p
n
m
k
F1
F2
F3
F9
D7
q
b
c
F8
F4
F5
F6
F7
del \b\p
F10
F11
35
Symbolic Links
For read/write access Symbolic link is the
same as actual link For deletion Only
symbolic link is deleted
Root
D1
a
z
b
D2
D3
c
n
a
p
l
r
t
D4
D5
D6
e
f
h
k
p
n
m
k
F1
F2
F3
F9
D7
q
b
c
F8
F4
F5
F6
F7
del \a\t\e
F10
F11
36
Symbolic Links
For read/write access Symbolic link is the
same as actual link For deletion Only
symbolic link is deleted
Root
D1
a
z
b
D2
D3
c
n
a
l
r
t
D4
D5
D6
f
h
k
n
m
k
F1
F2
F3
F9
D7
q
b
c
Phantom References
F4
F5
F6
F7
del \a\t\e
F10
F11
37
Symbolic Links
For read/write access Symbolic link is the
same as actual link For deletion Only
symbolic link is deleted
Root
rd \a
D1
a
z
b
D2
D3
c
n
a
p
l
r
t
D4
D5
D6
e
f
h
k
p
n
m
k
F1
F2
F3
F9
D7
q
b
c
F8
F4
F5
F6
F7
F10
F11
38
Symbolic Links
For read/write access Symbolic link is the
same as actual link For deletion Only
symbolic link is deleted
Root
rd \a
D1
b
D3
c
n
a
F1
F2
F3
Phantom References
39
Symbolic Links
For read/write access Symbolic link is the
same as actual link For deletion Only
symbolic link is deleted
Root
rd \a\l\m\q
D1
a
z
b
D2
D3
c
n
a
p
l
r
t
D4
D5
D6
e
f
h
k
p
n
m
k
F1
F2
F3
F9
D7
q
b
c
F8
F4
F5
F6
F7
F10
F11
40
Symbolic Links
For read/write access Symbolic link is the
same as actual link For deletion Only
symbolic link is deleted
Root
del \a\l\m\q
D1
a
z
b
D2
D3
c
n
a
p
l
r
t
D4
D5
D6
e
f
h
k
p
n
m
k
F1
F2
F3
F9
D7
b
c
F8
F4
F5
F6
F7
F10
F11
41
File/Directory Naming Path Names

• Concatenated local names with delimiter
• . or / or \
• Absolute path name start with root
• / or \
• Relative path name Start with current directory
• .
• Notation to move upward
• ..

42
Operations on File Directories
Original
Copied

• Create/delete
• List
• sorting, wild cards, recursion, information
  displayed
• Change (current, working, default) directory
• path name, home directory (default)
• Move
• Rename
• Change protection
• Create/delete link (symbolic)
• Find/search routines

43
Implementation of File Directories

• What information to keep in each entry?
• Only symbolic name and pointer to descriptor
• Needs an extra disk access to descriptor
• All descriptive information
• Directory can become very large
• How to organize entries within directory?
• Fixed-size array of slots or a linked list
• Easy insertion/deletion
• Search is sequential
• Hash table
• Easy insertion/deletion/search
• Hard to determine the table size
• B-tree (balanced, but sequential access can be
  slow)
• B-tree (balanced and with good sequential access)

44
File System and Input/OutputLecture 10 File
Systems

• Basic File System

45
Basic File System

• Open/close files
• access right verification
• maintain open file tables (OFT)

46
Open/Close Files

• Retrieve and set up descriptive information for
  efficient access.

Block-oriented I/O in UNIX
Low-level I/O in Windows
Stream I/O in Windows
FILE fopen( const char filename, const char
mode ) int fclose( FILE stream ) size_t
fread( void buffer, size_t size, size_t count,
FILE stream ) size_t fwrite( const void
buffer, size_t size, size_t count, FILE stream
)
47
File Descriptors (Unix i-node)

• Owner id
• File type
• Protection information
• Mapping to physical disk blocks
• Time of creation, last use, last modification
• Reference counter

48
Open File Table

• To keep track the currently open files of the
  system.
• Managed by the open/close functions

49
Open

• Verify access rights
• Allocate OFT entry
• Allocate read/write buffers
• Fill in OFT entry
• Initialization (e.g., current position)
• Information from descriptor (e.g. file length,
  disk location)
• Pointers to allocated buffers
• Return OFT index

50
Close

• Flush modified buffers to disk
• Release buffers
• Update file descriptor, e.g.,
• file length
• disk location
• usage information (e.g., date of last
  access/modification)
• Free OFT entry

51
ExampleOpen File Tables in Unix

• Unbuffered access
• fdopen(name,rw,) statread(fd,mem,n)
  statwrite(fd,mem,n)
• Buffered access
• fpfopen(name,rwa) creadc(fp)

52
ExampleOpen File Tables in Unix

• Unbuffered access
• fdopen(name,rw,) statread(fd,mem,n)
  statwrite(fd,mem,n)
• Buffered access
• fpfopen(name,rwa) creadc(fp)

53
File System and Input/OutputLecture 10 File
Systems

• Physical Organization Methods

54
Main Secondary Storage Media
disks
tapes
From the file system of views, they all
considered as a device with 1D sequence of
logical blocks by numbering them from 0 to n?1.
55
Device Organization Methods

• Contiguous organization
• Linked Organization
• Indexed Organization

56
Contiguous Organization

• Contiguous organization
• Linked Organization
• Indexed Organization

Device Organization Methods

• Simple implementation
• Fast sequential access (minimal arm movement)
• Insert/delete is difficult
• How much space to allocate initially
• External fragmentation

57
Linked Organization

• Contiguous organization
• Linked Organization
• Indexed Organization

Device Organization Methods

• Simple insert/delete, no external fragmentation
• Sequential access less efficient (seek latency)
• Direct access not possible
• Poor reliability (when chain breaks)

58
Linked Organization Variations

• Contiguous organization
• Linked Organization
• Indexed Organization

Device Organization Methods

• Variation 1 Keep pointers segregated
• Variation 2 Link sequences of adjacent blocks

59
Indexed Organization Variations

• Contiguous organization
• Linked Organization
• Indexed Organization

Device Organization Methods

• Multi-level index hierarchy
• Primary index points to secondary indices
• Problem number of disk accesses increases with
  depth of hierarchy
• Incremental indexing
• Fixed number of entries at top-level index
• When insufficient, allocate additional index
  levels
• E.g., Unix ? 3-level expansion (see next)

60
Example Unix -- 3-Level Expansion

• Contiguous organization
• Linked Organization
• Indexed Organization

Device Organization Methods
Incremental indexing
61
Free Storage Space Management
Similar to main memory management.

• Linked list organization
• Linking individual blocks ? inefficient
• Multiple of blocks are allocated/released at a
  time
• No block clustering to minimize seek operations
• Linking groups of consecutive blocks
• Bitmap organization
• Analogous to main memory

62
File System and Input/OutputLecture 10 File
Systems

• Distributed
• File Systems

63
Principles of Distributed File Systems

• Sharing
• to support sharing of files and resources in the
  form of persistent storage over a network.
• Transparencies
• to present a unifying view of all files to the
  user
• Location transparent
• Location independent

64
Directory structures

• Directory structures differentiated by
• Global vs. Local naming
• Single global structure or different for each
  user?
• Location transparency
• Does the path name reveal anything about machine
  or server?
• Location independence
• When a file moves between machines, does its path
  name change?

65
Global Directory Structure
Server S1
Server S2
66
Global Directory Structure
How to build a global directory?
Server S1
Server S2
67
Global Directory Structure
How to build a global directory?
Global root directory
S1
S2
Server S1
Server S2
local root directory (/)
local root directory (/)
68
How to Name a File/Directory?
Problem with Combine under new common
root Using / for new root invalidates
existing local names.
Global root directory
S1
S2
Server S1
Server S2
local root directory (/)
local root directory (/)
root directory (/)
root directory (/)
usr
mp
? ? ?
u1
? ? ?
? ? ?
? ? ?
/S1/usr/u1
69
How to Name a File/Directory?
Problem with Combine under new common
root Using / for new root invalidates
existing local names.

• Solution (Unix United)
• Use / for local root
• Use .. to move to new root
• Names are not location transparent

Global root directory
S1
S2
Server S1
Server S2
local root directory (/)
local root directory (/)
root directory (/)
root directory (/)
usr
mp
? ? ?
u1
? ? ?
? ? ?
? ? ?
/S1/usr/u1