Part I: Introduction

1
Sharing Unix File Systems

• NFS (Network File System ) allows you to share
  file systems
• Reasons for sharing file system
• Transparent to user
• User can keep use their familiar commands
• Access the same file from multiple nodes.
• To provide disk space to diskless clients
• To prevent duplication
• To provide centrally supported programs and data
• To share data among users
• Simplifies central support tasks
• Such as backup.
• Space usage monitor

2
NFS

• Candidates could be
• Home dirs,
• Web page,
• /usr/local and other common utilities
• Documentations
• Temporary huge space request
• Source code repository

3
NFS protocol versions

• NFS was introduced by Sun Microsystems in 1985
• The original public release of NFS was version 2
• In 1990s, version 3
• Increases performance
• Makes writes safely asynchronous
• Better support for large files
• version 4
• No ancillary protocols integrated locking and
  mount
• Compound operations bundle multiple RPC
  together in a single exchange
• Strong security uses RPCSSEC_GSS API
• Require use of transport protocols that offer
  congestion control hence NFS v4 will not
  support UDP transport.

4
More about NFS

• NFS run on top of Suns RPC (Remote Procedure
  Call) protocol
• RPC provides a system-independent way for
  processes to communicate in a client-server
  fashion over a network.
• Some NFS versions run on top of UDP and TCP
• Why?
• WebNFS
• Promote the use of NFS over the Internet
• A extention of NFS protocol that allow easier
  access to servers and clients through Internet
  firewalls
• A WebNFS-enhanced web browser can use an NFS URL
  to access data directory from the server
• nfs/www.YourCompany.com/

5
Security and NFS

• NFS protocol was originally designed with
  essentially no concerns for security
• Tight control over shared filesystems
• Dont export any filesystems if anyone that you
  dont trust has root access on a client host
• If you have a firewall, block port 2049, block
  SunRPC portmap deamon port 111
• Kerberos to authenticate to ensure that remote
  users really are who they say they are.
• Ipsec to encrypt the data

6
Network File System

• NFS consists of a number of components
• Mounting protocol
• Mount server
• Daemons that coordinate basic file service
• Several diagnostic utilities
• Basically two parts
• Client
• Mount a directory
• Server
• Export a directory

7
Server-side NFS

• Mount a filesystem
• Daemon mountd
• Access files
• Daemon nfsd
• Both daemon started at boottime
• Mountd and nfsd use the same database ( (xtab on
  most systems, sharetab on Solaris)
• There are commands (exportfs, share) to
  add/remove the entries in the database
• Exportfs reads /etc/exports file
• Run exportfs a
• To remove entries, run exportfs u

8
Server-side NFS

• On Solaris, /etc/dfs/dfstab is a shell script
• Run shareall command, which equivalent to sh
  /etc/dfs/dfstab
• To remove entries, run unshare
• Solaris also provides command exports, which
  translates export options to share/unshare
  commands
• Any directory can be exported
• Client can mount a subdirectories of an exported
  directory
• Example On HP-UX 11i
• You can do above, but you may not to mount
  another sub dir from the same exported directory.
• Each device to be exported separately
• For example
• /user is a separate partition, / can be exported
  without exporting /users

9
NFS Daemons on Solaris

• The following daemons are running on server side
• nfsd nservers
• mountd
• nfslogd
• The following daemons are running on both server
  side and client side
• rquotad
• lockd
• statd
• How to start the daemons
• Daemons are under /usr/lib/nfs
• Daemons are started at boot time by
  /etc/init.d/nfs.client and nfs.server
• nfs.client starts statd and lockd
• nfs.server starts nfsd (16 copies default),
  mountd, nfslogd
• inetd start rquotad.

10
Configure NFS on Solaris

• The share command to export a dir
• share F nfs -o options pathname
• Options are
• rw
• Export read-write to the entire world
• rwaccesslist
• Export read-only with access only by listed hosts
• ro
• roaccesslist
• rootlist
• Lists hosts permitted to access this filesystem
  as root
• Otherwise, root access from a client is
  equivalent to access by nobody (UID 2)

11
Configure NFS on Solaris

• anonuid
• defines the uid used for users who do not provide
  a valid user ID.
• nosub
• Forbids clients to mount subdirectories of the
  exported directory
• nosuid
• Prevents setuid and setgid files from being
  created via NFS
• Access list format
• Separated by
• Identify computers by
• individual hostnames/IP
• Domain, .domain
• NIS netgroup, groupname
• Network, _at_network/prefix

12
Configure NFS on Solaris

• Examples on Colossus /etc/dfs/dfstab
• share -F nfs -o
• rootaa.aaa.aa
• rwkeeper.csl.mtu.educslscifilanguagepizzaic
  ucsfishlabcec
• /home/csdept
• share -F nfs -o
• rootxx.xxx.xxyy.yy.yyy
• rwcsllanguagescifipizzaicucsfishlabphyspe
  cialceccs.mtu.edu
• /export/major

13
Configuring NFS on Linux

• /etc/exports
• The NFS server configuration file
• Controls which files/dirs are exported
• Access control
• The general format of entries
• directory host(option)
• Host can be
• empty, means every host
• individual host, name or IP address
• Domains, wrotethebook.com
• Networks, IP address/mask(prefix length)
• Netgroups, _at_group1
• Option can be
• ro
• rw

14
LINUX NFS Permission Control

• NFS server trusts local authentication
• USE GID/UID to do regular Unix permission
  control.
• Mismatch uid/gid will be a unexpected problem
• What about root?
• Do you want root users on NFS client to act like
  root users on NFS servers? Probably Not.
• NFS prevents by default setting root_squash
• Mapping root to nobody UID/GID
• Set no_root_squash to allow it.
• Option all_squash
• Map all uids and gids to the anonymous user.
  Useful for NFS-exportd public FTP directories,
  news spool directories, etc.

15
LINUX NFS

• Assign anonymous a UID/GID using option
• anonuidXXX, anongidXXX
• This option is primarily useful for PC/NFS
  clients, where you might want all requests appear
  to be from one user.
• UID/GID mapping file using option
• map_staticfilename
• Command exportfs,
• build /var/lib/nfs/xtab
• export all in /etc/exports file using exportfs
  a
• Sync /etc/exports and /var/bin/nfs/xtab
  exportfs r
• Temporarily export a filesystem
• exportfs fox/usr/local o rw
• remove from export list exportfs u
  fox/usr/local
• remove all export list exportfs ua

16
Client-side NFS

• Mounting remote filesystems
• Use mount command to establish temporary network
  mounts
• List in /etc/fstab or /etc/vfstabb
• Automatic mounting services such as automount
• what are exported or where
• Ask administrator
• Find out yourself
• what have been exported from a host
• showmount e hostname

17
Mounting Remote Filesystems

• Using mount command
• Basic format
• mount hostnameremote-dir local_dir
• hostname is the NFS server
• local_dir must exist already
• Using umount command
• umount local_dirremote_dir

18
Mounting NFS

• On Solaris, entries in /etc/vfstab are mounted
  by mountall during system startup
• On Linux, entries in /etc/fstab are mounted via
  mount a in startup files.
• NFS mount flags ( page 502)
• Options
• rw
• ro
• bg
• hard
• cause the operations that try to access it to
  block until the server comes backup
• Soft
• What about a job that has been running 18 hours
  and will be done in an hour and aborted?

19
Mounting NFS

• More mounting options
• intr
• Allow user to interrupt blocked operations
• nointr
• retransn
• Specifies the number of times to repeat a request
  before returning an error on a soft-mountd
  filesystem
• timeon
• Set the timeout period for requests
• rsize
• 8K for the same network
• wsize
• vers
• tcp

20
Common mount options on Linux
Option Function
async
auto When a is used
dev Allow character and block special files on this file system
exec
noauto Dont mount even with -a
noexec
nosuid Dont allow setuid, setgid
nouser Only root can mount
remount
ro
rw
suid Allow programs to run setuid or setgid
sync
user Ordinary users to mount the file system
atime Update inode
noatime
defaults Rw, suid, dev, exec, auto, nouser, and async
21
Dedicated NFS file Servers

• Fast, reliable file service is one of the most
  important elements
• Dedicated NFS servers
• Optimized for file service
• Storage scale smoothly
• Reliable, simplified software
• Redundant hardware
• Support different file formats for different
  platforms, such as NFS, CIFS
• Easier to administer than UNIX file servers
• Good backup and checkpoint facilities

22
NFS Automounter

• Automatically mounts NFS when needed
• Why automount?
• Maintaining /etc/fstab can be tedious
• Minimizes the number of active mount points to
  reduce/Avoid chaos when servers crashes
• How?
• Mount a virtual filesystem driver on the
  directories for automatic mount to occur
• Kernel-resident filesystem driver called autfs is
  used
• When the user references a directory withing the
  virtual filesystem, the automounter intercepts
  the reference and mount the actual filesystem the
  user is trying to reach.

23
NFS Automounter

• Daemons
• automountd and automount are started by
  /etc/init.d/autofs
• Automount reads the configuration file, setup
  autofs mounts
• References to automounted filesystems are handled
  by a separate daemon automountd
• On solaris 10, you can start automounter
• svcadm enable system/filesystem/autofs
• The actual script is under /lib/svc/method/svc-aut
  ofs

24
automount

• automount
• Basic configuration files
• Master map
• Direct maps
• Indirect maps
• Direct and indirect maps provide information
  about filesystems that are to be automounted
• A master map lists direct and indirect maps that
  automount should pay attention to.
• Only one master map that can be active
• The default one is /etc/auto_master

25
automount

• Indirect maps
• Automount several filesystems underneath a common
  directory
• The path is specified in the master maps, not in
  the indirect map itself
• The name indirect map will be used as
  subdirectory to install the mount
• Example, indirect map
• Info ro chimchimL/chimchim/info
• Using ls a directory full of indirect mounts can
  be confusing
• automount does not show the subdirectories until
  their contents have been accessed
• What does ls do?

26
automount

• Direct maps
• List the maps that do not share a common prefix
• Such as
• /cs/tools ro anchor/cs/tools
• /usr/src chimchim/usr/src
• Each implemented with a separate autofs mount
• Require slightly overhead
• But mount point and directory structure are
  always accessible by commands such as ls

27
automount

• master map
• format mount-point map-name option
• List the direct and indirect map
• For indirect map, it specify root directory used
  by the mounts defined in the map.
• /- is for direct maps
• Option set the defaults for all mounts within the
  map
• ruihong_at_cslserver nis more /etc/auto_master
• cs and csl don't use auto_home and auto yet.
• /net -hosts -nosuid,nobrowse
• /home auto_home -nobrowse
• /local auto_local -nobrowse
• /fisher auto_fisher
• /- auto_direct

28
automatic automount

• Have automount to figure it out itself
• Query mountd running on a remote server to find
  out what filesystems the server exports
• Use host as a map name, automount will map
  remote hostsexports into the specified automount
  directory
• Example
• The serer chimchim exported /usr/share/man
• The clients master map has a line
• /net -hosts nosuid,soft
• The mount point will be
• /net/chimchim/usr/share/man
• It does enumerate all possible hosts
• It waits for individual subdirectory names to be
  references,then runs off and mounts the exported
  filesystem

29
automatic automount

• Key substitutions
• Impress a degree of regularity on the automounter
  maps
• The ampersand () expands to the matched key
  value in a map
• The asterisk() is a default case.
• Example a indirect map
• usr1 -rw thud/export/home/usr1
• usr2 -rw thud/export/home/usr2
• usr3 -rw thud/export/home/usr3
• usr4 -rw thud/export/home/usr4
• Can be rewritten as only one line
• -rw thud/export/home/

30
NFS Automounter

• Automount can use maps from NIS server.
• Other type maps, like auto_home mount
• Master map for automounter
• auto_master
• /xfn -xfn
• /net -hosts -nosuid
• /home auto_home
• /- auto_direct
• Home directory map for automounter
• auto_home
• craig almond/export/home/craig
• pecan/export/home/

31
Practice

• In Lab4, we created a dir called /research. As a
  group, lets try the following tasks
• Export /research, so
• some hosts in your group as rw
• some hosts in your group as ro
• one extra host have root mapping
• Access the exported from /net/hostname/home
• Mount /research from some hosts in your group
• Automount the exported /research to /research
  from some hosts in your group

32
Network Information Service

• NIS
• Was originally called the Sun Yellow Pages
• Is an administrative database
• What information does database store?
• Provides central control and automatic
  dissemination of important files.

33
Network Information Service

• The unit of sharing in NIS is the record, not the
  file.
• A record is usually corresponds to one line
• Mast server maintains the authoritative copies of
  system files
• Are kept in their original locations and formats
• Edited with a text editor before
• A server process makes the contents of the file
  available over the network.
• A server and its clients constitute an NIS
  domain
• associated with A set of maps
• Different from DNS domain

34
Network Information Service

• On the master server
• Data file are preprocessed into database files
  (called maps) by a hashing library
• One key associated with each entry
• A system file may have to be translated into
  several NIS maps
• Example
• /etc/passwd is translated into two different NIS
  Maps
• passwd.byname
• passwd.byuid
• The record order is not preserved.

35
Network Information Service

• On the slave server
• Relieve load on the master and keep clients
  functioning when master is down.
• Replicate the network maps from master server
• On the client
• Does not distinguish between the master server
  and slaves
• Traditional
• Use IP broadcasting to locate servers
• Place at least one NIS server on every physical
  network
• Some implementation allows to specify the server
  name instead of broadcasting

36
Network Information Service

• Databases are called NIS maps
• /etc/passwd
• /etc/group
• /etc/netgroup
• /etc/hosts
• /etc/networks
• /etc/protocols
• /etc/services
• /etc/aliases
• /etc/auto_master

37
Network Information Service

• Netgroups
• Sets of users, machines, and nets for easy
  reference in other system files
• The format of netgroup entry is
• Groupname list-of-members
• Members are separated by whitespace. A member is
• Either a netgroup name
• Or a tripnet of the form
• (hostname, username, nisdomainname)
• a empty value in a field matching any value in
  that field
• Netgroup can be used in several system files that
  define permissions
• Configuring NFS export ( /etc/exports file)
• Restrict remote logins and shell access (
  /etc/hosts.equiv file)
• Local login access ( /etc/passwd file)

38
Network Information Service

• Exmaples
• /etc/netgroup
• grouplab \
• (goofy.csl.mtu.edu,,) (mickey.csl.mtu.edu,,) \
• (minnie.csl.mtu.edu,,) (donald.csl.mtu.edu,,) \
• (simba.csl.mtu.edu,,) (nala.csl.mtu.edu,,) \
• (tramp.csl.mtu.edu,,)
• noncs1121d \
• (,msbenson,) (,behart,) (,adrogers,) (,agwalthe,)
  (,adpeters,) (,ammurrel,) \
• (,aeblechi,) (,jaeul,) (,srkelley,) (,daperry,)
  (,anlevend,) (,frtuntla,) \
• (,cwminnic,) (,rrmilkov,) (,iatrifon,)
  (,ksrivast,) (,talange,) (,mtnielse,) \
• (,csdummy,)
• /etc/dfs/dfstab
• share -F nfs -o
• rootaa.aaa.aa
• rwkeeper.csl.mtu.educslscifilanguagepizzaic
  ucsfishlabcec

39
Network Information Service

• How NIS works
• NISs data files are stored in one directory
• Usually /var/yp
• Each NIS map is stored in a hashed format in a
  subdirectory named for the NIS domain
• Exact Map files names depends on the hashing
  library being used.
• For example
• On cslserver, under /var/yp/csl.mtu.edu, there
  are ndbm files
• password.byname.dir
• password.byname.pag
• auto.direct.dir
• auto.direct.pag

40
Network Information Service

• There is a makefile under /var/yp
• Which calls makedbm to generate NIS maps from
  flat files.
• After you modify a system file, cd to NIS dir
  which is /var/yp, and run make.
• Make checks the modification date and rebuild
  some maps.
• Slave server run ypxfr command regularly as cron
  to pull the map copies
• Server runs yppush to instruct each slave server
  to execute ypxfr.
• Yppush is used in makefile after rebuilding a
  updated map.
• a special map called ypservers
• no correspond flat file
• constructed automatically

41
Network Information Service

• After the initial configuration
• Deamon ypserv running only on the servers
• Accept queries from clients and answers them by
  looking up information in the hashed map files
• Deamon ypbind running on every machine, include
  servers
• C library contacts the local ypbind to answer a
  query
• provided /etc/nsswitch.conf says to check with
  nis
• Ypbind locates a ypserv and returns its identify
  to the C library
• C library contacts the server directly

42

• NIS commands and daemons (page 527)
• ypserv NIS server daemon
• ypbind NIS client daemon
• domainname Sets the NIS domain a machine is in
• ypxfr Downloads current version of a map from
  master
• ypxfrd Serves request from ypxfr (runs on
  master server)
• yppush Makes slave servers update their
  versions of a map
• makedbm builds a hashed map from a flat file
• ypmake rebuild map
• ypinit Configure the host as a master or slave
  server
• ypset Makes ypbind connect to a particular
  server
• ypwhich find out which server the current host
  is using
• ypcat Prints the values contained in an NIS map
• ypmatch prints map entries for a specified key
• yppasswd Changes a password on the NIS master
  server
• ypchfn Changes GECOS info on the NIS master
  server
• ypchsh Changes a login shell on NIS master
  server
• yppassdd Server for yppasswd, ypchsh, and
  ypchfn

43
Set UP NIS

• Procedure
• Prepare the source file
• Initialize master server
• Run ypinit -m
• Starting NIS service
• Run ypserv
• Initialize slave server
• Set domainname
• Run ypinit command with c and master serve ip.
• Setup NIS clients
• Setup default domainname
• /etc/nsswitch.conf
• Run ypinit c
• Start ypbind daemon
• Run ypbind

44
NIS

• Case study
• Lets take a look at where the startup files for
  starting NIS server and NIS client.
• /etc/init.d/rpc
• /usr/lib/netsvc/yp/ypstart (ypstop)
• Maps
• /var/yp/csl.mtu.edu
• Lets take a look at the make file

45
NIS

• Work with DNS
• Hostname can be converted to IP address by DNS,
  NIS, and the host file.
• /etc/nsswitch.conf
• Define the order in which the sources are
  searched. Choices are
• Dns
• Nis
• Files
• For example
• hosts dns nis files
• networks nis NOTFOUNDreturn files
• services nis files
• protocols nis files

46
NIS

• NIS is a completely new software product and
  structured entirely differently from NIS
• Suns new administrative database released in
  early 90s
• Provides all the functionality of NIS
• Improved security
• NIS does not authenticate servers or clients
• NIS use secure DES
• NIS provide same access to everyone in the domain
• NIS provide access levels to different users
• NIS is a hierarchical decentralized
  architecture.
• NIS has enhanced data structures.
• Multiple columns table and relation query between
  tables
• NIS is complex and not as popular as NIS

47
NIS practice

• Refer to the Sun document
• System Administration Guide Naming and
  Directory Service
• section NIS Setup and Administration
• As a group
• Set up a NIS server and a slave server
• On the NIS server
• create a user account named visitor1 with home
  dir /research/visitor1
• Set up a NIS client
• Modify /etc/nsswitch.conf
• To have passwd line to include nis

48
Copying files around

• Not elegant solution, but it works
• Example
• A change to a startup script
• A new version of software
• File copying systems can use push model or
  pull model
• rdist push files
• rsync push files more securely
• expect pull files

49
Copying files around

• Rdist
• The easiest way to distribute files from a
  central server.
• Sort of like make
• A text specification of the files to be
  distributed
• Rdist copies file when they are out of date.
• Preserves owner, group, mode and modification
  time of files.
• When updating an existing file, it first deletes
  the old version before installing the new
• Makes rdist suitable for transferring executables
  that might be in use during the update.

50
Copying files around

• It runs on rsh and relies on rsh-style
  authentication to gain access to remote system
• The protocol can also be changed from command
  line
• rdist -P /usr/local/bin/ssh -f myDistfile
• Control file has the statements of the form
• Label pathnames -gt destination commands
• Where
• label associate a name with the statement
• pathname lists of files to be copied
• destination list of hosts to copy files to
• commands modify the behavior ( default is to
  copy files to the equivalent paths)

51
Copying files around
EXAMPLE SYS_FILES (/etc/passwd /etc/group
/etc/mail/aliases) GET_ALL (chimchim lollopop
barkadon) GET_SOME (whammo spiff) all
SYS_FILES -gt GET_ALL notify
barb special /etc/mail/aliases
/usr/bin/newaliases some SYS_FILES -gt
GET_SOME except /etc/mail/aliases notify
eddie_at_spiff
52
Copying files around

• Rsync push files
• Flexiblie and faster replacement of rcp
• Only transmit the differences
• Support for copying links, devices,owners, groups
  and permissions
• Can use any transparent remote shell rsh, ssh
• Can use rsync daemon mode
• No controlfile
• Must run repeatedly to transfer a set of files to
  multiple hosts.
• Example
• rsync gopt password-file/etc/rsync.pwd
  /etc/passwd lollipopsysfiles
• rsync gopt e ssh /etc/passwd /etc/shadow
  lollipop/etc

53
Copying files around

• Expect pull files
• Expect is a TCL extension scripting language
• Allows to you write control scripts for
  interactive programs
• The output produced by each operation is examined
  to determine what input should be send next
• Use expect to retrieve and install them
• Fundamental expect commands are
• spawn start up a subprocess to control
• send feed input to a subprocess
• expect take action depending on a subprocesss
  output
• Example Make system files available via FTP from
  central server

54
Copying files around
Example ftps the /etc/passwd file from the
machine netserver spawn /usr/bin/ftp
netserver while 1 expect Name send
netclient\r Password send
netclientpassword\r ftpgt
break failed send_user Cant log in .\r
exit 1 timeout send_user Timeout problem.
\r exit 2 send lcd /etc\r expect
ftpgt send cd pub/sysfiles\r expect ftpgt
send get passwd\r expect ftpgt send
quit\r, send_user \r exit 0
55
Summary

• NFS
• NIS
• Copying file around