Workbook 10 Network Applications

1
Workbook 10Network Applications
Pace Center for Business and Technology
2
Chapter 1.  An Introduction to TCP/IP Networking

• Key Concepts
• Most Linux networking services are designed
  around a client-server relationship.
• Network server applications are generally
  designed to be "always running", starting
  automatically as a system boots, and only
  shutting down when the system does. Generally,
  only the root user may manage server processes.
• Network client applications are generally running
  only when in use, and may be started by any user.
  
• Most Linux network servers and clients
  communicate using the TCP/IP protocol.
• The TCP/IP address of both the client process and
  the server process consists of an IP address and
  a port.
• Network servers usually use assigned, "well
  known" ports, as cataloged in the /etc/services
  file. Network clients generally use randomly
  assigned ports. Often, well know ports reside in
  the range of privileged ports, below port number
  1024.
• The hostname command can be to examine a
  machine's current IP address, while the netstat
  -tuna command can be used to examine all open
  ports.

3
Clients, Servers, and the TCP/IP Protocol

• Most networking applications today are designed
  around a client-server relationship. The
  networking client is usually an application
  acting on behalf of a person trying to accomplish
  a particular task, such as a web browser
  accessing a URL, or the rdate command asking a
  time server for the current time. The networking
  server is generally an application that is
  providing some service, such as supplying the
  content of web pages, or supplying the current
  time.
• The design of (applications acting as) network
  clients and (applications acting as) network
  servers differs dramatically. In order to
  appreciate the differences, we will compare them
  to the agents in a more familiar client-server
  relationship, a customer buying a candybar from a
  salesperson.

4
The Server

• Servers are Highly Available
• Just as a salesperson must always be running the
  register, even when customers are not around,
  processes implementing networking services need
  to be running, ready to supply services upon
  request. Usually, processes implementing network
  services are started at boottime, and continue to
  run until the machine is shutdown. In Linux (and
  Unix), such processes are often referred to as
  daemons. Generally, only the root user may start
  or shutdown processes acting as network servers.
• Servers have Well Known Locations
• In addition to being available when a customer
  wants service, salespeople stay where they know
  customers can find them. Just as customers can
  look up the locations of unfamiliar candybar
  salesmen in telephone books and find them by
  street address, networking clients can look up
  the locations of unfamiliar network servers using
  a hostname, which is converted into the IP
  Address used to access the service.

5
The Server

• The Client
• In contrast, the candybar customer needs to be
  neither highly available nor well known. A
  customer doesn't hang around a store from dawn
  until dusk just in case he decides he wants a
  candybar. Likewise, clients generally do not need
  to to stay at well known locations. Our customer
  doesn't stay at home all day, just in case
  somebody wants to come by to sell him a candybar.
  
• Processes that implement network clients are
  started by normal users, and generally are only
  running as long a necessary to complete a task.
  When someone breaks for lunch, he closes his web
  browser. Also, client applications do not need to
  have fixed addresses, but can move from place to
  place. More on this next.

6
The Server

• TCP/IP Addresses
• Every process which is participating in a TCP/IP
  conversation must have an IP Address, just as
  every participant in a phone conversation must
  have a phone number. Additionally, every process
  in a TCP/IP conversation must have port number,
  whose closest analogy might be a telephone
  extension associated with a phone number.
• Computers on a network are identified by IP
  address. The IP address comes in the form of four
  integers, each ranging from 0 to 255 (not
  coincidentally, the amount of information that
  can be encoded in one byte of memory), and
  traditionally written separated by periods, such
  as 192.168.0.3. This representation is often
  informally referred to as a dotted quad.
• Processes on computers are identified by a port
  number, which is an integer ranging from 1 to
  65535 (not coincidentally, the amount of
  information that can be encoded in two bytes of
  memory). Whenever a process wants to participate
  in a TCP/IP conversation with another process, it
  must first be assigned a port number by the
  kernel.
• The TCP/IP protocol allows two processes, each
  identified by a combination of an IP address and
  port number, to locate one another. The IP
  address is used to locate the machine that the
  process is running on (this is the "IP" part of
  the protocol), and the port number is used to
  locate the correct process on the machine (this
  is the "TCP" part).

7
Sockets

• In order to illustrate a typical TCP/IP
  transaction, we will examine the conversation
  between a fictitious student's mozilla web
  browser, running on the machine
  station3.example.com, which translates into an IP
  address of 123.45.67.89, and the httpd web server
  running at academy.redhat.com, which translates
  into an IP address of 66.187.232.51. The process
  usually resembles the following.
• As the machine academy.redhat.com is booted, the
  httpd process is started. It first allocates a
  socket, binds it to the port 80, and begins
  listening for connections.
• At some point later, perhaps measured in minutes,
  perhaps days, the mozilla process is started on
  the machine station3.example.com. It also
  allocates a socket, and requests to connect to
  port 80 of the machine 66.187.232.51. Because it
  did not request a particular port number, the
  kernel provides a random one, say 12345. As it
  requests the connection, it provides its own IP
  address and (randomly assigned) port number to
  the server.
• The server chooses to accept the connection. The
  established socket is now identified by the
  combined IP address and port number of both the
  client and server.

8
Sockets

• Once the socket is established, the mozilla
  process and the httpd process can read
  information from and write information to one
  another as easily as reading and writing from a
  file. (Remember... "everything is a file", even
  network connections! For most practical purposes,
  a socket is just another file descriptor.) The
  highlighted verbs in this section, bind, listen,
  connect, accept, and even read and write, are
  well defined terms in Linux (and Unix). They also
  are the names of the programming system calls
  that implement each step.

9
Sockets

• In the TCP/IP protocol, a socket is defined by
  the combined IP address and port number of both
  the server and the client. For example, what if
  our student was running multiple versions of
  Mozilla, each making requests of
  academy.redhat.com, or what if multiple users
  were using the machine station3.example.com
  simultaneously, all accessing academy.redhat.com?
  How would the web server keep straight which
  conversation it was having with which client?
  Each client process would be assigned a distinct
  port number, and therefore converse with the
  httpd daemon using a distinct socket.

10
More about Ports

• Well Known Services and the /etc/services File
• In our example, we mentioned that the httpd
  process requested to bind to port 80, and in turn
  the mozilla process requested to connect to port
  80 of the server. How did each agree that port 80
  was the appropriate port for the web server?
  Traditional Internet services, such as a web
  server, or ftp server, or SMTP (email) server,
  are referred to as well known services. On Linux
  (and Unix) machines, a catalog of well known
  services, and the ports assigned to them, is
  maintained in the file /etc/services.
• Notice that both the client and server need to
  agree on the appropriate port number, so the
  /etc/services file is just as important on the
  client's machine as on the server's. Just because
  a service is listed in the /etc/services file
  does not mean that you are implementing (or even
  capable of implementing) that well known service.

11
Privileged Ports

• Unlike clients, processes implementing network
  servers generally request which port they would
  like to bind to. Only one process may bind to a
  port at any given time (Why must this be?).
  Otherwise, how would a client distinguish which
  process it would like to connect to? Ports less
  than 1024 are referred to as privileged ports,
  and treated specially by the kernel. Only
  processes running as the root user may bind to
  privileged ports. (This helps ensure that, if
  elvis had an account on the machine
  academy.redhat.com, he couldn't start up a rogue
  version of a web server which might hand out
  false information.) Originally, the well known
  ports and the privileged ports were meant to
  coincide, but in practice there are more well
  known ports than privileged ones.

12
Determining Current TCP/IP Networking Services

• Using hostname to Display the Current IP Address
• The hostname command, without arguments, displays
  a machine's current hostname. With the -i command
  line switch, machine's IP address is displayed
  instead.
• What if there are multiple IP addresses?
• The design of the hostname command is a little
  misguided, because machines can easily have more
  than one IP address (one for each of multiple
  network interface cards, for instance.) In such
  situations, there is no reason why any one IP
  address should be privileged over the others. For
  historical reasons, however, the kernel keeps
  track of a parameter it refers to as its
  hostname, and the hostname -i command displays
  the IP address associated with it.

13
Using the netstat Command to Display Open Ports

• When a port is used by a socket, it is referred
  to as an open port. The netstat command can be
  used to display a variety of networking
  information, including open ports.
• Unfortunately, when called with no command line
  switches, the netstat command's output is
  overwhelmed by the least interesting information,
  local "Unix" sockets that are used to communicate
  between processes on the same machine. When
  called with the following command line switches,
  however, more interesting information is
  displayed.

14
Using the netstat Command to Display Open Ports

• Many more switches are available, as a quick look
  at the netstat(8) man page will reveal. The above
  switches were chosen from the many, not only
  because when combined, they produce interesting
  output, but also because they are easy to
  remember just think "tuna".
• When invoked with the above switches, netstat's
  output is akin to the following.

15
Using the netstat Command to Display Open Ports

• Of the many lines, we may weed away those
  associated with the udp protocol, and focus
  instead on columns 4, 5, and 6 of the rows
  associated with tcp.

16
Special IP Addresses
17
Understanding IPv6 Addresses

• The IP protocol which we discussed above, with
  friendly IP addresses such as "66.187.232.51", is
  known as Internet Protocol version 4, or "IPv4",
  and is by far the dominant IP protocol in use
  today. With the IPv4 protocol, you can have
  around 4 billion distinct IP addresses. When the
  IPv4 protocol was developed in the early 1970's,
  this was plenty of addresses, and the IPv4
  protocol has served us well. By learning a few
  tricks such as masquerading, where many machines
  can "hide behind" a single public IP address, we
  still manage to live with it well. As appliances
  become more intelligent, however, and networking
  (particularly wireless networking) becomes
  cheaper, we are approaching a day where every
  toaster is going to want its own IP address, and
  4 billion address wont' be enough. Changes are on
  the horizon.
• These changes are coming in the form of Internet
  Protocol Version 6, or "IPv6". Among many other
  changes, the IPv6 protocol takes the obvious
  step, and makes IP addresses bigger. Four times
  bigger. An IPv6 address, in full form, looks like
  fe80000000000000021122fffe334411. Instead
  of writing addresses as familiar decimal numbers
  (like "127"), they are written in hexadecimal
  numbers (like "fe33").

18
Understanding IPv6 Addresses

• Some merciful conventions in representing IPv6
  addresses help.
• Once within an address, a series of zero
  segments, such as 000000000000, can be replaced
  with a double colon ().
• For any segment which begins with leading zeros,
  such as 0211, the leading zeros can be dropped.
• With these two shortcuts in mind, the above
  address could be written a little more friendly
  as fe8021122fffe334411.

19
Understanding IPv6 Addresses

• Red Hat Enterprise Linux is getting ready for a
  future transition to IPv6, and many applications
  can already handle either IPv4 or IPv6
  connections. For our purposes, because it is not
  yet widely adopted, we're going to ignore IPv6 as
  much as possible. Fortunately, there's just a few
  forms of IPv6 addresses we need to be able to
  recognize, summarized in the table below.
• Whenever these forms of IPv6 address are
  encountered ( ffff192.168.0.1, 1, and ),
  you can simply think of them as their IPv4
  equivalents ( 192.168.0.1, 127.0.0.1, and
  0.0.0.0, respectively).

20
Understanding IPv6 Addresses

• Listening Sockets
• Listening are sockets are sockets owned by a
  server, before any clients have come along. For
  example, at the end of step one of our sample
  TCP/IP connection above, the httpd process would
  have a socket open in the listening state. Notice
  in the above output that for listening sockets,
  only the local half of the address is defined.
• Established Sockets
• As the name implies, established sockets have
  both a client process and a server process with
  established communication.
• Pulling it All Together

21
Understanding IPv6 Addresses

• These two sockets are listening for connections,
  but only on the loopback address. They must
  belong to services expecting to receive
  connections from other processes on the local
  machine, but not from other machines. To
  determine what services these ports belong to, we
  do some greping from the /etc/services file.
• Apparently, whatever process has claimed port 25
  is listening for email clients. This is probably
  the sendmail daemon. The process listening on
  port 631 is listening for print clients. This is
  probably the cupsd printing daemon. Both of these
  services are discussed in more detail in this
  Workbook.

22
Understanding IPv6 Addresses

• These lines reflect both halves of an established
  connection between two processes, both on the
  local machine (notice the loopback IP address for
  both of them). The first is bound to port 59330
  (probably a randomly assigned client port), and
  the second to the port 631. Some process on the
  local machine must be communicating with the
  cupsd daemon.
• Our final extracted lines represent established
  connections between clients on remote machines,
  and services on our local machine's external
  interface (192.168.122.156). The first is a
  connection to an IPv6 aware service on port 22.
  Again, we try a little grepping to look up the
  well known service associated with port 22.
• Apparently, this line represents an active
  connection between a ssh client on a remote
  machine with IP address 192.168.122.1, and a sshd
  daemon on our local machine.
• The latter is a connection to an IPv4 only
  service bound to port 653, probably an NFS
  related service.

23
Online Exercises Chapter 1.  An Introduction to
TCP/IP Networking

• Online Exercises
• Lab Exercise
• Objective Gain familiarity with TCP/IP
  configuration and activity.
• Estimated Time 10 mins.
• Specification
• Create the file /lab11.1/ipaddr, which contains
  your machine's IP address, as reported by the
  hostname command, as its single word.
• Create the file /lab11.1/listening_ports, which
  contains a list of all ports less then 1024 on
  your current machine which are open in the
  listening state, one port per line.
• Deliverables
• The file /lab11.1/ipaddr, which contains your
  machine's current IP address (as reported by the
  hostname command) as its single word.
• The file /lab11.1/listening_ports, which
  contains a list of all ports less then 1024 on
  your current machine which are open in the
  listening state, one port per line.

24
Chapter 2.  Linux Printing

• Key Concepts
• Red Hat Enterprise Linux uses the CUPS printing
  system for managing printers.
• The CUPS printing system is designed around the
  concept of a print queue, which combines a
  spooling directory, a filter, and a print device.
  
• The lpstat command browses available print
  queues.
• The lpr, lpq, and lprm commands are used to
  submit ("request") print jobs, query for
  outstanding jobs, and remove pending print jobs,
  respectively.
• Print submission applications examine the PRINTER
  environment variable to determine the default
  print queue.
• The lp and cancel commands behave similarly to
  the lpr and lprm commands.
• The cupsd daemon supports a web interface, which
  can be accessed at http//localhost631.

25
Introducing CUPS

• Introducing CUPS
• Red Hat Enterprise Linux uses the Common Unix
  Printing System (CUPS) for managing printers.
  Rather than interacting with a printer directly,
  users submit print requests to print queues which
  are managed by the cupsd daemon. Print requests
  which are pending in a print queue are referred
  to as print jobs. Once a job has been submitted
  to the queue, users may return immediately to
  whatever tasks are at hand. If the printer is
  busy with another document, or out of paper, or
  unreachable over the network, the cupsd daemon
  will monitor the situation, and send (or resend)
  the print job to the printer as it becomes
  available.
• The cupsd daemon uses the Internet Printing
  Protocol (IPP), which is a direct extension of
  the HTTP protocol designed to allow print queue
  management in an operating system independent
  manner. As a result, CUPS management has much in
  common with web server management.

26
Introducing CUPS

• The following figure identifies the elements
  which participate in Linux printing. The various
  elements are discussed in more detail below.

27
Print Queues

• A print queue is a combination of the following
  elements.
• A spooling directory, where pending jobs may be
  temporarily stored.
• A series of filters which reformat various
  formats of input files into a format appropriate
  for whatever back end device is connected to the
  queue.
• A back end device, such as a locally attached
  printer, or a print queue defined on a remote
  machine. The following lists some of the various
  back end devices supported by CUPS.
• locally attached parallel port printers
• locally attached USB printers
• networked printers using the LPD interface
• networked printers using the JetDirect interface
• IPP based print queues on remote machines
• LPD based print queues on remote machines
• SMB (Microsoft) network print services

28
Browsing Available Print Queues lpstat

• Print queues are either available because they
  have been defined on the local machine, or
  discovered using CUPS's ability to browse the
  local network for published printers.
• The lpstat command can be used to scan available
  print queues from the command line. The following
  command line switches may be used to qualify the
  lpstat command.

29
Browsing Available Print Queues lpstat

• See the lpstat(1) man page for more details. In
  the following, elvis is discovering that his
  system's default print queue is named simply
  "printer", and that he has several print queues
  available to him, which seem to all refer to IPP
  based print queues on a local print server.

30
Submitting and Managing Jobs lpr, lpq,and lprm

• CUPS uses traditional UNIX commands to interface
  with the print system lpr for submitting files
  and data to be printed, lpq for examining the
  status of outstanding print jobs, and lprm for
  removing pending print jobs from the queue. All
  three commands use the following techniques for
  specifying which print queue to use, in the
  specified order.
• If the -P command line switch is found, its
  argument is used to specify the print queue.
• If -P is not used, then if the PRINTER
  environment variable exists, it is used to define
  the default print queue.
• Otherwise, the system default print queue is
  used.

31
Submitting Jobs with lpr

• Jobs may be submitted with the lpr command. Any
  arguments are interpreted as files to submit. If
  no arguments are specified, standard in is read
  instead. The following options may be used to
  qualify the lpr command.
• As an example, in the following, blondie uses the
  lpr command to print the file README, using the
  sales print queue.

32
Monitoring Jobs with lpq
The lpq command lists pending jobs in a queue. In
the following example, blondie will submit the
output of the df command to the printer legal,
and then examine the contents of the queue.
33
Removing Jobs with lprm
Blondie suspects that something is wrong with the
legal printer, and decides to start using the
sales printer as her default printer. She first
sets up the PRINTER environment variable to
reflect her new preferences, the uses the lprm
command to remove her job from the legal queue.
Notice in the first lpq command, and
the last lpr command, the PRINTER environment
variable implicitly specified the sales print
queue.
34
Alternate Front End Commands lp and cancel
In Unix, in the days preceding CUPS, two parallel
implementations of printing infrastructure were
implemented. The first used the three commands
introduced above, namely lpr, lpq, and lprm. The
other used lp, lpstat, and cancel for analogous
roles. We have already seen that lpstat is
supported, and is the preferred tool for
discovering available print queues. The commands
lp and cancel are also available as slight
variations of the lpr and lprm commands. Consult
the shared lp(1) man page for details.
35
The CUPS Web Interface
Lastly, we would be remiss to leave the topic of
CUPS without mentioning the native web interface
provided by the cupsd daemon. As mentioned, most
clients interact with the cupsd daemon using the
IPP protocol, which is an extension of the HTTP
protocol. Because of the similarities, the cupsd
daemon behaves in may ways like a web daemon,
including the serving of CGI style management
pages. In order to view CUPS's management pages,
point a web browser to the localhost address, but
override the default port 80 with the CUPS
daemons well known service port, 631. The cupsd
daemon will return with a CUPS "homepage", from
where printers and print jobs can be browsed, and
copious online documentation is available.
36
Chapter 3.  Managing Print Files

• Key Concepts
• The primary printing format in Linux is
  PostScript.
• Utilities such as ps2pdf and pdf2ps can convert
  PostScript to PDF and back.
• evince previews PostScript and PDF documents.
• enscript converts text file into decorated
  PostScript.
• mpage can rearrange individual pages from a
  PostScript document.

37
PostScript

• In Linux, most printers expect to receive either
  ASCII text, or graphics using the PostScript
  format. Unlike most graphics formats, PostScript
  is actually a scripting language which has been
  tailored to the task of rendering graphics on the
  printed page. The PostScript Language's
  sophistication allows it to perform powerful
  tasks, but a PostScript interpreter must be used
  to render PostScript files as images.
• Many printers implement native PostScript
  interpreters, and are referred to as PostScript
  printers. Whenever a PostScript printer receives
  a text file which begins with the characters
  !PS, the remainder of the file is interpreted as
  a PostScript script, rather than printed as ASCII
  text directly. (Note the similarity to Unix's
  !/bin/bash scripting mechanism).

38
PostScript

• In Linux (and Unix), an application called
  Ghostscript, or simply gs, implements a
  PostScript interpreter. Implementing a PostScript
  interpreter is a significant task, and although
  several applications in Linux can be used to view
  or manipulate PostScript files, almost all use
  Ghostscript as the back end to perform the actual
  rendering of PostScript into more accessible
  graphics formats.
• Rather than using the low level Ghostscript
  interpreter directly, the easy to use evince
  application is usually used to view PostScript
  documents.

39
Viewing PostScript Documents with the evince
Document Viewer.

• Many applications will print PostScript directly
  to files instead of delivering them to a print
  queue. For example, in the following dialog, by
  selecting "Print to File", the firefox web
  browser is being asked to print the current web
  page not to a print queue, but to a PostScript
  file titled example.ps.

40
Viewing PostScript Documents with the evince
Document Viewer.

• As the head command illustrates, a PostScript
  file is a simple text file beginning with the
  PostScript "magic" !PS.
• The PostScript file can be viewed with the evince
  document viewer.

41
Decorating Text for Printing with enscript

• The enscript utility converts text files into
  PostScript, often decorating the text with syntax
  highlighting (referred to as pretty printing), a
  header, or formatting multiple text pages per
  printed page (referred to as printing 2-up, 4-up,
  etc..).
• A little awkwardly, the enscript command sends a
  text file directly to the lpr command by default,
  with the result that enscripted files are
  immediately printed. The -o command line switch
  is often used to specify an output PostScript
  file instead.
• As an example, elvis could easily create a
  PostScript version of the GPL license.

42
Decorating Text for Printing with enscript

• So far, the formatting has been fairly minimal A
  title, a date, and a page number. The enscript
  command comes with a host of more sophisticated
  formatting options, however. Some of the more
  commonly used command line switches are found in
  the following table.

43
Decorating Text for Printing with enscript
44
Decorating Text for Printing with enscript

• Many more options exist. Consult the enscript(1)
  man page for more information. As an example, the
  following command line would render the C header
  file malloc.h as PostScript with 2 columns per
  page, rotated, and decorated with a fancy header
  and syntax highlighting.

45
Rearranging PostScript with mpage

• The mpage command can be used to extract pages
  from the middle of a multi-page PostScript
  document, or reformat the document to be printed
  with multiple pages per printed sheet.
• While designed to operate on PostScript, mpage
  can be fed plain text files as well, which it
  will simply render as PostScript before handling.
  Arguments are considered input files, with the
  output is directed to standard out.

46
Rearranging PostScript with mpage
47
Rearranging PostScript with mpage

• As an example, mpage could be used to convert the
  layout of the gpl.ps file generated earlier to "4
  UP".

48
Converting PostScript to PDF and PDF to
PostScript

• Unfortunately, on operating systems which are not
  Unix based, PostScript documents are pretty
  useless. The rest of the world deals with the
  alternate PDF format. Fortunately, Red Hat
  Enterprise Linux contains easy to use utilities
  that allow PostScript documents to be converted
  to PDF documents (ps2pdf), and vice versa
  (pdf2ps).
• The syntax of the ps2pdf command is trivial,
  where the first argument is the input PostScript
  filename (or a - to imply standard in), and the
  second argument is the output PDF filename (or a
  - to imply standard out). The pdf2ps command
  works similarly.
• As a quick example, in order to share his work
  with a friend who prefers PDF documents, elvis
  now uses the ps2pdf command to convert his
  malloc.ps file into PDF format, which is of
  comparable quality, but dramatically more
  compact.

49
Converting PostScript to PDF and PDF to
PostScript

• Note that as a nice benefit, the PDF format
  allows evince to display document thumbnails in a
  side pane.

50
Chapter 4.  Email Overview

• Key Concepts
• Email Management involves an MUA (Mail User
  Agent), which is used to present newly delivered
  mail to a user, and allow the user to compose new
  responses, and an MTA (Mail Transport Agent),
  which manages the background task of exchanging
  email with remote machines.
• Depending on the details of a computer's Internet
  access, the job of receiving email may be
  delegated to a mailbox server, which would then
  allow user's to access their delivered email
  using the POP or IMAP protocols.
• Again depending on the details of a computer's
  Internet access, the job of delivering email may
  be delegated to a remote outgoing SMTP server.
• Generally, locally delivered but unread mail is
  spooled in the file /var/spool/mail/USER, where
  USER is the username of the recipient.
• One of the simplest MUA's is the mail command.

51
Using Email

• The Simple Solution
• The simplest solution is appropriate for people
  using computers which have permanent, well known
  Internet connections. At the time that many of
  the protocols defining how email delivery should
  be handled were developed, this was more of the
  standard instead of the exception. The solution
  involves two separate applications whose roles
  are identified by TLA's (Three Letter Acronyms).
  The first application is referred to as the MTA,
  or Mail Transport Agent the second is known as
  the MUA, or Mail User Agent.

52
The MTA (Mail Transport Agent)

• The MTA generally operates in the background,
  performing the work of the local post office. The
  MTA receives email to be delivered from programs
  on the local machine, determines from the
  recipient's address the appropriate machine to
  contact, and attempts to connect to a
  complementary MTA running on the recipients
  machine, who should be listening to port 25. If
  the sender's MTA cannot contact the receiver's
  MTA, the mail is spooled on the sender's machine,
  and the sender's MTA tries again at a later time.
  
• The MTA also binds to the local port 25, where it
  receives connections from other MTA's. When it
  receives mail from a remote MTA which is destined
  for a user on the local machine, it receives the
  mail, and stores it in a mail spool which is
  referred to the user's inbox. In Linux (and
  Unix), the default inbox for a user is
  /var/spool/mail/username, so that mail awaiting
  the user elvis would be stored in the file
  /var/spool/mail/elvis.
• The default MTA in Red Hat Enterprise Linux is a
  daemon called sendmail.

53
The MUA (Mail User Agent)

• The MUA is the application most user's think
  about when they think about email. The MUA
  retrieves delivered mail from a user's mail spool
  (inbox), and presents it to the user as "New
  Mail". The MUA allows users to compose responses
  or new mail messages, and passes these messages
  to the local MTA for delivery.
• Red Hat Enterprise Linux ships with a wide
  selection of MUAs, several of which will be
  introduced in this and the next lesson.

54
Mailbox Servers

• While simple, the previous solution requires
  that, first of all, users are receiving email on
  a machine that maintains a persistent Internet
  connection, and secondly, the machine has a well
  known host name and is accessible to others. Many
  users, such as people using a "dial up" or "High
  Speed" connection from an ISP (Internet Service
  Provider), or people using machines behind an
  institution's firewall, are not in this
  situation. Another solution has evolved to serve
  people in these situations Mailbox servers.

55
Mailbox Servers

• In the diagram above, we assume that elvis is
  using a "high speed" Internet connection which he
  has subscribed to from the company "ISP.Net".
  When he connects to the Internet, his ISP issues
  his machine an IP address, but elvis cannot
  predict ahead of time which IP address he will
  receive. The hostname which the ISP has assigned
  to the IP address is probably unattractive, such
  as dhcp-191-93.isx.isp.net, so even if elvis were
  guaranteed to receive the same IP address each
  time, he would not want to advertise his
  machine's hostname as his email address.
• Instead, elvis takes advantage of an "email
  account" which his ISP offers him. Very likely,
  this account exists on a Linux or Unix machine
  owned by the ISP, which has a permanent
  connection to the Internet, and is assigned a
  hostname such as pop.isp.net. The ISP has
  arranged all mail destined to addresses of the
  form user_at_isp.net to be delivered to the MTA
  running on this machine. When the MTA running on
  pop.isp.net receives mail for elvis, it stores
  the delivered mail into a mail spool dedicated to
  elvis (very likely in the file /var/spool/mail/elv
  is), and the email is considered delivered.
• Because the ISP's machine pop.isp.net has a
  permanent, well known connection to the Internet,
  it is a much better candidate for receiving email
  than elvis's machine at home. Sitting at home,
  however, elvis still needs access to the email
  sitting in his "inbox" on pop.isp.net. Usually,
  this access is provided in the from of a POP
  (Post Office Protocol) or IMAP (Internet Mail
  Access Protocol) server.

56
Mail Servers

• POP servers
• POP servers perform a very simple service. They
  allow users to access the user's single mail
  spool, and transfer its contents to their local
  MUA. POP servers generally bind to port 110, and
  require that any POP client authenticate itself
  using a username/password pair. Most modern MUA's
  act as POP clients, and can be configured to
  retrieve mail from a specified POP server.
• If a mailbox server is implementing the POP
  service, it usually implies that the mailbox
  server is not intending to provide permanent
  storage for a user's email, but instead just
  cache it temporarily until the user can "pop" it
  down to a local machine.
• IMAP servers
• IMAP servers generally provide clients with more
  sophisticated mailbox management. IMAP users may
  maintain several folders on the mailbox server,
  not just their individual inbox. In general, an
  IMAP server implies that a user's email is
  intended to be permanently stored on the mailbox
  server, and users will occasionally connect with
  a MUA from a remote machine to "browse" their
  mail. Generally, IMAP servers are found in
  institutional and corporate environments. IMAP
  daemons bind to port 143.

57
Mail Servers

• Sending Mail
• Because elvis's machine maintains an almost
  continuous connection to the Internet, he is
  willing to still use his local MUA to deliver
  mail. If his local MUA is temporarily unable to
  connect to the recipient's machine, the MUA will
  spool the mail locally and try again later.
• Red Hat Enterprise Linux Default Configuration
• By default, Red Hat Enterprise Linux is
  configured appropriately for this scenario. The
  local MTA is started, but it will not accept
  connections over port 25 (except for from the
  loopback address, 127.0.0.1). It serves merely to
  deliver outgoing email. Users are assumed to be
  accessing their email from a POP or IMAP server.
• The default configuration can be changed, but the
  necessary configuration is beyond the scope of
  this course.

58
Outgoing Mail Servers

• For machines that have only transient connections
  to the Internet, attempting to deliver email
  using the local MTA may not be appropriate.
  Instead, users of "dial up" connections and the
  like often use what is referred to as an SMTP
  (Simple Mail Transfer Protocol) Server, or
  outgoing mail server. Many ISPs and institutions
  provide outgoing SMTP servers, often with names
  like smtp.isp.net.
• The MTA on the SMTP server is willing to accept
  mail from "local" machines, even though it is not
  the final recipient for the email. Instead, the
  SMTP server relays the mail, forwarding it on to
  its destination. If any temporary problems occur,
  the spooling and redelivery attempts of the
  pending mail is now the responsibility of the
  SMTP server.
• Many MUA's allow users the option of specifying a
  remote host to act as the user's SMTP server (as
  opposed to forwarding mail to the local MTA for
  delivery). 1

59
Local Delivery

• All of the above scenarios assume that a user is
  sending email from his machine to a recipient on
  a remote machine. Linux (and Unix) also allows
  users on a local machine to deliver email to one
  another, where email is addressed simply to a
  username, such as blondie.
• For local delivery, no POP servers or SMTP
  servers are required, as the email is instantly
  delivered by the local MTA.

60
Local Delivery

• Sending Email with mail
• The mail command can be used to deliver mail to
  recipients who are specified as arguments on the
  command line. The body of the message is read
  from standard in (which may be read from a pipe,
  a redirected file, or from the keyboard directly,
  where CTRLD ("EOF") is used to indicate the end
  of the message).
• The command line switches in the following table
  can be used to specify a subject line, recipients
  to "Cc", etc.
• As an example, in the following, elvis mails to
  blondie the contents of the file lyrics.txt.

61
Using mail to Read Mail

• The mail command can also be used to read mail
  from a user's local mail spool. The interface is
  primitive, however, and usually other MUA's are
  used instead. Details can be found in the mail
  man page. These days, the most common use of the
  mail command is as a quick and easy way to send
  mail.
• In the following example, blondie is using the
  mail command, without arguments, to view the mail
  in her inbox, and then delete the message and
  quit.

62
Chapter 6.  Network Diagnostic Applications

• Key Concepts
• /sbin/ifconfig displays local IP configuration.
• ping confirms low level network connectivity
  between hosts.
• host makes direct DNS queries.
• The netstat -tuna command lists currently active
  network services and connections.
• /sbin/traceroute can diagnose routing problems.

63
Required Configuration for the Internet Protocol

• IP Address
• Linux (and Unix) represents every networking
  device attached to a machine (such as an Ethernet
  card, a Token Ring card, a modem used for dialup
  connections, etc...) as a network interface.
  Before an interface can be used to send or
  receive traffic, it must be configured with an IP
  address which serves as the interface's identity.
  
• Default Gateway
• The mechanics of the IP protocol organizes
  machines into sub networks, or subnets. All
  machines on a given subnet may exchange
  information directly. IP subnets are in turn
  linked to other subnets by machines acting as
  routers. A router has multiple network
  interfaces, usually each participating in a
  distinct subnet. In order to communicate with a
  host on another subnet, the data must be passed
  to a router, which (with the help of other
  routers) routes the information to the
  appropriate subnet, and from there to the
  appropriate host.
• In order to communicate with machines outside of
  your local subnet, your machine must know the
  identity of a nearby router. The router used to
  route packets outside of your local subnet is
  commonly referred to as a default gateway.

64
Required Configuration for the Internet Protocol

• Nameserver
• Other machines on the Internet are in turn
  identified by their IP address. People tend to
  prefer to think in terms of names instead of
  numbers, however, so a protocol has been
  developed to assign names to IP addresses. The
  protocol is called Domain Name Service, or more
  commonly DNS.
• The DNS protocol requires that every machine have
  available one or more DNS servers (commonly
  called nameservers), which can serve as both a
  database for assigning name to local IP
  addresses, and also a starting point for
  determining IP addresses for domain names for
  which the server does not have direct knowledge.

65
Determining Your IP Address(es) /sbin/ifconfig

• A previous Lesson introduced the hostname -i
  command, which displays the IP address of your
  local machine. In reality, a "machine" does not
  have an IP address, a machine's network
  interfaces do. This Lesson discusses the topic of
  IP addresses and network interfaces in more
  detail.
• In Linux (and Unix), every network device is
  represented as a network interface. (For once, we
  encounter something that is not a file!) Linux
  names interfaces according to the type of device
  it represents. The following table lists some of
  the more commonly encountered interface names
  used in Linux.
• In each case, n is replaced with a distinct
  integer for each instance of a given device
  attached to a machine.

66
Confirming Network Interface Configuration

• The ifconfig command displays the configuration
  of all active network interfaces. Because the
  command is generally used by root to configure
  interfaces, it lives within the /sbin directory,
  which is outside of the default PATH for standard
  users. Standard users can use the command to view
  interface configuration information, however, by
  using an absolute reference, as in the following
  example.

67
Confirming Network Interface Configuration

• The ifconfig command displays a stanza of IP
  configuration information and usage statistics
  for each active network interface. In most
  situations, users should expect to see two
  stanzas reported. One stanza contains the
  configuration for an attached Ethernet card,
  while the other shows the configuration for the
  virtual loopback device. The important line is
  the second line, which displays the IP address
  assigned to the interface. If the line containing
  the IP address is missing, or if the IP address
  does not look reasonable for your local network
  configuration, you can expect to have trouble
  accessing the network.

68
Determining Your Default Gateway /sbin/route

• As mentioned in this Lesson's introduction,
  communicating with hosts on your local subnet
  uses different procedures than communicating with
  hosts on a separate subnet. The Linux kernel,
  like all Unix kernels, maintains an internal
  table which defines which machines should be
  considered local, and what gateway should be used
  to help communicate with those machines which are
  not. This table is called the routing table.
• If you are a standard user, the route command can
  be used to display the system's routing table. If
  you are the root user, the command can be used to
  manipulate the table as well. Like the ifconfig
  command, the route command lives in the /sbin
  directory, so standard users should invoke it
  using an absolute reference.

69
Determining Your Default Gateway /sbin/route

• A standard routing table displays two types
  entries. The first type defines which subnets
  should be considered local subnets. In general,
  there should be one line specifying a subnet for
  each active interface. In the output above, the
  first line defines the subnet associated with the
  Ethernet interface (with an IP address of
  192.168.0.51), and the second line defines the
  subnet associated with the loopback interface
  (with an IP address of 127.0.0.1).
• The second type of entry, which is used to define
  gateways, is more relevant to our discussion.
  Gateway entries can be distinguished because a
  host is defined in the second column ("Gateway"),
  and the fourth column ("Flags") contains a "G".
  Every routing table should contain an entry for a
  "default" gateway, an the second column should
  contain the gateway's hostname.

70
Determining Your Default Gateway /sbin/route

• The same information can be displayed using IP
  addresses instead of hostnames using /sbin/route
  -n.

71
Determining Your Nameserver(s)

• Domain Name Service allows users to refer to
  networked computers using hostnames instead of IP
  addresses. Unlike the other two aspects of
  network configuration, a nameserver is to some
  extent optional. In order to communicate with
  other machines at all, your host must have an IP
  address. In order to communicate with machines
  outside of your subnet, you must have a default
  gateway. If users are willing to refer to every
  machine by IP address instead of hostname,
  however, your machine can communicate using the
  IP protocol without ever consulting a nameserver.
  
• In practice, however, nameservers seem more of a
  necessity. (Do you have an easier time
  remembering academy.redhat.com, or
  66.187.232.51?) Converting a hostname into an IP
  address is often referred to as resolving an
  address, and the library which implements
  nameservice is called the resolv library. When
  the library attempts to resolves an address,
  there are generally two resources available.

72
Static DNS Configuration /etc/hosts

• The first resource is a simple text file called
  the /etc/hosts file. While only root can edit the
  file, any user can observe it.
• The format of the file is as simple as it looks.
  The first token on a line should be an IP
  address, and subsequent tokens are hostnames
  which should resolve to the IP address. The
  standard Unix comment character () is also
  supported.
• If a machine is only communicating with a few
  machines, or if an administrator wants to create
  shortcut hostnames (such as s), or if she would
  like to override the local nameserver, entries
  can be added to the /etc/hosts file using a
  simple text editor.
• Obviously, this technique does not scale well.
  You cannot expect the local /etc/hosts file to
  provide every answer.

73
Dynamic DNS Configuration /etc/resolv.conf

• When the local /etc/hosts cannot provide the
  answer, the resolv library consults a nameserver.
  In order to determine which machine running a
  nameserver to consult, it examines the resolv
  library's configuration file, /etc/resolv.conf.
• The /etc/resolv.conf configuration file uses
  lines which begin with the keyword nameserver to
  specify the IP addresses of machines that are
  running nameservers. If multiple nameservers are
  specified, the first one will be used by default.
  If it is unavailable, the second will be used,
  and so on. Accordingly, the first listed
  nameserver is sometimes called the primary
  nameserver, the second listed server the
  secondary nameserver, and so on.
• Notice that a nameserver does not need to be a
  nearby machine. Any machine which is implementing
  nameservice and allows you to query it can be
  used as a nameserver. In practice, using a local
  nameserver leads to better performance.

74
Network Diagnostic Utilities

• Diagnostic utilities that can be used to ensure
  that the configurations are working appropriately
  with your local network environment.
• Confirming IP Connectivity ping
• The ping command can be used to confirm IP
  connectivity between two hosts. The first
  argument to ping can be either the hostname or
  the IP address of the machine you would like to
  contact.
• The ping command will continue to run until
  canceled with the CTRLC control sequence.
• If no response is returned from the ping command,
  you cannot expect any higher level networking
  communication to occur. One common culprit is an
  Ethernet cable which has become physically
  disconnected from the machine or from the wall.
• Inconsistencies in the rate at which lines are
  displayed or discontinuities in the sequence
  numbers are generally indicative of a congested
  network, or a noisy connection, and you can
  generally expect poor network performance of
  higher level protocols.

75
Examining Routing /usr/sbin/traceroute

• When connecting to a machine outside of your
  subnet, your packet is passed from router to
  router as it traverses various subnets, until
  finally the packet is delivered to the subnet
  which contains the destination machine. The path
  of the packet, as it is passed from router to
  router, can be traced with the /usr/sbin/tracerout
  e command.
• The traceroute command is generally called with a
  single argument, the hostname or IP address of
  the destination machine.

76
Examining Routing /usr/sbin/traceroute

• Often, you will find that packets take surprising
  paths to get from one place to another. The
  number of routers that your packet passes through
  is generally referred to as the number of hops
  the packet has made. The packet above took 17
  hops to reach its destination.
• If your packet cannot complete the first hop,
  your machine's default gateway has probably not
  been correctly defined. If your traceroute
  terminates within the first couple of hops
  (without reaching the final destination), the
  problem is a misconfigured local router, and your
  local network administrator can probably help
  solve the problem. If the traceroute peters out
  more than four or five hops away, the problem is
  probably outside of your local network
  administrator's control.

77
Performing DNS Queries Manually host

• The host command can be used to perform DNS
  queries directly. With a single argument, the
  host command will simply report the requested DNS
  resolution.
• If the -a command line switch is included, the
  host command displays detailed information about
  the query performed, and the response received,
  in "resource record" format. Additionally, the
  final line identifies the nameserver who resolved
  the request, and the amount of time the
  resolution required.

78
Chapter 7  Terminal Based Web and FTP Clients

• Key Concepts
• The Elinks terminal HTTP client, invoked as the
  command links, can be used to browse web pages
  from a non graphical environment.
• The curl application is a non-interactive
  multi-protocol client (including HTTP, HTTPS, and
  FTP) which can be used to transfer files in batch
  mode.
• The wget application is a non-interactive
  multi-protocol client (including HTTP, HTTPS, and
  FTP) which can be used to recursively transfer
  web pages, recursing through all pages the web
  page references.

79
Chapter 7  Terminal Based Web and FTP Clients

• Why Terminal Based Clients?
• Users are probably familiar with sophisticated
  graphical applications such a mozilla for
  browsing the web, or nautilus for accessing FTP
  servers. In addition to these applications, Linux
  provides a collection of robust command line base
  clients as well. Why would someone be tempted to
  use command line based clients instead of point
  and click applications?
• The obvious reason If you are not running an X
  graphical environment, they are the only
  utilities available.
• Command line utilities are scriptable. If you are
  repeatedly pulling information from the same
  location, you might consider writing a script to
  do the repetitive work for you.
• Command line utilities require less resources. If
  you are trying to perform a long download on an
  otherwise busy machines, a curl process which
  consumes 4 megabytes of memory is much more
  attractive than a mozilla process which consumes
  40 megabytes of memory.
• Simplicity tends to encourage stability. When
  downloading large files, such as 700 Megabyte ISO
  images, many of the large graphical applications
  can become unstable. (The list of culprits
  includes Internet Explorer). Command line clients
  such as curl tend to produce more stable results.
  

80
Starting Elinks

• The Elinks text HTTP client (web browser) is
  designed to render web pages in a terminal,
  emulating the layout of a graphical web browser.
  As much as possible, tables and forms are
  rendered to scale. Note that in the following
  display of a web page associated with Red Hat
  Network, frames and tables are rendered
  appropriately.
• Elinks is started as the links command,
  specifying a URL to load as an argument, as in
  the following.

81
Browsing Web Pages with Elinks

• When using links, the screen is active, and
  links may be navigated using arrow keys or the
  mouse. By using the ESC key, a menu is created
  which allows a user to access bookmarks, history,
  and customization dialogs.
• Although the -dump command line switch can be
  used to non-interactively render a page to
  standard out, and the -source command line switch
  will do the same to the "RAW" content of the URL
  (often dumping HTML), the support for scripted
  interactions is fairly unsophisticated when
  compared to other utilities such as curl. The
  links browser's strength is its ability to
  provide a comfortable web browsing experience in
  a text environment.

82
Configuring Elinks to use a Proxy Server

• If the HTTP_PROXY environment variable is set,
  Elinks will use the specified proxy server, where
  (like the Elinks browser) the proxy server can be
  specified using the following syntax.
• Fetching Web Resources with curl
• In contrast the the links command, the curl
  command line web client makes no effort to render
  or interactively browse web pages. Instead, the
  development of curl has focused on a robust
  collection of command line switches that makes it
  well suited for the automated retrieval of files
  which are published using a web or ftp server.
  This text would have a hard time describing curl
  any more effectively or succinctly than the first
  two paragraphs of the curl(1) man page

83
Starting curl

• When called with a URL as its argument, the curl
  command retrieves the URL, and dumps its contents
  to standard out. Additionally, some timing
  information is written to standard error, which
  is discarded in the following example.

84
Command Line Switches for the curl Command

• The following table lists a few of the command
  line switches which can be used to modify curl's
  behavior. Consult the curl(1) man page for a
  complete list.

85
Command Line Switches for the curl Command

• As an example, the following command line could
  be used to download the ISO images for the Fedora
  Core 1 release
• The following details should be noted about the
  curl command line.
• Notice the use of "brace expansion syntax" in the
  referenced URL. The curl command will attempt to
  download three files, yarrow-i386-disc1.iso,
  yarrow-i386-disc2.iso, and yarrow-i386-disc3.iso.
  
• Because of the -O command line switch, the files
  will be downloaded to the local directory with
  the same names.
• Notice the time left at the current throughput
  13 hours! If the command needs to be aborted and
  later restarted, the -C - switch tells curl to
  examine any already existing partial downloads,
  and pick up the transfer where it left off.
• The -L command line switch tells curl to follow
  any redirects issued by the web server. (Although
  not evident from the output, curl was redirected
  to a FTP server.)

86
Configuring curl to use a Proxy Server

• If the http_proxy environment variable is set,
  curl will use the specified proxy server, where
  the proxy server can be specified using the
  following syntax.
• Recursively Downloading Web Resources with wget
• The wget command line web client is designed to
  non-interactively use the HTTP, HTTPS, and FTP
  protocols to download resources, possibly
  recursing to pages referenced from downloaded
  pages.
• Starting wget
• When called with a URL as its argument, the wget
  command retrieves the contents of the URL and
  stores it in a local file in the current working
  directory of the same name (or index.html if the
  URL specified a directory).
• The following table lists a few of the command
  line switches which can be used to modify wget's
  behavior. Consult the wget(1) man page for a
  complete list.

87
Command Line Switches for the wget Command

• wget

88
Command Line Switches for the wget Command

• As an example, the following command line could
  be used to mirror the content of the
  http//www.redhat.com/training site.

89
Command Line Switches for the wget Command

• And off it goes. After a little observation, it
  should become obvious that when called
  recursively, the wget command often recurses
  beyond expectations. After canceling the previous
  command with a CTRLC, the following directories
  demonstrate that wget strayed beyond the bounds
  of the www.redhat.com/training website.

90
Configuring wget to use a Proxy Server

• Like the curl command, wget uses the http_proxy
  environment variable to specify a proxy server,
  again using the following syntax.

91
Chapter 8.  Remote Shell Commands

• Key Concepts
• Remote shell applications allow users to execute
  arbitrary commands on remote machines, and have
  standard out returned locally. Alternately, an
  interactive shell can be started.
• The Secure Shell application provides a remote
  shell, where all transactions are encrypted, and
  users can be authenticated by traditional
  passwords or using a public key authentication
  scheme.
• In order to use the public key authentication
  scheme, a public- private key pair must be
  generated with the ssh-keygen command.
• Because Secure shell servers have their own
  public-private key pairs, servers can be
  authenticated to clients as well as clients
  authenticated to servers.

92
The Original Remote Shell rsh and Rhosts
Authentication

• Remote Shells with rsh
• Linux (and Unix) shells are intentionally
  designed with simple interfaces they read in