Application Protocols

1
Chapter 2
Application Protocols
2
Domain Name Service

• defines a hierarchical naming standard for the
  Internet
• top-level-domains (TLDs)
• "old-style" .com, .edu, .net, .mil, .gov
• ccTLDs .au, .us, .gb
• "new generic" TLDs .info, .biz
• second-level-domains
• company.com, com.au
• third-level-domains
• bond.edu.au

3
Registration Administration

• country code top level domains (ccTLDs)
• country by country
• Australia auDA
• TLDs
• was administrated by Network Solutions
• Internic
• now several 'accredited registrars'
• accredited by ICANN
• dispute resolution at infringements

4
DNS Function

• maps domain names to IP addresses
• aka "forward lookup"
• eg. kirk.bond.edu.au -gt A 131.244.1.1
• 'A record' holds address information in database
• maps aliases to other names
• 'CNAME records' list canonical names
• eg. www.bond.edu.au -gt CNAME redback.bond.edu.au

5
DNS Function

• maps domain names to mailserver names
• eg. webmaster_at_bond.edu.au -gt MX staff.bond.edu.au
• 'MX record' in database
• maps IP addresses to host names
• aka "reverse lookup"
• 'PTR record in subzone of IN-ADDR.arpa
• eg. 1.1.244.131.in-addr.arpa -gt PTR
  kirk.Bond.edu.au

6
DNS Database

• database contains zones
• everything what is known about a domain
• host names, IP addresses, subdomains....
• every zone is made up by records
• 'A records' for addresses
• 'MX records' for mail exchange references
• 'NS records' specifying the authoritative DNS for
  a zone
• every record has TTL (time-to-live) for caching

7
DNS Distributed Operation

• iterative query
• server answers either directly or gives reference
  to other DNS server
• recursive query
• server asks other DNSs itself
• more costly for the server, security issues
• zone transfer
• (trusted) DNSs exchange complete 'zones'
• for replication mainly

8
DNS Distributed Operation (cont'd)

• data dynamics
• read often, write seldom
• fault tolerance
• more than one name server for a zone
• caching, replication
• relaxed consistency, things "converge eventually"

9
Domain Name Service (cont'd)
QUESTIONS bond.edu.au, type A, class
IN ANSWERS bond.edu.au. 86400 A 131.244.1.1
AUTHORITY RECORDS bond.edu.au. 86400 NS kirk.b
ond.edu.au. bond.edu.au. 86400 NS minerva.its.bond
.edu.au. bond.edu.au. 86400 NS diablo.onthenet.com
.au. ADDITIONAL RECORDS kirk.bond.edu.au. 864
00 A 131.244.1.1 minerva.its.bond.edu.au. 86400 A
131.244.14.1 diablo.onthenet.com.au. 75661 A 203.1
0.89.2 bond.edu.au. 86400 MX 10 kirk.bond.edu.au.
10
Telnet

• for interactive access to a remote machine
• "Internet terminal"
• Catalogs, databases, services, etc. via Telnet -gt
  primitive information systems (although mostly
  displaced by the web)
• e.g. telnet pac.carl.org
• Colorado Alliance of Research Libraries
• TCP-based protocol, default server port 23
• multiplexes control data and user comm. into one
  TCP stream
• Application telnet lthostgt ltportgt

11
Telnet process model

• telnet shell.it.bond.edu.au
• Green Hat Linux release 5.5 (Manhattan)
• Kernel 2.20.36 on an 68000
• login jruser
• Password
• tcshgt

12
Telnet's pros and cons

• Standardized (RFC 854)
• Application useful to debug text-based protocols
  (HTTP, SMTP, NNTP, POP, ....)
• but 1 keystroke 3 TCP packets
• Data and Passwords in the clear
• like most other remote login tools
• simple to eavesdrop
• don't use it for interactive work!
• use SSH for that

13
Telnet for Debugging HTTP

• gt telnet www.bond.edu.au http
• Trying 203.144.16.237...
• Connected to redback.Bond.edu.au.
• Escape character is ''.
• GET / HTTP/1.0
• HTTP/1.1 200 OK
• Server Netscape-Enterprise/3.0
• Date Tue, 16 Apr 2002 113421 GMT
• Content-type text/html
• Last-modified Sun, 26 May 2002 235955 GMT
• Content-length 22944
• Accept-ranges bytes
• Connection close
• lthtmlgt
• ltheadgt
• lttitlegtBond Universitylt/titlegt
• .....

14
SSH

• interactive access to a remote computer, like
  TELNET
• but encrypted data transfer
• password not sniffable
• nifty authentication tricks possible
• arbitrary TCP traffic can be tunneled
• not fully standardized
• not as widespread yet, less applications and
  servers
• no replacement for TELNET as a debugging tool

15
File Transfer Protocol

• Bi-directional file transfer (binary, ASCII)
• ftp lthostgt (default port 21)
• Interactive access (file, directory commands)
• Authentication via username/password
• Anonymous FTP
• login as anonymous/ltemail addressgt

16
FTP process model
client
server
control connection
server protocol interface
(FTP commands, FTP replies)
file system
file system
server data transfer function
data connection
17
FTP sample session

• gt ftp mirror.aarnet.edu.au
• Connected to ....
• 220 AARNet Mirror FTP server ready.
• Name (mirror.aarnet.edu.auaz) ftp
• 331 Anonymous login ok
• Password az_at_bond.edu.au
• 230 Anonymous access granted, restrictions apply.
• ftpgt ls
• 150 Opening ASCII mode data connection for file
  list
• drwxr-xr-x 13 mirror mirror 4096 May 26
  2117 pub
• drwxr-xr-x 7 root root 512 Feb 28
  0047 raid
• 226 Transfer complete.
• ftpgt bye
• 221 Goodbye.

18
FTP commands

• User commands, understood by the client app.
• get, put, mget, mput
• cd, dir, lcd, prompt, ascii, binary
• Protocol commands exchanged by client and server
• USER, PASS, QUIT, RETR, STOR, CWD, LIST

19
FTP replies

• 3-digit code (ASCII) optional message
• Example replies
• 125 Data connection already open transfer
  starting.
• 200 Command OK.
• 331 Username OK, password required.
• 425 Can't open data connection.
• 500 Syntax error (unrecognized command).
• QUIT -gt 221 Goodbye

20
FTP's pros and cons

• Pro
• Standardized (RFC 959)
• Anonymous FTP
• Contra
• Passwords and data in the clear
• Very hard to get through a firewall securely

21
Electronic Mail
client
sender
TCP port 25
server
receiver
22
Protocols involved in email

• Simple Mail Transfer Protocol (SMTP)
• mail exchange between MTAs
• TCP protocol, port 25
• binary transport still not guaranteed
• Post Office Protocol (POP)
• retrieve mail from a mail drop
• TCP , port 110 (POP3)
• Internet Message Access Protocol (IMAP)
• access and manage remote mail drop
• TCP, port 143 (IMAP4)

23
SMTP sample session

• 220 athena.bond.edu.au -- Server ESMTP
• helo cluon.it.bond.edu.au
• 250 athena.bond.edu.au OK, cluon.it.bond.edu.au
  131.244.8.139.
• mail from ltaz_at_bond.edu.augt
• 250 2.5.0 Address Ok.
• rcpt to ltaz_at_bond.edu.augt
• 250 2.1.5 az_at_bond.edu.au OK.
• data
• 354 Enter mail, end with a single ".".
• bla
• .
• 250 2.5.0 Ok.
• quit
• 221 2.3.0 Bye received. Goodbye.

24
The corresponding email

• Email format specified in RFC 2821 and RFC 2822
• Received from conversion-daemon.staff.bond.edu.au
  by staff.bond.edu.au (iPlanet Messaging Server
  5.1 (built May 7 2001)) id lt0GXB00G01M0GH6_at_staff.
  bond.edu.augt for azangerl_at_ims-ms-daemon (ORCPT
  az_at_bond.edu.au) Fri, 07 Jun 2002 155154 1000
  (EST)
• Received from cluon.it.bond.edu.au
  (cluon.it.bond.edu.au 131.244.8.139) by
  staff.bond.edu.au (iPlanet Messaging Server 5.1
  (built May 7 2001)) with SMTP id
  lt0GXB00GN0MXK2G_at_staff.bond.edu.augt for
  azangerl_at_ims-ms-daemon (ORCPT az_at_bond.edu.au)
  Fri, 07 Jun 2002 155154 1000 (EST)
• Date Fri, 07 Jun 2002 155153 1000 (EST)
• From az_at_staff.bond.edu.au
• To az_at_staff.bond.edu.au
• Message-id lt0GXB00GN4MY02G_at_staff.bond.edu.augt
• bla

25
SMTP commands

• Minimal SMTP command set (RFC 2821)
• HELO client identification
• MAIL FROM identify sender
• RCPT TO identify receiver
• DATA start taking the message
• QUIT terminate mail exchange
• RSET abort current mail transaction, reset
• VRFY verify recipient address
• NOOP do nothing

26
SMTP replies

• 3-digit code (ASCII) optional message
• Example replies
• 250 ltaddressgt... Sender ok
• 354 Enter mail, end with "." on a line by itself
• 421 ltdomaingt Service not available, closing
  transmission channel
• 500 Syntax error (unrecognized command).
• 221 ltdomaingt closing connection (after QUIT)

27
SMTP future

• Extended SMTP (ESMTP) RFC 1425
• 220 athena.bond.edu.au -- Server ESMTP
• ehlo cluon.it.bond.edu.au
• 250-athena.bond.edu.au
• 250-8BITMIME
• 250-EXPN
• 250-STARTTLS
• 250-AUTH LOGIN PLAIN
• 250-AUTHLOGIN
• 250 SIZE 4096000
• .....lots of other capability codes
• SIZE willing to take mail up to size x, RFC1427
• 8BITMIME "8bit clean"
• EXPN expand address (do not send mail)
• AUTH, STARTTLS authentication at the server

28
Mail and the DNS

• MX resource record in the DNS
• Defines mail relay and precedence for domains
  (hosts)

tcsh gt nslookup gt set qtmx gt sun.com sun.com
preference 40, mail exchanger
mondzo.sun.com sun.com preference 5, mail
exchanger venus.Sun.COM sun.com preference 5,
mail exchanger lukla.Sun.COM sun.com preference
15, mail exchanger mercury.Sun.COM sun.com
preference 40, mail exchanger
mars.sun.com mondzo.sun.com internet address
192.18.100.1 venus.Sun.COM internet address
192.9.25.5 ....... gt
29
Anatomy of an email
Received from mail.bond.edu.au by localhost
with POP3 (fetchmail-4.7.5) for rho_at_localhost
(single-drop) Tue, 01 Feb 2000 000006 1000
(EST) Received from ocean-xterm4.it.bond.edu.au
(localhost 127.0.0.1) by ocean-xterm4.it.bond.e
du.au (8.9.3/8.9.3) with ESMTP id VAA17801 Mon,
31 Jan 2000 215254 1000 Message-Id
lt200001311152.VAA17801_at_ocean-xterm4.it.bond.edu.au
gt X-Mailer exmh version 2.1.1 10/15/1999 To
some_student_at_student.bond.edu.au Cc
robert_barta_at_bond.edu.au Subject Re proposal
for Special topics in Networking / Network
Administration In-reply-to Your message of
"Fri, 28 Jan 2000 113056 1000."
lt000901bf692f5414ec603c00a8c0_at_crippsygt
Reply-To rho_at_bond.edu.au Precedence
normal From Robert Barta ltrho_at_bond.edu.augt Mime-V
ersion 1.0 Date Mon, 31 Jan 2000 215254
1000 Sender rho_at_bond.edu.au that's the text in
the body of the email
30
Post Office Protocol

• Retrieve mail from a mail drop
• Default port 110 (POP3)

client
server
TCP connection
client POP interface
31
POP sample session

• OK POP3 kirk.bond.edu.au v7.59 server ready
• USER rho
• OK User name accepted, password please
• PASS ratzfatz
• OK Mailbox open, 2 messages
• LIST
• OK Mailbox scan listing follows
• 1 1485
• 2 2281
• .
• RETR 1
• OK 1485 octets
• Received from kirk.bond.edu.au (....
• the mail
• .
• DELE 1
• OK Message deleted
• DELE 3
• -ERR No such message

32
POP commands and replies

• Main commands (RFC 1939)
• USER specify user name (APOP name digest)
• PASS users password (cleartext!)
• QUIT end session
• STAT number of messages, size of mailbox
• LIST list mailbox/message, size number of
  message
• RETR retrieve a message
• DELE mark message for deletion
• NOOP do nothing
• RSET unmark message for deletion
• Replies (RFC 1939)
• OK lttextgt
• -ERR lttextgt

33
Internet Message Access Protocol

• POPs shortcomings
• password in the clear (if APOP not supported)
• transfers mail to local system
• network access ? Mail scattered on several
  computers.
• leave mail on POP server
• unstructured
• IMAP
• supports access and management of email on a mail
  server (RFC 2060)

34
IMAP goals

• Keep mail on the mail server
• mail needs not be transferred back and forth
• Manipulation of remote mailboxes as if they were
  local
• Access management from gt 1 computer
• full access to mail while travelling
• consistent access from home, work, etc.
• Concurrent access to shared mailboxes
• Access modes
• on-/offline, disconnected

35
IMAP access modes

• Offline similar to POP
• Online access, manage and manipulate mail on a
  server (mail is left on the server)
• Disconnected (mail is left on the server)
• make a cache copy of selected messages and
  disconnect
• manipulate cache copy
• reconnect to the server and resynchronize

36
IMAP can do more than POP

• List/create/delete/rename remote folders
• Support for folder hierarchies
• Manipulate remote folders ! INBOX
• Append messages to remote folder
• Standard and user-defined status flags (seen,
  answered, draft, ...)
• Simultaneous update and update recovery in shared
  folders
• New mail notification
• Determine message structure without downloading
• Selective fetching of individual MIME body parts
• Server-based searching and selection to minimize
  data transfer
• Negotiated extensions gt extend capabilities

37
IMAP problems

• Online mode no IMAP server gt no access to "old"
  mail
• Complex protocol gt harder to implement, more
  security problems, less stable?
• elder Mail clients support POP rather than IMAP
• Password and data in the clear
• various extensions support MD (message digest
  auth.) or CRAM (challenge-response authentication
  methods)
• with MD,CRAM at least the password not sent in
  clear

38
Multipurpose Internet Mail Extension

• "old-style" Mail body according to RFC 2822
• only 7 bit ASCII text allowed
• max line length 1000 (SMTP DATA command)
• how to transport non-text content?
• MIME (RFC 2045-2049)
• overcome RFC 822 limitations
• requires no change to MTAs
• structured mail content

39
MIME header fields (1/3)

• MIME-Version
• version of the Internet message body format
  standard in use
• MIME-version 1.0
• Content-Type
• describe the data contained in the body gt user
  agent can pick an appropriate agent to present
  the data to the user
• Content-Type text/plain

40
MIME header fields (2/3)

• Content-Transfer-Encoding
• describe encoding of binary data (non-7-bit) into
  a 7 bit short line format (SMTP, RFC 2821)
• Content-Transfer-Encoding base64
• Content-ID
• one body may want to reference another
• gt unique id required
• Content-ID ltid42_at_ocean-xterm4.it.bond...gt

41
MIME header fields (3/3)

• Content-Description
• associate some descriptive information with a
  given message body
• Content-Description Mail message body

42
MIME example (screenshot)
43
MIME example (raw content)

• Subject test
• Content-Type multipart/mixed boundary"border_1"
  
• This is a multi-part message in MIME format.
• --border_1
• Content-Type multipart/alternative
  boundary"border_2"
• --border_2
• Content-Type text/plain charsetus-ascii
• Content-Transfer-Encoding 7bit
• Content-Description Mail message body
• Just a nice test mail
• --border_2
• Content-Type text/html charsetus-ascii
• Content-Transfer-Encoding 7bit
• ltHTMLgt Just a nice test mail lt/HTMLgt
• --border_2--

44
Some MIME content types
Type Subtypes
text plain, html, xml
application octet-stream, postscript, pdf, zip
image jpeg, png, gif
audio mpeg, midi, basic
45
eMail issues, non-technical

• Be conservative in what you send and liberal in
  what you receive.
• HTML is for the web, email is to be TEXT ONLY.
• Quoting
• don't fullquote or topquote, ever.
• snip irrelevant stuff
• make "gt" your quote character or switch clients.
• McQ limit for mail signatures
• (from Usenet regular McQuary, alt.fan.warlord)
• up to 4 lines, 80 characters each

46
eMail issues (cont'd)

• don't waste bandwidth
• Meta viruses
• "There is a dangerous mail going round. Do not
  open it, it will format your harddisk. Send this
  to all your friends."
• Attachments
• proprietary formats are evil.
• size does matter

47
Spam

• Spam (UBE/UCE)
• commercials via email, "make money fast"
• mass marketing, 30 million email addresses on
  CDs
• illegal in many countries
• bandwidth consumption, consumer pays

48
Spam, personal behaviour

• DO NOT EVER reply to spam
• address is forged or defunct or a trap
• you validate your address to the spammer
• your spam intake will multiply
• use "snake skin" addresses
• gmx.net, hotmail.com, ....
• always complain to sender's ISP
• with COMPLETE mail including ALL headers
• ferreting the right info out is tedious work

49
Usenet News

• "A global distributed blackboard on top of other
  networks."
• "A World-wide discussion forum which is divided
  into hierarchical newsgroups dedicated to defined
  topics."

50
News basics

• Over 60000 newsgroups ( topics), many DOA
• Newsgroups are hierarchically structured
• e.g. comp.lang.java, sci.math.research
• Users can subscribe to a set of newsgroups
• Users can write new articles (or postings)
• Users can post them to a (set of) newsgroup(s)
• all subscribers see the postings
• Users can reply to existing postings
• thread (discussion)

51
Newsreader screenshot
52
Newsgroup hierarchies

• "big 8" comp, hum, misc, news, rec, sci, soc,
  talk
• the unregulated arena alt
• regional groups
• prefixed with ccTLD
• au.rec.cars
• other, more locali groups

53
Types of Newsgroups

• Reading and posting allowed
• everybody can read and post articles
• Moderated
• everybody can read articles
• articles to be posted are mailed to the moderator
  who decides whether they are being posted
• Read-only
• everybody can read articles
• posting requires special authorization
• Restricted circulation groups

54
Articles

• very similar to email format
• standardized in RFC 1036, changes proposed in
  "son-of-1036", ongoing discussions since years
• Articles can be posted to one or more newsgroups
  (crossposting)
• use with extreme care!
• usual rule there's exactly ONE group where
  article is on topic.
• If an article is crossposted, it should name a
  "Follow-Up" newsgroup to direct responses

55
Anatomy of an Article

• Newsgroups news.announce.newusersPath
  snafu.priv.at!pizzaschleicher.snafu.priv.at!newsfe
  ed.Austria.EU.net!newsfeed.kpnqwest.at!news-hub.s
  iol.net!cyclone.bc.net!newsfeed.stanford.edu!pres
  by.edu!nanadmin
• From nan-admin_at_presby.edu (news.announce.newusers
  moderators)
• Subject Welcome to newsgroups and Usenet!
• Sender nanadmin_at_presby.edu (news.announce.newuser
  s Moderators)
• Message-ID lt01-welcome.txt.1023537904_at_presby.edugt
  
• Supersedes lt01-welcome.txt.1022933103_at_presby.edugt
  
• Approved nan-admin_at_presby.edu
• Date Sat, 8 Jun 2002 120504 GMT
• Organization Presbyterian College, Clinton SC,
  USA
• Lines 46
• Welcome to newsgroups and Usenet! The newsgroup
  news.announce.newusers contains a collection of
  articles with basic information about newsgroups
  and pointers to further sources of information...

56
News distribution
Network News Transport Protocol (NNTP)
Network News Reader Protocol (NNRP)
57
News Servers

• Distribute articles and admin info (new
  newsgroups) via NNTP
• Store, offer and distribute a certain subset of
  all available newsgroups
• Clients can only read offered newsgroups (via
  NNRP)
• Index and expire articles (disk space !)
• full newsfeed gt200GB per day!
• Standard software INN (InterNetNews), diablo

58
Network News Transport Protocol

• Standardized in RFC 977
• Stream-based (TCP)
• SMTP-like commands and responses
• Example commands
• ARTICLE, HEAD, BODY msgID
• GROUP ltnewsgroupgt, LIST, LISTGROUP
• POST, NEWNEWS

59
NNTP Responses

• 3 digit code (ASCII) optional message
• Last line of multi-line responses .
• very similar to SMTP

60
NNTP sample session

• 200 snafu.priv.at InterNetNews NNRP server INN
  2.3.2 ready (posting ok).
• group news.announce.newusers
• 211 6 133 139 news.announce.newusers
• xover 139
• 224 139 fields follow
• 139 Welcome to newsgroups and Usenet!
  nan-admin_at_presby.edu (news.announce.newusers
  moderators) Sat, 8 Jun 2002 120504 GMT
  lt01-welcome.txt.1023537904_at_presby.edugt
  2715 46 Xref snafu.priv.at
  news.announce.newusers139
• .
• article 139
• 220 139 lt01-welcome.txt.1023537904_at_presby.edugt
  article
• Newsgroups news.announce.newusers
• From nan-admin_at_presby.edu (news.announce.newusers
  moderators)
• Subject Welcome to newsgroups and Usenet!
• ...
• .
• quit
• 205 .

61
News Problems

• deliver not guaranteed (best effort)
• reply may be received before article
• scalability
• every article is copied to all other News servers
• high bandwidth and disk space requirements

62
Directory Services

• Highly important as the Internet grows
• Certificates, identification
• Resources
• White pages (Example X.500)
• What is the phone number of John Smith in Austin,
  TX ?
• Yellow pages (Example X.500)
• Who offers a certain service in a certain area ?

63
X.500

• Directory services of the OSI model
• Global, distributed database
• Directory Information Base - DIB
• Holds entities consisting of a set of attributes
• object classes (inheritance)
• Entities, attributes and their types are freely
  definable

64
Directory Information Tree
root
CUS
CAU
OWollongong
ODEC
OBond
OUEngineering
OULegal
OUIt
OUIts
CNChris Moore
CNMarshall Rose
CNAnita Paque
OUUnix Guerilla
65
Directory System Agent

• Directory set of interacting DSAs
• DSA holds a set of naming contexts (NCs)

NC 2
NC 1
DSA
NC 3
66
Directory Access

• (Lightweight) Directory Access Protocol (LDAP)
• ldap//ldap.umich.edu/oUniversity20of20Michigan
  ,cUS
• Directory System Protocol (DSP)

DSA
Directory User Agent (DUA)
LDAP
DSP
DSA
DSA
DSP
DSA
67
LDAP

• Leight-weight Directory Access Protocol
• single-client/single-server
• atop TCP/IP
• Database
• object-oriented
• hierarchical
• distributed

68
LDAP Employments

• Organizational Address Book
• phone, email, services
• Intranet White/Yellow Pages
• NIS, yp, ldapd
• Mail Routing (address masqerading)
• Internet White/Yellow Pages

69
LDAP Basics

• Objects
• attribute/value pairs (AVAs)
• cnRobert Barta
• rfc822rho_at_bond.edu.au
• descriptionLecturer
• syntaxes (mail, photo, URL)
• objectClasses
• person
• organisationalRole

70
LDAP Basics (contd)

• relative distinguished name (RDN)
• cnRobert Barta, descConsultant
• set of AVAs
• distinguished name (DN)
• cAU, oBond University, cnRobert Barta
• path from DIT root
• aliases
• cAU, oBond University, cnLecturer for INFT130

71
LDAP Basics (contd)

• directory information tree (DIT)
• distributed over various servers (contexts)
• countries
• organizations
• organizational units
• persons
• localities
• .....

72
A brief WWW History
1945 V. Bush As we may think (memex, linked info)
1980 T. Berners-Lee Enquire system
1989 T. Berners-Lee Proposal for a global hypertext system (CERN)
1991 T. Berners-Lee 1st browser/editor (CERN)
1993 M. Andreessen Mosaic, NCSA server (NCSA)
1994 Netscape is founded
1996 Microsoft invents the Internet
1997 The browser war starts
1996-? Myriads of languages, protocols, standards
Now WWW cell phones PDAs ...
73
WWW Conceptually

• The WWW is a synchronous, distributed, multiple
  client, multiple server hyper-media system.
• HyperText Transfer Protocol (HTTP)
• HyperText Markup Language (HTML)

hypertext link
hypertext link
Web server
Web server
Web server
TCP port 80
TCP port 80
TCP port 80
HTTP connections
Web client (browser)
74
URIs, URLs URNs

• URI
• Naming scheme and syntax
• URL
• URIs for objects accessed with existing protocols
• URN
• persistent object names (resolution protocols)
• Hyperlinks on the WWW
• are represented as URLs

75
Uniform Resource Locator

• ltschemegtltscheme-specific-partgt
• Schemes
• ftp, http, mailto, news, telnet, file, ....
• Common Internet Scheme Syntax
• //ltusergtltpasswordgt_at_lthostgtltportgt/ltpathgtlt
  fragmentgt

76
URL Examples

• http//james.bond.edu.au/courses/inft11135_at_021/ind
  ex.mc
• ftp//ftp.bond.edu.au/
• ftp//anonymous_at_ftp.gnu.org/gnu/
• ftp//anonymous_at_ftp.gnu.org/gnu/
• ftp//hugomagumba_at_ftp.gnu.org/gnu/
• mailtorho_at_bond.edu.au
• telnet//hugo_at_shell.it.bond.edu.au
• file///home/users/hugo/file1.txt

77
HTTP Interaction Pattern
Web server
Web client (browser)
Open TCP connection
Request
Send HTTP request
Check request
- Syntax OK ?
- Document exists ?
- Authentication required ?
Send reply
Reply
Receive reply
Close TCP connection
Interpret reply
Display document
time
78
HTTP requests (1/2)

• HTTP 0.9
• GET /this/and/that/ ltCRLFgt
• HTTP 1.0
• GET /this/and/that/ HTTP/1.0 ltCRLFgt
• Date Sun, 12 Dec 2001 182628 GMT ltCRLFgt
• From client_at_somewhere.com ltCRLFgt
• Referrer http//www.gnu.org/index.html ltCRLFgt
• ltCRLFgt
• ltCRLFgt

79
HTTP requests (2/2)

• HTTP 1.1
• GET /this/and/that/ HTTP/1.1 ltCRLFgt
• Host www.gnu.org ltCRLFgt
• Date Sun, 12 Dec 2001 182628 GMT ltCRLFgt
• From client_at_somewhere.com ltCRLFgt
• Referrer http//www.gnu.org/index.html ltCRLFgt
• If-Modified-Since Sun, 8 Dec 2001 140000 GMT
  ltCRLFgt
• User-Agent Mozilla/4.5 ltCRLFgt
• ltCRLFgt

Host header allows the server to differentiate
between ambiguous URLs, such as the "/" or root
URL for multiple hostnames on a single IP address
(soft virtual servers)
80
Content Type Negotiation

• Client sends list of media types acceptable for
  the response
• Accept text/q0.3, text/htmlq0.7,
  text/htmllevel1,
• text/htmllevel2q0.4, /q0.5

81
Server-side Processing (1/4)

• Is the requested document available ?
• 404 Not Found
• 302 Found
• 303 See Other
• 307 Temporary Redirect
• Check file system access restrictions
• 403 Forbidden
• Check access restrictions (username, password,
  etc.)
• 401 Unauthorized

82
Server-side Processing (2/4)

• Is the requested URL a directory ?
• yes gt does "index.html exist ?
• yes gt read "index.html
• no gt generate directory listing on the fly
• no gt read file
• Determine MIME type for response (consult
  "mime.types)

83
Server-side Processing (3/4)

• Is it an executable program ?
• yes gt start it (output of the program is the
  reply to the original request)
• no gt return file contents ( admin headers,
  such as content-type, length, etc.)

84
mime.types
This file controls what Internet media types are
sent to the client for given file extension(s).
Sending the correct media type to the client is
important so they know how to handle the content
of the file.

• application/postscript ai eps ps
• application/x-dvi dvi
• application/x-javascript js
• audio/basic au snd
• audio/mpeg mpga mp2 mp3
• image/gif gif
• image/jpeg jpeg jpg jpe
• text/html html htm
• text/plain asc txt
• text/xml xml
• text/x-server-parsed-html shtml

85
A typical Reply

• HTTP/1.1 200 OK
• Date Mon, 13 Dec 2001 100329 GMT
• Server Apache/1.3.6 (Unix) mod_ssl/2.2.7
  SSLeay/0.9.0b
• Last-Modified Mon, 06 Dec 2001 100733 GMT
• ETag "9f807-3291-384b8ae5
• Accept-Ranges bytes
• Content-Length 12945
• Content-Type text/html
• lt!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN"gt
• ltHTMLgt
• ltHEADgtltTITLEgtPooh's Asylumlt/TITLEgtlt/HEADgt
• ltH1gtWelcome to Pooh's Asylumlt/H1gt
• ....

86
Client-side Processing (1/2)

• 200 OK gt display document
• Browser can display document
• Helper application
• Plug-in
• 302 Found gt Fetch again
• 303 See Other, 307 Temporary Redirect
• 4xx or 5xx gt Ooops
• error message
• 401 gt Authorization Basic QWxfhUdasksIqexaQ

87
Helper Applications

• New process
• Handles content unknown to browser
• .mailcap

88
Client-side Processing (2/2)

• HTML document ?
• Inline images ?
• get inline image via a new TCP connection
• External Javascript file ?
• get it via a new TCP connection
• run it
• Applet ?
• load applet
• start local Java and run applet
• Frames ?
• analyze frame structure
• get dependent documents / frames

89
Basic Authentication (RFC2617)
Web server
Web client (browser)
GET /auth/doc.html HTTP/1.1
Send HTTP request
Check request
gt Authentication required
Send challenge
Receive reply
HTTP/1.1 401 Authorization Required WWW-Authentica
te Basic RealmProveIt?
Send credentials (user-id, passwd in Base 64))
Check credentials if OK, send document
GET /auth/doc.html HTTP/1.1 Authorization Basic
QWxhzGBRpbjpvc
Send document
Display document
time
90
Basic Authentication Config

• .htaccess

91
HTTP Requests (Syntax)
Request Request-Line ((
general-header
request-header
entity-header ) ltCRLFgt)
ltCRLFgt message-body
Request-Line Method ltSPgt Request-URI ltSPgt
HTTP-Version ltCRLFgt
92
Methods (1/5)

• GET
• retrieve document
• parameters can be encoded in URL
• http//james.bond.edu.au/robot.mc?codelwhatever
• HEAD
• like GET but only returns the meta-information
  contained in the HTTP headers
• does not return the document itself !

93
Methods (2/5)

• POST
• add entity enclosed in the request as a new
  subordinate of the resource identified by the
  URL, i.e., URL identifies the resource that will
  handle the enclosed entity
• annotation of existing resources
• posting a message to a newsgroup, mailing list,
  etc.
• a block of data (as a result of submitting a
  form)
• Parameters come as message body in request
• (application/x-www-form-urlencoded)

94
Methods (3/5)

• OPTION communication options available for URL

95
Methods (4/5)

• PUT
• store enclosed entity under the given URL
• create entity if necessary
• DELETE
• delete the entity given by URL
• CONNECT
• reserved (proxy gt tunnel)

96
Methods (5/5)

• TRACE Is there anybody out there ?

97
HTTP Responses (Syntax)
Response Status-Line ((
general-header
response-header
entity-header ) ltCRLFgt)
ltCRLFgt message-body
Status-Line HTTP-Version ltSPgt Status-Code
ltSPgt Reason-Phrase ltCRLFgt
98
HTTP Status codes
Reply Description
1xx  Informational request received, continuing process
2xx   Success - action successfully received, understood, and accepted
3xx  Redirection - further action must be taken to complete request
4xx  Client error request contains bad syntax or cannot be fulfilled
5xx Server error server failed to fulfill an apparently valid request
99
Conditional GET

• GET /this/and/that/ HTTP/1.1
• Host www.gnu.org
• Date Sun, 12 Dec 2001 182628 GMT
• From client_at_somewhere.com
• Referrer http//www.gnu.org/index.html
• If-Modified-Since Sun, 12 Dec 1999 140000 GMT
• User-Agent Mozilla/4.5

100
Redirect

• GET /this/and/that/NotHere.html HTTP/1.1
• Host www.gnu.org
• Date Sun, 12 Dec 2001 183001 GMT
• User-Agent Mozilla/4.5

101
Advantages of HTTP 1.1 (1/2)

• Persistent connections
• before 1 TCP connection per inline image
• pipeline requests and responses over 1 TCP
  connection
• default behavior, but negotiable (Connection
  close)
• Hostname identification
• Content Negotiation
• quality factors
• language
• Compression (gzip, compress, ..)
• Multi-part messages
• transfer encoding, content encoding

102
Advantages of HTTP 1.1 (2/2)

• Byte ranges
• Entity Tags
• Request and response chains
• Proxies (HTTP-aware)
• Gateways (application-aware)
• Tunnels (e.g., firewalls)
• chain members can handle multiple clients/servers
• chain members may cache responses
• Elaborate support for caches and proxies