Web Server Traffic Analysis

1
Web Server Traffic Analysis
2
Web Traffic

• Users requesting files from different web servers
  generate web traffic - or the flow of data
  through the internet.

3
Why Measure Web Traffic?

• Measure Usage -- Costs can be high. Are people
  actually looking at the site?
• Who, What, Where, When
• Whos your audience?
• What parts of your site are accessed the most?
• When is the busiest time of day?
• Where are your visitors coming from?
• How much bandwidth is being used?
• Sites that sell advertising have payouts that
  rely on the influx of hits

4
What Are Log Files?

• Whenever an HTTP request is made to a web server,
  information regarding the request are stored in a
  file.
• Information logged includes file that was
  accessed, date/time, point of origin (IP
  address), type of browser used, and errors
  encountered.

5
Types of Log Files

• Error Log -- Tracks problems accessing files
• Access Logs -- Tracks http requests
• Referer Logs - Tracks the link from which the
  user arrived at your page
• Agent Log -- Tracks type of browser used
• Most Log files are stored in industry standard
  NCSA/CERN filetypes

6
What Log File Analyzers Do

• Parse data in log files and generates reports,
  graphs, and charts on a regular basis.
• Usually these programs give a visual idea of the
  amount and quality of traffic your site receives

7
Simple Analyzers

• Shows number of hits over a given time
• Amount of data transferred
• Busiest point of origin
• Busiest times of day

8
Complex Analyzers

• Generates Audit Reports that justify rates
  charged to advertisers
• Network oriented reports analyzes server and
  network load
• Helps to justify costs for marketing purposes

9
External Services

• Some sites use third party companies to track and
  analyze traffic
• This relieves the load on the web server because
  it doesnt have to process the log files
• Gives credibility to your results

10
In House Services

• Some sites track and analyze traffic locally
  within the organization
• Information may need to be confidential
• May operate on a server other than the web server

11
Criteria for Selecting an Analyzer

• Is it powerful enough? Will it choke on large
  files?
• Does it provide the information you need?
• How easy is it to install and use?
• Does it only run locally? Can it be located on
  another server or network?
• Does the package have a built in database? If
  not, can it export to Access, Lotus 123, or
  Paradox?
• Can it resolve IP addresses to generate host
  names of visitors?
• How much does it cost? (Ranges between 200 to
  several thousand dollars)

12
What Log Analyzers Tell You

• How to increase traffic to your site
• What pages are read the most
• Visitor demographics
• What Files are being downloaded?
• Any broken links?
• When is the busiest time of day/week/month for
  your site?

13
Product Reviews

• NetIntellect 2.1
• NetTracker 2.0
• WebTrends 3.0

14
NetIntellect 2.1

• Generates 2D 3D charts
• Has automated tools for retrieving databases and
  log files via remote FTP
• Priced at 199
• Accepts at least 20 different log formats, and
  due to remote FTP capabilities, interfaces
  seamlessly with Mac and Unix platforms as well as
  NT
• Reports can be saved in HTML, text, Word, Excel,
  and 123 formats
• 80 predefined charts, graphs, and tables
• Wizards help simply report generation

15
NetTracker 2.0

• Reports can be viewed from anywhere without
  special software
• Installs on server, but delivers results remotely
  to Java capable browsers
• Generates reports displaying basic data -- page
  usage, busiest times, referring URLs, and visitor
  profiles

16
WebTrends 3.0

• Rated the best
• One of the most widely used packages -- Fourtune
  100 firms use it to process 40 million daily hits
  and log files up to 4GB
• SPEED. Parses large log files quickly
• Runs as a background service in NT, maintaining a
  runtime index that the user can search through
• Creates reports ever 5-10 minutes, and results
  can be emailed to any address you specify
• Reports are generated in HTML
• Over 60 tables and charts to choose from