Host and Callback Tracking in OpenAFS

1
Host and Callback Tracking in OpenAFS

• Jeffrey Altman, Secure Endpoints, Inc
• Derrick Brashear, Sine Nomine Associates

2
AFS concepts

• Cell The unit of administrative control for AFS
  filesystems. (An organization or part of one)
• Volume A relocatable (path- and storage-wise)
  piece of the AFS filesystem.
• FID A volume, vnode (file or directory) and
  uniquifier (an incrementing version) which
  corresponds uniquely to one revision of one
  object.

3
AFS concepts

• UUID a (theoretically) globally unique
  identifier on each client and server.
• Used to track when were talking to the same
  machine at a different address.

4
10 mile view of AFS

• AFS uses UDP as its data transport.
• This means there is no concept of connected
• The fileserver must track the client so that as
  the client moves, or goes away, status can be
  maintained or purged as appropriate.
• Clients may have many addresses (host and port
  pairs), treated equivalently when requests are
  handled.
• Exactly one, the callback connection address,
  will be used for messages which originate at the
  server.

5
10 mile view of AFS

• Client-server architecture
• Clients work like traditional network FS
• Except they cache.
• In order to maintain cache coherency, help from
  the server is needed.

6
100 foot view of AFS

• To avoid unnecessary network traffic, the
  fileserver tracks what clients are interested in
  which files, and offers a callback, which is a
  coherency guarantee for a given duration.
• If the file changes before that time expires, the
  client is notified. This is a callback break.
  It will be sent via the callback connection.
• A client can at break or expiration re-obtain a
  callback. Data is re-fetched only if needed.

7
6 inch view of AFS

• Callbacks are granted on a sliding time scale.
  More clients interested in a file means shorter
  callbacks.
• This is probably dumb.
• Should be based upon the likelihood of change.
• For .readonly volumes, callbacks are tracked at
  the volume level.

8
Callback buckets

• Since we lower the duration of a callback as more
  clients become interested, we keep track of FIDs
  in buckets. When too many people for the given
  bucket care, it goes in the next bucket.
• Buckets are 4 hours (up to 7 users), 1 hour (up
  to 15 users), and descend to 7 minutes above 63
  users.

9
Host Tracking

• InitCallBackState, WhoAreYou, TellMeAboutYourself
  (explain)
• The addresses we receive from the client can not
  be trusted.
• Given NAT, it is likely that all of them are
  useless.
• We should be able to reply to the sender, though.

10
Enter the UUID

• ProbeUUID (explain)
• Remember its always safe to use the address
  which sent to us.
• Update the address list with the UUID as this
  changes.

11
The callback connection

• (details about how and when we update it)

12
Timeouts

• No, you dont have to sit in the corner.
• The common case is a 57 second delay, during
  which retransmits happen at slowing rates, until
  we decide the other end is gone.
• Ideally, state is tracked so the client never has
  to elapse this time idle.

13
More

• Perhaps a flow chart of the steps the server
  takes when receiving a connection from a client
• When are callback breaks issued on files,
  directories, mountpoints, symlinks, and volumes?

14

• How do the clients track callbacks?
• Debugging callback issues?
• Where are the bottlenecks?
• What can be done to reduce the bottlenecks and
  improve performance?

15

• How do NATs affect callback management?
• What are delayed callbacks?