Lazy Asynchronous IO For EventDriven Servers

1
Lazy Asynchronous I/O For Event-Driven Servers

• Khaled Elmeleegy, Anupam Chanda and Alan L. Cox
• Department of Computer Science
• Rice University, Houston, Texas.
• Willy Zwaenepoel
• School of Computer and Communication Sciences
• EPFL, Lausanne, Switzerland.

2
Event-Driven Architecture

• Event-driven architecture is widely used for
  servers
• Performance
• Scalability

3
Problem

• Developing event-driven servers is hard for many
  reasons
• We focus on problems with non-blocking I/O
• Incomplete coverage
• Burden of state maintenance

4
Lazy Asynchronous I/O (LAIO)

• Addresses problems with non-blocking I/O
• Universality
• Covers all I/O operations
• Simplicity
• Requires less code
• High performance for event-driven servers
• Meets or exceeds alternatives

5
Outline

• Background
• Lazy Asynchronous I/O (LAIO)
• Evaluation
• Conclusions

6
Outline

• Background
• Event-driven servers
• Lazy Asynchronous I/O (LAIO)
• Evaluation
• Conclusions

7
Event-Driven Servers

• Event loop processes incoming events
• For each incoming event, it dispatches its
  handler
• Single thread of execution

Event Loop
Handler 1
Handler 2
Handler k
8
Event Handler
I/O operation (Network/Disk)
To event loop
Complete handling event
9
Event Handler

• If the I/O operation blocks
• The server stalls

I/O operation (Network/Disk)
Block
To event loop
Complete handling event
10
Outline

• Background
• Lazy Asynchronous I/O (LAIO)
• API
• Example of usage
• Implementation
• Evaluation
• Conclusions

11
LAIO API
12
laio_syscall()

• Lazily converts any system call into an
  asynchronous call
• If the system call doesnt block
• laio_syscall() returns immediately
• With return value of system call
• Else if it blocks
• laio_syscall() returns immediately
• With return value -1
• errno set to EINPROGRESS
• Background LAIO operation

13
laio_gethandle()

• Returns a handle representing the last issued
  LAIO operation
• If operation didnt block, NULL is returned

14
laio_poll()

• Returns a count of completed background LAIO
  operations
• Fills an array with completion entries
• One for each operation
• Each completion entry has
• Handle
• Return value
• Error value

15
Event Handler With LAIO

• If operation completes
• Handler continues execution

laio_syscall()
Operation completed
No
To event loop
Yes
Complete handling event
16
Event Handler With LAIO

• If operation doesnt complete
• laio_syscall() returns immediately
• Handler records LAIO handle
• Returns to event loop
• Completion notification arrives later

laio_syscall()
Operation completed
No
Handle laio_gethandle()
To event loop
Yes
Complete handling event
17
LAIO Implementation

• LAIO uses scheduler activations
• Scheduler activations
• The kernel delivers an upcall when an operation
• Blocks - laio_syscall()
• Unblocks - laio_poll()

18
laio_syscall() Non-blocking case
Application
laio_syscall()

• Save context
• Enable upcalls

Issue operation
System call blocks?
Library
No

• Disable upcalls
• Return retval

19
laio_syscall() Blocking case
Application
laio_syscall()
Library

• Save context
• Enable upcalls

Issue operation
System call blocks?
Yes
20
laio_syscall() Blocking case
Application
laio_syscall()
Library

• Save context
• Enable upcalls

Issue operation
System call blocks?
Yes
Kernel
Background laio_syscall
Upcall on a new thread
21
laio_syscall() Blocking case
Application
laio_syscall()
Library

• Save context
• Enable upcalls

Issue operation
upcall handler
System call blocks?
Library
Steals old stack using stored context
Yes
Kernel
Background laio operation
Upcall on a new thread
22
laio_syscall() Blocking case
Application
laio_syscall()

• Disable upcalls
• errno EINPROGRESS
• Return -1

Library

• Save context
• Enable upcalls

Issue operation
upcall handler
System call blocks?
Library
Steals old stack using stored context
Yes
Kernel
Background laio operation
Upcall on a new thread
23
Unblocking case

• List of completions is retrieved by the
  application using laio_poll()

Kernel

• Background laio operation completes, thread dies
• Upcall on the current thread

24
Outline

• Background
• Lazy Asynchronous I/O (LAIO)
• Evaluation
• Methodology
• Experiments
• LAIO vs. conventional non-blocking I/O
• LAIO vs. AMPED
• Programming complexity
• Conclusions

25
Methodology

• Flash web server
• Intel Xeon 2.4 GHz with 2 GB memory
• Gigabit Ethernet between machines
• FreeBSD 5.2-CURRENT
• Two web workloads
• Rice 1.1 GB footprint
• Berkeley 6.4 GB footprint

26
Experiments LAIO vs. conventional non-blocking
I/O
Compare performance
27
Performance Large Workload
28
Performance Small Workload
29
Why Be Lazy?

• Most potentially blocking operations dont
  actually block
• Experiments 73 - 86 of such operations dont
  block
• Lower overhead
• Micro-benchmark AIO is 3.2 times slower than
  LAIO for non-blocking operations

30
Experiments LAIO vs. AMPED
Compare performance
31
Asymmetric Multi-process Event-Driven (AMPED)
Architecture

• Event-driven core
• Potentially blocking I/O handed off to a helper
• Helper can block as it runs on a separate thread
  of execution

Event Loop
Helper 1
Handler 1
Helper 2
Handler 2
Helper n
Handler k
32
Flash-NB-AMPED

• Stock version of flash
• Two relevant helpers
• File read reads a file from disk
• Name conversion checks for a files existence
  and permissions

33
Performance of LAIO vs. AMPED
34
Performance of LAIO vs. AMPED
35
Programming Complexity

• Where performance is comparable Flash-LAIO-LAIO
  vs. Flash-NB-AMPED
• Flash-LAIO-LAIO is simpler
• No helpers
• No state maintenance

36
State Maintenance With Non-blocking I/O

• If operation does not complete
• Handler returns to event loop

handler
I/O operation
Operation completed
No
To event loop
Yes
Complete handling event
37
State Maintenance With Non-blocking I/O

• If operation does not complete
• Handler returns to event loop
• Operation is continued later

handler
I/O operation
Operation completed
No
To event loop
Yes
Complete handling event
38
State Maintenance With Non-blocking I/O

• If operation does not complete
• Handler returns to event loop
• Operation is continued later
• This may require storing state

handler
I/O operation
Operation completed
No
To event loop
Store state
Yes
Complete handling event
39
Lines of Code Comparison
9.5 reduction in lines of code
40
Conclusions

• LAIO is universal
• Supports all system calls
• LAIO is simpler
• Used uniformly
• No state maintenance
• No helpers
• Less lines of code
• LAIO meets or exceeds the performance of other
  alternatives

41
LAIO source can be found at http//www.cs.rice.
edu/kdiaa/laio