Counting syscall in Varnish Cache

Way back when we did a rough count on how many syscalls Varnish uses to deliver a piece of cache content. I think Anders counted seven calls per hit, roughly based on some system counters. I just did a recount, just to see how we are doing. After all, calling the kernel is one of the more expensive things to do in a program. Here is what I found.

Edit: I think the original claim was 7 system calls + 5 locks.

The first two syscalls - the acceptor threads

[pid 26106] poll([{fd=8, events=POLLIN}, {fd=9, events=POLLIN}], 2, ....

[pid 26106] accept(9, <unfinished ...>)

It polls the file descriptors and accepts any connection that has been made. Then the file descriptor is handled over to the designated worker thread that. There is a lock, probably the one on the thread pool, that is set before a worker thread is picked.

[pid 26106] futex(0x7fb7fba9fc1c, FUTEX_WAKE_OP_PRIVATE, 1, 1,...

The worker threads

Then the worker thread takes a couple of locks and does a ioctl of some sort.
[pid 26202] futex(0x7fb835e123a8, FUTEX_WAKE_PRIVATE, 1
[pid 26202] ioctl(12, FIONBIO
[pid 26202] futex(0x6797a0, FUTEX_WAIT_PRIVATE, 2, NULL
[pid 26202] futex(0x6797a0, FUTEX_WAKE_PRIVATE, 1) = 0

Then a poll and then the request is read.

[pid 26202] poll([{fd=12, events=POLLIN}], 1, 50
[pid 26202] read(12, "GET / HTTP/1.0\r\n"..., 32767) = 82

After we got the request there is the small matter of serving it:

[pid 26202] writev(12, [{"HTTP/1.1", 8}, {" ", 1}, {"200", 3}, {" ", 1}, {"OK", 2}, {"\r\n", 2}, {"Server: Apache/2.2.14 (Ubuntu)", 30}, {"\r\n", 2}, {"V

And clean up. Close the connection and lock the thread pool before putting itself back in the pool.

[pid 26202] close(12
[pid 26202] futex(0x7fb7fba9fc1c, FUTEX_WAIT_PRIVATE, 453, NULL


To sum it up. 3 syscalls in the acceptor thread. 9 in the worker thread. 12 in all.

Edit: Of those 12, 5 are locks. So, 7 + 5 seems still to be the case. :-)

Topics: Varnish Cache, Open Source

4/12/12 11:19 AM by Per Buer

All things Varnish related

The Varnish blog is where the our team writes about all things related to Varnish Cache and Varnish Software...or simply vents.


Recent Posts

Posts by Topic

see all

Varnish Software Blog