Friday, July 13, 2012

lophttpd trickery

I made some optimizations for lophttpd to dynamically
adjust the chunksize that is passed to sendfile().
Its based on the amount of data that is already
in the TCP send queue. The problem is that the web
server can be too fast in pushing data to the client,
so the send queue starts filling until its full and
the connection times out and is dropped as poll() wont
report possible events on it. You can adjust your send
buffer sizes (up to some extent), but increasing the timeout is no good idea,
as evil clients might just not read the pending data.
This solution avoids that and still gives you enough tuning
parameters (-S, -N, -DSTATIC_SEND_SIZE_COMPUTATION) in case
you know better.:D Much of the performance parameters
depend on the actual hardware/network configuration
(fast CPU vs. slow network, fast CPU and Gigabit LAN,
slow receivers etc. etc.). For few connections like
1000 simultaneous downloads, this is no issue at all.
The fun begins with 10k connections and its not about
CPU usage, but having as few connection drops
as possible for long lasting downloads. However the
default values should fit for most scenarios.

If you have a good test environment for c10k, let me know.