What is "sendfile"?
Many Unix kernels provide a function called sendfile(2),
a system call which provides a "zero-copy" way of copying data from one
file (or socket) descriptor to another. The phrase "zero-copy" refers to the
fact that all of the copying of data between the two descriptors is done
entirely by the kernel, with no copying of data into userspace buffers.
The normal way of copying data between files involves using the
read(2) system call to read data from the source descriptor
into a userspace buffer, then calling the write(2) system call
to write that buffer to the destination descriptor. This copying of the
data twice (once into the userland buffer, and once out from that
userland buffer) imposes some performance and resource penalties. The
sendfile(2) avoids these penalties by avoiding any use of
userland buffers; it also results in a single system call (and thus only
one context switch), rather than the series of
read(2)/write(2) system calls (each system call
requiring a context switch) usually used for data copying.
Unix kernels handle a socket as just another file descriptor; this means
that sendfile(2) can be used to efficiently copy data from
a file on disk to a network socket, e.g. for downloading a file.
Use of sendfile(2) in ProFTPD
For the above reasons, ProFTPD will, by default, attempt to use the
sendfile(2) function for all downloads. However, there are
a few cases where ProFTPD will specifically avoid the use of
sendfile(2):
TransferRate
directive
MODE Z data compression is being used (via the
mod_deflate module)
sendfile(2) can also be explicitly configured at
run-time by using the following in your proftpd.conf file:
UseSendfile off
Sendfile support in the compiled proftpd daemon can also be
disabled at compile time, by using the --disable-sendfile
configure option, e.g.:
$ ./configure --disable-sendfile ...This is not recommended unless necessary.
Known Issues with sendfile(2)
As useful as the sendfile(2) function can be, there are
unfortunately cases where bad implementations of the function can cause
problems. These problems manifest as broken or aborted downloads, or as
downloaded data being corrupted, or even as downloaded files being larger
than the original file.
The sendfile(2) function requires support from both the Unix
kernel and from the filesystem drivers which handle the filesystems
for the file descriptors being used. The following operating systems
have had reports of issues when using sendfile(2):
There have been cases where it was the filesystems, rather than the kernels,
which appeared to have been the culprits in sendfile(2)
problems:
tmpfs on Linux)
proftpd.conf.
Bugs in certain network cards have been reported on Linux, where the use
of sendfile(2) triggers TCP checksum offloading bugs on these
cards when using IPv6.
Bug 3081 also
demonstrates an interesting sendfile(2) issue. It is a special
case where the FTP client and server are on the same machine,
sendfile(2) is used when downloading a file, and the client is
downloading the file to the exact same filesystem path from which the
server is reading the file. When this happens, proftpd will
"hang". The fix for this situation is relatively simple: "Don't do that."
If your Unix kernel and your filesystems work together to support
the use of sendfile(2) properly, you may run into one last
potential issue with ProFTPD's ftptop and ftpwho
utilities when sendfile(2) is used (at least for versions of
proftpd prior to 1.3.4rc1). These utilities rely on
data in ProFTPD's ScoreboardFile
for providing their information, including transfer rates. The progress
of a file being downloaded is tracked in the ScoreboardFile
as part of the download process; specifically, as part of the normal
read(2)/write(2) loop. When the
sendfile(2) function is used, the ScoreboardFile is
not updated with the progress (since all of the data copying is being handled
by the kernel, not by proftpd code at that point). Thus
for downloads which use sendfile(2), ftptop and
ftpwho often report the transfer rate as "Inf".
The transfer rates for uploads are still reported properly by
ftptop and ftpwho, however.
In proftpd-1.3.4rc1 and later, the UseSendfile
directive changed. Now you can use UseSendfile in
<Directory> sections and .ftpaccess files.
This allows sites to disable use of sendfile(2) just for specific
directories/filesystems where it might be a problem, e.g.:
<Directory /path/to/nfs/files>
UseSendfile off
</Directory>
Instead of the usual on/off parameters, the UseSendfile
directive can also take a byte length, or a file size percentage, as parameters.
These parameters can be used so that sendfile(2) is used for
more optimal downloads while still letting ftptop and
ftpwho display download progress rates. To use this, the
best way to use specify a file size percentage such as 10% or 25%; this means
that sendfile(2) will be used for each 10% or 25% percent of
the size of the file being downloaded:
<Directory />
UseSendfile 10%
</Directory>