What is Memcache?
Memcache (or "memcached") is an open-source,
high performance memory object caching system. A simple (and effective)
key/value store accessible, efficiently, over the network.
How Can Memcache Be Useful for ProFTPD?
Like any high-performance object store, memcached
offers several
possibilities to a server like proftpd
. Many sites use
memcached
for caching; it can also be used as an efficient
shared storage mechanism, for sharing data among many different servers. And
for ProFTPD specifically, the shared storage aspect is what is most useful.
Things like SSL/TLS sessions can be cached and shared across a pool of
proftpd
servers, as can ban lists for badly-behaved clients.
Enabling Memcache Support for ProFTPD
OK, so you are interested enough in the possibilities that
memcached
offers that you want to try it out. Excellent! To
do this, you will first need to make sure to build your proftpd
executable using the --enable-memcache
configure option. The
--enable-memcache
configure option automatically adds the
mod_memcache
module to
your proftpd
build.
The mod_memcache
module uses the libmemcached
library for talking to
memcached
servers. If your libmemcached
library is
installed in a non-standard location, you may need to tell the ProFTPD build
system where to find the libmemcached
header files and libraries
using the --with-includes
and --with-libraries
configure options.
There are other modules which make use of memcached support when available,
such as
mod_tls_memcache
.
Thus to take advantage of modules like this, putting everything together, your
configure command might look like this:
$ ./configure --enable-memcache \ --with-modules=...:mod_tls_memcache:... \ --with-includes=/path/to/libmemcached/include \ --with-libraries=/path/to/libmemcached/lib
Configuring mod_memcache
Now that you have compiled proftpd
with the
mod_memcache
module, you need to add the necessary
mod_memcache
directives to your proftpd.conf
.
The following example demonstrates this:
<IfModule mod_memcache.c> # Enable mod_memcache MemcacheEngine on # Tell mod_memcache where to log its messages MemcacheLog /path/to/proftpd/memcache.log # Tell mod_memcache where to find the memcached servers MemcacheServers 192.168.0.10:11211 192.168.0.11:11211 </IfModule>If you wish to see more detailed logging, at least while you are setting up your memcached servers for ProFTPD, you can enable trace logging for the
memcache
trace channel using e.g.:
TraceLog /path/to/proftpd/trace.log Trace DEFAULT:10 memcache:20
Using Memcache for Shared Storage
You have now compiled support for memcached into ProFTPD, and you have told the
mod_memcache
module where to find your memcached
servers. Is that all you need to do? No. Now you need to tell
proftpd
modules which bits of data to store in your
memcached
servers.
Currently, only two modules can take advantage of memcached
support: mod_ban
and
mod_tls_memcache
.
First, let us examine mod_ban
and how it would use
memcached
. The mod_ban
module manages ban lists,
lists of clients/users which have been banned for various reasons. These
lists are stored in shared memory by default; this works for a single
proftpd
server, but if a badly behaved client is banned by one
proftpd
server in pool of servers, that client can then connect to
a different server which might not have a ban for that client -- and the client
then gets another chance to be naughty. To configure mod_ban
so
that it stores its ban lists in memcached
, simply use the following
in your proftpd.conf
:
<IfModule mod_ban.c> BanEngine on # ...other mod_ban directives... # Tell mod_ban to store its ban lists using memcache BanCache memcache </IfModule>With this,
mod_ban
will use memcached
(as well as
shared memory) for reading/writing its ban lists. And this, in turn, means
that other proftpd
servers' mod_ban
modules can see
those bans, and reject the badly behaved clients across the pool/cluster.
The mod_tls_memcache
module uses memcached
servers
for storing SSL/TLS sessions; SSL/TLS session caching can greatly improve
SSL/TLS session handshake times, particularly for data transfers using
SSL/TLS. If you have a pool of proftpd
servers, and you have
FTPS clients which may connect to a different node every time, caching the
SSL/TLS session data in a shared storage mechanism like memcached
can be quite beneficial.
To use memcached
for SSL/TLS session caching, then, you use the
TLSSessionCache
directive of the mod_tls
module, using something like this
in your proftpd.conf
:
<IfModule mod_tls.c> TLSEngine on # ...other mod_tls directives... <IfModule mod_tls_memcache.c> # Tell mod_tls to cache sessions using memcached TLSSessionCache memcache: </IfModule> </IfModule>That's it. The
mod_tls
module now knows to give the SSL/TLS
session data to mod_tls_memcache
, and mod_tls_memcache
knows how to talk to the memcached
servers using
mod_memcache
.
Frequently Asked Questions
Question: If I don't use memcache, are there other
ways for sharing data (such as ban lists) among different proftpd
instances?
Answer: It might be possible using mod_sql
and some SQLLogInfo
directives, but that would only work for very
specific information. For sharing things like ban lists and SSL/TLS sessions
across a cluster of proftpd
servers, Memcache (or
Redis) support is recommended.
Question: Can I use mod_memcache
to cache frequently accessed files, similar to nginx+memcache
?
Answer: No. And in reality, caching of files like that
will probably not give you the same performance gain for FTP transfers as it
can for HTTP transfers.
Why not? Many HTTP transfers are for dynamically generated pages; the cost of
generating each page is expensive, and the generated content may not change
that frequently (relative to the rate of requests). FTP transfers, by contrast,
are for static files; FTP servers do not (usually) dynamically generate
the bytes of the files being downloaded. The cost of reading files from disk
is probably less than reading files from memcached
over
the network, even a LAN.
Now the above may not be true in all cases -- there may be FTP servers
serving files from network-mounted filesystems (e.g. NFS, CIFS
et al). And for these very specific cases, having a cache of frequently
access files on closer storage such as local disk (or memcached
)
could make a big difference; please contact the ProFTPD Project if you find
yourself in this situation, and we will see what can be done to help.
Question: Why do I see the following error when
proftpd
starts up?
mod_tls_memcache/0.1: notice: unable to register 'memcache' SSL session cache: Memcache support not enabledAnswer: This message means that your
proftpd
server has mod_tls_memcache
built and
loaded, but your proftpd
server was not built
with memcache support (i.e. the --enable-memcache
configure
option was not used when compiling proftpd
).
The above is not a fatal or worrisome error; it is merely pointing out that some of your modules want to use a feature that was not enabled.