Memcached can now survive restarts by storing its data on Persistent Memory (PMem) instead of just volatile DRAM.
Let’s see it in action. Imagine you have a Memcached instance running with a standard DRAM cache. You load it up with some data:
# Assume memcached is running on port 11211
echo "set mykey 0 60 5" | nc localhost 11211
echo "hello" | nc localhost 11211
If you then kill that Memcached process (pkill memcached) and restart it (memcached -p 11211), that mykey will be gone.
Now, let’s set up Memcached to use PMem. The key is the -P flag for the memcached binary, specifying the path to a PMem device file. You’ll need to have PMem installed and configured on your system. This usually involves ndctl commands to create a region and a namespace.
First, create a PMem namespace (e.g., /dev/pmem0):
# This assumes you have a PMem region already created by ndctl
sudo ndctl create-namespace -m fsdax -u -e <your_pmem_region_id>
Then, create a file within that namespace that Memcached will use:
sudo mkfs.ext4 /dev/pmem0 # Or xfs, depending on your setup
sudo mkdir /mnt/pmem_cache
sudo mount -o dax /dev/pmem0 /mnt/pmem_cache
sudo touch /mnt/pmem_cache/memcached.pmem
sudo chown nobody:nobody /mnt/pmem_cache/memcached.pmem # Or the user memcached runs as
Now, start Memcached pointing to this file:
memcached -p 11211 -P /mnt/pmem_cache/memcached.pmem -m 64 # -m is DRAM, -P uses PMem
Load data again:
echo "set mykey 0 60 5" | nc localhost 11211
echo "hello" | nc localhost 11211
Now, kill and restart Memcached:
pkill memcached
memcached -p 11211 -P /mnt/pmem_cache/memcached.pmem -m 64
Retrieve the data:
echo "get mykey" | nc localhost 11211
You should see:
VALUE mykey 0 5
hello
END
The data survived!
The core problem Memcached with PMem solves is the fragility of in-memory caches. Traditional caches live in DRAM, which is volatile. When the server restarts, the cache is wiped clean, forcing the application to go back to the slower primary data source (like a database) to repopulate the cache. This leads to performance degradation during the "cache warming" period. PMem offers byte-addressable memory that is non-volatile, meaning it retains its contents even when power is lost. By using a file on a PMem namespace as its backing store, Memcached can quickly re-attach to its cached data after a restart, drastically reducing or eliminating the cache warming downtime.
Internally, Memcached uses the libpmemobj library to manage the PMem file. This library handles the complexities of Direct Access (DAX) mode, allowing the CPU to directly access the PMem device as if it were regular RAM, but with persistence. When Memcached writes data, it’s written to the PMem file. Upon restart, it maps this file back into memory, and the data is immediately available. The -P flag tells Memcached where to find this persistent backing store. The -m flag still controls the size of the volatile DRAM cache, which can be used for frequently accessed "hot" data, while the PMem acts as a larger, persistent "warm" or "cold" store.
The size of the PMem file you create directly dictates how much data Memcached can persist. If you create a 1GB PMem file, Memcached can store up to 1GB of data persistently. The -m flag still dictates the size of the DRAM cache. A common strategy is to have a smaller DRAM cache (-m) for the most active items and a larger PMem backing store (-P) for less active but still frequently accessed items that you want to survive restarts. This hybrid approach balances performance and persistence.
One critical aspect often overlooked is the filesystem and mount options when using PMem. For Memcached to achieve the highest performance with PMem, the underlying filesystem must be mounted with the dax (Direct Access) option. Without dax, I/O operations would go through the standard page cache mechanism of the host OS, introducing latency and negating many of the benefits of PMem. The fsdax mode for namespaces, combined with a mount -o dax command, ensures that Memcached can bypass the OS page cache and access the PMem device directly. This direct mapping is what provides the near DRAM-like access speeds while retaining persistence.
The next challenge is ensuring your application can effectively leverage this persistent cache, especially with distributed Memcached deployments.