Reassigning slab classes in Memcached on the fly is how you dynamically adjust memory allocation without restarting the service, preventing a major performance hit from a cold cache.
Let’s see it in action. Imagine you have a Memcached instance with two slabs, 1 and 2. Slab 1 is for smaller items (say, up to 1KB), and slab 2 is for larger items (up to 8KB).
# Initial state (hypothetical, viewable via stats slabs)
# Slab 1: 100MB allocated, 80MB used
# Slab 2: 200MB allocated, 180MB used
You notice your application is frequently evicting items from slab 1 because it’s getting too full, even though slab 2 has plenty of free space. You want to take 50MB from slab 2 and give it to slab 1.
The command to do this is rebalance:
echo "rebalance <from_class> <to_class> <bytes>" | nc localhost 11211
So, to move 50MB (which is 50 * 1024 * 1024 bytes) from slab class 2 to slab class 1, you’d run:
echo "rebalance 2 1 52428800" | nc localhost 11211
After a short while, stats slabs would show something like this:
# After rebalance (hypothetical)
# Slab 1: 150MB allocated, 130MB used
# Slab 2: 150MB allocated, 130MB used
The magic here is that Memcached doesn’t stop serving requests. It carefully moves pages of memory between the specified slab classes. This is crucial for high-throughput systems where downtime or even a brief pause for cache re-population is unacceptable.
The core problem Memcached slab reassignment solves is the static nature of memory allocation at startup. When Memcached starts, you tell it the total memory and how to divide it into slab classes based on item size. If your application’s access patterns change, or if you miscalculate the initial distribution, you’re stuck with that allocation until the next restart. This leads to either wasted memory (large slabs with little data) or performance degradation due to excessive evictions (small slabs overflowing). rebalance provides an on-the-fly solution.
Internally, Memcached manages memory in fixed-size "pages" (typically 1MB). Slab classes are collections of these pages. When you rebalance, Memcached doesn’t move individual items. Instead, it moves entire pages from the source slab class to the destination slab class. If a page in the source slab is still holding items, Memcached will attempt to move those items to other pages within the same source slab class first. If it can’t, it will allocate new pages for the destination slab class and move the items there. This page-level operation is what allows it to be relatively fast and non-disruptive. The rebalance command signals the server to initiate this process.
The rebalance command requires you to know the slab class IDs and the exact number of bytes to transfer. You can view current slab allocations and their usage with stats slabs. Slab class IDs are sequential integers starting from 0. The bytes argument is a direct byte count, not a MB or GB notation. If you get the byte count wrong, you might move too little or too much memory, potentially causing the opposite problem. For instance, moving too much memory to a slab class that doesn’t have many items will result in a lot of unused allocated memory within that slab.
A common pitfall is assuming rebalance is instantaneous. It’s an asynchronous operation. The command initiates the process, and Memcached works on it in the background. You won’t see the full effect immediately, especially with large transfers or if the source slab has many items to relocate. Monitoring stats slabs periodically will show the gradual shift in allocated memory.
The next hurdle you’ll likely encounter is understanding how to monitor the effectiveness of your slab rebalancing. You’ll want to watch eviction rates per slab class. If evictions from a slab class decrease significantly after a rebalance, you’ve likely made a good adjustment. Conversely, if a slab class starts showing high evictions after receiving memory, it might indicate that the item sizes being stored in it are larger than anticipated, or that you haven’t allocated enough memory overall.