Caching in a monolith isn’t just about speed; it’s about fundamentally changing the shape of your application’s data access.
Imagine a monolith that needs to fetch user profiles. Without caching, every request for a user profile hits the database.
# Example without caching
def get_user_profile(user_id):
db_connection = connect_to_database()
profile_data = db_connection.execute("SELECT * FROM users WHERE id = ?", (user_id,))
return profile_data
This works, but if user_id=123 is requested 100 times a second, that’s 100 database hits per second for the same data.
Now, let’s introduce a simple in-memory cache.
# Example with in-memory caching
import time
user_profile_cache = {} # A simple dictionary as our cache
def get_user_profile_cached(user_id):
if user_id in user_profile_cache:
print(f"Cache HIT for user_id: {user_id}")
return user_profile_cache[user_id]
print(f"Cache MISS for user_id: {user_id}")
db_connection = connect_to_database()
profile_data = db_connection.execute("SELECT * FROM users WHERE id = ?", (user_id,))
# Store in cache for future requests
user_profile_cache[user_id] = profile_data
return profile_data
When get_user_profile_cached(123) is called the first time, it’s a "miss." The data is fetched from the database and then stored in user_profile_cache. The next time get_user_profile_cached(123) is called, it’s a "hit," and the data is returned directly from the dictionary, bypassing the database entirely. The critical difference is that the application code itself is now responsible for managing this data lifecycle.
The primary problem caching solves is reducing latency and load on downstream services, most commonly databases. By serving frequently accessed, relatively static data from a faster, closer store, you significantly decrease the number of expensive I/O operations. This isn’t just about making one request faster; it’s about making all requests that can be served from the cache faster, and freeing up the downstream service to handle the requests it absolutely must.
Internally, caching in a monolith operates on a few core principles: key-value storage, eviction policies, and cache invalidation.
- Key-Value Storage: Data is stored and retrieved using a unique key. For user profiles, the
user_idis a natural key. For a list of products, a composite key likecategory:electronics:sort:price_ascmight be used. - Eviction Policies: Caches have finite memory. When the cache is full, older or less frequently used items must be removed to make space for new ones. Common policies include:
- LRU (Least Recently Used): Evicts the item that hasn’t been accessed for the longest time.
- LFU (Least Frequently Used): Evicts the item that has been accessed the fewest times.
- TTL (Time To Live): Items expire after a set duration.
- Cache Invalidation: This is the hardest part. When the source data changes (e.g., a user updates their profile), the cached version becomes stale and must be removed or updated.
Consider a scenario where you’re fetching a list of product categories. The categories rarely change.
# Example with TTL-based caching
import time
import redis # Assuming a Redis cache
def get_product_categories(redis_client):
cache_key = "product_categories"
cached_categories = redis_client.get(cache_key)
if cached_categories:
print("Cache HIT for product categories")
return json.loads(cached_categories)
print("Cache MISS for product categories")
# Fetch from database
db_connection = connect_to_database()
categories = db_connection.execute("SELECT name FROM categories")
# Store in Redis with a 1-hour TTL
redis_client.setex(cache_key, 3600, json.dumps(categories)) # 3600 seconds = 1 hour
return categories
Here, redis_client.setex(cache_key, 3600, ...) tells Redis to store the data for cache_key and automatically delete it after 3600 seconds. This is a form of automatic invalidation.
The levers you control are primarily:
- What data to cache: Frequently accessed, slow-to-retrieve, and relatively static data.
- Cache key design: How to uniquely identify cached data.
- Cache duration/TTL: How long data remains valid.
- Cache invalidation strategy: How to handle data changes.
- Cache implementation: In-memory (like Python dictionaries,
functools.lru_cache), or external services (Redis, Memcached).
The most common mistake developers make with caching is assuming that simply adding a cache layer means the data is always fresh. This leads to serving stale data, which can be far worse than slow data. For instance, if a user’s permission changes, but their permission data is cached for 5 minutes, they will continue to have the old permissions for that duration. This makes explicit invalidation strategies, or very short TTLs, critical for data that changes frequently or where staleness has significant consequences.
The next layer of complexity you’ll encounter is distributed caching and cache coherency issues across multiple application instances.