AWS ElastiCache: Stop Querying Your Database Like It's 2010 💾⚡
AWS ElastiCache: Stop Querying Your Database Like It's 2010 💾⚡
Hot take: Your database is not a cache. Stop treating it like one.
I learned this the hard way during a flash sale on an e-commerce platform I'd architected. 8:00 AM, sale goes live, traffic spikes 10x. RDS CPU hits 100%. Response times hit 30 seconds. The product catalog — the exact same 200 products that hadn't changed in a week — being queried from the database 14,000 times per minute.
Every single query. Hitting the database. Fetching identical data. Over and over. Like some kind of digital Groundhog Day. 😩
AWS ElastiCache with Redis fixed it. Let me show you how.
What ElastiCache Actually Is 🤔
Amazon ElastiCache is a managed in-memory cache service. You pick Redis or Memcached (pick Redis, always Redis), and AWS handles provisioning, patching, failover, and backups.
Why Redis?
Memcached: Just key-value storage. Simple. Fine.
Redis: Key-value + sorted sets + pub/sub + streams + TTL per key + atomic ops
+ Lua scripting + geospatial indexes + ...
Redis is to Memcached what a Swiss Army knife is to a butter knife. Just use Redis. 🗡️
Where it fits in your stack:
User Request
↓
API Gateway → Lambda
↓
ElastiCache (check cache first) ← cache HIT: return instantly ✅
↓ cache MISS
RDS / DynamoDB (expensive, slow)
↓
Store result in cache, then return
That cache hit? Sub-millisecond. Your database query? 20-80ms on a good day, 3 seconds during a traffic spike. Do the math.
Setting Up ElastiCache (The Non-Painful Way) 🚀
In production, I've deployed ElastiCache clusters via Terraform, but let me show the AWS CLI approach:
# Create a Redis cluster (single node for dev, good for getting started)
aws elasticache create-cache-cluster \
--cache-cluster-id my-app-cache \
--cache-node-type cache.t3.micro \
--engine redis \
--num-cache-nodes 1 \
--cache-subnet-group-name my-subnet-group
# Get your endpoint
aws elasticache describe-cache-clusters \
--cache-cluster-id my-app-cache \
--show-cache-node-info \
--query 'CacheClusters[0].CacheNodes[0].Endpoint'
Critical gotcha: ElastiCache lives inside a VPC. Your Lambda must also be in that VPC to connect. I've made the mistake of deploying a Lambda outside the VPC and wondering why the connection timed out for 30 seconds before the cold start failed. 🤦
# Put your Lambda in the same VPC
aws lambda update-function-configuration \
--function-name my-function \
--vpc-config SubnetIds=subnet-abc123,SecurityGroupIds=sg-xyz789
And yes, VPC-attached Lambdas have longer cold starts. That's the tradeoff. ElastiCache isn't worth much if you can't connect to it.
The Connection Pooling Problem With Lambda 🎭
Here's where Lambda + ElastiCache gets spicy.
Every Lambda invocation that creates a new Redis connection is expensive. Redis connections aren't free — each one takes ~2MB of RAM on the Redis server, and creating one takes ~10ms.
The naive approach:
// BAD - Creates a new connection on EVERY invocation
exports.handler = async (event) => {
const redis = new Redis(process.env.REDIS_URL); // 10ms just to connect
const value = await redis.get('my-key');
await redis.quit(); // Closing connection is wasteful!
return value;
};
At 1,000 concurrent Lambdas, you've got 1,000 Redis connections. Redis starts sweating. Then falls over. 💀
The correct approach — connection outside the handler:
const Redis = require('ioredis');
// Connection created ONCE per Lambda container, reused across invocations
const redis = new Redis({
host: process.env.REDIS_HOST,
port: 6379,
maxRetriesPerRequest: 3,
enableReadyCheck: false, // Faster startup
lazyConnect: true // Only connect when first command runs
});
exports.handler = async (event) => {
// Redis connection already warm — reused from previous invocation!
const cached = await redis.get(`product:${event.productId}`);
if (cached) {
return JSON.parse(cached);
}
const product = await db.findProduct(event.productId);
await redis.setex(`product:${event.productId}`, 300, JSON.stringify(product));
return product;
};
When architecting on AWS, I learned: Lambda containers stay warm for 15-30 minutes. That Redis connection object you create at module level gets reused for free. This cut our connection overhead by ~85% under load. 🎯
Cluster Mode: When One Node Isn't Enough 🔧
For production e-commerce, I run Replication Group — one primary node plus read replicas. Not cluster mode with sharding (that's overkill until you're at serious scale).
# Production-grade: Primary + 2 read replicas, multi-AZ
aws elasticache create-replication-group \
--replication-group-id prod-cache \
--description "Production Redis cache" \
--num-cache-clusters 3 \
--cache-node-type cache.r6g.large \
--automatic-failover-enabled \
--multi-az-enabled \
--engine redis
Why replicas matter:
Writes → Primary node (serialized, consistent)
Reads → Read replicas (spread the load, near-free)
Failover→ Automatic (replica promoted if primary dies, ~60 seconds)
A serverless pattern that saved us: Separate your read endpoint from your write endpoint in the Lambda environment variables. During a traffic spike, read-heavy endpoints (product catalog, search results) get routed to replicas and leave the primary free for cache writes and session updates.
const writeRedis = new Redis({ host: process.env.REDIS_PRIMARY });
const readRedis = new Redis({ host: process.env.REDIS_READER });
// Product catalog reads → replica
const product = await readRedis.get(`product:${id}`);
// Session writes → primary
await writeRedis.setex(`session:${userId}`, 3600, sessionData);
Eviction Policies: What Gets Deleted When RAM Fills Up 🗑️
This one catches everyone off guard. When your cache is full, Redis has to delete something to make room. Which something? That depends on your eviction policy.
Common options:
noeviction → Returns error when full. Your app crashes. Bad.
allkeys-lru → Deletes least recently used key. Usually what you want.
volatile-lru → Only deletes keys with TTL set (my go-to for mixed caches)
allkeys-random → Deletes random key. Chaos. Don't use this.
In production, I've deployed volatile-lru with explicit TTLs on everything important. This way, product catalog (5 min TTL) gets evicted before session data (1 hour TTL) when RAM gets tight.
aws elasticache modify-cache-cluster \
--cache-cluster-id my-app-cache \
--cache-parameter-group-name redis-maxmemory-volatile-lru
Real-world gotcha: I once had a cache with noeviction and an undersized node. During a marketing email blast, product page traffic spiked, cache filled up, Redis started returning errors, Lambda crashed, RDS got hammered directly, and then RDS also fell over. Four failure modes in 90 seconds. I still have the Slack thread framed on my wall. 🖼️
Caching Strategies That Actually Work 💡
Not everything should be cached the same way:
Strategy 1: Cache-aside (the workhorse)
async function getProduct(id) {
let product = await redis.get(`product:${id}`);
if (!product) {
product = await db.findProduct(id);
await redis.setex(`product:${id}`, 300, JSON.stringify(product));
}
return JSON.parse(product);
}
Strategy 2: Write-through (for critical data)
async function updateProduct(id, data) {
await db.updateProduct(id, data);
// Always update cache on write — cache is never stale
await redis.setex(`product:${id}`, 300, JSON.stringify(data));
}
Strategy 3: TTL-based invalidation (for stuff that changes on a schedule)
// Product catalog refreshes nightly anyway
await redis.setex('catalog:featured', 3600, JSON.stringify(featured)); // 1 hour TTL
When architecting on AWS, I learned: Use cache-aside for most things, write-through for anything where stale data causes real problems (like inventory counts, pricing), and short TTLs on anything security-sensitive.
The Real Cost Breakdown 💸
Let's talk money, because ElastiCache has an interesting cost story:
cache.t3.micro → $0.017/hour → ~$12/month (dev, tiny traffic)
cache.t3.small → $0.034/hour → ~$25/month (small production)
cache.r6g.large → $0.166/hour → ~$120/month (real production)
cache.r6g.xlarge → $0.332/hour → ~$240/month (high traffic)
Sounds expensive? Here's the real math:
Before ElastiCache (our flash sale disaster):
RDS db.r5.large: $240/month
Extra read replicas: $480/month (we panicked and added 2 more)
Data transfer: $60/month
Total: $780/month
After ElastiCache:
cache.r6g.large: $120/month (handles 95% of reads)
RDS db.r5.large: $240/month (handles writes + cache misses)
Total: $360/month
Savings: $420/month. ElastiCache paid for itself 3.5x over. 🎉
Cost gotchas:
- You pay for reserved nodes 24/7 even if your Lambda scales to zero at night
- Multi-AZ doubles your node cost (worth it for production)
- Data transfer between Lambda and ElastiCache in the same AZ is free — cross-AZ charges apply
Common Mistakes I Made (So You Don't Have To) 🪤
Mistake #1: Caching everything forever
// BAD - No TTL means items never expire
await redis.set('user:profile:123', JSON.stringify(user));
// GOOD - Always set a TTL
await redis.setex('user:profile:123', 900, JSON.stringify(user)); // 15 minutes
Mistake #2: Not handling cache failures gracefully
// BAD - If Redis is down, your whole app is down
const product = JSON.parse(await redis.get(`product:${id}`));
// GOOD - Cache is an optimization, not a requirement
async function getProduct(id) {
try {
const cached = await redis.get(`product:${id}`);
if (cached) return JSON.parse(cached);
} catch (err) {
console.error('Cache miss (Redis error):', err.message);
// Fall through to database
}
return db.findProduct(id); // Always works without cache
}
Mistake #3: Storing sensitive data in cache without encryption
ElastiCache supports in-transit and at-rest encryption. Use both. I don't care if "it's internal" — session tokens and user data do not belong in an unencrypted cache. Period. 🔒
aws elasticache create-replication-group \
--replication-group-id prod-cache \
--at-rest-encryption-enabled \
--transit-encryption-enabled \
...
Monitoring: Know Before Your Users Know 📊
The three metrics I watch religiously:
# Cache hit ratio - should be >80% or caching is pointless
CacheHits / (CacheHits + CacheMisses) * 100
# Evictions - non-zero means your cache is undersized
CloudWatch metric: Evictions → alarm if >0 for 5 minutes
# Connection count - Lambda concurrency × connections per container
CloudWatch metric: CurrConnections → alarm if approaching max_connections
A serverless pattern that saved us: We set a CloudWatch alarm on cache hit rate dropping below 75%. Three times, that alarm fired before users complained. Twice it was a Lambda deployment that broke cache key format. Once it was our marketing team manually clearing product prices in the DB without clearing cache. All caught within minutes. 📱
TL;DR: ElastiCache Survival Guide 🎯
- Put your Redis connection outside the Lambda handler — one connection per container, not per invocation
- Use volatile-lru eviction — or your cache fills up and explodes
- Always set TTLs — on everything. No exceptions.
- Separate read/write endpoints — route reads to replicas, protect your primary
- Fail gracefully — cache is an optimization, not a hard dependency
- Enable encryption — in-transit AND at-rest. Non-negotiable.
- Monitor hit rate — if it drops below 80%, something broke
ElastiCache is genuinely one of the best ROI services on AWS. A $25/month cache node can absorb traffic spikes that would otherwise require $200/month of extra database capacity.
Set it up before your next flash sale. Your RDS will thank you. Your boss will thank you. Your bank account will thank you. ☁️
Still hitting your database 14,000 times a minute? Connect with me on LinkedIn — I've got the battle scars and CloudWatch graphs to prove ElastiCache is worth every penny.
Want to see the full caching architecture? Check out my GitHub for real-world Redis patterns from production e-commerce systems!
Now go put a cache in front of that database before your next big sale. ⚡
P.S. The flash sale RDS incident? We added ElastiCache, rewrote the product catalog fetching, and had a second flash sale two weeks later. Same traffic spike. RDS CPU: 12%. Response times: under 200ms. Best Monday of my career. 🚀