0x55aa
Back to Blog

Kubernetes Deployment Strategies: Ship Code Without Waking Up at 3 AM šŸš€šŸ˜“

|
6 min read

Kubernetes Deployment Strategies: Ship Code Without Waking Up at 3 AM šŸš€šŸ˜“

True story: It was a Friday afternoon. I had a "tiny" frontend fix — a two-line CSS change. I pushed it, Kubernetes did a Recreate deployment, and for exactly 47 seconds our entire app was a blank white page. Those 47 seconds cost us a very uncomfortable Slack message from the CEO.

CEO: "Is the site down?"

Me: (typing with shaking hands) "It's… being deployed? šŸ˜…"

CEO: "On a Friday?"

Me: (deletes Slack app)

That was the day I became a true believer in proper Kubernetes deployment strategies. Today I'm going to save you from that same fate.

The Problem with "Just Deploy It" 🤦

By default, Kubernetes gives you a Recreate strategy — kill all old pods, then start new ones. It's fast, simple, and absolutely brutal. You get a guaranteed downtime window every single deploy. Great for local dev, terrible for production.

The good news: Kubernetes ships with better options baked in, and with a few YAML lines you can sleep through your Friday deployments.

Strategy 1: Rolling Updates — The Safe Default šŸŽ¢

Rolling updates replace pods gradually, one (or a few) at a time. Old pods stay alive until new ones are healthy. This is Kubernetes' default strategy and handles 80% of use cases.

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 4
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1        # Spin up 1 extra pod during rollout
      maxUnavailable: 0  # Never kill an old pod until a new one is Ready
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: my-app
        image: my-app:v2.1.0
        readinessProbe:
          httpGet:
            path: /healthz
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 5

The critical pieces here are maxSurge: 1 and maxUnavailable: 0. This tells Kubernetes: "Never leave me with fewer healthy pods than I started with." That readinessProbe is equally vital — without it, Kubernetes can't tell if your new pod is actually working before it kills an old one.

Real lesson learned: I once deployed without a readiness probe and Kubernetes happily sent traffic to a pod that was still running database migrations. Users got 500 errors for 2 minutes before I noticed. Add the probe. Always.

Strategy 2: Blue/Green — The Confidence Booster šŸ’™šŸ’š

Rolling updates are great, but what if your new version has a subtle bug that only shows up under real traffic? By the time you notice, half your users have already hit it.

Blue/Green deployments run your old version (blue) and new version (green) simultaneously. You flip traffic only when green proves healthy.

# service.yaml — the traffic switch
apiVersion: v1
kind: Service
metadata:
  name: my-app
spec:
  selector:
    app: my-app
    version: blue   # <-- Just change this to "green" to flip traffic
  ports:
  - port: 80
    targetPort: 8080
---
# green-deployment.yaml — deploy this while blue is still live
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app-green
spec:
  replicas: 4
  selector:
    matchLabels:
      app: my-app
      version: green
  template:
    metadata:
      labels:
        app: my-app
        version: green
    spec:
      containers:
      - name: my-app
        image: my-app:v2.2.0

The flip is a single kubectl apply or a one-liner:

kubectl patch service my-app -p '{"spec":{"selector":{"version":"green"}}}'

And rollback? Just patch it back to blue. Instant. No drama. No 3 AM pages.

Downside: You're running double the pods during the transition. For teams with tight cloud budgets, this stings. But for mission-critical deploys, the peace of mind is worth the temporary cost.

Strategy 3: Canary Releases — The Careful Scientist 🐤

Sometimes you're shipping something risky — a new payment flow, a refactored auth system, a feature the CEO built himself. You want real user traffic to test it, but you don't want all users to be your guinea pigs.

Canary releases send a small percentage of traffic to the new version while the rest still hits the stable version.

The cleanest way to do this in vanilla Kubernetes is with replica ratios:

# stable: 9 replicas → ~90% of traffic
# canary: 1 replica  → ~10% of traffic

# stable-deployment.yaml
spec:
  replicas: 9
  selector:
    matchLabels:
      app: my-app   # Both deployments share this label
  template:
    metadata:
      labels:
        app: my-app
        track: stable

---
# canary-deployment.yaml
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app   # Same label — same Service picks up both!
  template:
    metadata:
      labels:
        app: my-app
        track: canary

Your Service selector only needs app: my-app — Kubernetes naturally distributes traffic across all matching pods. 9 stable pods + 1 canary pod ā‰ˆ 10% canary traffic. Monitor your error rates, response times, and business metrics. If canary looks good after 30 minutes, scale it up and scale stable down.

Pro tip: Pair canary deployments with Prometheus + Grafana dashboards. Watch your p99 latency and error rate during the canary window. If either spikes, scale canary to zero before most users ever notice. This is why observability is not optional.

Choosing the Right Strategy šŸŽÆ

Strategy Downtime Rollback Speed Cost Best For
Recreate Yes Fast Low Dev/staging only
Rolling Zero Medium Low Most production deploys
Blue/Green Zero Instant Double during switch High-stakes releases
Canary Zero Fast Slightly higher Risky features, A/B tests

For most teams: Rolling for everyday deploys, Blue/Green for major version bumps, Canary for anything that touches money or authentication.

The Golden Rules šŸ…

  1. Always define readiness probes. Kubernetes is trusting you — trust your pods back.
  2. Set resource requests and limits. A pod with no limits will eat all CPU during a bug and take down its neighbors.
  3. Test your rollback procedure. A rollback strategy you've never rehearsed is a rollback strategy that will fail at the worst moment.
  4. Never deploy on Fridays — but if you must, use Blue/Green so you can roll back in 10 seconds and still make it to dinner.

Wrapping Up

Kubernetes gives you the tools to deploy like a professional. The difference between "we took down prod" and "we deployed 47 times today with zero incidents" is usually just 20 lines of YAML and a healthy respect for readiness probes.

Start with a proper Rolling Update config today. Add Blue/Green when you have a high-stakes release coming. Graduate to Canary when you want to sleep through your own launch.

Your 3 AM on-call rotation will thank you.


Have a Kubernetes war story or a deployment strategy tip I missed? Drop it in the comments — or better yet, open a PR on this post. Let's build better deploys together. šŸ› ļø

Thanks for reading!

Back to all posts