0x55aa
Back to Blog

🏃 Self-Hosted Runners: When They're Worth the Trouble (and When They're Just a Second Job)

|
6 min read

Every team hits the same moment. CI is slow, the GitHub-hosted minutes bill is climbing, and someone says "why don't we just run our own runners?" It sounds simple — spin up a VM, install the agent, point your workflow at it, done. Cheaper, faster, more control. What's not to like?

A lot, actually. Self-hosted runners aren't a CI feature, they're a small fleet of servers you now own. And "own" means patching, scaling, securing, and getting paged when the fleet quietly runs out of disk at 2am on a Tuesday. I've watched teams adopt them for the right reasons and teams adopt them because a Hacker News thread made it sound easy. The difference between those two outcomes usually comes down to whether anyone actually did the math first.

What you're actually buying

GitHub-hosted runners give you a fresh VM every run: no state, no drift, no "works on my runner" surprises, and someone else handles the OS patching and the AMI updates. You pay per minute, and slow-but-predictable is baked in.

Self-hosted runners flip every one of those trade-offs:

  • Speed: no VM cold start, and you can pin dependencies/caches on disk instead of re-downloading them every run. For a monorepo with a huge node_modules or Docker layer cache, this alone can cut build times in half.
  • Hardware: need a GPU for a training job, or ARM for native builds? Hosted runners won't give you that without paying for specialty tiers. Your own box will.
  • Network locality: runners living inside your VPC can talk to a private RDS instance or internal artifact registry without a bastion or VPN hop.
  • Cost, at scale: GitHub-hosted minutes are billed per-minute after the free tier, and they add up fast for high-frequency monorepo CI. A handful of always-on EC2 spot instances can be dramatically cheaper — but only past a real usage threshold.

None of that is free. It's a straight trade of "pay GitHub in minutes" for "pay yourself in operational toil."

The part everyone underestimates: security

This is the one that bites people. GitHub's own docs are blunt about it: don't use self-hosted runners on public repositories. Anyone who opens a PR can get their workflow YAML executed on your runner, and if that runner has network access to your internal systems, you've just handed out a foothold. For private repos it's less catastrophic, but "less catastrophic" isn't the bar you want.

The concrete failure mode looks like this — a runner that persists state between jobs because nobody set up ephemeral instances:

# .github/workflows/build.yml
jobs:
  build:
    runs-on: [self-hosted, linux, x64]
    steps:
      - uses: actions/checkout@v4
      - run: npm ci && npm run build
      # if this runner isn't wiped after the job,
      # the next PR's build shares its filesystem,
      # its cached credentials, its leftover env vars

A malicious or just-buggy PR from a contributor can leave artifacts, planted binaries, or a reverse shell sitting on disk for the next job that lands on that same box. The fix isn't "trust your contributors more" — it's ephemeral runners: spin one up per job, tear it down after, no shared state, ever. AWS's own terraform-aws-github-runner and GitHub's autoscaling runner set for Kubernetes both default to this model for a reason.

Where self-hosted actually pays off

I lean toward self-hosted when at least two of these are true, not one:

  1. You're burning real money on hosted minutes. Not "it feels expensive" — pull the actual GitHub Actions usage report and compare it against a t3.xlarge running 24/7. Sometimes hosted is still cheaper once you account for idle capacity you'd need to provision.
  2. Your build is I/O or cache bound, not CPU bound. If 8 of your 10 minutes are npm install or pulling Docker layers, a warm local cache fixes that instantly. If it's 8 minutes of actual compilation, more runners of the same speed won't help — you need faster hardware, which hosted tiers can also sell you.
  3. You need hardware hosted runners don't offer — GPUs, specific chip architectures, or license-gated software that can't leave your network.
  4. You already have the platform team to run it. Autoscaling, patching, and security monitoring for a runner fleet is a real workload. If nobody owns it, it becomes the org's most stale, most vulnerable set of long-lived VMs within a year.

At Cubet, we ended up self-hosting runners for exactly one pipeline: a build that pulled from an internal Nexus registry sitting behind a VPN with no public endpoint. Hosted runners simply couldn't reach it without an expensive network bridge. Everything else stayed on GitHub-hosted, because the math didn't justify running our own fleet for pipelines that were already fast and cheap.

A middle ground worth knowing about

If the appeal is "faster and cheaper" rather than "network access I can't get any other way," check whether your CI provider has larger hosted runner tiers first — GitHub sells bigger hosted machines with the same ephemeral, zero-maintenance model, just at a higher per-minute rate. It's often the better deal once you price in the engineer-hours to run a fleet yourself:

jobs:
  build:
    runs-on: ubuntu-latest-8-cores   # bigger hosted runner, still zero ops

That one-line change solves the "our runners are slow" complaint for a lot of teams without anyone becoming a part-time fleet operator.

The actual decision

Self-hosted runners are worth it when you have a specific, hard constraint that hosted infrastructure genuinely can't meet — private network access, specialty hardware, or usage volume where the cost delta is undeniable on paper. They are not worth it as a default speed upgrade, and they are actively dangerous on public repos without ephemeral isolation.

Before you provision anything: pull your actual CI minutes bill, time where your build minutes actually go, and ask whether a bigger hosted runner or a better cache strategy solves it first. If the answer is still "no, we need our own boxes," go in knowing you've just adopted a small fleet of servers — plan the patching and the ephemeral teardown from day one, not after the first incident.

What's your self-hosted runner story — worth it, or a fleet you're quietly trying to decommission? I'd genuinely like to know which side of that line most teams land on.

Share:LinkedInXHacker News

Related reading

Thanks for reading!

Back to all posts