How Do I Reduce Startup Time of Heavy Pods in K8s? 🤯

If you've worked with Kubernetes for a while and tried spinning up pods with heavy images, you've probably hit this: a pod that takes minutes to reach Running. And then the obvious question follows.

How can I get the pods to start up early and reduce the time it currently takes from minutes to some seconds?

If you've ever asked that, you're not alone. I worked on exactly this problem and want to share the journey — the options I found, and the trade-offs that ruled each one in or out — while building an image-caching solution for heavy pods on EKS that eventually brought pull time down from minutes to a few seconds.

The stack this covers: an EKS cluster, ECR as the registry, Karpenter as the cluster autoscaler, the Helm and kubectl CLIs, and k9s for watching it all happen.

Here are the four options I evaluated, in the order I found them — and why I did or didn't go with each one.

ECR Pull Through Cache

The first solution I came across, and a natural one given this stack already pulls images from ECR: Pull Through Cache lets you pull from a registry in the same AWS region instead of reaching out to the public internet directly. Images get cached automatically into ECR repositories whenever the upstream public images update.

Verdict: it's a good technique for cutting cross-region latency, but it doesn't actually solve the problem I was trying to solve — the image still has to be pulled onto the node the first time a pod lands there. Skip it for this specific problem.

eStargz

The Stargz Snapshotter community built a lazy-pulling technique where a container can start running before the image finishes pulling — only the chunks it actually needs get fetched on demand. eStargz is the lazily-pullable image format that project proposes.

Verdict: an innovative idea, but it requires converting every existing image into the eStargz format. That conversion requirement was a deal-breaker for me personally, so I had to give this one up.

Spegel, kube-fledged

I learned about Spegel and kube-fledged from a great Discord community — a shoutout to them 📣. Spegel is a stateless application deployed as a DaemonSet on every node in the cluster; instances forward image-pull requests to each other, so a node can pull a cached layer from a peer node instead of the registry.

Verdict: both Spegel and kube-fledged looked promising, but I decided not to pick either one — the repositories didn't have enough active maintainers behind them for me to be comfortable depending on them.

AWS Bottlerocket OS + Karpenter

Bottlerocket is an OS AWS built specifically for running EKS workloads. What makes it useful here is that it ships with two separate volumes: a Root volume for OS data, and a Data volume for images and ephemeral storage.

The idea: provision a node on Bottlerocket, pull the heavy images onto it, take a snapshot of the Data volume, then point Karpenter at a custom AMI backed by that snapshot — so every new node Karpenter spins up already has the heavy images baked in, no pull required.

Verdict: this one actually solves the problem — no lazy pulling, no format conversion, no dependency on a thinly maintained project. The caveat is that building and maintaining that custom snapshot pipeline is genuinely time-consuming ⏰, which is the trade-off to weigh against the minutes it saves on every pod startup.

Where this leaves the decision

• ECR Pull Through Cache: real latency win, doesn't touch the actual startup-time problem.
• eStargz: solves it in principle, blocked by the cost of converting every image to its format.
• Spegel / kube-fledged: solves it in principle, blocked by maintainer risk on the underlying projects.
• Bottlerocket OS + Karpenter: the one that actually delivers minutes → seconds, at the cost of the upfront engineering time to build and maintain the snapshot pipeline.

Of the four, Bottlerocket + Karpenter is the one that actually closes the gap — the rest either don't solve the real problem or carry a dependency risk I wasn't willing to take on. The honest trade-off to go in with is time: it's the only option here that isn't a quick flip of a setting, it's infrastructure you build and own.