↑
logo

Show HN: Warmctl – Fast cold starts for vLLM inference

Posted by kbesplv |an hour ago |1 comments

kbesplv an hour ago

Hi HN!

I’m building this GPU checkpoint & restore tool to optimize vLLM cold start time for GPU neoclouds

It’s dead simple at UX:

- Run warmctl build once in your CI/CD

- warmctl push the warm image to any OCI registry

- warmctl up on every new replica on GPU hosts

That’s it, so you can serve even Deepseek 4.1 or GLM 5.3 < 60 seconds

an hour ago

Comment deleted