↑
logo

Overmind: Cut PyTorch model load time 15s to 0.2s with a zero-copy shared cache

Posted by westurner |an hour ago |1 comments

westurner an hour ago

meshy-dev/overmind: Cut PyTorch model loading time from 15s to 0.2s with zero-copy shared memory caching.

> Fast Model loading: First load caches to shared memory; subsequent loads are ~5x faster

> Process-agnostic - Cache persists across process restarts via a background server

> Non-intrusive - Just add one line of code; no changes to model loading logic