logo

Show HN: Tok/s on a 35B MoE model using a $100 AMD crypto APU and Vulkan

Posted by akandr |3 hours ago |1 comments

akandr 3 hours ago

Hi HN. I got my hands on an AMD BC-250 (the obscure GFX1013 chip, a repurposed PS5 APU used for crypto mining). Since ROCm officially ignores this hardware, I had to bypass it entirely using Vulkan and tweak the Linux kernel's TTM pages_limit to unlock the full 16GB of Unified Memory. Result: It runs a 35B MoE model at 38 tok/s and FLUX.2 for image generation. It's essentially a poor man's Mac Studio for Edge AI. Visit the page to see some benchmarks.