Running Kimi K3 on a M1 Max

Posted by tito |2 hours ago |50 comments

jrhizor a few seconds ago

Super cool, and I appreciate the upfront speed disclaimer

antirez 2 hours ago[3 more]

SSD streaming on an M5 Max 128GB: https://x.com/antirez/status/2082136334160818528

Soon decent speed across two Mac Studios with 512GB of RAM.

ALLTaken an hour ago[1 more]

Exactly my machine 64GB M1 Max So happy about this! ♡

idk how people access (soldout) and even afford 512GB RAM MacStudio's. Isn't it $40k or so?

mips_avatar 2 hours ago

Would be interesting to see how fast it would be on 4x mac studio 512gb machines.

acmnrs 2 hours ago[1 more]

The title should probably be edited to specify "M1 Max" instead of "M1 Mac". You aren't running K3 on a base M1 anytime soon. Either way, still a very impressive project.

nlessard 2 hours ago[2 more]

Anyone who knows the state of NVMe hardware more than me know if this would obliterate the lifespan of your drive? Seems like the biggest limitation to me (some people are probably fine with letting their Macs churn over the weekend).

Azantys 2 hours ago[6 more]

0.01 tk/s is unusable for anything, you would wait a whole day for just 1000 token of output, what is the point of projects like this?

denysvitali an hour ago

60s/token - if only there was a way to drop that "s" this would be amazing

hmokiguess an hour ago[1 more]

Now set it up with an agent and a permanent `/goal` to say it cannot stop until it has solved for speed, then leave it on and livestream so we can all see when it becomes exponential. Could have the Eternal Jukebox playing in the background!

tjwebbnorfolk 2 hours ago[2 more]

> ~60–76 s/token

I don't know if I'd call this "running"

als0 2 hours ago[2 more]

Says it requires a 2TB disk? Must it be internal NVMe?

lostmsu 2 hours ago

under 0.02 tok/s

piterrro 2 hours ago[2 more]

Will it fit on ESP32??

onesandofgrain an hour ago

Cool gimmick

brcmthrowaway an hour ago

Why not train another smaller LLM to give the same answers as Kimi K3?

fenestella an hour ago

Comment deleted