If this is true, the hyperscalers are toast

This logic seems mad. If people only need SLMs then hyperscalers can also centrally host higher-efficiency models, and still gain efficiencies of scale and convenience over hosting locally.

The paper focuses on "intelligence per watt (IPW)", as a way to compare SLMs vs LLMs.

What might happen is that a chunk of the market, whatever its size will be, will end up going to SLMs run on iphones or Macbooks, and eat some of the revenues from LLMs, because not everyone needs the most powerful LLM all the time.

From what’s presented this seems to be the lower end of Q&A and reasoning tasks and not long horizon agentic work. I agree that the search engine replacement AI usage is something that can run anywhere (though it’s still better run in the cloud for speed, context length, sandboxing and convenience) but this isn’t the engine of AI growth.

Also, the average consumer is not going to be running a local model until they are built into the hardware they already buy and when they are, who is supplying the weights? They’ll likely be shipped as an ASIC (or MSIC) at that point anyways. Those will use a licensed model from the current leaders. The whole argument sounds like saying that cloud services shouldn’t be profitable because everyone has a computer at home or to meme “we have AI at home”.

If you are like Google or Apple and you are delivering AI to a mass market unwilling to pay a lot for it, you are absolutely going to drive AI processing to endpoint devices. You are also going to spend what it takes in R&D make a hybrid system that knows when to use local compute or cloud compute. That's going to be the bulk of the workload.

Haven't the SLMs been distilled using the LLMs?

If this is correct I see a future where the hyperscalers are funded by the businesses integrating siloed SLMs in their software.

Also the defence/intelligence industry will always want to keep an edge so don't be surprised if they stick around and we see favourable regulations for them similarly to how the government turns a blind eye to social media platforms because they increase the footprint of mass surveillance.

I wouldn't be surprised if the hyperscalers became software auditors and any piece of critical software was required to have a regulated security audit before it could enter production. Selling the poison and the cure is a great business model.

A remaining advantage of large language models is that as they get larger, they tend to hallucinate less, simply because the odds of the training set containing a desired answer improve with size. If a solid "I don't know" detector is developed for inference, then you can try a small language model first.

An implication is that successful research in "I don't know" detection could destroy hundreds of billions in shareholder value.

"If", sure.

How many developers here don't see a difference between the latest LLMs and SLMs they can run on their own computer? I tried running a smaller model locally, and it's not usable for me.

I know people like to "predict" things, so that if they happen they can then say "I am a visionary, I predicted it" and start their blog posts with "as I predicted long ago (because I am a visionary), ...".

> The research report estimates that the addressable market in the US for SLMs has grown to about $10tn or one-third of the entire US GDP of $30tn. There isn’t much left for LLMs to thrive in, and every year, their advantage over SLMs is shrinking.

I stopped counting the number of times "estimates" said that a market would absolutely explode, and it absolutely didn't. Those are in the business of being a broken clock.

If something better comes, it will be better. Sure. And we would like to have something better, because it would be better.

> If their results are true, then we will hardly need any data centres in the future, and the hyperscalers are wasting hundreds of billions of dollars in investments.

Haha, no. They get sufficiently powered and watered industrial warehouses close to where the successful people live. That’s a jackpot for developers, although it destroys the neighborhood as part of the deal.

Cool, they scored well on all the "make complex calculations and I'll vibe check your results based on my own domain experience" things I use the average LLM chatbot for.

So maybe in 10yr I'll be able to run a SLM on a 5yo laptop and not have Google or whoever hoover up everything.

As much as I want local and open-weights models to succeed, nothing beats a paid frontier model for now. Anybody who claims otherwise is simply not a daily user of such models. So this "investor" here should invest sime time in actually using the various LLM models and get a real taste of what it's like.