The Closing of the Frontier

This feels really premature. The announcement was a week ago. The “this model is too powerful for the general public” sounds like marketing to me.

Give it a few months and it will be just another model they are selling, but the NEWER model is just too powerful for the public.

> But it is another thing entirely to share access only with enterprise partners such as Crowdstrike, Cisco, and Microsoft, which are known to have massive security incidents regularly.

The stated purpose of Glasswing is to give infra and security orgs the chance to close holes and improve their security. In that context, it seems odd to call for not providing them access with the justification being that they have security breaches sometimes.

Mythos may or may not itself be opened to the public at some point, but I would charitably expect that Anthropic plans that a future model at least as good as Mythos Preview will be, and the limited release for Mythos is intended to make that eventuality safer by having most of the existing holes patched.

  > A 16-year-old with no credentials and no capital could just do things. The world of bits offered the freedom to build without being drowned in arbitrary constraints, in a way that didn’t require assembling vast capital or prestige or connections, where your creativity and work could speak for itself, and you had agency.

This is now truer than it ever was.

Anthropic should provide a specific service where they attack a businesses infrastructure using this frontier model and then issue a report of all vulnerabilities found. I could imagine it would be quite lucrative.

Much better than hiding it away where it can't help anyone.

The problem is Anthropic doesn't have the compute to deploy this model to scale to everyone yet. Dario didn't believe that they needed as much compute, OpenAI is going to have much more compute unlocked this year and especially next year.

The opportunity from the early days of the American frontier is not typical. Instead, it's the brief burst of unrestrained growth as a better-adapted organization (the US, software companies) expands into, and expands, a niche--cannibalizing the previous occupant (Native Americans, older stagnant companies.) At times growth is so rapid that individuals are able to advance the frontier, but if the field stagnates, individuals will be outcompeted by corporations.

So, opportunity for individuals comes from disruption. Creative destruction is good up to a point, but it results from advancing capabilities. Technological advances compound and accelerate exponentially. Eventually we reach the point where any malcontent can destroy the world by snapping their fingers. At some point we need to place restrictions on the capabilities accessible to individuals. We have reached that point with nuclear weapons, and I think it is sensible to believe that AI is reaching that point as well.

Most important point in the piece (though I’m not sure if the historical analogy to the grid holds, given local electricity production has been unavailable for the majority of the history of the grid)

> You can generate your own electricity with a solar panel (think local models), but most people would rather pay a utility bill. And the power company doesn’t decide, on the basis of pedigree, who is worthy of electricity. Intelligence should work similarly, where the capabilities you can access scale may scale with vetting and due process, but the presumption should be access. Add safety guardrails to restrict dangerous use; start by making them overly trigger-happy if you must, and calibrate over time. But the default should be to allow entry.

I admit I didn't read the entire post (I honestly think authors really need to come to terms with the fact that we now live in a world of information excess, and pithiness is more important than ever), but I wouldn't feel too bad yet given there was a recent front page HN post about how free, open models could actually catch all the issues Mythos did, it just required a little more orchestration. E.g. see https://aisle.com/blog/ai-cybersecurity-after-mythos-the-jag... for a detailed analysis.

It's long been conventional wisdom that you shouldn't write your own crypto libraries - leave that to experts. But excellent open source libraries are available, which do get reviewed by experts. And if you're willing to study, maybe you can learn enough about cryptography to become one of the experts?

I'm wondering what other security-sensitive software that might become true of in the era of Mythos-or-better AI's?

There will still be open source projects that anyone could learn enough to contribute to, but maybe starting from scratch and writing your own becomes less feasible if you aren't attracting enough attention to get attention from people with access to the best AI's?

For example, Linux patches are going to get expert reviews, but maybe your homegrown OS won't?

You never needed a godzilla or a megatron to get on with your life. But the sellers of those monsters would make every attempt, in connivance with the authorities, to make it a basic necessity to use their services. That's a survival strategy for the monsters. The owners can't keep the monsters in cages for too long, even if the owner is a state actor.

My bet that this is just a model with a slimmer safety alignment which yields results in 100 agent running in parallel configuration.

Maybe not. You stated "Every country will pursue it as far as it can, and given the multipolar world we are back in, and our recent record with treaties and commitments, I do not believe there will be global alignment on risk reduction" and that is true. While Anthropic may hold of releasing Mythos I don't think they will for long. As long as there is someone somewhere in the world that releases a competing model then Anthropic will be forced to release Mythos. This also assumes that its not a marketing tactic from Anthropic in the first place, build suspense before releasing it.

I would like to see more countries capable of producing frontier models. At the moment we have two in the world but many countries are building their own national models and AI infrastructure and may join the race.

Having a multipolar world may actually result in more freedom in gaining access to frontier models.

As a teenager, visual studio cost 650 dollars or so. More money than I had ever held by a wide margin.

I interpreted their "too dangerous to release" comment as a statement about the current situation. If the model is truly as capable as they say, and the security issues so numerous, it makes sense to hold out until the biggest targets have been patched.

It'd only take one company deciding to not worry about safety, to change the calculus back to "we have to release this to stay competitive".

I think we are going to look at the era between 2019-2025 as a very rare blip in the history of public AI access. Regardless of whether fears about Mythos end up being justified, the clear trend is

1. AI models are becoming better and better at causing massively disruptive effects, leaving up larger and larger liabilities, especially as laws and regulations are being passed/proposed which would put the responsibility of some mass disruption/hacking event on the company which serves the model that made it possible

2. The relative advantage of serving an AI model for inference in exchange for money is waning compared to the advantage of using that model internally for purposes which accrue money/power/leverage for that AI company. Why serve a model at 30 dollars/million tokens when you've discovered you can use that model to run a simulated Quant firm with a net profit of 300 dollars/million tokens? Why offer the model to companies so they can find zero-day exploits, when you can find them yourself and sell the discovery to companies which would may millions to avoid this exploit being taken advantage of?

3. Why serve models so another wrapper company like Cursor can make billions off your tokens, and then try to train their own models as fast as possible, trained on your outputs so they aren't dependent on you? The entire AI startup industry and like 90% of YC batches depend on being able to serve frontier models at a profit, mediated through some wrapper, why can't OpenAI/ANthropic, once their models are good enough to handle the ideation/organizational problem, become their own incubator for thousands of AI run startups, running on models way better than the public has access to?

As a consequence, there is less and less incentive over time to offer models as an API to the public.

I don’t understand this sentiment at all - the older models are still available and still highly capable.

It's not clear to me if the author talk about European invasion as the colonization pattern behind pretended American frontier, as it was lands that never any human had reached before.

For pete's sake stop with the thinkpeices already.. its only been an week and the model isnt even out yet!!!!!

OMG this generation - we can't separete the outrage from reality anymore.

Meanwhile 3000 people have died arbitrarily in Iran War -while we navel gaze.

The richer are just getting scammed faster this time around

> already happening with recursive self improvement

Are any AI labs claiming this?

I doubt that Mythos is just so wonderful and not able to be replicated that we are actually being 'closed off' from frontier models.

For example, the people who Anthropic "trusts" with this "dangerous" model are a handful of fortune 500 companies? Seriously? Those are the people we trust?

We are going to have access to this within 6 months, and if we don't, someone else will offer an equivalent. Anthropic hasn't walked to the edge of the abyss only to be like "let the CEO's handle this!"

It is simply not the edge of the abyss.

I'm old enough to remember claims that gpt-2 is too powerful to release.

I had a similar thought, though not as extreme, the second they started nerfing and filtering models

Their intensions were good, they always are, but the minute you decide to nerf something powerful for someone, it means someone out there has access to the full blown, unnerfed version

Which means there are powerful people out there using AI in ways or for activities in which you will never be allowed to anyway

So yeah, this is just more of the same

In other, underreported news, companies like AirBnB are using open source models. Anthropic and OpenAI have a six months to year advantage over Qwen and other models. We reached the point a while ago where Anthropic models where good enough, and so now, inevitably, we’ve reached the point where open models are good enough. The boasting of models so good they can’t be shared was propaganda to frame the conversation. But for anyone paying attention, what matters is that open models are now good enough.

While I don't think Mythos is so powerful that it justifies containment permanently - I wonder how it might work when there is such an AI that can justify containment.

What if this new model can start proving Millenial problems and provide insights in other fields that was not possible before?

My intuition says that a model that is as good will also be equally well aligned -- but it is still highly risky to give it to the general public because all you need is one jailbreak from bad actors.

At that point I think society would change so dramatically that "access to general public" would be a non issue. Rather, time would be spent on making abundance happen - you might think of the political struggles, economics and new ventures.

Its a bit sad that democratised access is not provided because of negative sum possibilities like cybersecurity.

if it is an actual leap, expect more 90% is good enough open weights to hit the next 6 months. Theres no reason competitors will hold underperforming models if they have to retool. might as well do more marketing.

> Even though the American dream is nearly dead

That dream was always a lie. But in the past, people could purchase more in parity. You only need to look at income versus housing cost in, say, Canada.

Realistically there should not exist any superrich, but this seems hard to change. That means there needs to be a different society be given as promise. Other countries manage that. In the USA they have the orange oligarch who said a while ago how there is no money for health care because he has to invade countries and wage war. So much for the "no more wars" promise.

> The Anthropic Mythos announcement is the first time in my life I’ve felt truly poor. Maybe because I grew up on the internet and it was the one permissionless place where you could have leverage and a shot at uncapped exploration and ambition. That is now changing with the gap between models that are publicly available vs those reserved for the already wealthy and pre-established.

The Internet was developed by the US state sector and handed off to the private sector in the 90’s. Then it worked as an open space until it didn’t any more. Predictably driven by corporate interests.

> In 1893, Frederick Jackson Turner argued that much that is distinctive about America was shaped by the existence of free land to the West where anyone could start over, and that this condition infused America with its characteristic liberty, egalitarianism, rejection of feudalistic hierarchy, self-sufficiency, and ambition.

A more asinine comparison could not have been picked.