Who does Anubis actually stop?

Hmm, I don’t think I agree. The author is claiming that Anubis is meant to stop individuals using LLMs, but I don’t think that’s its purpose. I believe its purpose is to reduce mass scraping of data that puts excessive load on systems. It does that by increasing the cost of scraping, not by preventing it entirely.

Specifically, bad actors were ignoring robots.txt and rotating IPs to make blocking difficult. Anubis serves to make new connections more expensive, so that people will reuse a connection/cookie, and then falls back to normal means of preventing bad actors.

(That mass scraping is the result of AI training companies, yes, but it’s the mass scraping that’s the problem, not the LLMs.)

Anubis's primary goal is to prevent web scrapers from DDoSing a website. It's not meant to be an unbeatable challenge or only allow humans like Google's more privacy-invasive captchas.

You do the proof of work, you get the content. Not all web scrapers are willing to do the work, which reduces the strain put on web servers.

It's by no means a perfect system. It's goals in part prevent it from doing so. It tries to not be too annoying for humans, to not block real users, and not be privacy invasive.

> The exact adversary Anubis targets defeats it trivially.

Wrong, anubis stops mass crawling of web pages, by requiring a proof of work, which makes accessing these websites more expensive.

Anubis was not built to stop individual users with llms.

This mistakes the goal. It's not to block the scrapers, but to discourage excessive (and costly) scraping.

The Anubis cookies are bound to particular IP address. The scrapers are often using a large set of IP addresses, so they'll be paying a far higher cost than this suggests.

I was curious about the name:

> Anubis is a Web AI Firewall Utility that weighs the soul of your connection[1] using one or more challenges in order to protect upstream resources from scraper bots.

* https://anubis.techaro.lol/docs/

> The Weighing of the Heart would take place in Duat (the Underworld), in which the dead were judged by Anubis, using a feather, representing Ma'at, the goddess of truth and justice responsible for maintaining order in the universe. The heart was the seat of the life-spirit (ka). Hearts heavier than the feather of Ma'at were rejected and eaten by Ammit, the Devourer of Souls.

* https://en.wikipedia.org/wiki/Weighing_of_souls#Ancient_Egyp...

We don't know who is doing the really dumb global scraping attack, but that's who it's meant to stop.

Funny, I'm working on a simple tool that pulls the atom feed of the latest patches from lore with Python, and I just slip a User-Agent header into the requests.get() and it works great.

Exactly, it's the same as Cloudflare captchas where only certain blessed devices and browsers can seem to actually pass it. Ironically, adding Anubis accelerates the death of the open web.

perhaps the author, instead of the "I'm so clever" theoretical arguments from the client side, actually implemented Anubis on the server side and observe the results.

I have set it up on a few sites being relentlessly hammered by clearly idiotic bot traffic, and it drops bot the traffic levels from insane to manageable. I don't even care if the traffic is AI or bots, if the bot has gone to the same effort as the author has the bot may even just access the site in a responsible manner and that's fine.

Well, my call on this thing being useless waste of time of everyone involved was correct. Compute wasted on AI tokens alone is probably far more than some cpu for token solving, better invest time into caching it well (or blocking agentic traffic entirely if that's your jam)

Uh wait, a cookie that can be reused for a time to fetch the rest of the content? That's the exact opposite of what I'd intuitively want for this system...