logo

Self-Hosted Inference for Agents

Posted by rafaepta |3 hours ago |3 comments

Supermancho 2 minutes ago

> SIE is an open-source inference engine that runs the models behind every agent task through one API: search and retrieval, document-to-markdown conversion, structured output, content safety, and the agent loop itself. It replaces the patchwork of a separate model server per task with one system that serves 100+ models, loading each on demand.

3rd time reading this and it's still confusing.

> an open-source inference engine that runs the models behind every agent task through one API

"the models"? - it means "various models that are called by an agent, per task" (then lists all the tasks that it covers)

> "It replaces the patchwork of a separate model server per task with one system that serves 100+ models, loading each on demand."

Which says nothing about how it works.

The About block is poorly written.

wyzer 3 hours ago[1 more]

Parent deprecated?