Codex Security

Hey HN, Michael here, co-founder of Promptfoo and one of the people working on the Codex Security CLI at OpenAI.

Thanks for checking this out and for flagging the auth issues. We just open-sourced it, and there's still plenty for us to improve. Expect the product to evolve quickly.

If you try it, I'd really appreciate hearing what works well and what you think we should improve. Happy to answer questions here.

CLI docs: https://learn.chatgpt.com/docs/security/cli

EDIT: If you'd like to help make this better, we're hiring: https://openai.com/careers/full-stack-software-engineer-cybe...

Just ran it on a small repo. It ran for almost an hour and then got interrupted. It drained half my weekly usage on a Pro plan.

  npx codex-security scan .
  [00:00] Preparing scan
  [00:00] Authentication: stored Codex credentials.
  [00:03] Preparing scan
  [01:20] Running scan
  [01:20] Preflight: worker delegation supported (up to 8 worker slots).
  [52:47] Running scan
  codex-security: Could not save the Codex Security scan: Repository HEAD changed while the scan was running. Start a new scan.
  codex-security: Partial output was kept at ...

It's interesting how much of the value here is providing the english Skill definitions that tell the LLM what to do: https://github.com/openai/codex-security/tree/main/sdk/types...

Some of approaches there could be useful in other contexts. OAI has the compute to experiment with different prompts and I'd expect these to be somewhat optimized.

security tools from AI companies feel like fire departments run by arsonists. useful, sure, but you can't help noticing who benefits from all the fires

Update: As far as I understand, this was already available as a Codex plugin. The main news is that OpenAI has now open-sourced it, and development is still moving quickly.

Alibaba just open sourced their version of a CLI code review tool too.

https://github.com/alibaba/open-code-review

I seem to have gotten a bunch of you are trying to stuff we don't allow errors.. very annoying.

Can they explain what types of projects it works on and how does it check I own it? Like will it just not work on Linux kernel even on my own patches to it?

How can I trust this wont go rogue and hack Hugging Face?

Looks great but the CLI output is not particularly interesting while the scan is running. I wish it could show token usage, some kind of progress, etc.

How does it work? Does the tool upload code to ChatGPT for analysis? That may not be allowed for some corporate projects.

I wonder if tools like this will put companies like snyk out of business. We use snyk at work and I have not been satisfied.

be careful , your code will go to the cloud/ai using this

Just getting auth issues so far...

from plugin to main focus, wow

I don't think there's much to this other than it being a convenient CI wrapper around their existing models?

Edit: there's a little bit more meat here: https://github.com/openai/codex-security/tree/main/sdk/types...

would love it to see it h2h against https://github.com/usestrix/strix (45k stars)

I was actually discussing solutions for this with my coworkers—building white-hat security agents. It seems like openai/codex-security could simplify a lot of that, or at least provide a version of Codex that's purpose-built for security workflows. Really exciting news!

Allow only OpenAi key? Requires Cyber registration? Yes. Yes. Useless.

hey

[flagged]

I got my agent to analyze it and do a write up here:

https://wasnotwas.com/writing/inside-openai-codex-security/

The scanner is the least interesting part of this. The harness around it is the product: dedup across runs, false-positive tracking, budget controls, CI gating. That is the layer where we'll see most interesting innovations in my opinion.

I'm building AQ, a coding harness for teams and the pattern is identical. For a while, I thought the raw model is the answer and quickly changed my mind. Purpose built harnesses are way more powerful than it sounds.