Adversec

Teams shipping LLM features inherit a class of bugs that unit tests don't catch. A prompt injection hidden in a document. A jailbreak that works on Tuesday's model and not Monday's. A system prompt that leaks with the right question.

Testing for these was mostly ad hoc, or locked inside the red teams of the big labs. Smaller teams couldn't run that kind of program on their own.

A small, focused API. You point it at your LLM endpoint, it runs a battery of adversarial probes, and you get back a structured report of what broke and why.

The probe library is the core of it: categorised by attack type, versioned, and built to change as new jailbreaks show up in the wild. Versioned probe sets mean you can run the same attack against two models, or the same model twice, and compare.

Every probe Adversec runs is the same experiment: change the input a little and see whether the output changes a lot. A model that refuses a request and then complies when three words move is not a model you can predict. Red-teaming is how you find that out before a user does.

By the time I shipped it, frontier models had picked up a lot of built-in defences, and you can ask a model to red-team itself. Sometimes that's enough. A purpose-built tool with a stable API, versioned probes and side-by-side comparisons across models is a different thing from one-off prompting. It's also how I learned to build infrastructure other developers depend on: auth, rate limits, structured errors, docs.

I designed a crewneck for a theoretical Adversec drop. Adversec doesn't have the user base to justify a run, so it stays a design. Shipping the product comes first.