Ironside

When an AI application fails, the final answer rarely explains enough. You need to see what the agent received, which tools it called, and what came back to check whether the answer matches the evidence.

Ironside stores that record as a trace. It accepts traces from your application or supported session importers, provides a viewer for inspecting them, and lets you export the data. You can use it for debugging on its own or as a source of examples for an evaluator.

Keep enough context to learn

A trace covers one piece of work and can include the request, model responses, tool calls, timing, and usage information. The sequence often explains a failure that the final response alone cannot.

Ironside preserves raw ingest events in object storage and makes traces queryable through ClickHouse. Project-scoped credentials separate machine access from your browser session. The goal is to keep the original record useful even when you change the tools that analyze it.

Capture agent sessions

For application instrumentation, Ironside accepts OpenTelemetry traces, native JSON, and Langfuse-compatible ingest. Its Node.js SDK includes OpenAI Chat Completions and Anthropic wrappers, including streaming support.

For coding-agent work, Overclock’s Eval Stack includes importers for Claude Code and Codex session logs. A configured hook or scheduled import can send completed session records to Ironside without copying each conversation by hand.

These imports happen after records reach the session log. They are not a live feed of everything an agent does, and fields missing from the source log stay missing. Review the importer documentation before configuring capture, including its redaction and coverage limits.

Start with one project

You can also build a setup prompt for your coding agent, starting with one session or a question you want to evaluate.

You need Docker for the standard self-hosted setup. It starts the API, worker, web interface, and their storage services.

git clone https://github.com/luka-zivkovic/ironside.git
cd ironside
docker compose up -d --build

Once the services are healthy, create the one-time owner setup capability:

docker compose exec api node apps/api/dist/src/scripts/owner-setup.js

Open http://localhost:8080/setup, create the owner, and add a project. Its Connections page provides scoped credentials and setup snippets. Send one trace, open it in the viewer, and check that the input, output, and tool sequence contain what you expect.

Use the self-hosting guide for credentials, backup, retention, and deployment configuration.

Use the record elsewhere

A trace store becomes more useful when you can bring its records to another tool. Ironside supports object-storage exports, OpenTelemetry forwarding, webhooks, and API access. You can investigate a run directly, analyze exported data, or send selected examples to Rubrist.

That evaluation can inform a change to your prompt, model, or workflow. It does not need to live inside the trace store, and you do not need to adopt Rubrist to use Ironside.

Know the current limits

Ironside is pre-release. Its current schema policy still permits baseline changes that require a reset. Check the release roadmap before relying on it for persistent production data, and use disposable data for an initial trial.

Ironside preserves and moves evidence; it does not establish that a skill is correct or that a release should ship.

Once you have useful examples, Rubrist helps you turn a specific failure into a check.