An AI release gate needs a third state

Most release gates inherit a binary shape: pass or fail. That works when the thing being measured is deterministic and the measurement itself is dependable. An AI evaluation pipeline has more ways to stop telling the truth.

The candidate may fail. The judge may time out. A provider can return only part of the expected evidence. A receipt can be internally inconsistent. A regression corpus can pass while saying nothing about representative production behavior.

Collapsing all of those cases into one boolean creates two dangerous shortcuts. A permissive gate turns missing evidence into approval. A strict gate turns infrastructure failure into a claim that the product regressed. Neither statement is supported by what actually happened.

I built Dailies around a third state:

• promote — complete, admissible evidence satisfies the declared policy;
• block — complete evidence establishes a policy violation attributable to the candidate; and
• inconclusive — required evidence is missing, failed, or cannot be verified.

That distinction sounds small. In practice, it changes what the release report needs to preserve.

A decision is only as broad as its evidence

A curated regression corpus, a sealed validation set, and a production sample answer different questions. Passing one cannot silently stand in for the others.

Dailies binds every decision to a declared evidence scope and the exact input bytes used in the run. A promotion therefore means that the candidate satisfied policy on that named scope. It is not a universal claim that the model is “good” or “safe.”

That constraint makes the output less dramatic and more useful. A release owner can see what was measured, what population the evidence is meant to represent, and where the claim stops.

Trust survives aggregation

Not every judge result has the same provenance. Dailies keeps three evidence classes visible:

• deterministic checks that can be reproduced locally;
• verified evidence with a pinned contract, identity, coverage, and digest; and
• self-reported responses from a generic HTTP judge.

The class is not a score. It describes what the release owner can verify. A provider cannot upgrade its own result simply by calling it trusted, and self-reported evidence cannot independently promote a release under the safe default policy.

Teams can admit that evidence during a migration, but the override and its reason remain in the report. The exception is part of the decision, not an invisible configuration detail.

Policy belongs after evaluation

An evaluator should measure a criterion. It should not decide whether a customer ships.

Dailies keeps that consequence in customer-owned policy. Criteria can be mandatory, blocking, advisory, or explicitly compensatory. Missing mandatory evidence stays incomplete. A blocking failure cannot be averaged away by a favorable advisory score. Compensation exists only when a policy defines the formula and compatible units.

The same assessment evidence can therefore support different valid decisions for different release owners without asking the evaluator to manufacture a deployment verdict.

The report explains the stop

A typical local run ends with an intentionally plain summary:

decision: promote | pass rate 80.0% (4/5), regressions 1, evaluated 5/5, errored 0
report: .../dailies-out/report.json
report: .../dailies-out/report.md

The machine-readable report retains the exact input identity, evidence scope, trust class, policy, coverage, attempts, errors, and decision precedence. The Markdown report makes the same chain inspectable by a person reviewing a pull request or release candidate.

If a required evidence channel fails integrity checks, the run stops as inconclusive. If the candidate itself cannot execute, it blocks. If complete admissible evidence establishes a blocking failure, unrelated missing evidence cannot rescue the release. Those precedence rules are explicit and tested rather than left to whichever error happened to arrive first.

Local first, off the serving path

Dailies is currently a local TypeScript CLI. It can run command or HTTP candidates and use deterministic checks, generic HTTP judges, or verified assessment evidence. It writes JSON and Markdown reports and maps the three decisions to stable CI exit codes.

It does not proxy production prompts. It contacts only the candidate and evidence endpoints configured for a release run. A hosted control plane or rollout integration may become useful later, but neither is required to make the decision model valuable now.

The source, contracts, fixtures, and decision tests are public on GitHub. The narrow promise is the important one: Dailies does not claim more certainty than the evidence can support.