You’ve updated an assistant’s prompt. The application still builds and its tests pass, but that doesn’t tell you whether it now mishandles examples it used to answer correctly. Before releasing it, run those examples and use the results to decide whether to ship.
Dailies is a command-line tool for that decision. It runs or collects evaluations, checks that the required evidence is present and usable, applies the policy you configured, and writes a report. It runs locally or in CI; your production traffic does not need to pass through it.
Three possible decisions
A failed evaluation request is different from an evaluated answer that failed. Dailies preserves that distinction:
Same release rule. Different evidence.
Your ruleAll three known behaviors must pass.
Every case meets the rule. This example set supports promotion.
The refund case fails a blocking rule. Investigate the candidate before releasing it.
The refund case has no result. Approval needs more evidence.
When several conditions apply, Dailies uses a defined decision precedence. The report explains which condition determined the result; a required evidence-integrity failure, for example, makes the decision inconclusive.
Run the starter example
You need Node.js 20 or newer. In your project, install Dailies and generate the starter files:
npm install --save-dev dailies
npx dailies init
npx dailies --config dailies.config.jsonThe starter runs three demonstration cases through a deterministic exact-match check. Open dailies-out/report.md to read the decision and dailies-out/report.json for structured evidence. This verifies the local workflow; it makes no claim about your application.
Evaluate your own change
Replace the starter cases and candidate with your own. A candidate can be a command or an HTTP endpoint. Each example needs a stable identity. The configuration names the exact dataset and its digest, a hash of the file’s contents, so a decision cannot silently refer to a different file.
Describe what the examples cover. A collection of known refund failures supports a claim about those failures; it does not represent every support conversation. If you change the JSONL file, update its configured SHA-256 digest too.
Then choose the policy. For a single criterion, that may mean a minimum pass rate and no regressions against known baseline labels. With a suite, criteria can have different roles: some block a release, some are required, and some are advisory. Keep those choices explicit instead of hiding them in an average score.
Choose your evidence source
The built-in exact-match evaluator is a useful starting point for deterministic checks. For more nuanced judgments, Dailies can use Rubrist receipts with verifiable evaluator identity and coverage, or a generic HTTP judge.
A generic judge’s result is marked self-reported. It cannot independently approve a release under the default policy. You can explicitly admit that evidence with a recorded reason, but doing so does not turn it into verified evidence. Dailies applies your rules; it does not establish that a judge understands the task.
Use the report in CI
Dailies exits with 0 for promote, 1 for block, and 2 for inconclusive. Your CI workflow can use those codes to stop a release while retaining both reports for review. Preserve the distinction between a product regression and a broken evaluation run when deciding what to investigate.
Reports can include inputs and model outputs, so store them with the same care as the examples they evaluate. Dailies is pre-1.0; use the configuration reference for the schema and integration you adopt.