Assay.
How it works

From a plan to a finished run, step by step.

A plan is the library. A run is one pass through it, against one environment, on one version of the plan. Everything Assay records hangs off the run.

Illustration: a single row of small glass sample vials in a steel rack on a laboratory bench, the vials at the far end lit violet.
The path of a run

Four steps from a document to a result.

  1. Import the plan

    The importer parses the markdown into sections, cases, and steps. A dry run reports the counts and the warnings; the real import stores version 1, or a new version if the text changed.

  2. Generate a run

    Pick the plan, its version, an environment, and a preset or filter. Assay copies the matching cases and steps into the run and records which version it came from.

  3. Work the cases

    Check off steps and record each case's result in the web app, or assign cases to people and agents who report them. A fail needs a description of what actually happened.

  4. Link the failures

    A failed case is filed as a Docket card with its steps and environment, or linked to a card that already exists. The run's progress shows resolved, passed, failed, and blocked counts.

Assay's Runs page: four runs against staging, three in progress and one passed, with a note that each run pins to the plan version it was generated from.Assay's Runs page: four runs against staging, three in progress and one passed, with a note that each run pins to the plan version it was generated from.
Runs: each one pinned to the plan version it came from. Enlarge to see the full screen.
Assay's run view: the case list with filters for all, outstanding, failed, P0, and mine, and the step checklist for the open case.Assay's run view: the case list with filters for all, outstanding, failed, P0, and mine, and the step checklist for the open case.
The run view: filter the cases, then work the steps. Enlarge to see the full screen.
Generating a run

Run the whole plan, or just the slice you need.

Presets are stored filters, not special code. You can also pass your own filter (sections, priorities, tags, defect tags, or case keys), and an explicit key wins over the preset.

Run presets
PresetWhat it generates
FullEvery case in the plan version.
P0 and P1Only the blocker and critical cases.
Known defectsOnly cases that carry a known-defect tag, the regression set to re-run after a fix.
Agent-executableOnly cases marked as safe for an agent to run.
Manual onlyCases with no automation reference.
Case statuses
Case statusMeaning
Not started, in progressWaiting, or someone (or an agent holding a lease) is on it.
PassEvery step held.
FailRecorded with what actually happened; the failing step is marked.
BlockedIt could not be run, with a note on why.
Not applicable, skippedSet aside on purpose.
Three rules Assay keeps

A run is evidence, so it can't move.

  1. Runs pin to a version

    Plan content is append-only. A run stays on the version it was generated from, so publishing a new version never changes a run that is under way or finished.

  2. Markdown in, markdown out

    Export produces a document a person would accept as the source. Editing in a text editor and importing again is a supported loop.

  3. Nothing fails silently

    A Docket call that fails, a lease that expires, or a queued side effect that won't deliver is recorded, logged as an error, and shown, not swallowed.

Assay's API also has per-run pass rates, never-executed cases, and flaky-case reports. The web app does not show them.