Test plans for the Renkara fleet

Manual test plans that actually get run.

A markdown test plan is a good document and a bad instrument. Assay keeps the document and adds the instrument: import the plan, run it step by step, and share the queue with agents.

Illustration: a precision brass balance on an assayer's workbench in a dim laboratory, small metal ingots beside it and a violet glow in one pan.
The film · 0:56

A plan you can keep, run, and prove.

Assay keeps your markdown test plan, pins every run to the plan version it came from, makes a failure say what happened, and hands the same run to people and agents.

Music and on-screen text, no narration; the captions describe the music and sound. Assay's real screens with fictional StrataForge Robotics data; the third plan version, the app's refusal message, the lease, the repeat comment, and the finished run are drawn and marked as illustrations.
Read the film’s text version

Title: Assay. Manual test plans that actually get run. 2026. A bright, energized electronic score runs under the whole film, with a soft effect on each count, ring, and card.

A test plan is a document: “A test plan is a document.” The first lines of a real plan scroll by as markdown, with headings and a table of cases. A checkbox tries to appear in the margin three times and can't. “Nobody can check it off.”

Keep the markdown: As the beat drops: “Keep the markdown.” Section A's markdown sits beside the real Plan page. One colored mark lands for each of the plan's 15 cases, by priority, while the counts climb to 5 sections, 15 cases, and 56 steps.

A parser, not a model: “A parser. Not a model.” Then “Same document. Same plan.”: two parses of the same markdown give the same content hash, b2366793, twice.

Generate a run: “Generate a run.” The run view's header for Fleet Console API 9.18.7 on staging, with 12 of 15 cases resolved. “It pins to a plan version.” Version 1 holds 14 cases, version 2 holds 15, and the run is linked to version 2. “A new version. The run stays.” A third version arrives in a drawn illustration, and the link stays on version 2.

A failure needs words: “D-02 fails at step 3.” The run view shows step 3 marked as failed. “Say what happened.” A drawn illustration shows the app's own refusal when Fail is pressed with an empty box, “A failing result needs a description of what actually happened — the defect card is generated from it.” Then the real note types in, a request that “returned 200 OK instead of 403 Forbidden.” “The defect is built from it.”

People and agents: As the beat drops again: “One run. People and agents.” The case list rings C-01, one open case for Emi Maeda, and D-01 and D-05, two open cases for the agent qa-agent-1. “An agent takes a lease.” The real Agents page shows qa-agent-1 holding a case with 29 minutes left and two heartbeats, “30 minutes, by default.” counts up, and the MCP calls type out beside it.

The lease runs out: “Then its session dies.” A drawn lease: case D-05 held by qa-agent-1 with a 30:00 ring, two heartbeats, and two of its four steps ticked. The agent grays out and the ring empties. “The case comes back.” The card slides into the queue with its ticks: “Back in the queue. 2 of 4 steps still ticked.”

Defects: “Linked to its card. Or filed.” The Defects page's D-02 row, linked to its Docket card, is ringed. “Hit it again. It comments.” In a drawn illustration, a card Assay filed gets Assay's own comment: “Recurred — occurrence #2.”, with a run, an environment, and “Reported automatically by Assay.”

Close: “Manual test plans that actually get run.” The run is 15 marks, one per case, with a key: 10 passed, 1 failed, 1 blocked, 3 open, and 12 of 15 resolved. Under the second line, in a tagged illustration, the three open marks turn violet and the count reads 15 of 15.

End card: The Assay mark, its name, its tagline, and the line “Renkara's own tool. Not for sale.”

Assay's run view with a failed case open: the failed step is marked, and the what-actually-happened box holds the description a defect is built from.Assay's run view with a failed case open: the failed step is marked, and the what-actually-happened box holds the description a defect is built from.
A failed case: which step broke, and what actually happened. Enlarge to see the full screen.
Working a run

A failure says which step broke.

Checking off steps is the difference between "A-07 failed" and "step 4 of A-07 failed, and here is what happened." A case can't be recorded as failed without that description, so the defect that follows is never an empty card.

Plans and versions

Your markdown goes in, and comes back out.

Assay reads the fleet's manual-test-plan format with a deterministic parser and keeps every version. Runs stay on the version they were generated from, so what you tested stays true after the plan changes, and export writes the plan back out as markdown.

Assay's plan detail page: a plan's sections and cases with priorities and a known-defect tag, and a note that the plan has two versions.Assay's plan detail page: a plan's sections and cases with priorities and a known-defect tag, and a note that the plan has two versions.
A plan with two versions: runs pin to the one they were generated from. Enlarge to see the full screen.
Assay's Agents page: two registered agents, one of them holding a live lease on a case with its time left and heartbeat count.Assay's Agents page: two registered agents, one of them holding a live lease on a case with its time left and heartbeat count.
Agents: the registry, and the leases held right now. Enlarge to see the full screen.
Agents and defects

Agents take cases off the queue. Dead ones give them back.

An agent claims one assigned case at a time on a lease and reports every step. If it stops, the lease expires and the case goes back to the queue with its reported steps intact. Failures then link to Docket, and anything that can't reach it is shown, not dropped.

Features

A test plan is a document. Assay makes it an instrument.

These are the jobs Assay does today. Plans come in as markdown through the importer or the MCP tools; people work runs in the web app; agents claim work through the API and MCP.

Import the markdown. Keep the markdown.

A real parser, not a model call

Sections, case tables, priorities, defect tags, the environment notes, and appendices are read by a deterministic parser, so the same document always gives the same plan.

Dry run by default
Import reports what it found (sections, cases, defect tags, and every warning) and writes nothing until you say so. Importing changed text adds a new version; importing the same text again changes nothing.
Exports back to markdown
Export writes a document a person would accept as the source, and the exporter's tests check that it imports back to the same plan. The plan is never trapped in the database.
Honest about what it guessed
Steps split on explicit numbering or semicolons. A split on sentence boundaries is a weaker guess, so the import report marks it for review, and a single step that reads like several is flagged and left whole.
Assay's Plans page listing six release test plans, each with its product and owner.Assay's Plans page listing six release test plans, each with its product and owner.
Plans imported from markdown, with product and owner. Enlarge to see the full screen.

Evidence that can't shift underneath you

Runs pin to a plan version

Plan versions are append-only and a run stays on the version it was generated from. Publish a new version mid-run and the run does not move.

Generate the slice you need
The whole plan, P0 and P1 only, the known-defect cases, the agent-executable cases, or the manual-only cases, or your own filter by section, priority, tag, defect tag, or case key.
Assay's Runs page listing four runs against staging, with the note that each run pins to the plan version it was generated from.Assay's Runs page listing four runs against staging, with the note that each run pins to the plan version it was generated from.
Runs: each pinned to the plan version it came from. Enlarge to see the full screen.

Step by step, not just case by case

Check off individual steps, then record the case as pass, fail, blocked, or not applicable. A failure can't be saved without a description of what actually happened.

Keyboard-first
Move between cases and steps, toggle a step, and record a result without the mouse. Shortcuts are ignored while you type in a text box, and the command palette (Command-K) jumps to a case or a page and carries the verbs for the case you're on.
Assay's run view: a filterable case list on the left and the step checklist for the open case on the right, with pass, fail, blocked, and not applicable buttons.Assay's run view: a filterable case list on the left and the step checklist for the open case on the right, with pass, fail, blocked, and not applicable buttons.
The run view: every case on the left, every step on the right. Enlarge to see the full screen.

Assign a case to a person. Or to an agent.

Assign one case or a whole slice

Hand cases to a person or an agent one at a time, or in bulk by case key, section, priority, or everything unassigned. Your own queue shows on the dashboard, and the run view has a Mine filter.

Agents claim real work
An agent claims the next case assigned to it, highest priority first, executes it, and reports each step. The claim hands over the steps, the run's environment settings, and the plan's environment notes in one response. Agent results are marked as agent results in the run.
Agents can't do damage
An agent key carries five scopes: read, claim, report, evidence, and comment. It can't author a plan, assign work, or administer anything, and no two agents are handed the same case.
Assay's dashboard listing the signed-in person's assigned cases under My queue, three active runs, and five plans.Assay's dashboard listing the signed-in person's assigned cases under My queue, three active runs, and five plans.
My queue: the cases assigned to you. Enlarge to see the full screen.

A dead session loses nothing

A claim is a lease, not a lock. It runs 30 minutes by default and can be extended up to 4 hours from the first claim. If the agent stops, a sweeper that runs every 30 seconds returns the case to the queue, and every step it already reported stays recorded.

Illustration with example data: after its session died and its lease ran out, case D-05 is back in the queue, with 2 of its 4 steps still ticked, and the agent qa-agent-1 grayed out beside it.Illustration with example data: after its session died and its lease ran out, case D-05 is back in the queue, with 2 of its 4 steps still ticked, and the agent qa-agent-1 grayed out beside it.

Illustration, not the Assay app, with example data. An agent, qa-agent-1, takes case D-05 on a 30-minute lease and ticks two of its four steps. Two heartbeats put the lease clock back to 30 minutes. Then the agent's session dies and the clock runs out, and Assay's sweeper, which runs every 30 seconds, returns the case to the queue. The two steps it reported are still ticked.

A failure has somewhere to go

Linked or filed in Docket

A failed case can be filed as a Docket card with its steps, expected and actual results, and environment, or linked to a card that already exists. A repeat of the same failure comments on its card instead of filing another.

Nothing fails silently
Side effects go through a durable outbox with bounded retries, so a Docket outage never blocks a tester's click. The Defects page leads with anything that never reached Docket and shows the error.
Evidence, with secrets stripped
Attach log excerpts, HTTP transcripts, JSON, text, links, or screenshots to a case or a step. Tokens, keys, and authorization headers are redacted before anything is stored, and the patterns that fired are recorded.
Say it in Chirp
When a Chirp webhook is configured, Assay posts when a run starts and finishes, when a P0 or P1 case fails, when a defect is filed, and when a run stalls. It batches the summary, so a long run is not a flood.
Assay's Defects page: two failures linked to existing Docket cards, each with its state and a link to the card.Assay's Defects page: two failures linked to existing Docket cards, each with its state and a link to the card.
Defects: every failure and the card it belongs to. Enlarge to see the full screen.
How it works

Import, generate, work, link

  1. Import

    Point Assay at a markdown test plan. It parses the sections, cases, steps, priorities, and defect tags, and reports what it found before it writes anything.

  2. Generate

    Create a run against an environment: the whole plan, just the P0s, or one section. The run pins to that plan version.

  3. Work

    Check off steps and resolve cases, or assign them to people and agents who claim and report them.

  4. Link

    A failed case becomes a Docket card, or is linked to the one that already exists, with a path back to the case.

Accessibility

Built so everyone can use Assay.

Assay's web app is built on the Renkara design system and keeps a keyboard map for working a run. Here is what the product supports today and what has actually been checked. We do not claim a WCAG conformance level for the product, and Assay has no automated accessibility scan yet.

  • Partly supported

    Keyboard operation

    A run can be worked from the keyboard: j and k move between cases, J and K between steps, Space toggles a step, and p, f, b, and n record pass, fail, blocked, or not applicable. Shortcuts are ignored while you type in a text box, and the shortcut sheet opens as a dialog. Command-K opens a command palette.

  • Partly supported

    Screen reader landmarks and labels

    Case status, assignee kind, and run progress carry text labels (the progress bar has a name and a value, the assignee icons are labeled, and filter buttons report whether they are pressed). Assay has not been run through axe-core and has not been tested with a screen reader.

  • Partly supported

    Contrast in light and dark

    Assay uses the Renkara design system's light and dark tokens. Its tests recompute the contrast of the violet accent on the light and dark card backgrounds and require 4.5 to 1. No other text or control has been measured, and there is no axe-core color-contrast scan.

  • Not yet tested

    Reduced motion

    Assay's interface uses the Renkara design system's motion presets, which are designed to honor the reduced-motion setting. Assay's own tests do not check this.

  • Not yet tested

    Text resize and zoom

    Resizing text to 200 percent has not been tested.

  • Partly supported

    Mobile and touch

    The dashboard, plans, runs, and run view fit a 375 by 667 phone screen with no sideways scrolling, and the result buttons are at least 36 pixels tall. Assay's own tests do not yet hold every control to 44 pixels.

Tech specs

Under the hood

Assay technical specification
Backend
  • FastAPI (Python), with REST under /api/v1
  • PostgreSQL through SQLAlchemy (async) and Alembic migrations
  • Append-only plan versions; runs pinned to a version
  • Leased agent claims on SELECT ... FOR UPDATE SKIP LOCKED, with a supervised sweeper
  • A durable outbox with bounded retries for every side effect
Web app
  • React 19 and TypeScript, built on the Renkara design system
  • A command palette and a keyboard map for working a run
  • Light and dark themes; the run view fits 375 by 667
  • Pages for the dashboard, plans, runs, defects, and agents
Execution
  • Scoped agent keys: read, claim, report, evidence, and comment only
  • Leases with heartbeats, a hard cap, and an attempt limit
  • Server-side redaction of tokens and keys in evidence
  • Optimistic concurrency: a simultaneous edit gets a conflict, not an overwrite
Integrations
  • An MCP server with 40 tools, relayed to agents by Roost
  • Docket cards: filed with de-duplication, linked, or suppressed
  • Chirp notices through an inbound webhook
  • Authentication by the Renkara identity service; agents use keys
Stands in for

Markdown test plans nobody could check off, Failures copied into a tracker by hand

Works with

Part of the Renkara fleet

MCP tools

Built for agents as well as people

Agents and people author plans as markdown, generate and read runs, assign work, claim and report cases, attach evidence, and link or file defects through Assay's MCP tools.

Plans

  • list_plans
  • get_plan
  • create_plan
  • update_plan
  • archive_plan
  • import_plan_markdown
  • export_plan_markdown
  • list_plan_versions
  • get_plan_version
  • publish_plan_version
  • diff_plan_versions

Runs

  • generate_run
  • list_runs
  • get_run
  • update_run
  • get_run_progress
  • export_run
  • list_run_cases
  • get_run_case

Assignment

  • assign_case
  • bulk_assign
  • get_my_queue
  • list_agents
  • register_agent

Execution

  • claim_next_assignment
  • heartbeat_claim
  • release_claim
  • report_step_result
  • report_case_result
  • reset_case
  • list_claims

Evidence

  • attach_evidence
  • list_evidence
  • retract_evidence

Defects

  • list_defects
  • file_defect
  • link_existing_defect
  • suppress_defect

Operations

  • check_service_health
  • get_diagnostics
FAQ

Questions people ask

Is Assay available to other teams?

Assay is part of Renkara's own tool fleet, and this site shows what it does and how it is built. It is not offered for sale here. Write to fleet@renkara.com with questions.

Can I write or import a plan in the web app?

Not today. Plans come in as markdown through the importer script or the MCP import tool, and an agent can write the markdown for you. The web app shows plans and their versions, and it is where runs are worked.

What happens if an agent dies in the middle of a case?

Its lease expires and the case returns to the queue, and every step it already reported stays recorded. A case that keeps losing its claim is blocked with a reason instead of looping.

Does a failure always file a Docket card?

Only when the plan has a Docket board and auto-filing on. A repeat of the same failure comments on the existing card, and a failure can instead be linked to a card that already exists or suppressed with a reason.

Is the data in the screenshots real?

No. Every screenshot is a capture of the running app against fictional StrataForge Robotics data.

Contact

Questions about this tool?

Write to Renkara Media Group and a person will answer.