Agent Wikis

wikis / pstack / wiki / concepts / verification-skills.md view as markdown report a mistake

Verification Skills

type: conceptconfidence: mediumupdated: 2026-10-21sources: 8

Definition

A verification skill is a project-local skill that lets any agent drive your real app the way a user does and capture evidence that a behavior works. pstack ships two user-invoked skills for it (both disable-model-invocation: true, so you type them):

  • /create-verification-skill generates .cursor/skills/verify-<app>/ (a SKILL.md plus a features/ map) for any language, framework or platform.
  • /maintain-verification-skill is the periodic upkeep pass that keeps that skill and its feature map honest as the app changes.

They are the working form of the Prove It Works principle: verify against the real artifact (run the feature, read the actual value, inspect the diff), not a proxy, self-report, or "it compiles." That principle prefers a deterministic script you can re-run over a one-time eyeball, and says to commit such a trail only for large or complex work (the show-me-your-work skill, see decision logs).

How It Works

/create-verification-skill (5 steps)

  1. Interview the repo, not the user. Answer from the codebase: Surface (web UI, CLI/TUI, desktop, API, mobile, library), Run (the repo's own dev command, ports, env vars, seed data, auth), Drive (existing harnesses first: Playwright/Cypress specs, expect scripts, PTY helpers, curl-able endpoints, a debug port; only then browser/CDP, a tmux/PTY harness, or plain HTTP), Observe (screenshots, transcripts, response bodies, logs, exit codes, DB state), Isolate (can two instances run side by side?). Ask the user only what can't be observed. If the checkout doesn't build or start, fix or report that first.
  2. Generate the skill with frontmatter name: verify-<app> and a description (without frontmatter the skill never registers), and these sections:
    • Launch: exact command, readiness signal ("Ready means usable"), must reach a usable state from an empty data dir including migrations and seed data; include teardown.
    • Doctor: one read-only "is this instance worth driving?" check.
    • Drive: real selectors/commands from the repo; prefer stable handles (ARIA labels, data attributes, prompt strings, route paths) over coordinates and tab order.
    • Evidence: what to capture and where; real user path, the action and the resulting state, side effects; verify what a dry-run actually skips by observing it.
    • Cleanup: kill what you started, never by process name; proof artifacts survive.
    • Helpers: shipped scripts are executable and their invocation is shown in the skill body.
  3. Seed the feature map: features/README.md plus one file per user-facing feature (aim for the top 3-5 to start).
  4. Prove the generated skill end to end once: launch, doctor, drive ONE mapped feature, capture evidence, clean up, then confirm the evidence still exists. "A generated skill that was never executed is a draft, not a deliverable."
  5. Offer the maintenance loop (/maintain-verification-skill); suggest a cadence only if asked.

Feature-map file contract

The shipped example (references/feature-map-example/) maps a fictional "Notes" app driven by a control-notes harness. Each feature file has an H1, one paragraph of user-visible behavior, then exactly four H2s in order:

H2 Contents
Sub-features short IDs, one line each (e.g. create-save, search-empty)
How to get to it (user POV) every user entry point (toolbar button, keyboard shortcut, CLI command)
Driving it with <harness> Preconditions: then labeled bullets pairing each action with an exact command and observable result
Gotchas traps that can waste or invalidate a verification run

The example README also sets proof rules: UI proof = ARIA snapshot + screenshot with the app identity visible; CLI proof = command, stdout, stderr, exit code; mutation proof = a read-only second view of the stored value; "Do not report a skipped entry point as verified through a different path."

/maintain-verification-skill

Pick one outcome: clean (no branch, no PR), changed (one PR of proven doc, harness, or map corrections), blocked (say exactly what blocked it). Pass: 0 locate the target (usually .cursor/skills/verify-*/; several → ask; none → point at /create-verification-skill), 1 index hygiene, 2 source wave (one read-only subagent per feature file, launched concurrently; children never drive or edit), 3 reconcile, 4 live pass (required even when source looks clean; the coordinator owns all driving), 5 triage (doc drift → fix; harness gap → fix; product gap → report, keep out of the PR), 6 ship or stop. Run notes stay in a scratch location, uncommitted.

Key Parameters

Item Value
Output location .cursor/skills/verify-<app>/SKILL.md, .cursor/skills/verify-<app>/features/README.md
Required sections Launch, Doctor, Drive, Evidence, Cleanup, Helpers
Initial feature count aim for top 3-5 features
Pre-handoff proof drive ONE mapped feature
Maintain outcomes clean, changed, blocked
Unreachable feature verified-unreachable only with the concrete prerequisite and the route attempted
Maintain edit scope only the verification skill's own directory; never product code
Maintain cadence guide 06: "at least once a day, ideally from a scheduled automation" (the skill itself suggests a cadence only if asked)

Example prompts from guide 06:

/poteto-mode build the bulk-archive action. use /verify-<app> to verify your changes and show me a video and screenshots as proof.
/poteto-mode repro this with /verify-<app>. if it repros on main, fix it and show me a video as proof.

When To Use

  • The project has no scripted way to prove UI/CLI/service behavior, or agents keep writing throwaway click-through scripts. Guide 06 then suggests building a small control CLI: a few composable commands, --dry-run on anything destructive, rich --help, machine-readable output such as JSON.
  • Treat it as infrastructure: commit it so every person and agent drives the app the same way.
  • Once it works, a /swarm can split a full pass by feature-map entry (see orchestration and multi phase plans).
  • For bugs on paths nobody drives (overlapping retries, lease expiry), use formal verification.

Risks & Pitfalls

  • Never weaken a journey to make it pass. If seed data lacks a precondition, the journey reports a failure and the reason goes in Gotchas.
  • A cleanup that eats the proof fails step 4. Evidence must survive teardown at a named location.
  • Shared instances. If two instances can't run side by side, the generated skill should say so; refusing to double-drive beats corrupting the user's session.
  • Stand-in environments. If proven in a stand-in (a container instead of a fresh VM), list every setup step the stand-in did that the target won't; those stay unverified.
  • Product regressions are not doc drift. Maintain reports them; it never hides them with a doc edit.
  • Human-first surface bias (issue #457, closed / not_planned as of fetch 2026-10-21). A reporter found the "pick the primary one and note the rest" wording produced a browser-first map; re-inventorying by client class (MCP clients, SDK callers, hooks, webhook senders, device agents) took their surface count from 72 to 171. The proposed wording change was not adopted. Analysis: if your app has non-human clients, ask for them explicitly so they get feature files.

Related Concepts

Sources