QA.io.vn

QA · IO · VN

Field edition·MIT licensed

The AI QE Workflow

A complete, battle-tested operating model for AI-driven quality engineering — the agent authors the tests, executes manual charters, triages CI, files defects, verifies fixes, and reports the week. Extracted from a production Playwright + Java suite. Genericized for yours.

MIT licensed·7 commands·7 pillars·stdlib-only Python connectors

Plate A — Verdicts

PASS FAIL BLOCKED

Honesty over green: a false PASS is worse than an honest FAIL.

Chapter One

Getting started

Three levels, each strictly safer than the next. Don't skip to the end — the whole point of the design is that you don't have to trust it blindly to get value from it.

  1. Observe only

    See what the agent would do, without it doing anything. Nobody needs to approve anything here — every command below is read-only by construction.

    /create <TICKET> --dry-run
    /manual-test <TICKET> --cases-only
    /verify <TICKET> --dry-run

    Move up when the test-case design consistently reflects your product on 2–3 real tickets.

  2. One ticket, real writes, you review everything

    Let the agent write code and comments for real, but keep every action reversible — nothing below advances a ticket.

    /create <TICKET>
    /manual-test <TICKET>
    /verify <TICKET> --no-close

    Move up when you're reviewing out of habit, not because you expect to catch something.

  3. Standing authorization, the full loop

    /verify, /verify-bug, and /manual-test now advance tickets to Done on their own. Get every box in TODO.md Part A approved first.

    /verify <TICKET>
    /verify-bug <BUG>
    /manual-test <TICKET>

    Steady state — this is what the first two levels exist to earn.

No GitHub access?

Some organizations firewall github.com. This page is served from the plain domain qa.io.vn — if you can read this, you can copy the block below into a terminal from the root of your test repository. It recreates the whole handbook and toolkit as local files, with no network access and nothing that touches GitHub. Click inside the box, select all, and copy — the button below does the same thing in one click where supported.

Copied to clipboard

macOS, Linux, WSL, or Git Bash. On native PowerShell, run it inside Git Bash or WSL instead. Safe to re-run.

Chapter Two

The lifecycle

One rulebook, seven commands, shared state. Every outward action sits behind a gate.

Figure 1 — the lifecycle of the AI QE workflow, drawn as a metro map. A ticket flows to /create, then through a three-times-green gate into CI. CI feeds /ci triage, which — behind a locally-green gate — writes to the bug store. Behind an interactive-confirm gate, /bug files the store; /verify-bug re-checks each bug and, behind a stable-green gate, advances it to Done. In parallel, /manual-test runs the ticket's charter and reaches Done only when every case passes. A dashed weekly line marks /report, reconstructed from the git diff. 3× green (UI) locally green confirm first stable green every case passes TICKET /create CI /ci BUG STORE /bug known-failures registry — a prior, never a verdict /verify-bug DONE /manual-test /report weekly — reconstructed from the git diff, not commit messages
Fig. 1The lifecycle. Solid tracks carry the work; nothing crosses a filled marker without meeting its condition.
  • gate — an outward action waits behind it
  • station
  • durable artifact
  • weekly cadence

The commands communicate through durable artifacts — a local bug store, a recurrence registry of known failures, manual test charters, and the tracker itself. Each command also states what it does not do: /ci never files Jira issues, /bug never re-triages, /manual-test writes no test code.

Chapter Three

The seven commands

A quick-reference spread: each entry names its stage, its work, and the gate that holds it.

/createAuthor coverage

From a dev ticket to automated tests: maps each acceptance criterion to the lowest sufficient layer, inspects the live product before writing a selector, tracks the work as a linked task.

Gate every new test green — 3× for UI.

/manual-testManual testing

Designs a manual test charter from the ticket's acceptance criteria, executes every case against the live product, records PASS/FAIL/BLOCKED with evidence, comments the result sheet on the ticket.

Gate advance to Done only when every case passes.

/verifyDelivery check

Audits whether automation actually asserts the ticket's expected result — a stale mapping counts as no coverage — authors what's missing, then closes the ticket.

Gate N clean runs (default 3), reruns off.

/verify-bugBug re-check

Re-runs a bug's mapped test; on stable green auto-advances the bug through the workflow to Done and posts the verdict.

Gate stable green of the mapped test.

/ciCI triage

Pulls the failed build, triages every failure on its own merits with the known-failures registry as a prior, fixes test bugs, writes ISTQB-style product-bug tickets.

Gate fixes count only when locally green; extended mode adds 3× + full suite.

/bugFile defects

Files the bug store into Jira — active sprint, mapped priority, idempotent. The one command that always asks before writing.

Gate interactive confirm.

/reportWeekly report

Reconstructs the week from the git diff, not commit messages, into one plain-English paragraph. Undercounts when unsure.

Gate counts must trace to the diff.

Chapter Four

The seven pillars

  1. Inspect before you assert

    Selectors and contracts come from the live product, never guessed from a similar page.

  2. Test at the lowest sufficient layer

    E2E only for what a browser alone can prove.

  3. Page Object discipline

    One page per class, constants not magic strings, fluent API.

  4. Deterministic waiting

    Hardcoded sleeps forbidden; every wait names its condition.

  5. Tests own their data

    Seed, assert, clean up; leave the environment as found.

  6. The stable-green gate

    Nothing closes on one lucky pass; reruns-off, three consecutive greens by default.

  7. Honesty over green

    Blocked is a legitimate verdict; a false PASS is worse than an honest FAIL.

Chapter Five

Adopt it

A working skeleton takes about a day.

  1. Write the project rulebook
  2. Stand up the connectorsJira, Teams, CI; gitignored env files.
  3. Map your tracker workflow once
  4. Create the token-saving live-inspection reference
  5. Install the commandskeep the gates verbatim.
  6. Start the shared state emptynever seed the registry with assumptions.
  7. Dry-run the loop end to endbefore trusting standing authorizations.

Also on the shelf: commands/· toolkit/· rules/PROJECT-INSTRUCTIONS-TEMPLATE.md