SKILL

Agent Usability Test Loop

Plain markdown at a stable URL. Fetch it, follow it.

Use when deploying the loopmaster.ai agent-usability-test template to run clean-room agent install tests and capture pass/fail evidence.

Open raw SKILL.md

Agent usability test loop skill

Use this pack to install or run agent-usability-test for clean-room documentation verification. The deterministic local check only proves the packet is complete; a public pass still needs transcript and rubric evidence.

Canonical references:

  • templates/agent-usability-test/loop.yaml — verification requirements, gates, state paths, and backend hints.
  • templates/agent-usability-test/AGENT-INSTALL.md — complete backend install guide.
  • templates/agent-usability-test/verification/clean-room-protocol.md — prompt/protocol for the test agent.
  • templates/agent-usability-test/verification/transcript-template.md — transcript capture format.
  • templates/agent-usability-test/verification/pass-fail-rubric.md — scoring rubric.

If this pack conflicts with templates/agent-usability-test/AGENT-INSTALL.md, follow the template guide and fix this pack.

Preflight from repo root

test -f templates/agent-usability-test/loop.yaml
test -f templates/agent-usability-test/AGENT-INSTALL.md
uv run --with check-jsonschema --with pyyaml check-jsonschema --schemafile spec/loop.schema.json templates/agent-usability-test/loop.yaml
python3 templates/agent-usability-test/verification/check_clean_room_packet.py

Expected result: manifest validation exits 0 and the packet checker prints CLEAN_ROOM_PACKET_OK.

Pick a backend

  • cron: useful for reminders and packet completeness checks; it must not self-certify a verified status.
  • GitHub Actions: can create a clean checkout/container and archive artifacts for reviewer inspection.
  • agent-kanban: recommended for maker/checker separation; assign the clean-room run to a reviewer task.

Use templates/agent-usability-test/AGENT-INSTALL.md for exact backend setup.

Real run checklist

Define the target commit and allowed packet before the agent starts. The packet must list exact docs the agent may read, including the relevant template loop.yaml, README.md, AGENT-INSTALL.md, and the verification protocol files. Do not give prior chat logs, hidden local context, or credential values.

Required evidence paths for template runs:

templates/<template-id>/verification/YYYY-MM-DD-agent-usability-transcript.md
templates/<template-id>/verification/YYYY-MM-DD-agent-usability-report.md

Expected result: transcript records environment, prompt, every file read, every command, exit code, output summary, ambiguity, and final decision.

Stop or roll back

  • cron: remove the reminder or packet-check entry.
  • GitHub Actions: disable or delete the verification workflow in a PR.
  • agent-kanban: pause/remove the recurring reviewer task.
  • Failed run: do not mark the target verified; keep the transcript/report and create follow-up work for each friction point.

First-week ownership checklist

  • Confirm each clean-room run starts from a fresh environment.
  • Confirm forbidden context stays out of the packet.
  • Confirm every conditional pass or failure has a follow-up task or PR comment.
  • Keep first public verified badge/status changes human-gated.