Loopmaster Resource Library — Quality Rubric
This rubric is applied by the resource-discovery loop (lm-disc-01) and by human curators when deciding whether an external resource earns a place in resources/resources.yaml. It exists to keep the library a *curated, annotated bibliography* — not a link dump.
Why a rubric
The Loopmaster library is differentiating only if every entry is worth a reader's time and honest to annotate. A rubric with teeth does two things: it raises the floor on quality, and it gives the discovery loop a repeatable scoring procedure instead of a vibe check. Resources that do not meet the bar are rejected — and at least one rejection is documented below to prove the rubric bites.
The five dimensions
Each dimension is scored 0, 1, or 2. Sum is the total (0–10).
| Score | Meaning |
|---|---|
| 2 | Strong — clearly meets the dimension. |
| 1 | Partial — meets it in a limited or shallow way. |
| 0 | Absent or actively violated. An entry with any 0 is excluded, regardless of total. |
Authority (0–2)
The author has shipped or operated real agent systems, or the work is peer-reviewed / from a credible lab or vendor engineering team. We credit practitioners who have run loops in production over speculative commentary. Vendor marketing with no technique scores 0.
Durability (0–2)
The resource teaches patterns or mental models that stay useful in 6+ months, not news or release notes tied to a model version. A framework API reference can score 2 if it documents stable abstractions; a "Claude 3.7 is out" announcement scores 0.
Actionability (0–2)
A reader can change how they build after reading. "Build loops with clear stop conditions" is actionable if it says *what* a stop condition looks like and *how* to declare one; the same sentence as a throwaway line in a hype post is not. Code, schemas, or concrete checklists earn the 2.
Uniqueness (0–2)
The resource says something the library does not already cover, or covers it from a materially different angle. We reject near-duplicates of already-included essays even if they are good. A resource that adds a new pattern, a new failure mode, or a new eval lens scores 2; a rehash of "ReAct is think-act-observe" scores 1 at most.
Agent relevance (0–2)
The resource bears on loop design, verification, guardrails, or maintenance — the things Loopmaster teaches. General LLM prompting or a model benchmark with no loop/agent angle scores low even if it is excellent in its own domain.
Inclusion bar
- Total ≥ 7 AND no dimension at 0.
- Both conditions must hold. A 7 with a 0 in authority is still rejected.
- The
rubric.justificationfield records why the score, in 2–3 sentences referencing the
resource's concrete content. The discovery loop emits a draft; the human (or delegated reviewer) edits it on merge.
Hard exclusions (automatic reject, do not score)
- Vendor marketing with no transferable technique.
- "AGI tomorrow" hype / speculative autonomy takes with no engineering substance.
- Content behind hard paywalls we cannot annotate honestly. (Soft paywalls / free-with-email
are fine; annotate the access note.)
- Anything we cannot annotate honestly — i.e. we cannot say what it says and why it matters
without inventing. If the discovery loop cannot summarize it, it is not ready.
Status taxonomy (the status field)
| status | meaning |
|---|---|
active | URL resolves, content still current. |
stale | >60 days since last_verified, or content visibly drifting. Visually flagged on the site, not dropped. |
dead | 404 or redirected to something unrelated. Link-audit loop opens a replacement task. |
superseded | Replaced by a newer entry; superseded_by names the replacement. Kept for provenance. |
---
Worked example A — ACCEPT
Candidate: Addy Osmani — "Loop Engineering" URL: https://addyosmani.com/blog/loop-engineering/
| Dimension | Score | Reasoning |
|---|---|---|
| Authority | 2 | Osmani is a senior practitioner (Google Chrome team) who writes from building, not observing. The post is widely cited by other practitioners (LangChain, Simon Willison). |
| Durability | 2 | Defines a meta-practice — designing the system that prompts the agent — that is framework- and model-version independent. Still cited a year+ later. |
| Actionability | 2 | Gives a concrete shape (find work → assign → check → record → decide next), names the recurring failure modes, and tells the reader what to change about how they work. You can build differently after reading. |
| Uniqueness | 2 | Coins the "loop engineering" framing that the field has converged on; no other entry in the seed library frames it this way. |
| Agent relevance | 2 | Directly about designing loops around agents — the core of what Loopmaster teaches. |
| Total | 10 |
Verdict: Accept. Annotation: "Defines loop engineering as designing the recurring system that finds work, assigns it, checks it, records state, and decides what runs next — not writing better prompts. Pairs with Osmani's harness-engineering post. Read this first if you are new to loops."
---
Worked example B — REJECT
Candidate: A vendor blog post titled "Our new agent is autonomous and will run your business." URL: https://example-vendor.com/blog/autonomous-agents-change-everything (illustrative)
| Dimension | Score | Reasoning |
|---|---|---|
| Authority | 1 | Authored by a vendor product team, but the post is marketing copy with no named engineer and no described implementation. |
| Durability | 0 | Tied to a product launch and model-version claims; the substance is "our agent is better" with no pattern that outlives the release. |
| Actionability | 0 | No technique the reader can apply. The only "lesson" is to buy the product. No schema, no checklist, no failure-mode analysis. |
| Uniqueness | 0 | Says nothing the library doesn't already cover — and what it gestures at is covered better by Anthropic, Osmani, and Willison. |
| Agent relevance | 1 | Topically about agents, but the content is positioning, not loop design, verification, or guardrails. |
| Total | 2 |
Verdict: Reject. Fails the no dimension at 0 rule (three zeros) and the ≥7 bar. This is the vendor-marketing-without-technique hard exclusion. The discovery loop should record the rejection in its report with the scores and a one-line reason, then move on. Keeping this example here proves the rubric does not rubber-stamp anything labeled "agent."
---
How the discovery loop uses this
lm-disc-01 runs weekly (see loops/resource-discovery.loop.yaml). Each run:
- Sweeps
resources/SOURCES.mdplus 2–3 rotating exploratory queries. - Dedupes against
resources/resources.yamland its own state file. - Scores each candidate against this rubric, emitting per-dimension scores +
justification. - Opens a proposal (PR or kanban card) with ready-to-merge YAML for candidates that meet the
bar, and a *rejected-candidates* section in the run report with at least one documented rejection to prove the rubric has teeth.
Proposing is autonomous. Publishing is gated — proposed entries merge to resources.yaml (and thus the site) only after human or delegated-reviewer approval. Every accepted entry carries its scores and provenance in the PR; that trail *is* the audit trail.