audit-tests — Diagnostic Test Auditor (7-Layer + RTM)
Invocation: "audit tests" · "find gaps" · "full sweep" · "7-layer audit" · "rtm check" · "test quality".
> Enforcement harness: this skill delegates all deterministic checks (hash-pin, escape-scan, CRAP, architecture, bias, Gherkin lint) to the in-repo installation of @intentsolutions/audit-harness. Node repos install via pnpm add -D @intentsolutions/audit-harness; other languages vendor via the package's install.sh. Never reference ~/.claude/... paths from a target repo's hooks or CI — the enforcement must travel with the code.
Read-only diagnostic skill. Classifies the repo, maps applicable layers from the 7-layer testing taxonomy, runs deterministic gates, builds requirements traceability (RTM / personas / journeys), writes TEST_AUDIT.md + updates tests/TESTING.md, and mandatorily hands off to implement-tests when P0/P1 gaps exist.
No filesystem mutations other than TEST_AUDIT.md (transient) and the observational sections of tests/TESTING.md, tests/RTM.md, tests/PERSONAS.md, tests/JOURNEYS.md.
Overview
Diagnostic-only test suite auditor for any Intent Solutions (or compatible) repo.
Classifies the repository, maps the 7-layer testing taxonomy, runs deterministic
gates via in-repo @intentsolutions/audit-harness, builds RTM/personas/journeys
traceability, and writes TEST_AUDIT.md plus observational sections of
tests/TESTING.md. When P0/P1 gaps exist it hands off to implement-tests
instead of implementing tests itself.
Prerequisites
- A git working tree for the target repository
- Ability to run
Read/Glob/Grep/Bash against that tree
- For deterministic gates: in-repo
audit-harness (or install path documented
for the language) — never wire target CI/hooks to ~/.claude/...
You are the engineer
Deterministic tools do the grading. The engineer owns the walls (features/*.feature, coverage thresholds, architecture rules, TESTING.md policy sections, RTM.md MoSCoW tags). The AI operates inside them and never grades itself — every gate is a tool that returns exit 0.
Ownership (audit-tests may READ; never modify)
| Artifact |
Owner |
features/*.feature |
Engineer — hash-pinned |
Policy sections of tests/TESTING.md |
Engineer — hash-pinn
Validate a SKILL.
ReadEditWriteBash(python3:*)Bash(j-rig:*)Bash(node:*)GlobGrepAskUserQuestion
Validate SKILL.md
Grade any SKILL.md file against the Intent Solutions rubric (validator v7.0 / schema 3.3.1). Four-tier validation: Tier 0 (locate), Tier 1 (standard or marketplace grading), Tier 2 (static production gate), Tier 3 (JRig behavioral eval — opt-in).
Source of truth: /skill-creator validation workflow + claude-code-plugins-plus-skills/000-docs/SCHEMA_CHANGELOG.md + j-rig-skill-binary-eval/ (Tier 3).
Overview
Schema 3.3.1 enforces the 8-field IS enterprise required-field set at marketplace tier (name, description, allowed-tools, version, author, license, compatibility, tags). Anthropic's spec floor (name + description only) sits underneath; the IS rubric sits on top. Modes:
- Standard (default): Mirrors
platform.claude.com/docs/en/agents-and-tools/agent-skills/overview exactly. Required: name, description. Everything else is silent unless invalid type/value. Fast (~10 sec).
- Marketplace (
--marketplace): 8-field enterprise required set + 100-point IS rubric. Missing required fields = ERROR, not warning. The --enterprise flag is a deprecated alias. Fast (~10 sec).
- Deep (
--deep): Intent Solutions Deep Evaluation Engine — 10 weighted dimensions, trust badges, Elo competitive ranking, optional LLM quality assessment via Groq. Fast (~30 sec).
- Thorough (
--thorough): Adds Tier 3 JRig behavioral eval on top of Tier 1+2. Runs 7-layer eval across Haiku/Sonnet/Opus. Slow (~10–30 min) and costs ~$2–5 per skill in API spend — opt-in only. Right for production-gating, not iterative authoring.
> Performance + cost note: Tiers 0–2 run in seconds and are free. Tier 3 (JRig) is opt-in because behavioral eval across the model matrix is genuinely expensive. Default invocations stay fast; --thorough is for the moment a skill is being promoted to production or marketplace-verified.
Prerequisites
- Python 3 with
pyyaml installed
- Validator script:
claude-code-plugins-plus-skills/scripts/validate-skills-schema.py (v7.0+)
- For
--thorough (Tier 3): JRig CLI on PATH (jrig --version returns ≥ v0.14.0). Install: cd ~/000-projects/j-rig-binary-eval && pnpm install && pnpm build && pnpm link --global. Tier 3 is opt-in; the rest of the skill works without JRig installed.
Kernel schema first (canonical machine spec)
Validate frontmatter structure against
Ready to use intent-labs-pack?
|