QA Report — Complex Tier + M5 Fix¶
QA Engineer: Quinn
Date: 2026-03-20
Spec reference: IMPLEMENTATION_SPEC.md (V1 + V2)
Scope: M5 fix (vv-dashboard-design examples), C1–C5 (Complex tier)
Overall Verdict¶
| ID | Skill | Result |
|---|---|---|
| M5 | vv-dashboard-design (fix) | ✅ PASS |
| C1 | doc-coauthoring | ✅ PASS (1 WARN) |
| C2 | webapp-testing | ✅ PASS (1 WARN) |
| C3 | vv-skill-creator | ✅ PASS |
| C4 | agent-dispatch | ✅ PASS |
| C5 | revenue-modeling | ✅ PASS (2 WARN) |
M5 Fix: vv-dashboard-design Examples¶
Checks¶
| # | Check | Result | Notes |
|---|---|---|---|
| 1 | references/examples/alert-feed.md exists |
✅ PASS | 9681 bytes |
| 2 | alert-feed.md contains severity color mappings | ✅ PASS | Maps critical=#c04040, warning, info, success via CSS tokens |
| 3 | alert-feed.md contains accessibility guidance | ✅ PASS | WCAG 2.1 SC 1.4.1 referenced; icon+color requirement documented |
| 4 | references/examples/empty-state.md exists |
✅ PASS | 7871 bytes |
| 5 | empty-state.md contains template JSX | ✅ PASS | Full EmptyState component in components/EmptyState.tsx, usage examples present |
M5 Fix: ✅ PASS — All items confirmed.
C1: doc-coauthoring¶
Skill path: skills/doc-coauthoring/
Checks¶
| # | Check | Result | Notes |
|---|---|---|---|
| 1 | SKILL.md exists | ✅ PASS | — |
| 2 | Valid frontmatter (name, description) | ✅ PASS | name: doc-coauthoring |
| 3 | Description under 1024 chars | ✅ PASS | Confirmed within limit |
| 4 | Negative trigger in frontmatter | ✅ PASS | "Do NOT use for quick single-paragraph responses, memory capture, or code documentation" |
| 5 | SKILL.md body contains exactly 3 stages, clearly labeled | ✅ PASS | Stage 1: Context Gathering, Stage 2: Refinement & Structure, Stage 3: Reader Testing |
| 6 | Stage 1 has ≥7 context questions | ✅ PASS | 8 numbered questions confirmed (doc type, audience, purpose, required sections, length, tone, urgency, constraints) |
| 7 | Stage 2 references references/quality-checklist.md |
✅ PASS | Referenced at lines 70 and 152 |
| 8 | Stage 3 references scripts/word-count.sh |
✅ PASS | Referenced at lines 103, 164, 167, 180 |
| 9 | references/doc-types.md exists |
✅ PASS | 9 files confirmed present |
| 10 | references/reader-test-protocol.md exists |
✅ PASS | — |
| 11 | references/tone-guide.md exists |
✅ PASS | Referenced in Stage 1 and body |
| 12 | references/quality-checklist.md exists |
✅ PASS | — |
| 13 | Example exists in references/examples/ |
✅ PASS | sample-spec-workflow.md exists — complete 3-stage worked example |
| 14 | scripts/word-count.sh exists |
✅ PASS | — |
| 15 | scripts/word-count.sh is executable |
✅ PASS | -rwx------ confirmed |
| 16 | word-count.sh produces output without crashing |
✅ PASS | Tested: test_doc.md: ~14 words |
Warnings¶
⚠ WARN — Example file naming differs from spec:
The spec file structure specifies references/examples/spec-example.md and references/examples/proposal-example.md (two separate annotated files). What was delivered is one consolidated file: references/examples/sample-spec-workflow.md. The content of this file is a complete, annotated 3-stage spec workflow — functionally it satisfies the intent and passes the task brief's check ("Example exists in references/examples/"). However, it does not satisfy the spec's AC1 ("All files exist at specified paths") or AC9 ("Both example files exist and are annotated") as written.
Resolution required: Either rename/split into the two spec-named files, or Forge updates the spec to accept the single consolidated example. As QA, I'm passing on functional criteria but flagging for Forge to adjudicate.
C1: ✅ PASS (1 WARN — example filename consolidation differs from spec, content satisfies intent)
C2: webapp-testing¶
Skill path: skills/webapp-testing/
Checks¶
| # | Check | Result | Notes |
|---|---|---|---|
| 1 | SKILL.md exists | ✅ PASS | — |
| 2 | Valid frontmatter (name, description) | ✅ PASS | name: webapp-testing |
| 3 | Description under 1024 chars | ✅ PASS | — |
| 4 | Negative trigger in frontmatter | ✅ PASS | "Do NOT use for unit tests (use npm test), API-only validation (use qa-validation validate.sh), or mobile app testing" |
| 5 | SKILL.md contains a decision tree | ✅ PASS | ## Decision Tree section with branching tree structure (Need to test a web UI? → branches for test suite, new tests, MC, baselines) |
| 6 | scripts/with_server.py exists |
✅ PASS | — |
| 7 | scripts/with_server.py is executable |
✅ PASS | -rwx------ confirmed |
| 8 | with_server.py --help does not crash |
✅ PASS | Prints correct usage with all args |
| 9 | with_server.py propagates test command exit code |
✅ PASS | test_exit_code = test_result.returncode → sys.exit(main()) confirmed |
| 10 | with_server.py terminates server in finally block |
✅ PASS | test_exit_code = 1 default; exits with test command code |
| 11 | scripts/run-tests.sh exists and is executable |
✅ PASS | -rwx------ confirmed |
| 12 | scripts/capture-baseline.sh exists and is executable |
✅ PASS | -rwx------ confirmed |
| 13 | capture-baseline.sh invokes companion Playwright spec (V2 fix) |
✅ PASS | Script explicitly invokes tests/capture-baselines.spec.ts per V2 spec |
| 14 | tests/capture-baselines.spec.ts exists (V2 requirement) |
✅ PASS | Present in webapp-testing/tests/ |
| 15 | All 3 reference files exist | ✅ PASS | playwright-patterns.md, mc-test-suite.md, troubleshooting.md — 3 files present |
| 16 | Reference file covers selector philosophy | ✅ PASS | playwright-patterns.md — "Selector Philosophy (In Priority Order)" section present |
| 17 | Reference file covers async patterns | ✅ PASS | waitForLoadState, waitForSelector patterns documented |
| 18 | Reference file covers error state testing | ✅ PASS | "Error State Testing" section present |
| 19 | mc-test-suite.md covers all current MC routes |
✅ PASS | /sigint, /agents, /tasks, /initiatives, /settings all present with assertions |
Warnings¶
⚠ WARN — Reference file names differ from spec:
The spec requires three specific files: references/playwright-setup.md, references/test-patterns.md, and references/mc-test-suite.md. What was delivered is references/playwright-patterns.md (which covers both setup and test patterns in one file), references/mc-test-suite.md, and references/troubleshooting.md (a bonus file not in the spec). The SKILL.md correctly references playwright-patterns.md throughout. The content requirements from the spec are met; only the file naming/split differs.
Resolution required: Rename/split playwright-patterns.md into playwright-setup.md + test-patterns.md to match spec ACs 8-9, or Forge accepts the consolidated approach. Not blocking on functional grounds.
C2: ✅ PASS (1 WARN — reference file names differ from spec; content satisfies all functional requirements)
C3: vv-skill-creator¶
Skill path: skills/vv-skill-creator/
Checks¶
| # | Check | Result | Notes |
|---|---|---|---|
| 1 | SKILL.md exists | ✅ PASS | — |
| 2 | Frontmatter name field is vv-skill-creator (NOT skill-creator) |
✅ PASS | V2 rename confirmed |
| 3 | Valid frontmatter, description under 1024 chars | ✅ PASS | 652 chars per validate-skill.sh output |
| 4 | Negative trigger present | ✅ PASS | "Do NOT use for executing skill workflows... Do NOT use for non-skill documentation" |
| 5 | scripts/validate-skill.sh exists |
✅ PASS | — |
| 6 | validate-skill.sh is executable |
✅ PASS | -rwx------ confirmed |
| 7 | validate-skill.sh passes bash -n syntax check |
✅ PASS | No syntax errors |
| 8 | Run validate-skill.sh against vv-skill-creator itself — mostly passes |
✅ PASS | 12/12 PASS — perfect score |
| 9 | Run validate-skill.sh against bad-skill-example — reports failures |
✅ PASS | 5 FAIL reported: name not kebab-case, no negative trigger, missing referenced file, non-executable script, folder/name mismatch |
| 10 | references/skill-checklist.md exists |
✅ PASS | 4688 bytes |
| 11 | references/frontmatter-guide.md exists |
✅ PASS | 9574 bytes |
| 12 | references/description-patterns.md exists |
✅ PASS | 10394 bytes |
| 13 | Additional: references/trigger-test-suite.md exists |
✅ PASS | — |
| 14 | Additional: references/vv-skill-standards.md exists |
✅ PASS | — |
| 15 | Additional: references/skill-spec-template.md exists |
✅ PASS | — |
| 16 | references/test-fixtures/bad-skill-example/ exists with intentional failures |
✅ PASS | non-executable helper.sh, missing reference file, bad name field all confirmed |
C3: ✅ PASS — All items confirmed. validate-skill.sh is production-ready.
C4: agent-dispatch¶
Skill path: skills/agent-dispatch/
Checks¶
| # | Check | Result | Notes |
|---|---|---|---|
| 1 | SKILL.md exists | ✅ PASS | — |
| 2 | Valid frontmatter (name, description) | ✅ PASS | name: agent-dispatch |
| 3 | Description under 1024 chars | ✅ PASS | — |
| 4 | Two negative triggers in frontmatter | ✅ PASS | "Do NOT use for actually executing tasks" + "Do NOT use for external API routing or OpenClaw model config" — both present on description line 10 |
| 5 | SKILL.md contains agent team with all 5 agents | ✅ PASS | Jules, Atlas, Melody, Quinn, Forge all in table |
| 6 | SKILL.md contains task routing decision tree | ✅ PASS | Structured as decision-tree section (not pure prose) |
| 7 | SKILL.md subagent prompt structure with required fields | ✅ PASS | — |
| 8 | SKILL.md has Examples section with multi-agent pipeline | ✅ PASS | — |
| 9 | SKILL.md has Troubleshooting section | ✅ PASS | — |
| 10 | references/agent-profiles.md covers all 5 agents |
✅ PASS | Jules, Atlas, Melody, Quinn, Forge each have full profile section (What they're optimized for, NOT for, prompt guidance) |
| 11 | references/model-routing.md covers ≥3 tiers |
✅ PASS | Heavy (Opus), Standard (Sonnet), Local (Qwen/GLM) — all documented |
| 12 | references/model-routing.md has "Last updated: 2026-03-20" (V2 requirement) |
✅ PASS | **Last updated:** 2026-03-20 confirmed in header |
| 13 | references/handoff-protocol.md exists |
✅ PASS | 5853 bytes |
C4: ✅ PASS — All items confirmed including V2 date-stamp requirement.
C5: revenue-modeling¶
Skill path: skills/revenue-modeling/
Checks¶
| # | Check | Result | Notes |
|---|---|---|---|
| 1 | SKILL.md exists | ✅ PASS | — |
| 2 | Valid frontmatter (name, description) | ✅ PASS | name: revenue-modeling |
| 3 | Description under 1024 chars | ✅ PASS | — |
| 4 | Two negative triggers in frontmatter | ✅ PASS | "Do NOT use for existing revenue tracking or invoicing" + "Do NOT use for cost estimation of API spend" |
| 5 | scripts/arr-calc.py exists and is executable |
✅ PASS | -rwx------ confirmed |
| 6 | arr-calc.py --help does not crash |
✅ PASS | Prints full usage with all args |
| 7 | Run with 3 tiers (named), 3 prices, 3 customers, 5% churn | ✅ PASS | --tiers "Starter,Pro,Enterprise" --prices "500,1000,2000" --customers "50,30,10" --churn 0.05 produces full 3-scenario projection table |
| 8 | Run with high churn (0.6) — prints warning, does NOT exit 1 | ✅ PASS | "WARNING: Monthly churn > 50% detected (60.0%)" printed; exit code 0 (correct per V2 spec) |
| 9 | Run with negative price — exits 1 | ✅ PASS | "ERROR: Prices cannot be negative." → exit code 1 confirmed |
| 10 | APA example files use PLACEHOLDER values (V2 requirement) | ✅ PASS | Both files contain header warning and [TBD — pending Atlas market research] throughout all numeric fields; no invented figures |
| 11 | references/arr-model-template.md exists |
✅ PASS | 3603 bytes; 3-scenario output table confirmed |
| 12 | references/market-sizing-guide.md exists |
✅ PASS | 5829 bytes |
| 13 | references/examples/apa-arr-model.md exists |
✅ PASS | Full structure with TBD placeholders |
| 14 | references/examples/apa-market-sizing.md exists |
✅ PASS | Full structure with TBD placeholders |
| 15 | SKILL.md defines TAM, SAM, SOM explicitly | ✅ PASS | — |
| 16 | ARR workflow covers 3-scenario requirement | ✅ PASS | Conservative/Base/Optimistic documented |
| 17 | ARR workflow includes critical assumption step | ✅ PASS | — |
| 18 | Troubleshooting section covers ≥4 failure scenarios | ✅ PASS | — |
Warnings¶
⚠ WARN 1 — Reference file name differs from spec:
Spec AC requires references/pricing-tier-guide.md. File delivered is references/pricing-frameworks.md. The content covers value-based pricing, tier structures, and anchoring principles (which is what the spec specified). The SKILL.md correctly references pricing-frameworks.md throughout. Not a functional gap, but a spec naming discrepancy.
⚠ WARN 2 — Task brief test invocation syntax inconsistency:
The task brief specifies --tiers 3 (treating it as a count). The script implements --tiers as comma-separated tier names per the spec body ("Starter,Pro,Enterprise"). Running --tiers 3 treats "3" as a single tier name and fails with a mismatch error. The script's implementation is correct per the spec body. The task brief's test case uses incorrect syntax. Verified with correct invocation (--tiers "Starter,Pro,Enterprise") — works correctly. This is a task brief bug, not a script bug.
C5: ✅ PASS (2 WARN — pricing file naming; task brief test syntax error — script itself is correct)
Issues Requiring Disposition¶
The following WARNs require Forge to adjudicate before closing the build:
| ID | Skill | WARN | Action Required |
|---|---|---|---|
| C1-W1 | doc-coauthoring | Single sample-spec-workflow.md instead of spec-example.md + proposal-example.md |
Forge: accept consolidated or Melody splits into 2 files |
| C2-W1 | webapp-testing | playwright-patterns.md instead of playwright-setup.md + test-patterns.md |
Forge: accept consolidated or Melody splits into 2 files |
| C5-W1 | revenue-modeling | pricing-frameworks.md instead of pricing-tier-guide.md |
Forge: accept rename or Melody renames file + updates SKILL.md references |
| C5-W2 | revenue-modeling | Task brief test invocation uses --tiers 3 (incorrect syntax per spec) |
Jules/Forge: update task brief to use correct invocation; no script change needed |
Post-C3 Retroactive Check: C1 AC11¶
Per V2 spec, after C3 QA passes, run validate-skill.sh against C1's SKILL.md:
vv-skill-creator Validation Report — 2026-03-20
Target: skills/doc-coauthoring
================================================
PASS SKILL.md exists at expected path
PASS Frontmatter delimiters present (opening and closing '---')
PASS name field: 'doc-coauthoring' is valid kebab-case
PASS description field: non-empty
PASS description length: 592 characters (under 1024)
PASS negative trigger present in description
PASS No XML/HTML tags in frontmatter
PASS SKILL.md word count: 1224 words (under 5000)
PASS Referenced files: all 6 referenced paths exist on disk
PASS All 1 script(s) in scripts/ are executable
PASS No README.md at skill root
PASS Folder name matches name field: 'doc-coauthoring'
Result: 12 PASS — skill passes all checks ✓
C1 AC11: ✅ PASS — doc-coauthoring passes all 12 validate-skill.sh checks with no FAILs.
Sign-off¶
All 5 Complex tier skills plus the M5 fix are functionally complete and meet the intent of the spec. Four WARNs require Forge disposition before final closure — none are functional blockers.
Quinn — QA Director, Vivere Vitalis
2026-03-20