AI-workflow generator · v0.20.0 · MIT
An AI‑workflow generator that publishes its own report card.
It reads your codebase and forges a project-specific workflow — rules, skills, commands, agents, MCP config, memory — then measures the pieces it ships against a pre-registered harness and commits the raw scores, weak grades included.
claude plugin marketplace add directiveforge/directiveforge claude plugin install directiveforge The re-measure
The proof, up front
Read the report card before you read the pitch.
Every row below traces to a committed artifact that ships in the repo. Open it and check. These are the metrics that carried an F at baseline, re-measured after the fix.
Scores — higher is better · 0 to 1.0
v1.0 single-call channel, same instrument as baseline
Wilson 95% CIs do not overlap
v1.0 channel
CIs do not overlap
v1.1 batched channel + equivalence block
10/10; Heroku trap now a correctly-flagged negative
Defects — removed
all 4 planted traps now cited only with a drift flag
backup + audit-trail manifest; the hard F-cap lifts
same 2 real dead links kept (exit 1); 13 FP → 0
On the measured slice: decision pack F B band brownfield-api F A band brownfield-docs F-cap lifted
derivation & scope — DELTA.mdRegressions ship too
The row that costs us, in the same table as the wins.
the boundary rewrite that killed a poem false-positive also lost a tone-check positive — a real trade, filed
We publish the row that costs us, because a report card that only shows the good numbers is marketing.
Measured, not claimed
The harness is pre-registered.
The measurement spec is committed to git before any run — a release verifier checks in history that the spec’s first commit predates the first results commit, so metrics can never be redefined after seeing the numbers. Every figure carries its n, its method, and a 95% CI — or an explicit “single-run, directional” caveat. A number that cannot be recomputed from a committed raw artifact does not ship.
A Fable-5 blind re-judge agreed with the rubric judge on 18/20 scores exactly (0.90) and 20/20 within ±1 (1.00).
What it is
A generator with a proof harness.
- A generator, not a catalog
- It reads your codebase and synthesizes a workflow for your stack and conventions — not a pile of prebuilt components to pick from.
- 22 measured skills
- Across three packs — decision (12), naming (6), design-elevation (4) — each with trigger and activation numbers in the harness results, not just prose.
- A proof harness
- It scores the generator on itself: artifact-level statistical scoring (trigger F1, activation repeatability with CIs, anchored rubrics) plus an end-to-end benchmark on synthetic fixture repos with answer keys.
- A vigilance loop
- It scans the AI ecosystem daily, synthesizes weekly, and integrates monthly — so the kit’s knowledge does not silently rot as models and frameworks move.
What it is not
And what it refuses to be.
- Not a CLI or a SaaS
- There is no binary and no server. The plugin installs skills; the generator is a prompt you paste; upgrades run through a manifest.
- Not a 200-skill catalog
- 22 skills, each with a published trigger score. Breadth is not the pitch; measurement is.
- Not infrastructure benchmarks
- No latency, RSS, or throughput numbers. The harness measures workflow-artifact quality and generator behavior on fixtures — a different axis entirely.
- Not autonomous
- The generator asks before it writes. Upgrade mode dry-runs by default. /report-friction never submits anything without your explicit review. Nothing edits your repo behind your back.
- Not self-congratulatory
- The baseline published F grades. The numbers are recorded before they are fixed, and the regressions ship in the same table as the wins.
60-second quickstart
Three steps to a measured workflow.
-
Install the skills and commands as a plugin
Two commands. This installs the 22 measured skills (decision / naming / design packs) and the workflow commands, including /report-friction.
claude plugin marketplace add directiveforge/directiveforgeclaude plugin install directiveforge -
Generate a project-specific workflow
Open a session at the root of your target project and paste generator/PROJECT_SETUP_PROMPT.md. It reads your codebase, asks a short profile, and writes your CLAUDE.md, .cursor/rules/*.mdc, .claude/commands/, agents, and MCP config.
generator/PROJECT_SETUP_PROMPT.md -
Cursor consumes skills as files
Copy templates/skills/<pack>/ into your project per the Cursor workflow.
workflows/WORKFLOW-CURSOR.md
Full walkthrough: QUICK_START.md
Honest limitations
Where the numbers stop.
None are softened; each links to where it lives in the repo.
- Not everything was re-measured
- The delta re-ran only the metrics that carried an F. Static gates (L1.1) and rubric scores (L1.4) were not re-measured post-fix; the greenfield fixture was deferred to public round 2. Absent, not zero. DELTA.md
- Single-run directionality
- Layer 2 and some claim rows are single re-measure runs. Where a metric is not CI’d, it is labelled directional, not proven.
- The self-checklist is our own instrument
- The L2.1 checklist measures self-consistency, not external quality — circular by construction. The answer-key-scored metrics (L2.2 / L2.4 / L2.6) are the external check.
- Judge-model dependence
- Trigger routing is simulated with Haiku and rubric scores are judged by Opus — both are LLM judges. A Fable-5 blind re-judge calibrates the rubric judge, but a different judge could move borderline rows. calibration-fable.md
- Synthetic-fixture boundary
- Layer 2 runs against synthetic reference repos with answer keys, not real projects. We make no claim that the kit improves real-project outcomes. Nobody in the ecosystem measures that yet — including us.
- A regression shipped this release
- disconfirming-evidence-first F1 0.9091 → 0.8889, disclosed above and filed for the next patch. Six residual defects are open and dispositioned. feedback/DISPOSITIONS-v0.19.0.md
Freshness
Knowledge rots silently. The cadence is public.
- Scan / synthesize / integrate
- A daily scan of a severity-tagged watchlist, a weekly synthesis that separates signal from noise, and a monthly integration through a reviewed session — never an auto-merge. workflows/KIT-VIGILANCE.md
- A weekly public digest
- What moved upstream, what it means for kit users, what shipped or is queued, and the open friction-report count with disposition status — on GitHub Releases. vigilance/PUBLIC-DIGEST-FORMAT.md
Feedback
Every report answered. Zero silent drops.
- /report-friction
- Harvests the defect (file:line, expected vs actual, severity), sanitizes paths and names, and stops at a review gate. Nothing is submitted until you approve it.
- The disposition guarantee
- Every report is answered in a public DISPOSITIONS file — fixed, deferred-with-reason, or rejected-with-reason. feedback/DISPOSITIONS-v0.19.0.md
For the skeptic
The questions a Hacker News reader asks first.
Is this a SaaS or a CLI I have to run?
Neither. There is no binary and no server. The plugin installs skills into Claude Code; the generator is a prompt you paste at your project root; upgrades run through a manifest. Nothing phones home.
Does it actually improve my real project?
We make no such claim. The harness measures generator behavior on synthetic fixture repos with answer keys — not real-project outcomes. Nobody in the ecosystem measures that yet, including us. What we publish is what we can recompute from a committed artifact.
What is a pre-registered harness?
The measurement spec is committed to git before any run. A release verifier checks in git history that the spec’s first commit predates the first results commit, so metrics cannot be redefined after seeing the numbers. Tuning a rubric to pass is a spec change, a new dated version, and a re-run — not a quiet edit.
Why publish your own F grades and a regression?
Because a report card that only shows the good numbers is marketing. The baseline F grades and the shipped regression sit in the same table as the wins — that is the credibility mechanism, not a footnote.
The F grades ship in the same table as the wins.
claude plugin marketplace add directiveforge/directiveforge claude plugin install directiveforge