Verify toolkit docs before publishing

[jottakka]

Summary

• run the generator's isolated type-check and test suite before publishing
• support an explicitly named branch for manual verification PRs
• avoid assigning normal production reviewers to those verification PRs

Test plan

• pnpm run toolkit-docs:check

---

Note

Low Risk
Changes are limited to CI scripts, workflow dispatch options, and tests; production doc generation paths are unchanged aside from an earlier validation gate.

Overview
The generate toolkit docs workflow now runs pnpm run toolkit-docs:check (TypeScript no-emit + Vitest for toolkit-docs-generator) instead of building the generator, so broken generator code fails before doc generation and PR creation.

Root package.json adds toolkit-docs:typecheck, toolkit-docs:test, and toolkit-docs:check so CI and local runs share the same gate.

Manual workflow dispatch accepts optional pr_branch: when set, the PR uses a [TEST] title, bases on the triggering ref, opens on that branch, and skips auto-requesting ArcadeAI/engineering-tools-and-dx reviewers. Scheduled/Porter runs keep the automation/toolkit-docs branch and normal review behavior.

Workflow markdown and workflow contract tests were updated to match (no build step; verification PR wiring).

^{Reviewed by Cursor Bugbot for commit 15bf213. Bugbot is set up for automated code reviews on this repo. Configure here.}

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
docs	Ready	Preview, Comment	Jul 16, 2026 8:20pm

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 15bf213. Configure here.}

[cursor]

Wrong Vitest config for generator

Medium Severity

The toolkit-docs:test script passes --config vitest.config.ts from the repo root, so Vitest loads the root config instead of toolkit-docs-generator/vitest.config.ts. The CI gate therefore skips that project’s test settings (including Vitest typechecking), while toolkit-docs:typecheck excludes **/*.test.ts.

 

^{Reviewed by Cursor Bugbot for commit 15bf213. Configure here.}

[cursor]

Stale gate blocks scheduled regen

Medium Severity

The new toolkit-docs:check step runs the full generator Vitest suite, including the committed-data stale-summaries test, before generate. If toolkit-docs-generator/data/toolkits on the checked-out ref already has summaryStale: true, validation fails and the workflow never runs generation—the same path that would refresh those summaries and open the auto PR.

Additional Locations (1)

• package.json#L13-L15

 

^{Reviewed by Cursor Bugbot for commit 15bf213. Configure here.}

[cursor]

Current version of PR was reviewed by /review-bugbot on Jul 16, 16:06 GMT-3. It flagged 1 finding.

Show 1 finding

1. workflow_dispatch inputs wrongly indented

.github/workflows/generate-toolkit-docs.yml:13 · High Severity

The new inputs block is indented at the same level as workflow_dispatch under on:, so it is a sibling trigger key rather than a child of workflow_dispatch. GitHub Actions only accepts inputs nested under on.workflow_dispatch; a top-level on.inputs entry is treated as an invalid webhook event and the workflow file fails validation.

^{Bugbot on commit 721b9bc is skipped.}