Harden toolkit-doc generation against destructive failures

[jottakka]

Summary

• reject incomplete, colliding, unsafe, or unwritable generator inputs before publishing
• preserve the last-known-good generated JSON when a run fails
• add deterministic verification for output safety and data-source snapshots

Test plan

• pnpm exec tsc --noEmit --project toolkit-docs-generator/tsconfig.json
• pnpm exec vitest run --config toolkit-docs-generator/vitest.config.ts

---

Note

Medium Risk
Changes affect the CI docs generation pipeline and incremental --skip-unchanged behavior; failures now skip overwrites and can fail the job, which is intentional but alters operational outcomes.

Overview
Hardens the toolkit docs generator so incremental runs cannot wipe committed JSON from bad API data or partial merge failures.

Snapshot consistency and mass-delete guard: createCachedToolkitDataSource reuses one fetchAllToolkitsData snapshot for change detection, progress, and merging. --skip-unchanged now documents a single /v1/tool_metadata fetch instead of summary-then-partial fetches. assertSafeCurrentToolkitSnapshot aborts when the current snapshot is empty but previous output had toolkits.

Failures preserve last-known-good output: MergeResult gains an error field; failed merges return the previous toolkit when available and are excluded from writes. The CLI exits non-zero on merge or write failures and only writes successful merges. requireCompleteData fails the run instead of emitting placeholder toolkits.

Safer output I/O: Toolkit IDs are validated (no path escape, reserved index, case-insensitive collisions). JSON is written atomically; index.json entries are sorted. Output directory clearing uses stricter clearSafeOutputDir rules.

Schema/docs: ToolkitSubPageSchema tightens custom sections and merged subPages. README/workflow docs add generator tests in CI and Anthropic as the primary LLM secret.

^{Reviewed by Cursor Bugbot for commit 7ffca25. Bugbot is set up for automated code reviews on this repo. Configure here.}

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
docs	Ready	Preview, Comment	Jul 16, 2026 8:34pm

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 7ffca25. Configure here.}

[cursor]

Overwrite then strict fail wipes output

High Severity

With requireCompleteData enabled, a single toolkit merge failure now throws an error, stopping the entire generation process. This can leave the output directory empty if cleared, or result in stale/incomplete output and an outdated index.json for other toolkits that successfully processed before the failure, particularly in batch or incremental runs.

Additional Locations (1)

• toolkit-docs-generator/src/cli/index.ts#L792-L798

 

^{Reviewed by Cursor Bugbot for commit 7ffca25. Configure here.}

[cursor]

Incremental writes skip collision checks

Medium Severity

Case-insensitive toolkit ID collision detection runs only in generateAll via validateToolkitFileNames. Incremental mode publishes through onToolkitComplete → generateToolkitFile, which never runs that batch check, so colliding IDs can still overwrite the same JSON file before publishing is rejected.

Additional Locations (1)

• toolkit-docs-generator/src/cli/index.ts#L1443-L1456

 

^{Reviewed by Cursor Bugbot for commit 7ffca25. Configure here.}