Estate audit — Wave 7: ground the dashboard (every pass backed by an executable check)#502
Merged
Merged
Conversation
…table check The compliance dashboard's `status = "pass"` was self-asserted prose. Wave 7 makes it machine-verified: DYADT applied to the scorecards themselves. - scorecard.schema.json: optional `check` field — a read-only shell command, run from the repo root, that exits 0 iff the requirement currently holds. Distinct from `system` (human-readable): `check` is what CI RUNS. - build-scorecards.sh --verify: RUNS every pass-row check. A claimed pass whose check fails is a HARD error; a fail-row whose check now passes is flagged as a stale-fail re-grade candidate (advisory); passes without checks are counted as self-asserted (visible debt). Dashboard gains a Grounded column + rollup. extract_checks handles TOML-escaped quotes (strip trailing quote + unescape — first-quote truncation silently rewrote commands). - Grounding sweep (per-spec fan-out, every check tested before adoption): 87/87 pass rows now grounded — 0 broken, 0 self-asserted. 82 checks added, 4 honest enforcement-gap flips (fact true on disk, previously unchecked), 4 re-grades (licence rows to manual-only per Manual-Only policy). Implementation gaps (~250 rows) deliberately stay honest `fail` — a check cannot invent missing engineering. - The grounding pass caught a real defect: avow-protocol's OSSF-scorecard evidence cited a NESTED workflow (dormant — GitHub only executes root .github/workflows); evidence corrected to the root workflows that actually run, and grounded against them. - registry-verify.yml: the dashboard step now runs --check --strict --verify, with remediation hints. Justfile: scorecards-verify recipe. - wave3-scorecards-test.sh: +4 assertions (broken pass fails, holding checks stay green, grounded count reported, stale-fail advisory); updated fixtures for the fully-grounded world; fixed a grep -q/SIGPIPE race under pipefail. MUST-passing moved 48 -> 47 by honest regrade (licence row to manual-only), not by loss. Licence/SPDX rows remain manual-only throughout; no SPDX edits. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0114ps6mY5jAH4SzbGxeuYjc
…gates Promotes the enforcement surfaces that could not block, each honestly (no manufactured red noise), each with a red-team proof it CAN fail. - hypatia-scan-reusable.yml: baseline-aware gate. When a caller commits .hypatia-baseline.json (+ ships scripts/apply-baseline.sh), findings are filtered through the baseline and any UNBASELINED finding >= high FAILS the job (apply-baseline.sh blocking mode — thresholds, expiry, downgrades all already implemented there). Repos without a baseline keep the honestly- labelled ADVISORY behaviour; the gate arms itself the moment a baseline lands (standards#399/#437/#446). - affinescript-verify.yml: SPLIT GATE replaces the blanket advisory. ADDED .affine files must compile — a failing added file fails the job (new code has no excuse); MODIFIED legacy ports stay advisory until the port backlog clears (LEGACY_BLOCKING flag to flip later); toolchain failures skip LOUDLY ("SKIPPED, not passed" — never a silent green claim). Job-level continue-on-error removed; verify step guards against a missing compiler checkout before cd. - registry-verify.yml: Mustfile enforcement now gates PRs (structural check + run-mustfile.sh) — previously the invariants executed only on push-to-main, so a breaking change was caught after merge, not before. - scripts/tests/wave8-gates-test.sh (11 assertions; Justfile: gates-test): unbaselined-high blocks / baselined passes / EXPIRED baseline entry stops suppressing / low severity below threshold; Mustfile gate passes on this repo and blocks on a failing critical invariant; workflow-shape checks (job-level continue-on-error gone, blocking branch present, loud skip). Licence/SPDX untouched (flag-only policy). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0114ps6mY5jAH4SzbGxeuYjc
hyperpolymath
marked this pull request as ready for review
July 17, 2026 23:07
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
Next phase of the estate program (umbrella #460; Waves 0–6 merged). The compliance dashboard was honest but self-asserted: a
status = "pass"was prose, not proof. Wave 7 closes that: DYADT applied to the scorecards themselves — every pass row now carries an executablecheckthat CI runs, and a claimed pass whose check doesn't hold fails the build.What's here
checkfield (scorecard.schema.json): a read-only command, run from the repo root, that exits 0 iff the requirement currently holds — distinct from the human-readablesystem.build-scorecards.sh --verify: runs every pass-row check. Broken pass → hard error. Fail-row whose check now passes →stale-failre-grade prompt (advisory). Pass without a check → counted self-asserted (visible debt). Dashboard gains a Grounded column + rollup line.82 checks added; 4 honest enforcement-gap flips (fact true on disk, previously unchecked); 4 licence-row re-grades to
manual-only(Manual-Only policy). The ~250 implementation-gap rows deliberately stay honestfail— a check cannot invent missing engineering.A real defect caught by grounding: avow-protocol's OSSF-scorecard evidence cited a nested workflow — which is dormant, since GitHub only executes root
.github/workflows/. Evidence corrected to the root workflows that actually run, then grounded against them.CI:
registry-verify.yml's dashboard step now runs--check --strict --verify.Justfile:scorecards-verify.Verification
wave3-scorecards-test.shgained 4 assertions (broken pass fails; holding checks stay green; grounded count reported; stale-fail advisory) and a fix for agrep -q/SIGPIPE race underpipefail.--verifyfull run: 87 grounded · 0 broken · 0 self-asserted · 0 stale-fail.--checksync; DYADT dogfood claims all confirmed.manual-only), not by loss.Notes
🤖 Generated with Claude Code
Generated by Claude Code