feat(storage,entity): path identity + speculation path→build mapping store by behinddwalls · Pull Request #331 · uber/submitqueue

behinddwalls · 2026-07-10T05:08:28Z

Summary

Why?

Integrating speculation needs the controllers to answer "which build belongs to this path" in both directions, without ever looking a row up by non-key attribute — the storage contract stays get/put-by-key so any KV backend can satisfy it. The build system mints its own build identifiers, and those stay the build store's primary key: the runner's ID is the natural name for the build row. What's missing is a durable link between a tree path and its build that exists independent of any in-flight message, hung on the path identity (SpeculationPathInfo.ID) introduced at the bottom of this stack.

What?

entity.SpeculationPathBuild is the new mapping entity ({PathID, BuildID, BatchID, CreatedAt, Version}, named after its speculation_path_build table) with a SpeculationPathBuildStore (Create/Get keyed by PathID): the forward path→build lookup, written only by the build controller, at most one build per path with ErrAlreadyExists making the existing row the truth on races. PathID is globally unique — the path ID contract on SpeculationPathInfo.ID is strengthened accordingly, since this table keys rows by it alone. BatchID makes the row self-describing without parsing PathID's format; Version follows the repo's optimistic-locking convention (write-once today, reserved for future conditional re-pointing flows). entity.Build keeps the runner-minted ID as its primary key and gains SpeculationPathID as a plain column — the reverse build→path lookup; the previously embedded SpeculationPath value is dropped as a redundant denormalized copy of what the tree already stores. SpeculationPath.Equal (order-sensitive Base + Head) provides structural path identity for the few controller spots where only structure can identify a path (deduplicating enumerator output, carrying entries over across re-enumeration).

MySQL gains the speculation_path_build table and the build table swaps speculation_path/runner_id for speculation_path_id; schema files are picked up automatically (the schema dir is globbed by both Bazel and the testutil ApplySchema helper).

Test Plan

✅ bazel test //submitqueue/entity/... //submitqueue/extension/storage/... //submitqueue/orchestrator/... and the storage integration suite exercises Create/Get round-trips and the duplicate-create race.

Issues

Stack

## Summary ### Why? The speculation seams being introduced up-stack (path scorer, selector, prioritizer) and durable links from other entities (a path→build mapping) all need to refer to one specific path inside a batch's speculation tree. Restating the full Base/Head split in every seam output couples those contracts to path structure and forces consumers to compare ordered slices; a single opaque identity lets each seam return only what matters — a path ID plus its verdict — and gives durable links a stable key to hang on. ### What? `entity.SpeculationPathInfo` gains `ID`: assigned by the controller when the path entry is first persisted, immutable thereafter, and unique within its tree. Its format is the controller's choice and carries no meaning — it is never parsed. Everything outside the tree names a path by this ID. The tree store persists it transparently through the JSON `paths` column, so the change is additive with no schema migration. ## Test Plan ✅ `bazel test //submitqueue/entity/...` — entity suite passes; the field rides the existing JSON round-trip in the tree store. ## Issues ## Stack 1. @ #337 1. #315 1. #316 1. #317 1. #320 1. #331 1. #332 1. #333

behinddwalls · 2026-07-12T21:07:28Z

🛑 Automatic stack rebase aborted — diff mismatch

The rebase of preetam/int/storage-build-lookup completed without conflicts, but the resulting code diff does not match the original. This means the rebase silently altered code content. The branch was not force-pushed.

Please rebase manually and verify the changes are correct.

## Summary Add the first speculation seam and its limit counterpart from the speculation RFC, as vendor-agnostic extension interfaces under submitqueue/extension/speculation/. enumerator: given a batch and its active dependencies ordered oldest-first (queue arrival order — the ordering that fixes which assumed-good Base prefixes exist), mechanically lists the candidate Base/Head speculation paths — pure, deterministic, and purely structural. It returns only []entity.SpeculationPath: identity, status, and score are controller-owned, and the controller wraps each path into the persisted tree entry (SpeculationPathInfo), keeping a clean ownership boundary between the seam and tree state. dependencylimit: the "how much" policy bounding how many active (in-flight) dependencies a batch may speculate over. It is the eligibility gate before enumeration; unlike the other speculation limits it is controller-held rather than injected into a seam, keeping the enumerator pure. The value is signal-driven, not a fixed constant. Each follows the repo extension contract (conflict.Analyzer reference shape): Factory.For(Config) (T, error) with Config carrying only QueueName; behavioral knobs and limit signals are integrator-injected at construction. Includes READMEs, gomock packages, and Makefile mock-gen wiring. Interfaces only; concrete impls and controller wiring are deferred. ## Test Plan ## Issues ## Stack 1. #337 1. @ #315 1. #316 1. #317 1. #320 1. #331 1. #332 1. #333

behinddwalls · 2026-07-12T21:07:58Z

🛑 Automatic stack rebase aborted — diff mismatch

The rebase of preetam/int/storage-build-lookup completed without conflicts, but the resulting code diff does not match the original. This means the rebase silently altered code content. The branch was not force-pushed.

Please rebase manually and verify the changes are correct.

Add the scorer seam from the speculation RFC, as a vendor-agnostic extension interface under submitqueue/extension/speculation/pathscorer/. The scorer computes each speculation path's predicted-success score from the current state: the per-batch scores of the path's base batches (entity.Batch.Score) and which of those dependencies have resolved (landed or build-passed), plus optionally other signals. It is a prediction over live state, so the controller re-runs it on every respeculate right after reconciling status, and persists the result; the scorer owns only the formula. The controller hands it the batch's speculation tree directly — the subject it scores. Any richer signal an implementation needs (dependency batch scores, historical pass rates) is injected at its Factory, not put in the signature. It never writes: its only output is per-path scores ([]entity.PathScore — path ID plus fresh score, an entity-level seam-output type alongside the path-decision type), which the controller merges into the tree and persists, staying the single writer of tree state; structure and status never pass through the scorer. Scores are probabilities in [0, 1] — the contract every implementation must satisfy, enforced by the controller on consume. This is the per-path scorer, distinct from the existing per-batch score stage (`extension/scorer`) that sets entity.Batch.Score — the path scorer consumes those to score whole paths. Follows the repo extension contract: Factory.For(Config) (Scorer, error) with Config carrying only QueueName. Includes README, gomock package, and a programmable fake. The speculation RFC's seam descriptions are updated to match the identity-keyed minimal-output contracts (and gain a design-decision entry for assigned path identity). Interface only; concrete impls and controller wiring are deferred. ## Stack 1. #337 1. #315 1. @ #316 1. #317 1. #320 1. #331 1. #332 1. #333

behinddwalls · 2026-07-12T21:08:19Z

🛑 Automatic stack rebase aborted — diff mismatch

The rebase of preetam/int/storage-build-lookup completed without conflicts, but the resulting code diff does not match the original. This means the rebase silently altered code content. The branch was not force-pushed.

Please rebase manually and verify the changes are correct.

Add the selector seam and its limit counterpart from the speculation RFC, as vendor-agnostic extension interfaces under submitqueue/extension/speculation/. selector: the controller hands it the batch's speculation tree; it returns per-path decisions (Promote/Cancel), each naming a path by its ID (entity.SpeculationPathInfo.ID), for the paths it chooses to act on — at most one decision per path, with conflicting duplicates treated by the controller as a policy bug (applied first, logged and skipped after). It is the policy seam — it reads the tree it is given and emits decisions only, never writing status, and does not decide merging. The controller maps each decision to a guarded status transition (Promote → Selected, Cancel → Cancelling/Cancelled) and persists it, staying the single writer of tree state. entity.PathDecision correspondingly names paths by PathID rather than restating the Base/Head split. selectionlimit: the "how much" policy bounding how many paths a batch may build in parallel. It is the selector's companion — the selector decides which paths, the limit decides how many. Injected into the selector at construction and called by it, keeping the selector interface limit-free; the value is dynamic, not a fixed constant. Each follows the repo extension contract: Factory.For(Config) (T, error) with Config carrying only QueueName. Includes READMEs, gomock packages, and programmable fakes. Interfaces only; concrete impls and controller wiring are deferred. ## Stack 1. #337 1. #315 1. #316 1. @ #317 1. #320 1. #331 1. #332 1. #333

behinddwalls · 2026-07-12T21:08:38Z

🛑 Automatic stack rebase aborted — diff mismatch

The rebase of preetam/int/storage-build-lookup completed without conflicts, but the resulting code diff does not match the original. This means the rebase silently altered code content. The branch was not force-pushed.

Please rebase manually and verify the changes are correct.

…320) Add the queue-wide prioritizer seam and its limit counterpart from the speculation RFC, as vendor-agnostic extension interfaces under submitqueue/extension/speculation/. prioritizer: selection is per batch and blind to other batches, so it cannot ration a shared build budget. The prioritizer sees every path across the queue's in-flight batches that runs or wants to run (Selected / Prioritized / Building, each carrying its ID and score), ranks them — scores are probabilities in [0, 1] per the path scorer's contract — and returns sparse decisions naming paths by ID: Promote to admit under the budget, Cancel to preempt, at most one decision per path. Whether it preempts at all is implementation policy (sticky-slots vs preemptive). It never writes: the controller maps each decision back to its tree and applies the status transition under that tree's optimistic lock. Prioritized thus means exactly "admitted under the queue's build budget, cleared to build, not yet building". prioritizationlimit: the "how much" policy bounding the queue's concurrent speculation builds — the budget the prioritizer admits against. Injected into the prioritizer at construction and applied by it, keeping the interface limit-free; the value is signal-driven, not a fixed constant. Each follows the repo extension contract: Factory.For(Config) (T, error) with Config carrying only QueueName. Includes READMEs, gomock packages, and programmable fakes. Interfaces only; concrete impls and controller wiring are deferred. ## Stack 1. #337 1. #315 1. #316 1. #317 1. @ #320 1. #331 1. #332 1. #333

behinddwalls · 2026-07-12T21:08:57Z

🛑 Automatic stack rebase aborted — diff mismatch

The rebase of preetam/int/storage-build-lookup completed without conflicts, but the resulting code diff does not match the original. This means the rebase silently altered code content. The branch was not force-pushed.

Please rebase manually and verify the changes are correct.

Integrating speculation needs the controllers to answer "which build belongs to this path" in both directions, without ever looking a row up by non-key attribute — the storage contract stays get/put-by-key so any KV backend can satisfy it. The build system mints its own build identifiers, and those stay the build store's primary key: the runner's ID is the natural name for the build row. What's missing is a durable link between a tree path and its build that exists independent of any in-flight message, hung on the path identity (SpeculationPathInfo.ID) introduced at the bottom of this stack. `entity.SpeculationPathBuild` is the new mapping entity ({PathID, BuildID, BatchID, CreatedAt, Version}, named after its `speculation_path_build` table) with a `SpeculationPathBuildStore` (Create/Get keyed by PathID): the forward path→build lookup, written only by the build controller, at most one build per path with ErrAlreadyExists making the existing row the truth on races. PathID is globally unique — the path ID contract on SpeculationPathInfo.ID is strengthened accordingly, since this table keys rows by it alone. BatchID makes the row self-describing without parsing PathID's format; Version follows the repo's optimistic-locking convention (write-once today, reserved for future conditional re-pointing flows). `entity.Build` keeps the runner-minted `ID` as its primary key and gains `SpeculationPathID` as a plain column — the reverse build→path lookup; the previously embedded `SpeculationPath` value is dropped as a redundant denormalized copy of what the tree already stores. `SpeculationPath.Equal` (order-sensitive Base + Head) provides structural path identity for the few controller spots where only structure can identify a path (deduplicating enumerator output, carrying entries over across re-enumeration). MySQL gains the `speculation_path_build` table and the build table swaps `speculation_path`/`runner_id` for `speculation_path_id`; schema files are picked up automatically (the schema dir is globbed by both Bazel and the testutil ApplySchema helper). ✅ `bazel test //submitqueue/entity/... //submitqueue/extension/storage/... //submitqueue/orchestrator/...` and the storage integration suite exercises Create/Get round-trips and the duplicate-create race.

…#332) ## Summary ### Why? The speculation seam contracts (enumerator, selector, prioritizer, limits) have no real implementations yet — only mocks and programmable fakes. Integrating the seams into the orchestrator needs defaults that reproduce the existing single-chain behavior exactly (one speculative path per batch: the full dependency chain, promoted and admitted unconditionally), so the pipeline can switch to tree-driven speculation with zero behavior change and the e2e suite staying green. The path scorer — the one seam with a real model rather than a parity stub — is split into its own follow-up change for focused review. ### What? Five impl packages under `submitqueue/extension/speculation/`, each a plain `New(...)` constructor returning the contract interface (no factories, no queue routing — that stays in wiring): - `enumerator/chain` — one path per batch: `{Base: deps in order, Head: batch}`; structure only, controller owns identity/status/score. - `selector/all` — promotes every candidate path, naming each by its path ID. - `prioritizer/sticky` — sticky slots: running/prioritized paths hold their slot, top-scored selected paths fill free budget (decisions by path ID), never preempts; deterministic tie-break for stable rounds. - `dependencylimit/static`, `prioritizationlimit/static` — fixed limits injected at construction. ## Test Plan ✅ `make gazelle && make fmt && bazel test //submitqueue/extension/speculation/...`. Unit coverage: chain ordering (incl. empty deps), candidate-only selection, sticky budget accounting (running paths consume slots, zero-free floor, tie-break determinism across permutations), static limit passthrough. ## Issues ## Stack 1. #331 1. @ #332 1. #333

## Summary ### Why? The pathscorer seam has no real implementation — only mocks and a programmable fake — yet its output is the ranking currency the selector and prioritizer act on. Speculation needs a default scorer whose scores mean something from the start: inert while trees hold a single path, but ranking chain-vs-fallback paths correctly the moment an enumerator produces alternatives. It is split out from the parity-stub impls because it is the one seam implementation with an actual model, worth reviewing on its own. ### What? `pathscorer/probability` — scores each path as the probability that exactly that path's assumption holds: `Score = p(head) × Π p(d) for base deps × Π (1−p(d)) for non-base head deps`, returned as path-ID-keyed `PathScore`s per the seam contract (probabilities in [0, 1] by construction). Each batch's probability resolves from its state, so resolved outcomes override predictions: `succeeded` → 1, `failed`/`cancelled` → 0, otherwise the score stage's predicted `Batch.Score` (the score stage sets it before a batch reaches speculation; best-effort `cancelling` keeps its prediction). A dead dependency therefore zeroes every path built on it and boosts every path that excluded it by the full `(1−p)=1` factor — score mass shifts to the paths consistent with reality, with no cross-path coupling. Folding outcomes into path *status* stays the controller's reconcile job; the scorer only recomputes scores, over every path regardless of status. Dependencies are read one at a time through the injected `storage.BatchStore` per the store's key/value contract, each batch loaded at most once per call; store errors return wrapped and unclassified. The README walks a three-path worked example (chain / drop-B / alone) before and after a dependency failure. ## Test Plan ✅ `make gazelle && make fmt && bazel test //submitqueue/extension/speculation/...`. Unit coverage: probability-formula table including all state-resolution cases (succeeded/failed/cancelled in and out of base, cancelling keeps prediction), a three-path dependency-failure scenario pinning the score-mass shift, per-dep single store read, error propagation, empty-tree short-circuit. ## Issues ## Stack 1. #331 1. #332 1. @ #333

behinddwalls added the ChangesPlanned label Jul 10, 2026

behinddwalls marked this pull request as ready for review July 10, 2026 05:11

behinddwalls requested review from a team and sbalabanov as code owners July 10, 2026 05:11

This was referenced Jul 10, 2026

feat(speculation): add parity default impls for the speculation seams #332

Merged

feat(speculation): add probability path scorer #333

Merged

behinddwalls force-pushed the preetam/ext/speculation-prioritizer branch from f3c68e1 to 0426ccb Compare July 10, 2026 19:24

behinddwalls force-pushed the preetam/int/storage-build-lookup branch from 2551cf4 to bbc4956 Compare July 10, 2026 19:24

behinddwalls force-pushed the preetam/ext/speculation-prioritizer branch from 0426ccb to 55d0b08 Compare July 11, 2026 05:03

behinddwalls force-pushed the preetam/int/storage-build-lookup branch from bbc4956 to e0c6857 Compare July 11, 2026 05:03

behinddwalls mentioned this pull request Jul 11, 2026

feat(entity): give speculation paths an assigned identity #337

Merged

behinddwalls force-pushed the preetam/ext/speculation-prioritizer branch from 55d0b08 to 87f2056 Compare July 11, 2026 05:24

behinddwalls force-pushed the preetam/int/storage-build-lookup branch from e0c6857 to 293ad91 Compare July 11, 2026 05:24

behinddwalls force-pushed the preetam/ext/speculation-prioritizer branch from 87f2056 to 03c3b63 Compare July 12, 2026 21:07

behinddwalls force-pushed the preetam/ext/speculation-prioritizer branch from 03c3b63 to 6f3e86b Compare July 12, 2026 21:07

behinddwalls force-pushed the preetam/ext/speculation-prioritizer branch from 6f3e86b to a2bd5b4 Compare July 12, 2026 21:08

behinddwalls force-pushed the preetam/ext/speculation-prioritizer branch from a2bd5b4 to 5c5dafc Compare July 12, 2026 21:08

behinddwalls changed the base branch from preetam/ext/speculation-prioritizer to main July 12, 2026 21:13

behinddwalls force-pushed the preetam/int/storage-build-lookup branch from 293ad91 to b095758 Compare July 12, 2026 21:13

sbalabanov requested changes Jul 13, 2026

View reviewed changes

behinddwalls force-pushed the preetam/int/storage-build-lookup branch from b095758 to 3feafaa Compare July 13, 2026 22:42

behinddwalls requested a review from sbalabanov July 13, 2026 22:43

albertywu reviewed Jul 13, 2026

View reviewed changes

Comment thread submitqueue/entity/build.go

albertywu approved these changes Jul 13, 2026

View reviewed changes

behinddwalls force-pushed the preetam/int/storage-build-lookup branch from 3feafaa to 9451b7d Compare July 14, 2026 01:24

behinddwalls merged commit c906860 into main Jul 14, 2026
15 checks passed

behinddwalls temporarily deployed to stack-rebase July 14, 2026 01:27 — with GitHub Actions Inactive

behinddwalls deleted the preetam/int/storage-build-lookup branch July 14, 2026 01:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(storage,entity): path identity + speculation path→build mapping store#331

feat(storage,entity): path identity + speculation path→build mapping store#331
behinddwalls merged 1 commit into
mainfrom
preetam/int/storage-build-lookup

behinddwalls commented Jul 10, 2026 •

edited

Loading

Uh oh!

behinddwalls commented Jul 12, 2026

Uh oh!

behinddwalls commented Jul 12, 2026

Uh oh!

behinddwalls commented Jul 12, 2026

Uh oh!

behinddwalls commented Jul 12, 2026

Uh oh!

behinddwalls commented Jul 12, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

behinddwalls commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Why?

What?

Test Plan

Issues

Stack

Uh oh!

behinddwalls commented Jul 12, 2026

Uh oh!

behinddwalls commented Jul 12, 2026

Uh oh!

behinddwalls commented Jul 12, 2026

Uh oh!

behinddwalls commented Jul 12, 2026

Uh oh!

behinddwalls commented Jul 12, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

behinddwalls commented Jul 10, 2026 •

edited

Loading