Skip to content

Add inline-script cache key hash utility (PEP 723 PR 1/16)#1634

Open
StellaHuang95 wants to merge 1 commit into
microsoft:mainfrom
StellaHuang95:pep723-pr1-cache-hash
Open

Add inline-script cache key hash utility (PEP 723 PR 1/16)#1634
StellaHuang95 wants to merge 1 commit into
microsoft:mainfrom
StellaHuang95:pep723-pr1-cache-hash

Conversation

@StellaHuang95

Copy link
Copy Markdown
Contributor

Part of #1602 (PEP 723 inline script env support). Design doc: #1601.

Roadmap context

This is PR 1 of 16 in the PEP 723 inline-script roadmap. The full plan lives in #1602.

Phase PR Status
Phase 1: Foundation PR 1: cache key hash utility this PR
PR 2: cache layout + meta.json sidecar in progress
PR 3: requires-python to interpreter selection in progress
Phase 2: Manager PR 4: InlineScriptEnvManager skeleton merged (#1610)
PR 5: create() happy path not started (needs 1, 2, 3, 4)
PR 6: create() uv-install fallback not started (needs 3, 5)
PR 7: persistence with get, set, and Memento in progress (needs 4)
PR 8: activation-time discovery not started (needs 2, 4, 7)
Phase 3: Routing PR 9: route PEP 723 scripts to the inline manager not started (needs 4, 7)
PR 10: per-script project registration not started (needs 9)
Phase 3.5: Cross-repo PRs 17-19: Pylance and vscode-python integration not started
Phase 4: UX PR 11: picker item; PR 12: bulk command not started
Phase 5: Lifecycle and polish PR 13: clear cache; PR 14: TTL; PR 15: telemetry; PR 16: status bar not started

PRs 1-3 are independent Phase 1 utilities that PR 5 will compose when it implements the environment creation path.

Why this PR

The cache key decides whether a PEP 723 script reuses an existing environment or builds a fresh one. Under the dependency-keyed design from Q4 of #1601, scripts using the same dependencies and interpreter should share an environment.

Trivial metadata edits such as changing package-name casing, separator style, whitespace, dependency order, or extras order should not fragment the cache.

This PR isolates that logic as pure functions with no filesystem access, global storage dependency, or extension activation behavior. The on-disk cache layout remains separate in PR 2.

What this PR does

Adds src/common/inlineScriptCacheKey.ts with four exports:

  1. normalizeDependency(dep)

    Canonicalizes common variants of the same requirement:

    • Applies PEP 503 normalization to project names and extras: lowercase and collapse runs of ., _, and - to a single -.
    • Sorts and deduplicates extras after normalization.
    • Removes whitespace around version comparison operators.
    • Preserves meaningful version and marker content.
    • Drops empty dependency entries when keys are computed.

    Examples:

    • Django and django normalize identically.
    • Flask-Login, flask_login, and flask.login normalize identically.
    • requests[Socks,security] becomes requests[security,socks].
    • requests < 3 becomes requests<3.

    This is intentionally not a complete PEP 508 parser. Inputs it cannot meaningfully parse remain deterministic without attempting semantic normalization.

  2. normalizeInterpreterPath(interpreterPath)

    Uses the existing normalizePath helper so Windows path casing and separator differences do not fragment the cache. POSIX path casing remains unchanged.

  3. computeCacheKey({ dependencies, interpreterPath })

    • Normalizes, deduplicates, and sorts dependencies.
    • Normalizes the interpreter path.
    • Builds a versioned, labelled payload.
    • Hashes the payload with SHA-256.
    • Returns the first 16 lowercase hexadecimal characters.

    The resulting key is deterministic and safe to use as a directory name under the cache root introduced by PR 2.

  4. CACHE_KEY_HEX_LENGTH

    Exposes the key length so callers and tests do not duplicate the value 16.

Caller contract

interpreterPath must be absolute, trimmed, and already resolved through symlinks, for example with fs.realpath().

The utility deliberately performs no I/O. Two different path strings that resolve to the same executable will therefore produce different keys unless the caller canonicalizes them first. The creation flow in PR 5 will own that resolution.

Cache behavior

The script path and raw requires-python value are intentionally not included directly in the key.

  • Two scripts with equivalent dependencies and the same selected interpreter share an environment.
  • Moving or renaming a script does not invalidate its environment.
  • Adding, removing, or repinning a dependency produces a new key.
  • Selecting a different interpreter produces a new key.
  • A later PR re-verifies requires-python on cache hits in case the constraint changes without changing the selected interpreter.

This supports the design's two outcomes: reuse an unchanged cache entry or build a fresh environment. There is no in-place dependency synchronization path.

Tests

Adds src/test/common/inlineScriptCacheKey.unit.test.ts with 37 unit tests covering:

  • PEP 503 project-name normalization.
  • Extras normalization, sorting, deduplication, and whitespace handling.
  • Version comparison operators and multi-clause specifiers.
  • Empty and duplicate dependency entries.
  • Dependency ordering and deterministic output.
  • Inputs that should and should not change the key.
  • Windows and POSIX interpreter-path behavior.
  • Fixed-length lowercase hexadecimal output.
  • Filesystem-safe output.
  • Pinned caller-contract behavior such as trailing whitespace in interpreter paths.

Platform behavior is stubbed through platformUtils, keeping the suite independent of the host operating system.

User impact

Zero.

Nothing in the extension calls these helpers yet. This PR adds no settings, commands, UI, logging, filesystem writes, or activation behavior. The utility is wired into environment creation in PR 5.

Design context

This implements the cache-key portion of Q4 in #1601: a pipx-style cache keyed by normalized dependencies and the selected interpreter.

PEP 503 normalization for project names and extras was added in response to review feedback on the design PR.

Pure utility for computing a deterministic cache key for a PEP 723
inline-script environment, given the script's declared dependencies
and the chosen Python interpreter path. First foundation PR for
inline-script env support; no behavior impact on its own -- nothing
in the extension calls these helpers yet.

What is in:

- src/common/inlineScriptCacheKey.ts
  - normalizeDependency(dep) -- folds common variants of the same
    PEP 723 requirement so trivial edits don't fragment the cache:
      * PEP 503 name normalization on the project name AND on each
        extra: lowercase, then collapse runs of [._-] to a single -.
        So `Django` ≡ `django`, `Flask-Login` ≡ `flask_login`,
        `requests[Socks]` ≡ `requests[socks]`,
        `pkg[socks_5]` ≡ `pkg[socks-5]`.
      * Extras are deduplicated and sorted alphabetically after
        canonicalization, so `requests[security,socks]` and
        `requests[socks,security]` hash to the same key.
      * Whitespace around comparison operators is stripped, so
        `requests <3` ≡ `requests<3`.
    Not a full PEP 508 parser -- only folds the superficial edits
    users are likely to make. Pinned-behavior tests document what
    is and isn't folded.
  - normalizeInterpreterPath(path) -- wraps the existing
    normalizePath helper so case/separator differences on Windows
    do not fragment the cache. POSIX paths pass through unchanged.
  - computeCacheKey({ dependencies, interpreterPath }) -- normalizes
    and dedupes deps, sorts, builds a versioned labelled-line
    payload, hashes with SHA-256, and truncates to 16 hex characters
    (64 bits -- well below practical collision risk for a per-user
    cache).

- src/test/common/inlineScriptCacheKey.unit.test.ts -- 37 unit tests
  covering the three exports, including pinned-behavior tests for
  asymmetries the docstrings call out (trailing whitespace in paths
  fragments) and explicit canonicalization tests for extras (PEP 503
  folding, sort, dedup, whitespace tolerance, empty-bracket collapse,
  composition with version specifiers).

Caller contract

The interpreter path must be absolute and symlink-resolved before
calling computeCacheKey. The function does no I/O, so symlink
canonicalization is the caller's responsibility -- a later PR will
do that at the resolution site.

Design context

Implements the cache-key half of PR 1 in the Phase 1 roadmap of
pep723_design_questions.md. Deps-keyed + interpreter-path-keyed
matches the pipx-style choice in Q4. The PEP 503 extras handling
was added in response to reviewer feedback on the design doc.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@StellaHuang95 StellaHuang95 added the feature-request Request for new features or functionality label Jul 14, 2026
@brettcannon

Copy link
Copy Markdown
Member

The normalization approach SGTM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature-request Request for new features or functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants