Add inline-script cache layout, meta.json sidecar, and env-usability guard (PEP 723 PR 2/16)#1635
Open
StellaHuang95 wants to merge 1 commit into
Open
Add inline-script cache layout, meta.json sidecar, and env-usability guard (PEP 723 PR 2/16)#1635StellaHuang95 wants to merge 1 commit into
StellaHuang95 wants to merge 1 commit into
Conversation
StellaHuang95
added
the
feature-request
…ity guard (PEP 723 PR 2/3)
Helpers that resolve the on-disk cache layout for PEP 723 inline-script
envs (under <globalStorageUri>/script-envs-v1/<hash>/), read/write the
.meta.json sidecar that lives at the root of every cached env, and
verify on cache hit that the base interpreter the env was built against
still exists on disk. Second foundation PR; pure utility, no behavior
change on its own.
What is in:
- src/common/inlineScriptCacheLayout.ts
- Path helpers: getScriptEnvCacheRoot, getScriptEnvDir,
getMetaJsonPath. All take globalStorageUri: Uri rather than
ExtensionContext so they are easy to test.
- InlineScriptEnvMeta interface with schemaVersion: 1. Minimal
shape: only fields with a concrete consumer in a planned PR
(scriptFsPath for cache cleanup, lastUsedAt for TTL, optional
requiresPython for cache-hit re-verify). Extras dropped on read
per the v1 evolution policy.
- readMetaJson(envDir) -- never throws. Returns undefined (with a
traceWarn that includes err.code) for: missing file, non-regular
file, oversize file (>1 MiB defensive cap), malformed JSON,
invalid shape, unknown schemaVersion, non-canonical ISO 8601
timestamps. Strict ISO check uses round-trip equality.
- writeMetaJson(envDir, meta) -- atomic temp-file + native
fs.rename (NOT fs-extra move(), which does remove+rename and
has both a no-sidecar window and a crash-loses-data failure
mode).
- CacheEntrySummary + selectStaleEntries(entries, now, ttlMs) --
pure selector for Q7s TTL eviction path. Zero I/O. Entries with
undefined lastUsedAt are never TTLed (deliberate: do not delete
what we do not understand).
- verifyEnvUsable(envDir): Promise<boolean> -- cache-hit guard
that confirms the base interpreter the cached env was built
against still exists on disk. The cache-key path hash detects
"user switched to a different Python" because the path changes;
it does NOT detect "user uninstalled the Python at the cached
path" (the cache directory and .meta.json are unchanged, only
the file at the launcher target is gone). Common triggers:
pyenv/uv python uninstall, brew/apt removal, Add/Remove
Programs.
* POSIX: stat <envDir>/bin/python. fs.stat follows symlinks so
a dead symlink to a removed base throws ENOENT.
* Windows: <envDir>/Scripts/python.exe is a COPY of the base
(not a symlink), so checking it does not help. Read
pyvenv.cfg, parse `home = <dir>`, stat <home>/python.exe.
* Never throws. Returns false + traceWarn(message) on any
failure mode (missing launcher, missing/malformed
pyvenv.cfg, non-regular file at the launcher path, EACCES
etc.). Mirrors readMetaJson's failure policy.
Internal helpers `statRegularFile` (POSIX/Windows shared
stat+isFile+error-tagging) and `parsePyvenvHome` (CRLF + extra
whitespace tolerated).
- META_SCHEMA_VERSION evolution policy documented inline.
- src/test/common/inlineScriptCacheLayout.unit.test.ts -- 44 unit
tests (real-fs, mirroring readMetaJson conventions):
* path-helper shape
* round-trip writeMetaJson/readMetaJson, concurrent writers,
and all rejection paths (ENOENT-specific warn, malformed JSON,
unknown schemaVersion, non-canonical ISO, size cap, etc.)
* selectStaleEntries boundaries (TTL=0, exact-equal age,
undefined lastUsedAt, future timestamps)
* verifyEnvUsable POSIX branch (isWindows stubbed to false):
bin/python exists / missing / is-a-directory; symlink-alive
and dead-symlink cases gated on process.platform !== 'win32'
* verifyEnvUsable Windows branch (isWindows stubbed to true):
home points to existing / removed python.exe, missing
pyvenv.cfg, no home= line, empty home value, whitespace and
CRLF tolerance.
Design context
Implements Q2 (disk location), Q3 (env folder contents + .meta.json
shape), Q4 step 4 (cache-hit interpreter-existence guard), and the
pure selector half of Q7 (TTL eviction) from
pep723_design_questions.md. The disk walk and deletion calls live in
a later PR. The verifyEnvUsable guard was added in response to
reviewer feedback on the design doc.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
StellaHuang95
force-pushed
the
pep723-pr2-cache-layout
branch
from
July 15, 2026 22:16
d352893 to
ee85edb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Roadmap context
This is PR 2 of 16 in the PEP 723 inline-script roadmap. The full plan lives in #1602.
meta.jsonsidecarrequires-pythonto interpreter selectionInlineScriptEnvManagerskeletoncreate()happy pathcreate()uv-install fallbackget,set, and MementoPRs 1-3 are independent Phase 1 utilities that PR 5 will compose when it implements the environment creation path.
Why this PR
Q2 and Q3 of #1601 place inline-script environments under
<globalStorageUri>/script-envs-v1/<cache-key>/and give each environment a.meta.jsonsidecar for lifecycle bookkeeping. Those contracts need to exist before the manager can create, discover, reuse, or clean up cached environments.This PR also implements the cache-hit usability guard from Q4 step 4. Hashing the selected interpreter path catches a switch to a different Python path, but it does not catch an interpreter being uninstalled from the same path. The guard detects that stale cache state before the extension attempts to reuse a broken environment.
The filesystem concerns are kept separate from PR 1's pure hashing logic and PR 3's interpreter-selection logic.
What this PR does
Adds
src/common/inlineScriptCacheLayout.tswith cache path helpers, typed sidecar I/O, stale-entry selection, and an environment-usability check.Cache layout
getScriptEnvCacheRoot(globalStorageUri)returns<globalStorageUri>/script-envs-v1/.getScriptEnvDir(globalStorageUri, cacheKey)returns the directory for one cached environment.getMetaJsonPath(envDir)returns<envDir>/.meta.json.INLINE_SCRIPT_CACHE_DIR_NAME,META_JSON_FILENAME, andMETA_SCHEMA_VERSIONcentralize the on-disk contract.The cache directory and metadata schema are both versioned. An incompatible future format should bump the cache directory suffix and schema version together rather than migrate environments in place.
Typed
.meta.jsonsidecarInlineScriptEnvMetacontains the fields with concrete downstream consumers:schemaVersion: validates the sidecar format.scriptFsPath: identifies the owning script for lifecycle cleanup.lastUsedAt: drives TTL eviction.requiresPython?: supports compatibility revalidation on cache hits.readMetaJson(envDir)never throws. It returnsundefinedwith a warning for missing or non-regular files, files larger than 1 MiB, malformed JSON, unknown schema versions, invalid field types, and non-canonical ISO timestamps. Unknown properties are dropped when the validated object is constructed.writeMetaJson(envDir, meta)ensures the directory exists, writes formatted JSON to a randomized sibling temporary file, and atomically renames it to.meta.json. If writing or renaming fails, it attempts to remove the temporary file and rethrows the original error.TTL selection
selectStaleEntries(entries, now, ttlMs)is a pure selector used by the future opportunistic cleanup path.lastUsedAtare retained because the extension should not delete data it cannot classify confidently.The function returns paths to delete; it performs no filesystem walk or deletion itself.
Cache-hit usability guard
verifyEnvUsable(envDir)checks whether the cached environment still has a usable base interpreter. It never throws; failures produce a warning and returnfalse.<envDir>/bin/python. Becausefs.statfollows symlinks, a dead launcher symlink is detected.home = ...frompyvenv.cfgand stats<home>/python.exe. Checking<envDir>/Scripts/python.exewould be insufficient because that launcher can remain after the base Python is uninstalled.This covers common removal paths such as
pyenv uninstall,uv python uninstall, Homebrew or apt removal, and Windows Add/Remove Programs.Failure policy
The helpers distinguish recoverable cache state from creation-time failures:
undefinedorfalserather than throwing, allowing callers to treat invalid cache state as a miss.Tests
Adds
src/test/common/inlineScriptCacheLayout.unit.test.tswith 44 focused unit tests covering:lastUsedAtvalues.pyvenv.cfgparsing and missing base interpreters.Required checks pass on this branch:
npm run lintnpm run compile-testsnpm run unittest: 1418 passing, 0 failing, 4 pendingUser impact
Zero.
Nothing in the extension calls these helpers yet. This PR creates no environments, writes nothing to global storage during activation, deletes no cache entries, and adds no settings, commands, UI, or logging on the default execution path.
PR 5 will use the layout and sidecar helpers during creation. PRs 7 and 8 will use them during reuse and activation-time discovery. PRs 13 and 14 will use the cleanup helpers.
Design context
This implements Q2 (disk location), Q3 (environment contents and sidecar metadata), Q4 step 4 (base-interpreter existence guard), and the pure selection portion of Q7 (TTL cleanup) from #1601.
The base-interpreter guard was added in response to review feedback on the design PR.