Environment
- t3 0.0.29-nightly.20260705.729 (
apps/server, npx-installed, launched via t3's own ssh-launch mechanism — the t3-generated ~/.t3/ssh-launch/<id>/run-t3.sh — on a Linux dev machine: veLinux 2 / Debian-based, kernel 5.15.152, 32 vCPU / 62.6 GiB RAM, no swap)
- Node via mise shims (engine range
^22.16 || ^23.11 || >=24.10)
- Server runs detached 24/7;
run-t3.sh performs no cd, so process.cwd() = the login home directory
Behavior observed
The provider status/refresh loop (checkClaudeProviderStatus → runClaudeCommand / makeClaudeEnvironment, probeCodexAppServerProvider) runs every 301–305 s, continuously, whether or not any client is connected. Per cycle it spawns, among others, a Claude capability/auth probe:
claude --output-format stream-json --verbose --input-format stream-json \
--setting-sources=user,project,local --permission-mode default --no-session-persistence
Two properties of this probe are problematic:
--setting-sources=user,... loads the user's global ~/.claude.json, including mcpServers — so every probe boots every user-configured stdio MCP server (in our case serena, a Python language-server frontend), just to check provider capability. That's ≈285 cold-starts per configured MCP server per day as a side effect of health checking. (The claude --version probe and the codex app-server probe do NOT load MCP servers — only the capability probe does; source-verified in the t3 dist bundle.)- cwd is inherited from the t3 server =
$HOME — MCP servers that key off the working directory (serena's --project-from-cwd) treat the entire home directory as their project.
Impact on our host
serena v1.5.3 has a bug (reported upstream separately: oraios/serena#1683 — a pre-stdio project auto-generation scan: a recursive symlink-following os.scandir walk with per-entry gitignore matching, minutes-long on large trees — ~200 s measured on our $HOME) that leaves serena blind to its client's exit until the scan completes. Our home directory is itself a git repository, so serena adopted the entire $HOME as its project root; at a ~300 s probe cadence, scanners accumulated once per-scan duration exceeded the probe interval under the growing load. Result: ≈84% of capability probes were followed by a surviving ~100%-CPU serena orphan (≈76 probes between the MCP-config change at 00:43 and the 07:09 snapshot; 66 surviving orphans, 64 of them born within 2 s of a probe).
Forensic correlation (atop process accounting, 10 s resolution): 64 of 66 surviving orphans were born within 2 seconds of a t3 Claude probe span (6 same-second, 38 at +1 s, 20 at +2 s). Over ~21 unattended hours the fleet reached 167 orphans (peak observation) consuming ~46 GiB RSS combined; the host (62.6 GiB usable RAM, no swap) froze and required a reboot.
While the runaway accumulation stems from serena's pre-stdio blindness, t3's probe design is what turned it into an outage on this host — an attribution inferred from the 2 s birth correlation and the source-verified spawn chain above — and it is heavyweight even when MCP servers are well-behaved.
Suggested directions
- Probe with MCP loading disabled:
--strict-mcp-config with an empty --mcp-config, or restrict --setting-sources so user-level mcpServers are not booted for a health check.
- Run probes from a dedicated benign cwd (e.g. an empty directory under
~/.t3), not the server's inherited cwd.
- Consider caching capability results and/or pausing probes while no client is connected (the server probed all weekend with, as far as we can tell, no client attached — inferred from the client machine's power logs and persisted session state, not from t3's own logs).
- Consider
PR_SET_PDEATHSIG/job-object semantics for any child the probe spawns, so probe children do not outlive the probe (subject to pdeathsig's direct-parent semantics and setup race).
Evidence available
~/.t3/userdata/logs/server.trace.ndjson* span counts: 92 Claude checks between 23:21:30 and 07:00:47 (≈7.7 h), i.e. every 301–305 s, all exit: Success- atop raw logs correlating probe spans to orphan birth times
- The probe argv recovered from the dist bundle and matched against live process records
- Public-source confirmation at tag
v0.0.29-nightly.20260705.729: capability probe uses settingSources: ["user","project","local"] and persistSession: false (apps/server/src/provider/Layers/ClaudeProvider.ts:599-602, spawned via @anthropic-ai/claude-agent-sdk query() with streaming input — matching the recovered stream-json argv); ClaudeProvider passes no cwd option, so the SDK child inherits the server's cwd, while the codex probe passes cwd: process.cwd() explicitly (CodexProvider.ts:504); the refresh interval is Duration.minutes(5) with a 5-minute probe cache TTL (ClaudeDriver.ts:60-61), driven by an unconditional Effect.forever loop with no connected-client gate (makeManagedServerProvider.ts:141-146); the generated run-t3.sh is pure exec with no cd (packages/ssh/src/tunnel.ts:413ff)
Related: the serena-side report of the same incident is oraios/serena#1683.
Environment
apps/server, npx-installed, launched via t3's own ssh-launch mechanism — the t3-generated~/.t3/ssh-launch/<id>/run-t3.sh— on a Linux dev machine: veLinux 2 / Debian-based, kernel 5.15.152, 32 vCPU / 62.6 GiB RAM, no swap)^22.16 || ^23.11 || >=24.10)run-t3.shperforms nocd, soprocess.cwd()= the login home directoryBehavior observed
The provider status/refresh loop (
checkClaudeProviderStatus→runClaudeCommand/makeClaudeEnvironment,probeCodexAppServerProvider) runs every 301–305 s, continuously, whether or not any client is connected. Per cycle it spawns, among others, a Claude capability/auth probe:Two properties of this probe are problematic:
--setting-sources=user,...loads the user's global~/.claude.json, includingmcpServers— so every probe boots every user-configured stdio MCP server (in our case serena, a Python language-server frontend), just to check provider capability. That's ≈285 cold-starts per configured MCP server per day as a side effect of health checking. (Theclaude --versionprobe and thecodex app-serverprobe do NOT load MCP servers — only the capability probe does; source-verified in the t3 dist bundle.)$HOME— MCP servers that key off the working directory (serena's--project-from-cwd) treat the entire home directory as their project.Impact on our host
serena v1.5.3 has a bug (reported upstream separately: oraios/serena#1683 — a pre-stdio project auto-generation scan: a recursive symlink-following
os.scandirwalk with per-entry gitignore matching, minutes-long on large trees — ~200 s measured on our$HOME) that leaves serena blind to its client's exit until the scan completes. Our home directory is itself a git repository, so serena adopted the entire$HOMEas its project root; at a ~300 s probe cadence, scanners accumulated once per-scan duration exceeded the probe interval under the growing load. Result: ≈84% of capability probes were followed by a surviving ~100%-CPU serena orphan (≈76 probes between the MCP-config change at 00:43 and the 07:09 snapshot; 66 surviving orphans, 64 of them born within 2 s of a probe).Forensic correlation (atop process accounting, 10 s resolution): 64 of 66 surviving orphans were born within 2 seconds of a t3 Claude probe span (6 same-second, 38 at +1 s, 20 at +2 s). Over ~21 unattended hours the fleet reached 167 orphans (peak observation) consuming ~46 GiB RSS combined; the host (62.6 GiB usable RAM, no swap) froze and required a reboot.
While the runaway accumulation stems from serena's pre-stdio blindness, t3's probe design is what turned it into an outage on this host — an attribution inferred from the 2 s birth correlation and the source-verified spawn chain above — and it is heavyweight even when MCP servers are well-behaved.
Suggested directions
--strict-mcp-configwith an empty--mcp-config, or restrict--setting-sourcesso user-levelmcpServersare not booted for a health check.~/.t3), not the server's inherited cwd.PR_SET_PDEATHSIG/job-object semantics for any child the probe spawns, so probe children do not outlive the probe (subject to pdeathsig's direct-parent semantics and setup race).Evidence available
~/.t3/userdata/logs/server.trace.ndjson*span counts: 92 Claude checks between 23:21:30 and 07:00:47 (≈7.7 h), i.e. every 301–305 s, allexit: Successv0.0.29-nightly.20260705.729: capability probe usessettingSources: ["user","project","local"]andpersistSession: false(apps/server/src/provider/Layers/ClaudeProvider.ts:599-602, spawned via@anthropic-ai/claude-agent-sdkquery()with streaming input — matching the recovered stream-json argv); ClaudeProvider passes nocwdoption, so the SDK child inherits the server's cwd, while the codex probe passescwd: process.cwd()explicitly (CodexProvider.ts:504); the refresh interval isDuration.minutes(5)with a 5-minute probe cache TTL (ClaudeDriver.ts:60-61), driven by an unconditionalEffect.foreverloop with no connected-client gate (makeManagedServerProvider.ts:141-146); the generatedrun-t3.shis pure exec with nocd(packages/ssh/src/tunnel.ts:413ff)Related: the serena-side report of the same incident is oraios/serena#1683.