Investigate flaky test-fs-stat-bigint on macos M1

[ruyadorno]

Test

test-fs-stat-bigint

Platform

macOS ARM64

Console output

node:internal/process/promises:289
            triggerUncaughtException(err, true /* fromPromise */);
            ^

AssertionError [ERR_ASSERTION]: Number version atimeMs = 1669309618654.8481, BigInt version atimeMs = 1669309618653n, Allowable delta = 1
    at verifyStats (/Users/ruyadorno/Documents/ruyadorno/node/test/parallel/test-fs-stat-bigint.js:71:7)
    at /Users/ruyadorno/Documents/ruyadorno/node/test/parallel/test-fs-stat-bigint.js:213:3 {
  generatedMessage: false,
  code: 'ERR_ASSERTION',
  actual: false,
  expected: true,
  operator: '=='
}

Build links

• Seen locally, reproduced during tests in preparation for a v19.x release, will update with a build link in case it happens during CI builds

Additional information

Failing consistently when running about 50 times in parallel execution, e.g:

hyperfine -m 50 "out/Release/node test/parallel/test-fs-stat-bigint.js"

[targos]

Is it specific to v19.x?

I cannot reproduce on main:

$ ./tools/test.py parallel/test-fs-stat-bigint --repeat 1000 -j 50
[00:10|% 100|+ 1000|-   0]: Done

[marco-ippolito]

This also happens to me today on macos m1 version 13.0 running the tests on main.

command:
./tools/test.py parallel/test-fs-stat-bigint --repeat 1000 -j 50

Node.js v20.0.0-pre
Command: out/Release/node /Users/marcoippolito/Documents/projects/forks/node/test/parallel/test-fs-stat-bigint.js
=== release test-fs-stat-bigint ===                   
Path: parallel/test-fs-stat-bigint
node:assert:400
    throw err;
    ^

AssertionError [ERR_ASSERTION]: Number version atimeMs = 1669634672268.1, BigInt version atimeMs = 1669634672248n, Allowable delta = 20
    at verifyStats (/Users/marcoippolito/Documents/projects/forks/node/test/parallel/test-fs-stat-bigint.js:71:7)
    at /Users/marcoippolito/Documents/projects/forks/node/test/parallel/test-fs-stat-bigint.js:158:7
    at /Users/marcoippolito/Documents/projects/forks/node/test/common/index.js:448:15
    at FSReqCallback.oncomplete (node:fs:199:5) {
  generatedMessage: false,
  code: 'ERR_ASSERTION',
  actual: false,
  expected: true,
  operator: '=='
}

[marco-ippolito]

it seems that both these assertions fail:

node/test/parallel/test-fs-stat-bigint.js

Lines 70 to 81 in 3bef549

	assert(
	msFromNum - Number(msFromBigInt) <= allowableDelta,
	`Number version ${key} = ${msFromNum}, ` +
	`BigInt version ${key} = ${msFromBigInt}n, ` +
	`Allowable delta = ${allowableDelta}`);
	assert(
	msFromNum - Number(msFromBigIntNs) <= allowableDelta,
	`Number version ${key} = ${msFromNum}, ` +
	`BigInt version ${nsKey} = ${nsFromBigInt}n` +
	` = ${msFromBigIntNs}ms, Allowable delta = ${allowableDelta}`);

the result of this operations fail inconsistently, exceeding the allowableDelta :
msFromNum - Number(msFromBigIntNs) <= allowableDelta
msFromNum - Number(msFromBigInt) <= allowableDelta
49.189697265625 <= 49

[jasnell]

I was able to reproduce the error in about 1% of local runs. With the change in #45925 it appeared to become less flaky locally but really difficult to say for sure.

[ruyadorno]

@jasnell just checked out #45925 locally and tried running the same thing, it's much less recurrent to get a failure now but it still does fail from time to time 😞

[jasnell]

grr, ok. hmm... the only thing I can think of is artificially setting the allowableDelta higher. What I did notice is that it would fail when the delta was either 0 or 1.

[marco-ippolito]

@jasnell I tried to increase the allowableDelta and it still fails

[jazelly]

I guess this is particularly on M1, since no one complains the CI for this test?

I noticed this on local m1 env as well. The flakiness rate is pretty high, that I can even manually reproduce it by repeatedly running the test (and hence not related to parallel FWIW).

If I do it for 1000 times. it's about 15%.

$ tools/test.py --repeat=1000 test/parallel/test-fs-stat-bigint.js
[00:33|% 100|+ 854|- 146]: Done

[KunalKumar-1]

I reporoduced it today and i guess with the latest version of macos Version 15.0.1 on M1 it passes All

Darwin Kunals-MacBook-Air-M1.local 24.0.0 Darwin Kernel Version 24.0.0: Tue Sep 24 23:36:26 PDT 2024; root:xnu-11215.1.12~1/RELEASE_ARM64_T8103 arm64

$  ./tools/test.py parallel/test-fs-stat-bigint --repeat 1000 -j 50
[00:20|% 100|+ 1000|-   0]: Done

All tests passed.

---

$ tools/test.py --repeat=1000 test/parallel/test-fs-stat-bigint.js
[00:22|% 100|+ 1000|-   0]: Done

All tests passed.