test-fs-watchfile failure on OSX - intermittent ?

[mhdawson]

• Version: master
• Platform: OSX
• Subsystem: fs

Failure in CI testing PR with changes unrelated to fs watch

https://ci.nodejs.org/job/node-test-commit-osx/10086/nodes=osx1010/console

not ok 664 parallel/test-fs-watchfile
 ---
 duration_ms: 60.45
 severity: fail
 stack: |-
 timeout
 ...

[mhdawson]

stress run of 200 https://ci.nodejs.org/job/node-stress-single-test/1234/

[Trott]

I can replicate this locally with this command:

tools/test.py --repeat 100 -j 64 test/parallel/test-fs-watchfile.js 

It doesn't trigger failures every time, but often enough that I haven't had to run it more than 5 times to get a failure yet.

This suggests that the issue is triggered by high load. Simple solution could be to move the test to sequential so it is not competing for resources with other tests.

One curiosity: When it times out with the above command line, it is always starting after the 36th or 37th successful run. Not sure what that means. But maybe someone more macOS savvy would. @nodejs/platform-macos

[refack]

Maybe #13111

[refack]

Probably my fault, I said it would work without common.refreshTmpDir()

[refack]

Have a PR borrowing

[chris--young]

@refack you had mentioned that having common.refreshTmpDir() in there could cause the test to fail on Windows. Is there anyway to avoid that?

Edit: After a quick test, it appears that this is still reproducible even after adding common.refreshTmpDir(). Still could be related to my changes though.

[refack]

Yeah, make sure that you rmdir before.
Did it in #13251

[Trott]

I could be wrong, but I don't think removing files plays a part here.

I think the issue is that (on macOS at least) this is a race condition:

 fs.watch(dir, common.mustCall(function(eventType, filename) {
 this._handle.close();
 assert.strictEqual(filename, 'foo.txt');
 }));

 fs.writeFile(`${dir}/foo.txt`, 'foo', common.mustCall(function(err) {
 if (err) assert.fail(err);
 }));

My recollection is that (on macOS at least) you are not guaranteed to start receiving events write after fs.watch() returns.

Wrapping the fs.writeFile() in a setTimeout() for 100ms makes the test reliable on my machine. I'm going to test more, but if it holds up, I'll experiment a bit more with process.nextTick(), setImmediate(), etc. before opening a PR.