watch mode test errors on test-osuosl-aix72-ppc64_be-3

[richardlau]

Builds on test-osuosl-aix72-ppc64_be-3 are consistently failing FS watch tests.

e.g. https://ci.nodejs.org/job/node-test-commit-aix/nodes=aix72-ppc64/56388/consoleFull

12:56:50 not ok 2167 parallel/test-fs-watch-recursive-delete
12:56:50   ---
12:56:50   duration_ms: 120141.74400
12:56:50   severity: fail
12:56:50   exitcode: -15
12:56:50   stack: |-
12:56:50     timeout
12:56:50   ...
...
13:12:56 not ok 4358 sequential/test-fs-watch
13:12:56   ---
13:12:56   duration_ms: 204.98800
13:12:56   severity: fail
13:12:56   exitcode: 1
13:12:56   stack: |-
13:12:56     node:internal/fs/watchers:255
13:12:56         throw error;
13:12:56         ^
13:12:56     
13:12:56     Error: ENOSPC: System limit for number of file watchers reached, watch '/home/iojs/build/.tmp.4357/watch.txt'
13:12:56         at FSWatcher.<computed> (node:internal/fs/watchers:247:19)
13:12:56         at Object.watch (node:fs:2549:36)
13:12:56         at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-aix/nodes/aix72-ppc64/test/sequential/test-fs-watch.js:61:22)
13:12:56         at Module._compile (node:internal/modules/cjs/loader:1734:14)
13:12:56         at Object..js (node:internal/modules/cjs/loader:1899:10)
13:12:56         at Module.load (node:internal/modules/cjs/loader:1469:32)
13:12:56         at Function._load (node:internal/modules/cjs/loader:1286:12)
13:12:56         at TracingChannel.traceSync (node:diagnostics_channel:322:14)
13:12:56         at wrapModuleLoad (node:internal/modules/cjs/loader:235:24)
13:12:56         at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:151:5) {
13:12:56       errno: -28,
13:12:56       syscall: 'watch',
13:12:56       code: 'ENOSPC',
13:12:56       path: '/home/iojs/build/.tmp.4357/watch.txt',
13:12:56       filename: '/home/iojs/build/.tmp.4357/watch.txt'
13:12:56     }
13:12:56     
13:12:57     Node.js v24.0.0-pre
13:12:57   ...
13:16:13 not ok 4429 sequential/test-watch-mode # TODO : Fix flaky test
13:16:13   ---
13:16:13   duration_ms: 120059.16100
13:16:13   severity: fail
13:16:13   exitcode: -15
13:16:13   stack: |-
13:16:13     timeout
...

[richardlau]

I'll take the machine offline in Jenkins for now.

[richardlau]

Going to try rebooting the machine.

[richardlau]

hmm. Machine rebooted but is now missing /home (i.e. it didn't mount the disk and /home is empty)😞.

[richardlau]

root@nodejs03:[/root]mount /home
Replaying log for /dev/hd1.
mount: /dev/hd1 on /home: Unformatted or incompatible media
The superblock on /dev/hd1 is dirty.  Run a full fsck to fix.
root@nodejs03:[/root]

👀

[richardlau]

Can't seem to run fsck as suggested:

root@nodejs03:[/root]fsck /dev/hd1



The current volume is: /dev/hd1
Primary superblock is valid.
Inode 16 has references to cross linked blocks.
The extent descriptors for inodes 2129024 through 2129055 are invalid.
        Inode Allocation Map 16,
        Inode Allocation Group 519,
        Extent Index 100
Errors detected in the file system inode allocation map.
fsck: 0507-278 Cannot continue.
File system is dirty.
root@nodejs03:[/root]