Repository navigation
IBM i CI failures #43509
Copy link
Copy link
Closed
Labels
ibm iIssues and PRs related to the IBM i platform.Issues and PRs related to the IBM i platform.
Description
It looks like the IBM CI has been red for as long as we've got history: https://ci.nodejs.org/job/node-test-commit-ibmi/
From the latest CI run, https://ci.nodejs.org/job/node-test-commit-ibmi/798/#showFailuresLink, filtering out the tests already marked flaky we have these failures:
- test.parallel/test-fs-readSync-position-validation test: allow EOVERFLOW errors in fs position tests #43510
- test.parallel/test-icu-env
- test.parallel/test-fs-read-position-validation test: allow EOVERFLOW errors in fs position tests #43510
- test.parallel/test-v8-serialize-leak test: skip test-v8-serialize-leak on IBM i #43511
test.parallel/test-fs-readSync-position-validation
node:fs:747
handleErrorFromBinding(ctx);
^
Error: EOVERFLOW: value too large for defined data type, read
at Object.readSync (node:fs:747:3)
at testValid (file:///home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi73-ppc64/test/parallel/test-fs-readSync-position-validation.mjs:20:8)
at file:///home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi73-ppc64/test/parallel/test-fs-readSync-position-validation.mjs:60:3
at ModuleJob.run (node:internal/modules/esm/module_job:197:25)
at async Promise.all (index 0)
at async ESMLoader.import (node:internal/modules/esm/loader:549:24)
at async loadESM (node:internal/process/esm_loader:83:5)
at async handleMainPromise (node:internal/modules/run_main:65:12) {
errno: -127,
syscall: 'read',
code: 'EOVERFLOW'
}
Node.js v19.0.0-pre
test.parallel/test-icu-env
timeout
test.parallel/test-fs-read-position-validation
node:internal/process/esm_loader:89
internalBinding('errors').triggerUncaughtException(
^
[Error: EOVERFLOW: value too large for defined data type, read] {
errno: -127,
code: 'EOVERFLOW',
syscall: 'read'
}
Node.js v19.0.0-pre
test.parallel/test-v8-serialize-leak
node:assert:399
throw err;
^
AssertionError [ERR_ASSERTION]: before=0 after=0
at Object.<anonymous> (/home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi73-ppc64/test/parallel/test-v8-serialize-leak.js:21:3)
at Module._compile (node:internal/modules/cjs/loader:1116:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1170:10)
at Module.load (node:internal/modules/cjs/loader:994:32)
at Module._load (node:internal/modules/cjs/loader:834:12)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
at node:internal/main/run_main_module:17:47 {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '=='
}
Node.js v19.0.0-pre
cc @nodejs/platform-ibmi
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
ibm iIssues and PRs related to the IBM i platform.Issues and PRs related to the IBM i platform.
Activity
richardlau commentedon Jun 20, 2022
These two are variations on #36925 (comment). I'll open a PR to add
EOVERFLOWto the allowed list of errors.richardlau commentedon Jun 20, 2022
I suspect this one is that IBM i doesn't (cannot?) report memory usage: #32043 (comment)
richardlau commentedon Jun 20, 2022
@V-for-Vasili, since you volunteered to look, please take a look at test-icu-env on IBM i. It was added back in May by #42683 and appears to be timing out on IBM i, e.g. https://ci.nodejs.org/job/node-test-commit-ibmi/798/nodes=ibmi73-ppc64/testReport/junit/(root)/test/parallel_test_icu_env/
test: allow EOVERFLOW errors in fs position tests
V-for-Vasili commentedon Jun 22, 2022
Looks like the whole test takes about 3m to complete on IBMi compared to about 11seconds on x64 linux.
All test assertions take comparable time to complete (there is not one that is significantly longer then the rest).
I think marking it SLOW could be an acceptable solution here.Does not look like SLOW is an option for Release build, PR to skip: #43559
test: skip test-v8-serialize-leak on IBM i
richardlau commentedon Jun 24, 2022
I edited the IBM i Jenkins job to increase the test timeout from the default 2 mins to 5 mins (via
TEST_CI_ARGS="-t 300").test-icu-envpassed in https://ci.nodejs.org/job/node-test-commit-ibmi/803/nodes=ibmi73-ppc64/#showFailuresLink 🎉It looks like we've gained two new failures in
V-for-Vasili commentedon Jun 24, 2022
I assume this is not needed then #43559 if we intend to keep the timeout 5minutes on ibmi?
richardlau commentedon Jun 24, 2022
Yeah, sorry about that. I'd be happy to keep the timeout on IBM i at 5 minutes (or even extend further if we find it necessary to do so).
richardlau commentedon Jun 29, 2022
The two new failures are problems opening a test fixture:
Details
richardlau commentedon Jun 29, 2022
I logged into the machine and was able to see that, for some reason, one of the test fixtures was not owned by the
iojsuser running the tests:I'm not entirely sure how that has happened -- I've changed the owner of that file back to
iojsand re-running: https://ci.nodejs.org/job/node-test-commit-ibmi/809/nodes=ibmi73-ppc64/richardlau commentedon Jun 29, 2022
Good news is the repl test failures have gone away. Bad news is that this run has yet another failure 😢.
https://ci.nodejs.org/job/node-test-commit-ibmi/809/nodes=ibmi73-ppc64/testReport/junit/(root)/test/parallel_test_http_pipeline_requests_connection_leak/
Trying again: https://ci.nodejs.org/job/node-test-commit-ibmi/810/nodes=ibmi73-ppc64/
12 remaining items
richardlau commentedon Aug 11, 2022
There's only one failure there that isn't already marked flaky --
test-http-pipeline-requests-connection-leak. This was evaluated earlier in this issue (#43509 (comment)) as being related to a set of already marked flaky tests (#39683). There are OS PTFs (i.e. patches) available to fix the underlying issue but as noted we only have them on the rzkh IBM i CI machine and haven't yet been able to apply them to the iinthecloud machines (the Build WG are unable to do this).I've opened a PR to mark
test-http-pipeline-requests-connection-leakas flaky on IBM i: #44215FTR I forced a CI run onto the rzkh machine which has thrown up a different, much smaller, set of failures: https://ci.nodejs.org/job/node-test-commit-ibmi/855/nodes=ibmi73-ppc64/
(the rzkh machine appears unreachable today so I haven't been able to repeat).
test: mark connection leak test flaky on IBM i
richardlau commentedon Aug 15, 2022
With #44215 we're down to one test that is failing the IBM i CI build:
https://ci.nodejs.org/job/node-test-commit-ibmi/862/nodes=ibmi73-ppc64/console
This test is already marked flaky on IBM i
node/test/sequential/sequential.status
Lines 30 to 31 in d7f1934
severity: fail) instead of flakes (severity: flaky).test: mark connection leak test flaky on IBM i
test: mark connection leak test flaky on IBM i
test: mark connection leak test flaky on IBM i
richardlau commentedon Sep 29, 2022
I inadvertently introduced new failures with #44148 that are being addressed by #44810.
I've opened #44821 for the remaining issue.
test: allow EOVERFLOW errors in fs position tests
test: skip test-v8-serialize-leak on IBM i
test: mark connection leak test flaky on IBM i
test: mark connection leak test flaky on IBM i