Skip to content

tests: clean up unintentionally skipped tests - #2867

Open
xymus wants to merge 13 commits into
nitlang:masterfrom
xymus:skip-less-tests
Open

xymus wants to merge 13 commits into
nitlang:masterfrom
xymus:skip-less-tests

Conversation

@xymus

@xymus xymus commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

My recent work revealed that Objective-C tests were skipped unintentionally, even on macOS. This was caused by the gitlab CI skip list applying too widely. Fix this issue and enable more skipped tests from patterns matching more than intended.

This enables more tests on the github CI, so let's update the CI config files or tooling to support them.

Fix the skip file for GitLab CI to actually apply only to GitLab CI.
A missing $ made the list apply everywhere and the Objective-C tests
never ran.

Signed-off-by: Alexis Laferrière <alexis.laf@xymus.net>
Match `mpi` only at the start of a word, which still covers `mpi_simple`
and the `lib/mpi` paths.

The bare `mpi` pattern also matched `compiler` and skipped:
- `base_compile`
- `error_annot_c_compiler`
- `test_annot_c_compiler`
- `test_annot_c_compiler_conditional`

Signed-off-by: Alexis Laferrière <alexis.laf@xymus.net>
Fix bare patterns in the skip files that skipped more than intended:
- `neo` also skipped `chameneosredux`, so it was never compiled on macOS.
  Replace it by `test_neo`, which still covers `test_neo` and `test_neo4j`.
- `nitc_args1` also skipped `nitc_args10`, `11` and `12` under the
  interpreters, these never ran on niti and nitvm.
- The `nit_args` also matched the `nitunit_args`.

Signed-off-by: Alexis Laferrière <alexis.laf@xymus.net>
The skip files listed tests that were renamed or deleted:
- `converter` and `actors_` in `exec.skip`
- `wiringPi` in `gitlab_ci.skip`
- `shoot_logic`, `test_docdown_args` and `test_ropebuffer` in `niti.skip`
  and `nitvm.skip`
- `test_docdown` and `^neo_doxygen` in `nitcg.skip`

`listfull.sh` still listed the deleted `shoot` example, `friendz` and
`neo_doxygen`, and globs on `examples/*/src/` that now match nothing.

Signed-off-by: Alexis Laferrière <alexis.laf@xymus.net>
Since glslang 11, the validator stops at the first error by default.
Let's use the `-C` flag, for cascading errors, which restores reporting all
errors.

Signed-off-by: Alexis Laferrière <alexis.laf@xymus.net>
With the `gitlab_ci.skip` fix more tests will run on GitHub and some
would fail. Install what they need on the Ubuntu runners:
- `clang` and `gobjc` for the Objective-C FFI
- `glslang-tools` for `test_glsl_validation`
- `emscripten` for `emscripten_nodejs`

Signed-off-by: Alexis Laferrière <alexis.laf@xymus.net>
A stray backtick in the path prevented uploading the `.res` files of the
tests-full jobs.

Signed-off-by: Alexis Laferrière <alexis.laf@xymus.net>
@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

Test Results

    67 files     343 suites   16m 21s ⏱️
14 271 tests 13 757 ✅ 514 💤 0 ❌
14 694 runs  14 166 ✅ 528 💤 0 ❌

Results for commit 0781a00.

♻️ This comment has been updated with latest results.

xymus added 3 commits October 1, 2026 10:27
Decode the POST keys like we do for POST values, and the keys of GET arguments.

Signed-off-by: Alexis Laferrière <alexis.laf@xymus.net>
The function takes the interface and the root of the Web site to serve.
This enables other programs to run the server.

The `/hello/:name` action also echoes the arguments of POST requests.

Signed-off-by: Alexis Laferrière <alexis.laf@xymus.net>
The curl tests relied on the availability and content of `example.org`.
The website content isn't stable and not designed for testing. Let's
instead launch a local nitcorn server for testing against.

Replace `curl_http` by three straightforward examples. Each exposes its
request as a function and expects a URL on the command line:
- `curl_http_get`
- `curl_http_post`
- `curl_http_download`

Intro `curl_http_local_server`, it runs `hello_world_server` from the
nitcorn examples in a thread and executes the three requests against it.

Signed-off-by: Alexis Laferrière <alexis.laf@xymus.net>
@xymus

xymus commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

The curl tests are failing again from a change in the contents of example.org. That website isn’t designed for such a use case, let’s instead use a local nitcorn server.

To make it cleaner, I’ve split up the big curl example (also acting as the test configured via the args file) into 3 simple examples and one test-oriented example. The test one launches a nitcorn server and the requests from the 3 examples against it.

Making the skip lists more specific enabled more tests, some were slow
and others were slow and revealed bugs.

Let's skip the slow ones again. `nitc_args10` to `nitc_args12` have nitc
compile `core`, which is very slow when nitc is interpreted.

But `nitc_args11` and `nitc_args12` also fail with niti on Linux. The
programs created by the interpreted nitc would crash at startup because
the interpreter mixes up local variables. Keeping this issue in the
reduced `base_var_assign_in_branch` for future investigation.

Signed-off-by: Alexis Laferrière <alexis.laf@xymus.net>
@xymus

xymus commented Oct 2, 2026

Copy link
Copy Markdown
Contributor Author

Skipping less tests revealed an issue in niti. When running the compiler in the interpreter, we would miscompile the nitc_args11 and 12 cases and the produced binary would crash at launch on Linux. macOS would likely avoid the crash because it initializes variables to null.

Keeping this issue for future investigation as base_var_assign_in_branch.nit. For now we bring back skipping these tests because they are too slow anyway and it was the original intent with the skipping.

xymus added 2 commits October 2, 2026 20:56
…more

The emscripten library included `gc.h`, it used to be provided by the
garbage collector emulated in JavaScript in the old Emscripten
toolchain. It has been removed from Emscripten since then, breaking the
numerous emscripten Nit clients.

The GC wasn't fully supported on emscripten even back then, see the
manual triggers. We could likely add Nit GC support to that platform by
building libgc ourselves like we do for iOS.

Signed-off-by: Alexis Laferrière <alexis.laf@xymus.net>
Add a missing check on `c_name` in the `==` service of the class
`CustomizedRuntimeFunction` to distinguish it from its subclass
`CustomizedThunkFunction`. In the rare case where the hash of a thunk
and its target function would lead to the same bucket, the missing check
on `c_name` could mix them up.

Let's also move including the `c_name` in the hash from
`CustomizedThunkFunction` to `CustomizedRuntimeFunction`. This is mostly
to align it with the equality check and prevent such errors in the
future. However, we need the `c_name` in the equality check of the
parent class `CustomizedRuntimeFunction` for it to be symmetric between
both classes.

This caused non-deterministic compilation errors in the generated C code
where we'd print a reference to the thunk without declaring it.

Signed-off-by: Alexis Laferrière <alexis.laf@xymus.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant