ci: Move FreeBSD/NetBSD module tests from x86_64-linux to aarch64-linux

[alexrp]

aarch64-linux runs take 1-2 hours even under load, while x86_64-linux runs are now pushing 8-9 hours under heavy load. Move some of the work to the former to avoid timeouts and get faster CI turnaround in general.

Also disable x86_64-linux-*x32 and mips64*-linux-*n32 tests in CI since these targets are slowly but surely on their way out. They can still be run locally with -Dtest-extra-targets.

[andrewrk]

You're starting to write a meta language inside the build options... let's not go down the path of having build options fundamentally change the meaning of other build options.

-Dskip-netbsd -Dskip-freebsd would be the simplest approach.

Also OK would be -Dtest-target-include=... -Dtest-target-exclude=... and have the inclusions always override the exclusions since everything is included by default.

[alexrp]

-Dskip-netbsd -Dskip-freebsd would be the simplest approach.

I thought about doing that at first, but I don't love the idea of adding options for every new target category that someone might be interested in skipping.

Also OK would be -Dtest-target-include=... -Dtest-target-exclude=... and have the inclusions always override the exclusions since everything is included by default.

You mean something like:

        var include = true;
        for (options.test_target_excludes) |include| {
            if (std.mem.indexOf(u8, triple_txt, exclude) != null) {
                include = false;
                break;
            }
        }
        if (options.test_target_includes.len != 0) {
            include = blk: for (options.test_target_includes) |include| {
                if (std.mem.indexOf(u8, triple_txt, include) != null) break :blk true;
            } else false;
        }
        if (!include) continue;

?

[andrewrk]

        const include = (for (options.test_target_excludes) |include| {
            if (std.mem.indexOf(u8, triple_txt, exclude) != null) break false;
        } else true) or for (options.test_target_includes) |include| {
            if (std.mem.indexOf(u8, triple_txt, include) != null) break true;
        } else false;

[mlugg]

That snippet doesn't really make sense. It makes -Dtest-target-include=foo pretty much entirely useless, since you also somehow need to exclude everything else, lest that first for return true giving include = true.

[mlugg]

FWIW, I think the typical way of doing this (at least, it's the thing I expect?) would be for excludes to override includes, and passing no include filters implicitly includes everything (still subject to exclusions). Probably easier to show programmatically and with examples:

const include = for (options.test_target_excludes) |exclude| {
    if (std.mem.indexOf(u8, triple_txt, exclude) != null) break false; // explicitly excluded
} else for (options.test_target_includes) |include| {
    if (std.mem.indexOf(u8, triple_txt, include) != null) break true; // explicitly included (and not excluded)
} else options.test_target_includes.len == 0; // include by default if no include filter given

then you get:

<no options>                 include all targets
-Dtest-target-include=foo    include targets matching 'foo'
-Dtest-target-exclude=foo    include targets not matching 'foo'
-Dtest-target-include=foo -Dtest-target-include=bar -Dtest-target-exclude=baz -Dtest-target-exclude=qux
  ^^ include targets which match 'foo' or 'bar' but do not match 'baz' nor 'qux'

[andrewrk]

somehow need to exclude everything else,

-Dtest-target-exclude="" (matches everything)

but I'm fine with the counter proposal suggested above

[andrewrk]

the CI scripts are very intentionally not specifying what to include, but what to skip.

[alexrp]

The behavior we want is to build all native targets + FreeBSD targets + NetBSD targets. We can't express that with just excludes. Or well, we can, but it's going to be a lot of excludes, and I'm not sure that'd be better than this?

[mlugg]

To be honest, I would also say that -Dskip-non-native isn't really saying what to skip; it's essentially just whitelisting "native". If we ignore the difference between writing native vs writing the "resolved native triple component" (e.g. aarch64 on an aarch64 host), then -Dtest-target-filter=aarch64-linux is precisely equivalent to -Dskip-non-native. If we stop ignoring that distinction, then -Dtest-target-filter=aarch64-linux clearly provides a superset of -Dskip-non-native.

In other words, you could essentially rephrase a single -Dtest-target-filter=xxxx as -Dskip-non-xxxx, and I'd argue that -Dskip-non-native is doing exactly that: "exclude all tests which do not match this filter" means exactly the same thing as "include all tests which do match this filter".

So, I struggle to buy that "the CI scripts are specifying what to skip, not what to include".

[alexrp]

Keep in mind also that if we do go the exclusion route for this script, we'd have to update it every time a new kind of target is added to the test matrix, e.g. serenity, openbsd, ...

[andrewrk]

That's the point. Everything is supposed to be tested by default, not testing things being explicit exceptions that are obviously noted.