Run unittests in debug mode at pull time and release mode on trunk builds

[swolchok]

Per discussion in discord with @mergennachin. Motivation was to have debug coverage to catch debug-only test failures.

[swolchok]

Stack from ghstack (oldest at bottom):

• -> Run unittests in debug mode at pull time and release mode on trunk builds #8550

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/8550

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures

As of commit 5dca304 with merge base e2f8624 ():

NEW FAILURES - The following jobs have failed:

• pull / unittest / macos / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• pull / unittest-buck / macos / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3
• trunk / unittest-release / macos / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[swolchok]

linux unittest jobs are green and mac test is broken on trunk, think this is ready to go

[mergennachin]

Shouldn't we also plumb all the way to
.ci/scripts/unittest-linux-cmake.sh and
https://github.com/pytorch/executorch/blob/main/test/run_oss_cpp_tests.sh?

[mergennachin]

for buck we are ignoring and using whatever's default?

[swolchok]

yes. I think that whether the code works with optimized/debug build flags is orthogonal to the choice of build system. (The ExecuTorch .buckconfig is configured to produce debug builds by default.)

[swolchok]

Shouldn't we also plumb all the way to
.ci/scripts/unittest-linux-cmake.sh and
https://github.com/pytorch/executorch/blob/main/test/run_oss_cpp_tests.sh?

As you can see in the linux release build logs and the linux debug build logs, CMAKE_BUILD_TYPE is correctly set as-is. CMake caches settings, and 1) unittest-linux-cmake.sh does not itself invoke cmake 2) the cmake invocation in run_oss_cpp_tests.sh doesn't set CMAKE_BUILD_TYPE explicitly.

[mergennachin]

Rebase and resolve the conflict

[mergennachin]

In unittest-macos.sh we are doing this, but why not here?

can we make it consistent/symmetric?

if [[ "$BUILD_TOOL" == "cmake" ]]; then
    .ci/scripts/unittest-macos-cmake.sh "$BUILD_MODE"
elif [[ "$BUILD_TOOL" == "buck2" ]]; then
    .ci/scripts/unittest-buck2.sh "$BUILD_MODE"

[swolchok]

good catch. fix is to remove it in unittest-macos.sh

[mergennachin]

.

[swolchok]

double-checking CMAKE_BUILD_TYPE in the unittest cmake jobs on this diff: all 3 mac jobs are good, all 3 linux jobs are good. found unittest-arm while looking; should talk about whether we need to give that one the same treatment. merging.