Skip to content

[llvm][llvm-lit] Hide --use-unique-output-file-name from --help #114812

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 12, 2024

Conversation

DavidSpickett
Copy link
Collaborator

I was too hasty landing an option whose only known use at this time is LLVM's own CI.

We may be able to remove it before the next branch that would be the next llvm-lit release outside of llvm, but the timing may not work out.

So I am hiding the option in case that were to happen.

I was too hasty landing an option whose only known use
at this time is LLVM's own CI.

We may be able to remove it before the next branch that would
be the next llvm-lit release outside of llvm, but the timing
may not work out.

So I am hiding the option in case that were to happen.
@llvmbot
Copy link
Member

llvmbot commented Nov 4, 2024

@llvm/pr-subscribers-testing-tools

Author: David Spickett (DavidSpickett)

Changes

I was too hasty landing an option whose only known use at this time is LLVM's own CI.

We may be able to remove it before the next branch that would be the next llvm-lit release outside of llvm, but the timing may not work out.

So I am hiding the option in case that were to happen.


Full diff: https://github.com/llvm/llvm-project/pull/114812.diff

1 Files Affected:

  • (modified) llvm/utils/lit/lit/cl_arguments.py (+7-4)
diff --git a/llvm/utils/lit/lit/cl_arguments.py b/llvm/utils/lit/lit/cl_arguments.py
index 3e5488f388ccfa..a401c9a09e081b 100644
--- a/llvm/utils/lit/lit/cl_arguments.py
+++ b/llvm/utils/lit/lit/cl_arguments.py
@@ -175,12 +175,15 @@ def parse_args():
         type=lit.reports.TimeTraceReport,
         help="Write Chrome tracing compatible JSON to the specified file",
     )
+    # This option only exists for the benefit of LLVM's Buildkite CI pipelines.
+    # As soon as it is not needed, it should be removed. Its help text would be:
+    # When enabled, lit will add a unique element to the output file name,
+    # before the extension. For example "results.xml" will become
+    # "results.<something>.xml". The "<something>" is not ordered in any
+    # way and is chosen so that existing files are not overwritten. [Default: Off]
     execution_group.add_argument(
         "--use-unique-output-file-name",
-        help="When enabled, lit will add a unique element to the output file name, "
-        'before the extension. For example "results.xml" will become '
-        '"results.<something>.xml". The "<something>" is not ordered in any '
-        "way and is chosen so that existing files are not overwritten. [Default: Off]",
+        help=argparse.SUPPRESS,
         action="store_true",
     )
     execution_group.add_argument(

@jh7370
Copy link
Collaborator

jh7370 commented Nov 4, 2024

Has there been some opposition to the option being public, or was your intention always for it to be internal to LLVM only?

@DavidSpickett
Copy link
Collaborator Author

DavidSpickett commented Nov 4, 2024

Has there been some opposition to the option being public, or was your intention always for it to be internal to LLVM only?

It's a bit of both, I shouldn't have merged it as soon as I did. My intention was that it would be broadly useful but at this time there is only one known user of it, which would be us i the Buildkite CI.

This was raised on follow up patches: #113447 (comment)

My survey of use on GitHub showed that while it has been a problem for external lit users, they have usually solved it by refactoring the pipeline which brings other benefits with it (#113447 (comment)). This is something LLVM should do as well, but I'm told we may be moving the CI soon, so if I want to improve things in the short term I needed something low impact.

I went through a bunch of alternatives that didn't need a new option, but there was only one other viable option which is wrapping lit in another script. Small chance we go with that and I revert the option altogether, still waiting for confirmation either way on that.

So this time I will not be so hasty merging this :)

If this goes ahead, my plan would be to remove this option as soon as LLVM's own CI does not need it. Hiding it means if it ends up in lit 20 on pypi then we have done as much as we can to make sure external users don't rely on it.

Copy link
Collaborator

@jh7370 jh7370 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. LGTM.

@DavidSpickett DavidSpickett merged commit 5dd9867 into llvm:main Nov 12, 2024
10 of 13 checks passed
@DavidSpickett DavidSpickett deleted the lit-hide-option branch November 12, 2024 11:40
@llvm-ci
Copy link
Collaborator

llvm-ci commented Nov 12, 2024

LLVM Buildbot has detected a new failure on builder libc-riscv64-debian-fullbuild-dbg running on libc-riscv64-debian while building llvm at step 4 "annotate".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/183/builds/6149

Here is the relevant piece of the build log for the reference
Step 4 (annotate) failure: 'python ../llvm-zorg/zorg/buildbot/builders/annotated/libc-linux.py ...' (failure)
...
[       OK ] LlvmLibcFStatTest.CreatAndReadMode (277 us)
[ RUN      ] LlvmLibcFStatTest.NonExistentFile
[       OK ] LlvmLibcFStatTest.NonExistentFile (4 us)
Ran 2 tests.  PASS: 2  FAIL: 0
[895/1016] Running unit test libc.test.src.sys.utsname.uname_test
[==========] Running 1 test from 1 test suite.
[ RUN      ] LlvmLibcUnameTest.GetMachineName
[       OK ] LlvmLibcUnameTest.GetMachineName (7 us)
Ran 1 tests.  PASS: 1  FAIL: 0
[896/1016] Running unit test libc.test.src.sys.statvfs.linux.fstatvfs_test
FAILED: projects/libc/test/src/sys/statvfs/linux/CMakeFiles/libc.test.src.sys.statvfs.linux.fstatvfs_test /home/libc_worker/libc-riscv64-debian/libc-riscv64-debian-fullbuild-dbg/build/projects/libc/test/src/sys/statvfs/linux/CMakeFiles/libc.test.src.sys.statvfs.linux.fstatvfs_test 
cd /home/libc_worker/libc-riscv64-debian/libc-riscv64-debian-fullbuild-dbg/build/projects/libc/test/src/sys/statvfs/linux && /home/libc_worker/libc-riscv64-debian/libc-riscv64-debian-fullbuild-dbg/build/projects/libc/test/src/sys/statvfs/linux/libc.test.src.sys.statvfs.linux.fstatvfs_test.__build__
[==========] Running 2 tests from 1 test suite.
[ RUN      ] LlvmLibcSysFStatvfsTest.FStatvfsBasic
[       OK ] LlvmLibcSysFStatvfsTest.FStatvfsBasic (33 us)
[ RUN      ] LlvmLibcSysFStatvfsTest.FStatvfsInvalidPath
/home/libc_worker/libc-riscv64-debian/libc-riscv64-debian-fullbuild-dbg/llvm-project/libc/test/src/sys/statvfs/linux/fstatvfs_test.cpp:40: FAILURE
Failed to match LIBC_NAMESPACE::mkdirat(AT_FDCWD, TEST_DIR, S_IRWXU) against Succeeds(0).
Expected return value to be equal to 0 but got -1.
Expected errno to be equal to "Success" but got "File exists".
[  FAILED  ] LlvmLibcSysFStatvfsTest.FStatvfsInvalidPath
Ran 2 tests.  PASS: 1  FAIL: 1
[897/1016] Running unit test libc.test.src.sys.statvfs.linux.statvfs_test
[==========] Running 2 tests from 1 test suite.
[ RUN      ] LlvmLibcSysStatvfsTest.StatvfsBasic
[       OK ] LlvmLibcSysStatvfsTest.StatvfsBasic (23 us)
[ RUN      ] LlvmLibcSysStatvfsTest.StatvfsInvalidPath
[       OK ] LlvmLibcSysStatvfsTest.StatvfsInvalidPath (473 us)
Ran 2 tests.  PASS: 2  FAIL: 0
[898/1016] Running unit test libc.test.src.sys.wait.waitpid_test
[==========] Running 1 test from 1 test suite.
[ RUN      ] LlvmLibcWaitPidTest.NoHangTest
[       OK ] LlvmLibcWaitPidTest.NoHangTest (8 us)
Ran 1 tests.  PASS: 1  FAIL: 0
[899/1016] Running unit test libc.test.src.sys.wait.wait4_test
[==========] Running 1 test from 1 test suite.
[ RUN      ] LlvmLibcwait4Test.NoHangTest
[       OK ] LlvmLibcwait4Test.NoHangTest (8 us)
Ran 1 tests.  PASS: 1  FAIL: 0
[900/1016] Running unit test libc.test.src.inttypes.strtoimax_test.__unit__
[==========] Running 7 tests from 1 test suite.
[ RUN      ] LlvmLibcStrtoimaxTest.InvalidBase
[       OK ] LlvmLibcStrtoimaxTest.InvalidBase (5 us)
[ RUN      ] LlvmLibcStrtoimaxTest.CleanBaseTenDecode
[       OK ] LlvmLibcStrtoimaxTest.CleanBaseTenDecode (28 us)
[ RUN      ] LlvmLibcStrtoimaxTest.MessyBaseTenDecode
[       OK ] LlvmLibcStrtoimaxTest.MessyBaseTenDecode (18 us)
[ RUN      ] LlvmLibcStrtoimaxTest.DecodeInOtherBases
[       OK ] LlvmLibcStrtoimaxTest.DecodeInOtherBases (1627 ms)
Step 8 (libc-unit-tests) failure: libc-unit-tests (failure)
...
[       OK ] LlvmLibcFStatTest.CreatAndReadMode (277 us)
[ RUN      ] LlvmLibcFStatTest.NonExistentFile
[       OK ] LlvmLibcFStatTest.NonExistentFile (4 us)
Ran 2 tests.  PASS: 2  FAIL: 0
[895/1016] Running unit test libc.test.src.sys.utsname.uname_test
[==========] Running 1 test from 1 test suite.
[ RUN      ] LlvmLibcUnameTest.GetMachineName
[       OK ] LlvmLibcUnameTest.GetMachineName (7 us)
Ran 1 tests.  PASS: 1  FAIL: 0
[896/1016] Running unit test libc.test.src.sys.statvfs.linux.fstatvfs_test
FAILED: projects/libc/test/src/sys/statvfs/linux/CMakeFiles/libc.test.src.sys.statvfs.linux.fstatvfs_test /home/libc_worker/libc-riscv64-debian/libc-riscv64-debian-fullbuild-dbg/build/projects/libc/test/src/sys/statvfs/linux/CMakeFiles/libc.test.src.sys.statvfs.linux.fstatvfs_test 
cd /home/libc_worker/libc-riscv64-debian/libc-riscv64-debian-fullbuild-dbg/build/projects/libc/test/src/sys/statvfs/linux && /home/libc_worker/libc-riscv64-debian/libc-riscv64-debian-fullbuild-dbg/build/projects/libc/test/src/sys/statvfs/linux/libc.test.src.sys.statvfs.linux.fstatvfs_test.__build__
[==========] Running 2 tests from 1 test suite.
[ RUN      ] LlvmLibcSysFStatvfsTest.FStatvfsBasic
[       OK ] LlvmLibcSysFStatvfsTest.FStatvfsBasic (33 us)
[ RUN      ] LlvmLibcSysFStatvfsTest.FStatvfsInvalidPath
/home/libc_worker/libc-riscv64-debian/libc-riscv64-debian-fullbuild-dbg/llvm-project/libc/test/src/sys/statvfs/linux/fstatvfs_test.cpp:40: FAILURE
Failed to match LIBC_NAMESPACE::mkdirat(AT_FDCWD, TEST_DIR, S_IRWXU) against Succeeds(0).
Expected return value to be equal to 0 but got -1.
Expected errno to be equal to "Success" but got "File exists".
[  FAILED  ] LlvmLibcSysFStatvfsTest.FStatvfsInvalidPath
Ran 2 tests.  PASS: 1  FAIL: 1
[897/1016] Running unit test libc.test.src.sys.statvfs.linux.statvfs_test
[==========] Running 2 tests from 1 test suite.
[ RUN      ] LlvmLibcSysStatvfsTest.StatvfsBasic
[       OK ] LlvmLibcSysStatvfsTest.StatvfsBasic (23 us)
[ RUN      ] LlvmLibcSysStatvfsTest.StatvfsInvalidPath
[       OK ] LlvmLibcSysStatvfsTest.StatvfsInvalidPath (473 us)
Ran 2 tests.  PASS: 2  FAIL: 0
[898/1016] Running unit test libc.test.src.sys.wait.waitpid_test
[==========] Running 1 test from 1 test suite.
[ RUN      ] LlvmLibcWaitPidTest.NoHangTest
[       OK ] LlvmLibcWaitPidTest.NoHangTest (8 us)
Ran 1 tests.  PASS: 1  FAIL: 0
[899/1016] Running unit test libc.test.src.sys.wait.wait4_test
[==========] Running 1 test from 1 test suite.
[ RUN      ] LlvmLibcwait4Test.NoHangTest
[       OK ] LlvmLibcwait4Test.NoHangTest (8 us)
Ran 1 tests.  PASS: 1  FAIL: 0
[900/1016] Running unit test libc.test.src.inttypes.strtoimax_test.__unit__
[==========] Running 7 tests from 1 test suite.
[ RUN      ] LlvmLibcStrtoimaxTest.InvalidBase
[       OK ] LlvmLibcStrtoimaxTest.InvalidBase (5 us)
[ RUN      ] LlvmLibcStrtoimaxTest.CleanBaseTenDecode
[       OK ] LlvmLibcStrtoimaxTest.CleanBaseTenDecode (28 us)
[ RUN      ] LlvmLibcStrtoimaxTest.MessyBaseTenDecode
[       OK ] LlvmLibcStrtoimaxTest.MessyBaseTenDecode (18 us)
[ RUN      ] LlvmLibcStrtoimaxTest.DecodeInOtherBases
[       OK ] LlvmLibcStrtoimaxTest.DecodeInOtherBases (1627 ms)

Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Nov 15, 2024
…#114812)

I was too hasty landing an option whose only known use at this time is
LLVM's own CI.

We may be able to remove it before the next branch that would be the
next llvm-lit release outside of llvm, but the timing may not work out.

So I am hiding the option in case that were to happen.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants