[doc] Add special case for unsupported test

[hiraditya]

That helps disable the test for all the targets

[llvmbot]

@llvm/pr-subscribers-testing-tools

Author: AdityaK (hiraditya)

Changes

That helps disable the test for all the targets

---

Full diff: https://github.com/llvm/llvm-project/pull/78858.diff

1 Files Affected:

• (modified) llvm/docs/TestingGuide.rst (+2-1)

diff --git a/llvm/docs/TestingGuide.rst b/llvm/docs/TestingGuide.rst
index a692e301fa2c41..802cb8d2e6db04 100644
--- a/llvm/docs/TestingGuide.rst
+++ b/llvm/docs/TestingGuide.rst
@@ -568,7 +568,8 @@ list of boolean expressions. The values in each expression may be:
 | ``UNSUPPORTED`` disables the test if any expression is true.
 | ``XFAIL`` expects the test to fail if any expression is true.
 
-As a special case, ``XFAIL: *`` is expected to fail everywhere.
+As a special case, ``XFAIL: *`` is expected to fail everywhere. Similarly,
+``UNSUPPORTED: target={{.*}}`` will disable the test everywhere.
 
 .. code-block:: llvm
 

[compnerd]

Minor suggestion to improve the docs further.

[compnerd]

It might be a good idea to expand on this a bit and explain that XFAIL runs the test but expects it to fail, where as UNSUPPORTED will skip the test entirely so you would never know if the state changes.

[hiraditya]

Coincidentally, this is documented exactly as you mentioned in https://github.com/llvm/llvm-project/blob/main/llvm/docs/TestingGuide.rst?plain=1#L596

[jh7370]

I'm not sure UNSUPPORTED: target={{.*}} is really a "special case": it's just a natural consequence of the use of a regex with the target= method. XFAIL: * on the other hand IS a special case, because it isn't a regex.

[hiraditya]

yeah, it is however useful to disable the test in all cases. lmk if you think this is still an interesting thing to document.

[compnerd]

@jh7370 I agree with you, but I guess I didn't read it as "this is also a special case", but rather as "this is another way to do this". I think that having the pointer to the fact that this is not great is important since we don't want to encourage people to disable tests I think.

[jh7370]

How about we rephrase this as: "To mark a test expected failing or unsupported for all cases, use XFAIL: * or UNSUPPORTED: target={{.}} as appropriate" or something to that effect (most importantly, simply drop the "as a special case" bit)?

I've definitely seen a use-case downstream for marking something as UNSUPPORTED for all cases: if a test from upstream has come down and is flaky (so doesn't pass/fail consistently and thus XFAIL won't work), it's important to mark it as such whilst it gets reported upstream, so that our continuous merge process doesn't get blocked.

[compnerd]

I like that suggestion @jh7370

[hiraditya]

I've definitely seen a use-case downstream for marking something as UNSUPPORTED for all cases

Same. it took me a bit to find this trick with UNSUPPORTED.

[jh7370]

Could you add [doc] to the start of the commit title, please? The way the title is currently it makes it look like there's some functional change to the testing infrastructure in this patch.

[jh7370]

I'm not sure UNSUPPORTED: target={{.*}} is really a "special case": it's just a natural consequence of the use of a regex with the target= method. XFAIL: * on the other hand IS a special case, because it isn't a regex.

[hiraditya]

Merging it, assuming there are no further suggestions. Let me know if it can be improved further and I can follow up with another PR.

[jh7370]

Nit: delete the "," after "Use"