[mlir][utils] Add script to verify canonicalizations against Alive2

[Hardcode84]

This script takes IR before and after canonicalization, translates it into llvm IR and converts it to format suitable for Alive2 https://alive2.llvm.org/ce/

This is primarily for arith canonicalizations verification, but technically it can be adapted for any dialect translatable to llvm.

Usage python verify_canon.py canonicalize.mlir -f func1 func2 ...

Example output: https://alive2.llvm.org/ce/z/KhQs4J

Initial discussion: #91646 (review)

[llvmbot]

@llvm/pr-subscribers-mlir

Author: Ivan Butygin (Hardcode84)

Changes

This script takes IR before and after canonicalization, translates it into llvm IR and converts it to format suitable for Alive2 https://alive2.llvm.org/ce/

This is primarily for arith canonicalizations verification, but technically it can be adapted for any dialect translatable to llvm.

Usage python verify_canon.py canonicalize.mlir func1 func2 ...

Example output: https://alive2.llvm.org/ce/z/KhQs4J

Initial discussion: #91646 (review)

---

Full diff: https://github.com/llvm/llvm-project/pull/91867.diff

1 Files Affected:

• (added) mlir/utils/verify-canon/verify_canon.py (+74)

diff --git a/mlir/utils/verify-canon/verify_canon.py b/mlir/utils/verify-canon/verify_canon.py
new file mode 100644
index 0000000000000..1dae30757f220
--- /dev/null
+++ b/mlir/utils/verify-canon/verify_canon.py
@@ -0,0 +1,74 @@
+# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+# See https://llvm.org/LICENSE.txt for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+# Run canonicalization, convert IR to LLVM and convert to format suitable to
+# verification against Alive2 https://alive2.llvm.org/ce/.
+
+import subprocess
+import tempfile
+import sys
+from pathlib import Path
+
+
+def filter_funcs(ir, funcs):
+    if not funcs:
+        return ir
+
+    funcs_str = ",".join(funcs)
+    return subprocess.check_output(
+        ["mlir-opt", f"--symbol-privatize=exclude={funcs_str}", "--symbol-dce"],
+        input=ir,
+    )
+
+
+def add_func_prefix(src, prefix):
+    return src.replace("@", "@" + prefix)
+
+
+def merge_ir(chunks):
+    files = []
+    for chunk in chunks:
+        tmp = tempfile.NamedTemporaryFile(suffix=".ll")
+        tmp.write(chunk)
+        tmp.flush()
+        files.append(tmp)
+
+    return subprocess.check_output(["llvm-link", "-S"] + [f.name for f in files])
+
+
+if __name__ == "__main__":
+    argv = sys.argv
+    if len(argv) < 2:
+        print(f"usage: {argv[0]} canonicalize.mlir [func1] [func2] ...")
+        exit(0)
+
+    file = argv[1]
+    funcs = argv[2:]
+
+    orig_ir = Path(file).read_bytes()
+    orig_ir = filter_funcs(orig_ir, funcs)
+
+    to_llvm_args = [
+        "--convert-arith-to-llvm",
+        "--convert-func-to-llvm",
+        "--convert-ub-to-llvm",
+        "--convert-vector-to-llvm",
+    ]
+    orig_args = ["mlir-opt"] + to_llvm_args
+    canon_args = ["mlir-opt", "-canonicalize"] + to_llvm_args
+    translate_args = ["mlir-translate", "-mlir-to-llvmir"]
+
+    orig = subprocess.check_output(orig_args, input=orig_ir)
+    canonicalized = subprocess.check_output(canon_args, input=orig_ir)
+
+    orig = subprocess.check_output(translate_args, input=orig)
+    canonicalized = subprocess.check_output(translate_args, input=canonicalized)
+
+    enc = "utf-8"
+    orig = bytes(add_func_prefix(orig.decode(enc), "src_"), enc)
+    canonicalized = bytes(add_func_prefix(canonicalized.decode(enc), "tgt_"), enc)
+
+    res = merge_ir([orig, canonicalized])
+
+    print(res.decode(enc))

[kuhar]

Thanks for writing this. I came up with a similar arith rewrite testing flow but never bothered to package it as a script, seems very useful!

[joker-eph]

These should all be handled by a single --convert-to-llvm, aren't they?

[Hardcode84]

done

[gysit]

I would probably go for an item list here to make it a bit less dense:

e.g.:

This script is a helper to verify canonicalization patterns using Alive2 https://alive2.llvm.org/ce/.
It performs the following steps:
- Filters out the provided test functions.
- Runs the canonicalization pass on the remaining functions.
- Lowers both the original and the canonicalized functions to LLVM IR.
- Prints the canonicalized and the original function side-by-side in a format that can be copied into Alive2 for verification.
Example: `python verify_canon.py canonicalize.mlir func1 func2 func3`

[Hardcode84]

done

[gysit]

Thanks!

LGTM from my end.

[gysit]

Suggested change

	# - Prints the canonicalized and the original function side-by-side in a format
	# - Prints the canonicalized and the original functions side-by-side in a format

Sorry my bad... This should be plural.

[jpienaar]

Just noticed this one, this is really nice. Hoping it flushes out more cases :)