[PGO][test] Enable continuous mode PGO tests on AIX #115987

w2yehia · 2024-11-13T04:30:51Z

No description provided.

llvmbot · 2024-11-13T04:31:27Z

@llvm/pr-subscribers-pgo

Author: Wael Yehia (w2yehia)

Changes

Full diff: https://github.com/llvm/llvm-project/pull/115987.diff

9 Files Affected:

(modified) compiler-rt/test/profile/ContinuousSyncMode/basic.c (+3-3)
(modified) compiler-rt/test/profile/ContinuousSyncMode/get-filename.c (+2-2)
(modified) compiler-rt/test/profile/ContinuousSyncMode/image-with-mcdc.c (+2-2)
(modified) compiler-rt/test/profile/ContinuousSyncMode/image-with-no-counters.c (+3-3)
(modified) compiler-rt/test/profile/ContinuousSyncMode/online-merging.c (+4-4)
(modified) compiler-rt/test/profile/ContinuousSyncMode/pid-substitution.c (+2-2)
(modified) compiler-rt/test/profile/ContinuousSyncMode/reset-default-profile.c (+1-1)
(modified) compiler-rt/test/profile/ContinuousSyncMode/set-filename.c (+2-2)
(modified) compiler-rt/test/profile/lit.cfg.py (+24)

diff --git a/compiler-rt/test/profile/ContinuousSyncMode/basic.c b/compiler-rt/test/profile/ContinuousSyncMode/basic.c
index 142a47a71ad8ac..39ee15fcd0214c 100644
--- a/compiler-rt/test/profile/ContinuousSyncMode/basic.c
+++ b/compiler-rt/test/profile/ContinuousSyncMode/basic.c
@@ -1,11 +1,11 @@
-// REQUIRES: darwin
+// REQUIRES: target={{.*(darwin|aix).*}}
 
-// RUN: %clang -fprofile-instr-generate -fcoverage-mapping -o %t.exe %s
+// RUN: %clang_profgen_cont -fcoverage-mapping -o %t.exe %s
 // RUN: echo "garbage" > %t.profraw
 // RUN: env LLVM_PROFILE_FILE="%c%t.profraw" %run %t.exe
 // RUN: llvm-profdata show --counts --all-functions %t.profraw | FileCheck %s -check-prefix=CHECK-COUNTS
 // RUN: llvm-profdata merge -o %t.profdata %t.profraw
-// RUN: llvm-cov report %t.exe -instr-profile %t.profdata | FileCheck %s -check-prefix=CHECK-COVERAGE
+// RUN: %if !target={{.*aix.*}} %{ llvm-cov report %t.exe -instr-profile %t.profdata | FileCheck %s -check-prefix=CHECK-COVERAGE %}
 
 // CHECK-COUNTS: Counters:
 // CHECK-COUNTS-NEXT:   main:
diff --git a/compiler-rt/test/profile/ContinuousSyncMode/get-filename.c b/compiler-rt/test/profile/ContinuousSyncMode/get-filename.c
index d70e11991e180a..40a0cc5ffd6886 100644
--- a/compiler-rt/test/profile/ContinuousSyncMode/get-filename.c
+++ b/compiler-rt/test/profile/ContinuousSyncMode/get-filename.c
@@ -1,6 +1,6 @@
-// REQUIRES: darwin
+// REQUIRES: target={{.*(darwin|aix).*}}
 
-// RUN: %clang_pgogen -o %t.exe %s
+// RUN: %clang_pgogen_cont -o %t.exe %s
 // RUN: env LLVM_PROFILE_FILE="%c%t.profraw" %run %t.exe %t.profraw
 // RUN: env LLVM_PROFILE_FILE="%t%c.profraw" %run %t.exe %t.profraw
 // RUN: env LLVM_PROFILE_FILE="%t.profraw%c" %run %t.exe %t.profraw
diff --git a/compiler-rt/test/profile/ContinuousSyncMode/image-with-mcdc.c b/compiler-rt/test/profile/ContinuousSyncMode/image-with-mcdc.c
index d2afe7048f37f4..d171badbf4d332 100644
--- a/compiler-rt/test/profile/ContinuousSyncMode/image-with-mcdc.c
+++ b/compiler-rt/test/profile/ContinuousSyncMode/image-with-mcdc.c
@@ -1,6 +1,6 @@
-// REQUIRES: darwin
+// REQUIRES: target={{.*(darwin|aix).*}}
 
-// RUN: %clang_profgen -fcoverage-mapping -fcoverage-mcdc -O3 -o %t.exe %s
+// RUN: %clang_profgen_cont -fcoverage-mapping -fcoverage-mcdc -O3 -o %t.exe %s
 // RUN: env LLVM_PROFILE_FILE="%c%t.profraw" %run %t.exe 3 3
 // RUN: llvm-profdata show --text --all-functions %t.profraw | FileCheck %s
 
diff --git a/compiler-rt/test/profile/ContinuousSyncMode/image-with-no-counters.c b/compiler-rt/test/profile/ContinuousSyncMode/image-with-no-counters.c
index f30e2255b2289b..4313ad30b7273f 100644
--- a/compiler-rt/test/profile/ContinuousSyncMode/image-with-no-counters.c
+++ b/compiler-rt/test/profile/ContinuousSyncMode/image-with-no-counters.c
@@ -1,8 +1,8 @@
-// REQUIRES: darwin
+// REQUIRES: target={{.*(darwin|aix).*}}
 
 // RUN: echo "static void dead_code(void) {}" > %t.dso.c
-// RUN: %clang_profgen -fcoverage-mapping -O3 -dynamiclib -o %t.dso.dylib %t.dso.c
-// RUN: %clang_profgen -fcoverage-mapping -O3 -o %t.exe %s %t.dso.dylib
+// RUN: %clang_profgen_cont -fcoverage-mapping -O3 %shared_lib_flag -o %t.dso.dylib %t.dso.c
+// RUN: %clang_profgen_cont -fcoverage-mapping -O3 -o %t.exe %s %t.dso.dylib
 // RUN: env LLVM_PROFILE_FILE="%c%t.profraw" %run %t.exe 2>&1 | count 0
 // RUN: llvm-profdata show --counts --all-functions %t.profraw | FileCheck %s
 
diff --git a/compiler-rt/test/profile/ContinuousSyncMode/online-merging.c b/compiler-rt/test/profile/ContinuousSyncMode/online-merging.c
index b3c33f339713c2..35b0cd0b05d1f7 100644
--- a/compiler-rt/test/profile/ContinuousSyncMode/online-merging.c
+++ b/compiler-rt/test/profile/ContinuousSyncMode/online-merging.c
@@ -1,4 +1,4 @@
-// REQUIRES: darwin
+// REQUIRES: target={{.*(darwin|aix).*}}
 
 // Test the online merging mode (%m) along with continuous mode (%c).
 //
@@ -8,9 +8,9 @@
 // Create two DSOs and a driver program that uses them.
 // RUN: echo "void dso1(void) {}" > dso1.c
 // RUN: echo "void dso2(void) {}" > dso2.c
-// RUN: %clang_pgogen -dynamiclib -o %t.dir/dso1.dylib dso1.c -mllvm -instrprof-atomic-counter-update-all=1
-// RUN: %clang_pgogen -dynamiclib -o %t.dir/dso2.dylib dso2.c -mllvm -instrprof-atomic-counter-update-all=1
-// RUN: %clang_pgogen -o main.exe %s %t.dir/dso1.dylib %t.dir/dso2.dylib -mllvm -instrprof-atomic-counter-update-all=1
+// RUN: %clang_pgogen_cont %shared_lib_flag -o %t.dir/dso1.dylib dso1.c -mllvm -instrprof-atomic-counter-update-all=1
+// RUN: %clang_pgogen_cont %shared_lib_flag -o %t.dir/dso2.dylib dso2.c -mllvm -instrprof-atomic-counter-update-all=1
+// RUN: %clang_pgogen_cont -o main.exe %s %t.dir/dso1.dylib %t.dir/dso2.dylib -mllvm -instrprof-atomic-counter-update-all=1
 //
 // === Round 1 ===
 // Test merging+continuous mode without any file contention.
diff --git a/compiler-rt/test/profile/ContinuousSyncMode/pid-substitution.c b/compiler-rt/test/profile/ContinuousSyncMode/pid-substitution.c
index b2b95f41f2bdf2..309b685a95c5bc 100644
--- a/compiler-rt/test/profile/ContinuousSyncMode/pid-substitution.c
+++ b/compiler-rt/test/profile/ContinuousSyncMode/pid-substitution.c
@@ -1,7 +1,7 @@
-// REQUIRES: darwin
+// REQUIRES: target={{.*(darwin|aix).*}}
 
 // RUN: rm -rf %t.dir && mkdir -p %t.dir
-// RUN: %clang_pgogen -o %t.exe %s
+// RUN: %clang_pgogen_cont -o %t.exe %s
 //
 // Note: %%p is needed here, not %p, because of lit's path substitution.
 // RUN: env LLVM_PROFILE_FILE="%t.dir/%c-%%p" %run %t.exe
diff --git a/compiler-rt/test/profile/ContinuousSyncMode/reset-default-profile.c b/compiler-rt/test/profile/ContinuousSyncMode/reset-default-profile.c
index 75af7684161c9b..fb35d77c434280 100644
--- a/compiler-rt/test/profile/ContinuousSyncMode/reset-default-profile.c
+++ b/compiler-rt/test/profile/ContinuousSyncMode/reset-default-profile.c
@@ -1,4 +1,4 @@
-// REQUIRES: darwin || linux
+// REQUIRES: target={{.*(darwin|linux|aix).*}}
 
 // Test when LLVM_PROFILE_FILE is set incorrectly, it should fall backs to use default.profraw without runtime error.
 
diff --git a/compiler-rt/test/profile/ContinuousSyncMode/set-filename.c b/compiler-rt/test/profile/ContinuousSyncMode/set-filename.c
index e6d5fd31ab1b60..106e12e4e3b6eb 100644
--- a/compiler-rt/test/profile/ContinuousSyncMode/set-filename.c
+++ b/compiler-rt/test/profile/ContinuousSyncMode/set-filename.c
@@ -1,6 +1,6 @@
-// REQUIRES: darwin
+// REQUIRES: target={{.*(darwin|aix).*}}
 
-// RUN: %clang_pgogen -o %t.exe %s
+// RUN: %clang_pgogen_cont -o %t.exe %s
 // RUN: env LLVM_PROFILE_FILE="%c%t.profraw" %run %t.exe %t.profraw %t.bad
 
 #include <string.h>
diff --git a/compiler-rt/test/profile/lit.cfg.py b/compiler-rt/test/profile/lit.cfg.py
index bb5e28d87bb0f9..bee3a989e83c4e 100644
--- a/compiler-rt/test/profile/lit.cfg.py
+++ b/compiler-rt/test/profile/lit.cfg.py
@@ -30,6 +30,9 @@ def get_required_attr(config, attr_name):
 
 target_is_msvc = bool(re.match(r".*-windows-msvc$", config.target_triple))
 
+# Whether continous profile collection (%c) requires runtime counter relocation on this platform
+runtime_reloc = bool(config.host_os in ["AIX"])
+
 if config.host_os in ["Linux"]:
     extra_link_flags = ["-ldl"]
 elif target_is_msvc:
@@ -154,6 +157,22 @@ def exclude_unsupported_files_for_aix(dirname):
     )
 )
 
+config.substitutions.append(
+    (
+        "%clang_profgen_cont",
+        build_invocation(clang_cflags) + " -fprofile-instr-generate " +
+          ("-mllvm -runtime-counter-relocation" if runtime_reloc else "")
+    )
+)
+
+config.substitutions.append(
+    (
+        "%clang_pgogen_cont",
+        build_invocation(clang_cflags) + " -fprofile-generate " +
+         ("-mllvm -runtime-counter-relocation" if runtime_reloc else "")
+    )
+)
+
 if config.host_os not in [
     "Windows",
     "Darwin",
@@ -166,6 +185,11 @@ def exclude_unsupported_files_for_aix(dirname):
 ]:
     config.unsupported = True
 
+config.substitutions.append(
+    ("%shared_lib_flag",
+     "-dynamiclib" if (config.host_os == "Darwin") else "-shared")
+)
+
 if config.host_os in ["AIX"]:
     config.available_features.add("system-aix")
     exclude_unsupported_files_for_aix(config.test_source_root)

github-actions · 2024-11-13T04:34:14Z

✅ With the latest revision this PR passed the Python code formatter.

w2yehia · 2024-11-13T05:12:42Z

I would appreciate some feedback on the changes in compiler-rt/test/profile/lit.cfg.py. Thank you.

compiler-rt/test/profile/lit.cfg.py

compiler-rt/test/profile/ContinuousSyncMode/basic.c

hubert-reinterpretcast

LGTM; thanks! You may want to address the Python formatter complaints.

chapuni · 2024-12-28T00:23:38Z

compiler-rt/test/profile/ContinuousSyncMode/image-with-no-counters.c

+// RUN: %clang_profgen_cont -fcoverage-mapping -O3 %shared_lib_flag -o %t.dso.dylib %t.dso.c
+// RUN: %clang_profgen_cont -fcoverage-mapping -O3 -o %t.exe %s %t.dso.dylib


I tried enabling this on Linux but couldn't. As far as I investigated, looks like bias var has a few issues. Does this really work on AIX?

it passes on AIX according to the latest buildbot, (check-all output)

Thanks. Let me know if you met any issues re. bias var.

what problem are you encountering on linux with this testcase?

bias is not emitted if no instrumentations are emitted. libprofile complains if no bias var is found with continuous mode at runtime.

d2f77eb#diff-71a5c7472b215c25675c554f2ee1f3bbc84f29aa47ba2d30543246ad79aeffc0R654-R657

I see. Yes, the test is passing by accident on AIX: the dynamic library is not being linked in (because it has no reachable code).

If I add reachable code and compile without PGO but link in the PGO library, then I get, at runtime, the following
LLVM Profile Error: Neither __llvm_profile_counter_bias nor __llvm_profile_bitmap_bias is defined

Based on #115987, with the introduction of `REQUIRES: continuous-mode`. Also Linux assumes `runtime_reloc`. FIXME: image-with-no-counters.c is still excluded.

llvmbot added compiler-rt PGO Profile Guided Optimizations labels Nov 13, 2024

w2yehia changed the title ~~[PGO][test] Enable continous mode PGO tests on AIX~~ [PGO][test] Enable continuous mode PGO tests on AIX Nov 13, 2024

w2yehia force-pushed the pgo_cont branch from acbdd6f to 96d0411 Compare November 13, 2024 05:05

w2yehia requested review from rorth, cachemeifyoucan and ZequanWu November 13, 2024 05:10

[PGO][test] Enable continuous mode PGO tests on AIX

96d0411

hubert-reinterpretcast reviewed Nov 13, 2024

View reviewed changes

compiler-rt/test/profile/lit.cfg.py Outdated Show resolved Hide resolved

hubert-reinterpretcast reviewed Nov 13, 2024

View reviewed changes

compiler-rt/test/profile/ContinuousSyncMode/basic.c Outdated Show resolved Hide resolved

w2yehia force-pushed the pgo_cont branch from e7d9b43 to 185013d Compare November 13, 2024 17:57

Wael Yehia added 2 commits November 13, 2024 18:10

code review

185013d

add comment and format ContinuousSyncMode/basic.c

d27b627

hubert-reinterpretcast approved these changes Nov 14, 2024

View reviewed changes

follow the suggested python formatting

d5d036b

w2yehia merged commit 4f2651c into llvm:main Nov 15, 2024
6 of 7 checks passed

w2yehia deleted the pgo_cont branch November 15, 2024 03:18

chapuni mentioned this pull request Dec 27, 2024

[profile] Enable testing Continuous mode on Linux #121238

Merged

chapuni reviewed Dec 28, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[PGO][test] Enable continuous mode PGO tests on AIX #115987

[PGO][test] Enable continuous mode PGO tests on AIX #115987

Uh oh!

w2yehia commented Nov 13, 2024

Uh oh!

llvmbot commented Nov 13, 2024

Uh oh!

github-actions bot commented Nov 13, 2024 •

edited

Loading

Uh oh!

w2yehia commented Nov 13, 2024

Uh oh!

Uh oh!

Uh oh!

hubert-reinterpretcast left a comment •

edited

Loading

Uh oh!

Uh oh!

chapuni Dec 28, 2024

Uh oh!

w2yehia Dec 29, 2024

Uh oh!

chapuni Dec 29, 2024

Uh oh!

w2yehia Dec 30, 2024

Uh oh!

chapuni Dec 30, 2024

Uh oh!

w2yehia Dec 30, 2024 •

edited

Loading

Uh oh!

Uh oh!

		// RUN: %clang_profgen_cont -fcoverage-mapping -O3 %shared_lib_flag -o %t.dso.dylib %t.dso.c
		// RUN: %clang_profgen_cont -fcoverage-mapping -O3 -o %t.exe %s %t.dso.dylib

[PGO][test] Enable continuous mode PGO tests on AIX #115987

[PGO][test] Enable continuous mode PGO tests on AIX #115987

Uh oh!

Conversation

w2yehia commented Nov 13, 2024

Uh oh!

llvmbot commented Nov 13, 2024

Uh oh!

github-actions bot commented Nov 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

w2yehia commented Nov 13, 2024

Uh oh!

Uh oh!

Uh oh!

hubert-reinterpretcast left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

chapuni Dec 28, 2024

Choose a reason for hiding this comment

Uh oh!

w2yehia Dec 29, 2024

Choose a reason for hiding this comment

Uh oh!

chapuni Dec 29, 2024

Choose a reason for hiding this comment

Uh oh!

w2yehia Dec 30, 2024

Choose a reason for hiding this comment

Uh oh!

chapuni Dec 30, 2024

Choose a reason for hiding this comment

Uh oh!

w2yehia Dec 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Nov 13, 2024 •

edited

Loading

hubert-reinterpretcast left a comment •

edited

Loading

w2yehia Dec 30, 2024 •

edited

Loading