Skip to content

[profile] Use fprofile-continuous in compiler-rt tests #126617

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Mar 15, 2025

Conversation

w2yehia
Copy link
Contributor

@w2yehia w2yehia commented Feb 10, 2025

PR #124353 introduced the clang option -fprofile-continuous to enable continuous mode.
Use this option in all compiler-rt tests, where applicable.

Changes can be summarized as follows:

  1. tests that use -fprofile-instr-generate (%clang_profgen), which is an option that takes profile file name, are changed like so:
-// RUN: %clang_profgen_cont <SOME-OPTIONS> -o %t.exe %s
-// RUN: env LLVM_PROFILE_FILE="%c%t.profraw" %run %t.exe
+// RUN: %clang_profgen=%t.profraw -fprofile-continuous <SOME-OPTIONS> -o %t.exe %s
+// RUN: %run %t.exe
  1. tests that use -fprofile-generate (%clang_pgogen), which is an option that takes a profile directory, are on case-by-case basis. Where the default name "default_%m.profraw" works, those tests were changed to use %clang_pgogen=<dir>, and the rest (set-filename.c and get-filename.c) continued to use the LLVM_PROFILE_FILE environment variable .
  2. set-file-object.c uses different filename for different run of the same executable, so it continued to use the LLVM_PROFILE_FILE environment variable.
  3. pid-substitution.c add a clang_profgen variation.

@llvmbot llvmbot added compiler-rt PGO Profile Guided Optimizations labels Feb 10, 2025
@llvmbot
Copy link
Member

llvmbot commented Feb 10, 2025

@llvm/pr-subscribers-pgo

Author: Wael Yehia (w2yehia)

Changes

PR #124353 introduced the clang option -fprofile-continuous to enable continuous mode.
Use this option in all compiler-rt tests, where applicable.


Full diff: https://github.com/llvm/llvm-project/pull/126617.diff

12 Files Affected:

  • (modified) compiler-rt/test/profile/ContinuousSyncMode/basic.c (+2-2)
  • (modified) compiler-rt/test/profile/ContinuousSyncMode/get-filename.c (+1-1)
  • (modified) compiler-rt/test/profile/ContinuousSyncMode/image-with-mcdc.c (+2-2)
  • (modified) compiler-rt/test/profile/ContinuousSyncMode/image-with-no-counters.c (+3-3)
  • (modified) compiler-rt/test/profile/ContinuousSyncMode/multi-threaded.cpp (+4-4)
  • (modified) compiler-rt/test/profile/ContinuousSyncMode/online-merging.c (+3-3)
  • (modified) compiler-rt/test/profile/ContinuousSyncMode/pid-substitution.c (+3-1)
  • (modified) compiler-rt/test/profile/ContinuousSyncMode/reset-default-profile.c (+1-1)
  • (modified) compiler-rt/test/profile/ContinuousSyncMode/runtime-counter-relocation.c (+2-2)
  • (modified) compiler-rt/test/profile/ContinuousSyncMode/set-file-object.c (+1-2)
  • (modified) compiler-rt/test/profile/ContinuousSyncMode/set-filename.c (+1-1)
  • (modified) compiler-rt/test/profile/lit.cfg.py (-27)
diff --git a/compiler-rt/test/profile/ContinuousSyncMode/basic.c b/compiler-rt/test/profile/ContinuousSyncMode/basic.c
index 531877b78a1a295..209ceec6a83f02d 100644
--- a/compiler-rt/test/profile/ContinuousSyncMode/basic.c
+++ b/compiler-rt/test/profile/ContinuousSyncMode/basic.c
@@ -1,8 +1,8 @@
 // REQUIRES: continuous-mode
 
-// RUN: %clang_profgen_cont -fcoverage-mapping -o %t.exe %s
+// RUN: %clang_profgen=%t.profraw -fprofile-continuous -fcoverage-mapping -o %t.exe %s
 // RUN: echo "garbage" > %t.profraw
-// RUN: env LLVM_PROFILE_FILE="%c%t.profraw" %run %t.exe
+// RUN: %run %t.exe
 // RUN: llvm-profdata show --counts --all-functions %t.profraw | FileCheck %s -check-prefix=CHECK-COUNTS
 // RUN: llvm-profdata merge -o %t.profdata %t.profraw
 //
diff --git a/compiler-rt/test/profile/ContinuousSyncMode/get-filename.c b/compiler-rt/test/profile/ContinuousSyncMode/get-filename.c
index e341dd429eb84db..ca9033703c62207 100644
--- a/compiler-rt/test/profile/ContinuousSyncMode/get-filename.c
+++ b/compiler-rt/test/profile/ContinuousSyncMode/get-filename.c
@@ -1,6 +1,6 @@
 // REQUIRES: continuous-mode
 
-// RUN: %clang_pgogen_cont -o %t.exe %s
+// RUN: %clang_pgogen -fprofile-continuous -o %t.exe %s
 // RUN: env LLVM_PROFILE_FILE="%c%t.profraw" %run %t.exe %t.profraw
 // RUN: env LLVM_PROFILE_FILE="%t%c.profraw" %run %t.exe %t.profraw
 // RUN: env LLVM_PROFILE_FILE="%t.profraw%c" %run %t.exe %t.profraw
diff --git a/compiler-rt/test/profile/ContinuousSyncMode/image-with-mcdc.c b/compiler-rt/test/profile/ContinuousSyncMode/image-with-mcdc.c
index fa24e26c4c53b8c..d4cae5ab249429c 100644
--- a/compiler-rt/test/profile/ContinuousSyncMode/image-with-mcdc.c
+++ b/compiler-rt/test/profile/ContinuousSyncMode/image-with-mcdc.c
@@ -1,7 +1,7 @@
 // REQUIRES: continuous-mode
 
-// RUN: %clang_profgen_cont -fcoverage-mapping -fcoverage-mcdc -O3 -o %t.exe %s
-// RUN: env LLVM_PROFILE_FILE="%c%t.profraw" %run %t.exe 3 3
+// RUN: %clang_profgen=%t.profraw -fprofile-continuous -fcoverage-mapping -fcoverage-mcdc -O3 -o %t.exe %s
+// RUN: %run %t.exe 3 3
 // RUN: llvm-profdata show --text --all-functions %t.profraw | FileCheck %s
 
 // CHECK: Num Bitmap Bytes:
diff --git a/compiler-rt/test/profile/ContinuousSyncMode/image-with-no-counters.c b/compiler-rt/test/profile/ContinuousSyncMode/image-with-no-counters.c
index 4313ad30b7273f0..c261248b35d4343 100644
--- a/compiler-rt/test/profile/ContinuousSyncMode/image-with-no-counters.c
+++ b/compiler-rt/test/profile/ContinuousSyncMode/image-with-no-counters.c
@@ -1,9 +1,9 @@
 // REQUIRES: target={{.*(darwin|aix).*}}
 
 // RUN: echo "static void dead_code(void) {}" > %t.dso.c
-// RUN: %clang_profgen_cont -fcoverage-mapping -O3 %shared_lib_flag -o %t.dso.dylib %t.dso.c
-// RUN: %clang_profgen_cont -fcoverage-mapping -O3 -o %t.exe %s %t.dso.dylib
-// RUN: env LLVM_PROFILE_FILE="%c%t.profraw" %run %t.exe 2>&1 | count 0
+// RUN: %clang_profgen=%t.profraw -fprofile-continuous -fcoverage-mapping -O3 %shared_lib_flag -o %t.dso.dylib %t.dso.c
+// RUN: %clang_profgen=%t.profraw -fprofile-continuous -fcoverage-mapping -O3 -o %t.exe %s %t.dso.dylib
+// RUN: %run %t.exe 2>&1 | count 0
 // RUN: llvm-profdata show --counts --all-functions %t.profraw | FileCheck %s
 
 // CHECK: Total functions: 1
diff --git a/compiler-rt/test/profile/ContinuousSyncMode/multi-threaded.cpp b/compiler-rt/test/profile/ContinuousSyncMode/multi-threaded.cpp
index aa0a46e0fc396d5..7387f47c13edbc1 100644
--- a/compiler-rt/test/profile/ContinuousSyncMode/multi-threaded.cpp
+++ b/compiler-rt/test/profile/ContinuousSyncMode/multi-threaded.cpp
@@ -1,9 +1,9 @@
 // REQUIRES: continuous-mode
 
-// RUN: rm -f %t.profraw
-// RUN: %clangxx_pgogen_cont -lpthread %s -o %t.exe -mllvm -disable-vp -fprofile-update=atomic
-// RUN: env LLVM_PROFILE_FILE="%c%t.profraw" %run %t.exe
-// RUN: llvm-profdata show --counts --function=accum  %t.profraw | FileCheck %s
+// RUN: rm -rf %t.dir
+// RUN: %clangxx_pgogen=%t.dir -fprofile-continuous -lpthread %s -o %t.exe -mllvm -disable-vp -fprofile-update=atomic
+// RUN: %run %t.exe
+// RUN: llvm-profdata show --counts --function=accum  %t.dir/default_*.profraw | FileCheck %s
 // CHECK:    Block counts: [100000, 4]
 
 #include <thread>
diff --git a/compiler-rt/test/profile/ContinuousSyncMode/online-merging.c b/compiler-rt/test/profile/ContinuousSyncMode/online-merging.c
index b11a098b4d2b7da..6c3e1b28514ea3a 100644
--- a/compiler-rt/test/profile/ContinuousSyncMode/online-merging.c
+++ b/compiler-rt/test/profile/ContinuousSyncMode/online-merging.c
@@ -9,9 +9,9 @@
 // Create two DSOs and a driver program that uses them.
 // RUN: echo "void dso1(void) {}" > dso1.c
 // RUN: echo "void dso2(void) {}" > dso2.c
-// RUN: %clang_pgogen_cont %shared_lib_flag -o %t.dir/dso1.dylib dso1.c -fprofile-update=atomic
-// RUN: %clang_pgogen_cont %shared_lib_flag -o %t.dir/dso2.dylib dso2.c -fprofile-update=atomic
-// RUN: %clang_pgogen_cont -o main.exe %s %t.dir/dso1.dylib %t.dir/dso2.dylib -fprofile-update=atomic
+// RUN: %clang_pgogen -fprofile-continuous %shared_lib_flag -o %t.dir/dso1.dylib dso1.c -fprofile-update=atomic
+// RUN: %clang_pgogen -fprofile-continuous %shared_lib_flag -o %t.dir/dso2.dylib dso2.c -fprofile-update=atomic
+// RUN: %clang_pgogen -fprofile-continuous -o main.exe %s %t.dir/dso1.dylib %t.dir/dso2.dylib -fprofile-update=atomic
 //
 // === Round 1 ===
 // Test merging+continuous mode without any file contention.
diff --git a/compiler-rt/test/profile/ContinuousSyncMode/pid-substitution.c b/compiler-rt/test/profile/ContinuousSyncMode/pid-substitution.c
index 8a00b28825cae6f..cc256ab16212996 100644
--- a/compiler-rt/test/profile/ContinuousSyncMode/pid-substitution.c
+++ b/compiler-rt/test/profile/ContinuousSyncMode/pid-substitution.c
@@ -1,9 +1,11 @@
 // REQUIRES: continuous-mode
 
 // RUN: rm -rf %t.dir && mkdir -p %t.dir
-// RUN: %clang_pgogen_cont -o %t.exe %s
 //
 // Note: %%p is needed here, not %p, because of lit's path substitution.
+// RUN: %clang_profgen=%t.dir/-%%p -fprofile-continuous -o %t.exe %s
+// RUN: %run %t.exe
+// RUN: %clang_pgogen -fprofile-continuous -o %t.exe %s
 // RUN: env LLVM_PROFILE_FILE="%t.dir/%c-%%p" %run %t.exe
 
 #include <stdlib.h>
diff --git a/compiler-rt/test/profile/ContinuousSyncMode/reset-default-profile.c b/compiler-rt/test/profile/ContinuousSyncMode/reset-default-profile.c
index fb35d77c4342804..b97459969f1749d 100644
--- a/compiler-rt/test/profile/ContinuousSyncMode/reset-default-profile.c
+++ b/compiler-rt/test/profile/ContinuousSyncMode/reset-default-profile.c
@@ -4,7 +4,7 @@
 
 // Create & cd into a temporary directory.
 // RUN: rm -rf %t.dir && mkdir -p %t.dir && cd %t.dir
-// RUN: %clang -fprofile-instr-generate -fcoverage-mapping -mllvm -runtime-counter-relocation=true -o %t.exe %s
+// RUN: %clang_profgen -fprofile-continuous -fcoverage-mapping -o %t.exe %s
 // RUN: env LLVM_PROFILE_FILE="incorrect-profile-name%m%c%c.profraw" %run %t.exe
 // RUN: ls -l | FileCheck %s
 
diff --git a/compiler-rt/test/profile/ContinuousSyncMode/runtime-counter-relocation.c b/compiler-rt/test/profile/ContinuousSyncMode/runtime-counter-relocation.c
index 6ec9077f4d6145c..01b25d2167cc095 100644
--- a/compiler-rt/test/profile/ContinuousSyncMode/runtime-counter-relocation.c
+++ b/compiler-rt/test/profile/ContinuousSyncMode/runtime-counter-relocation.c
@@ -1,8 +1,8 @@
 // REQUIRES: target={{.*(linux|solaris|windows-msvc|aix).*}}
 
-// RUN: %clang -fprofile-instr-generate -fcoverage-mapping -mllvm -runtime-counter-relocation=true -o %t.exe %s
+// RUN: %clang_profgen=%t.profraw -fprofile-continuous -fcoverage-mapping -o %t.exe %s
 // RUN: echo "garbage" > %t.profraw
-// RUN: env LLVM_PROFILE_FILE="%c%t.profraw" %run %t.exe
+// RUN: %run %t.exe
 // RUN: llvm-profdata show --counts --all-functions %t.profraw | FileCheck %s -check-prefix=CHECK-COUNTS
 // RUN: llvm-profdata merge -o %t.profdata %t.profraw
 // RUN: %if !target={{.*aix.*}} %{ llvm-cov report %t.exe -instr-profile %t.profdata | FileCheck %s -check-prefix=CHECK-COVERAGE %}
diff --git a/compiler-rt/test/profile/ContinuousSyncMode/set-file-object.c b/compiler-rt/test/profile/ContinuousSyncMode/set-file-object.c
index 321a69b4f23d0c3..5baa64baaafe3d1 100644
--- a/compiler-rt/test/profile/ContinuousSyncMode/set-file-object.c
+++ b/compiler-rt/test/profile/ContinuousSyncMode/set-file-object.c
@@ -4,8 +4,7 @@
 // Create & cd into a temporary directory.
 // RUN: rm -rf %t.dir && mkdir -p %t.dir && cd %t.dir
 
-// The -mllvm -runtime-counter-relocation=true flag has effect only on linux.
-// RUN: %clang -fprofile-instr-generate -fcoverage-mapping -fprofile-update=atomic -mllvm -runtime-counter-relocation=true -o main.exe %s
+// RUN: %clang_profgen -fprofile-continuous -fcoverage-mapping -fprofile-update=atomic -o main.exe %s
 
 // Test continuous mode with __llvm_profile_set_file_object with mergin disabled.
 // RUN: env LLVM_PROFILE_FILE="%t.dir/profdir/%c%mprofraw.old" %run  %t.dir/main.exe nomerge %t.dir/profdir/profraw.new 2>&1 | FileCheck %s -check-prefix=WARN
diff --git a/compiler-rt/test/profile/ContinuousSyncMode/set-filename.c b/compiler-rt/test/profile/ContinuousSyncMode/set-filename.c
index abc72646d16b42c..f3a76408ae47760 100644
--- a/compiler-rt/test/profile/ContinuousSyncMode/set-filename.c
+++ b/compiler-rt/test/profile/ContinuousSyncMode/set-filename.c
@@ -1,6 +1,6 @@
 // REQUIRES: continuous-mode
 
-// RUN: %clang_pgogen_cont -o %t.exe %s
+// RUN: %clang_pgogen -fprofile-continuous -o %t.exe %s
 // RUN: env LLVM_PROFILE_FILE="%c%t.profraw" %run %t.exe %t.profraw %t.bad
 
 #include <string.h>
diff --git a/compiler-rt/test/profile/lit.cfg.py b/compiler-rt/test/profile/lit.cfg.py
index fc2baf7c40b8fc9..c9a716abeccd8e9 100644
--- a/compiler-rt/test/profile/lit.cfg.py
+++ b/compiler-rt/test/profile/lit.cfg.py
@@ -30,9 +30,6 @@ def get_required_attr(config, attr_name):
 
 target_is_msvc = bool(re.match(r".*-windows-msvc$", config.target_triple))
 
-# Whether continous profile collection (%c) requires runtime counter relocation on this platform
-runtime_reloc = bool(config.host_os in ["AIX", "Linux"])
-
 if config.host_os in ["Linux"]:
     extra_link_flags = ["-ldl"]
 elif target_is_msvc:
@@ -97,14 +94,6 @@ def exclude_unsupported_files_for_aix(dirname):
 config.substitutions.append(
     ("%clang_profgen=", build_invocation(clang_cflags) + " -fprofile-instr-generate=")
 )
-config.substitutions.append(
-    (
-        "%clang_profgen_cont ",
-        build_invocation(clang_cflags)
-        + " -fprofile-instr-generate "
-        + ("-mllvm -runtime-counter-relocation " if runtime_reloc else ""),
-    )
-)
 config.substitutions.append(
     (
         "%clangxx_profgen ",
@@ -124,28 +113,12 @@ def exclude_unsupported_files_for_aix(dirname):
 config.substitutions.append(
     ("%clang_pgogen=", build_invocation(clang_cflags) + " -fprofile-generate=")
 )
-config.substitutions.append(
-    (
-        "%clang_pgogen_cont ",
-        build_invocation(clang_cflags)
-        + " -fprofile-generate "
-        + ("-mllvm -runtime-counter-relocation " if runtime_reloc else ""),
-    )
-)
 config.substitutions.append(
     ("%clangxx_pgogen ", build_invocation(clang_cxxflags) + " -fprofile-generate ")
 )
 config.substitutions.append(
     ("%clangxx_pgogen=", build_invocation(clang_cxxflags) + " -fprofile-generate=")
 )
-config.substitutions.append(
-    (
-        "%clangxx_pgogen_cont ",
-        build_invocation(clang_cxxflags)
-        + " -fprofile-generate "
-        + ("-mllvm -runtime-counter-relocation " if runtime_reloc else ""),
-    )
-)
 
 config.substitutions.append(
     ("%clang_cspgogen ", build_invocation(clang_cflags) + " -fcs-profile-generate ")

Copy link
Member

@anhtuyenibm anhtuyenibm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks, Wael @w2yehia

@dtellenbach dtellenbach self-requested a review March 14, 2025 18:30
Copy link
Member

@dtellenbach dtellenbach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I've not been following this really much, does Darwin not relying on counter relocation change anything with respect to using -fprofile-continuous?

Also, I think we should definitely have some tests that are using %c since that is a workflow which is established for many years and is expected to be used much more than the explicit flag. I would actually prefer to have both versions tested.

@w2yehia
Copy link
Contributor Author

w2yehia commented Mar 14, 2025

Thanks @dtellenbach

does Darwin not relying on counter relocation change anything with respect to using -fprofile-continuous?

-fprofile-continuous does not enable counter relocation on Darwin. It enables it on targets using ELF, on AIX, and on Windows.

Also, I think we should definitely have some tests that are using %c since that is a workflow which is established for many years and is expected to be used much more than the explicit flag. I would actually prefer to have both versions tested.

-fprofile-continuous doesn't replace %c as a method of enabling continuous mode in the runtime. The option is a convenience option that appends '%c' to the default profile name. The runtime is unchanged.
Given the above, I don't see the necessity to have both explicit filename setting and -fprofile-continuous in every continuous mode test. I think it's ok to have it in one test. What do you think?

After this PR, explicit filename setting with "%c" is used in ContinuousSyncMode/online-merging.c, ContinuousSyncMode/set-filename.c, ContinuousSyncMode/multiple-DSOs.c, and ContinuousSyncMode/pid-substitution.c.
Also tests that were testing profile file name parsing obviously remain untouched, for example: ContinuousSyncMode/reset-default-profile.c and ContinuousSyncMode/get-filename.c.

@dtellenbach
Copy link
Member

does Darwin not relying on counter relocation change anything with respect to using -fprofile-continuous?

-fprofile-continuous does not enable counter relocation on Darwin. It enables it on targets using ELF, on AIX, and on Windows.

Thanks!

Also, I think we should definitely have some tests that are using %c since that is a workflow which is established for many years and is expected to be used much more than the explicit flag. I would actually prefer to have both versions tested.

-fprofile-continuous doesn't replace %c as a method of enabling continuous mode in the runtime. The option is a convenience option that appends '%c' to the default profile name. The runtime is unchanged. Given the above, I don't see the necessity to have both explicit filename setting and -fprofile-continuous in every continuous mode test. I think it's ok to have it in one test. What do you think?

Agreed, doesn't have to be in every test.

After this PR, explicit filename setting with "%c" is used in ContinuousSyncMode/online-merging.c, ContinuousSyncMode/set-filename.c, ContinuousSyncMode/multiple-DSOs.c, and ContinuousSyncMode/pid-substitution.c. Also tests that were testing profile file name parsing obviously remain untouched, for example: ContinuousSyncMode/reset-default-profile.c and ContinuousSyncMode/get-filename.c.

Sounds good, as long as we have some tests that still use explicit %c that should be fine.

Thanks a lot!

Copy link
Member

@dtellenbach dtellenbach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@w2yehia w2yehia merged commit f83726e into llvm:main Mar 15, 2025
7 checks passed
@w2yehia w2yehia deleted the pgo_cont_tests branch March 15, 2025 16:01
aeubanks added a commit to aeubanks/llvm-project that referenced this pull request Mar 21, 2025
aeubanks added a commit to aeubanks/llvm-project that referenced this pull request Mar 24, 2025
aeubanks added a commit to aeubanks/llvm-project that referenced this pull request Mar 24, 2025
aeubanks added a commit to aeubanks/llvm-project that referenced this pull request Mar 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler-rt PGO Profile Guided Optimizations
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants