[libc++] Add testing configurations for libstdc++ and a native stdlib

[ldionne]

This allows running the test suite against the native Standard Library on most systems, and against libstdc++ installed at a custom location.

Of course, these configurations don't run 100% clean at the moment. In particular, running against the native stdlib is almost guaranteed not to work out-of-the-box, since the test suite generally contains tests for things that have been implemented on tip-of-trunk but not released to most major platforms yet. However, having an easy way to run the test suite against that library is still both useful and interesting.

[ldionne]

CC @jwakely

[llvmbot]

@llvm/pr-subscribers-libcxx

Author: Louis Dionne (ldionne)

Changes

This allows running the test suite against the native Standard Library on most systems, and against libstdc++ installed at a custom location.

Of course, these configurations don't run 100% clean at the moment. In particular, running against the native stdlib is almost guaranteed not to work out-of-the-box, since the test suite generally contains tests for things that have been implemented on tip-of-trunk but not released to most major platforms yet. However, having an easy way to run the test suite against that library is still both useful and interesting.

---

Full diff: https://github.com/llvm/llvm-project/pull/98539.diff

2 Files Affected:

• (added) libcxx/test/configs/stdlib-libstdc++.cfg.in (+57)
• (added) libcxx/test/configs/stdlib-native.cfg.in (+24)

diff --git a/libcxx/test/configs/stdlib-libstdc++.cfg.in b/libcxx/test/configs/stdlib-libstdc++.cfg.in
new file mode 100644
index 0000000000000..e849d9b9512a1
--- /dev/null
+++ b/libcxx/test/configs/stdlib-libstdc++.cfg.in
@@ -0,0 +1,57 @@
+#
+# This testing configuration runs the test suite using the libstdc++ Standard library.
+#
+# The additional '--param libstdcxx-install-prefix=<PATH>' must be provided when invoking
+# lit for the configuration to find the appropriate headers and library.
+#
+
+import os, site
+site.addsitedir(os.path.join('@LIBCXX_SOURCE_DIR@', 'utils'))
+import libcxx.test.params, libcxx.test.config, libcxx.test.dsl
+
+# Basic configuration of the test suite
+config.name = 'libstdc++'
+config.test_source_root = os.path.join('@LIBCXX_SOURCE_DIR@', 'test')
+config.test_format = libcxx.test.format.CxxStandardLibraryTest()
+config.recursiveExpansionLimit = 10
+config.test_exec_root = os.path.join('@CMAKE_BINARY_DIR@', 'test')
+
+# Additional parameters for libstdc++
+LIBSTDCXX_PARAMETERS = [
+    libcxx.test.dsl.Parameter(name='libstdcxx-install-prefix', type=str,
+        actions=lambda path: [libcxx.test.dsl.AddSubstitution('%{libstdcxx-install-prefix}', path)],
+        help="""
+        The installation prefix where libstdc++ was installed. This is used to find the libstdc++ headers,
+        link against its built library, etc.
+        """),
+]
+
+config.substitutions.append(('%{libcxx-dir}', '@LIBCXX_SOURCE_DIR@'))
+
+# Configure the compiler and flags
+config.compiler = "@CMAKE_CXX_COMPILER@"
+config.target_triple = "@LLVM_DEFAULT_TARGET_TRIPLE@"
+config.stdlib = 'libstdc++'
+
+config.substitutions.append(('%{flags}',
+    '-pthread' + (' -isysroot {}'.format('@CMAKE_OSX_SYSROOT@') if '@CMAKE_OSX_SYSROOT@' else '')
+))
+config.substitutions.append(('%{compile_flags}',
+    '-nostdinc++ -isystem %{libstdcxx-install-prefix}/include/c++/14 -isystem %{libstdcxx-install-prefix}/include/c++/14/aarch64-apple-darwin23 -I %{libcxx-dir}/test/support'
+))
+config.substitutions.append(('%{link_flags}',
+    '-nostdlib++ -L %{libstdcxx-install-prefix}/lib/gcc/14 -Wl,-rpath,%{libstdcxx-install-prefix}/lib/gcc/14 -lstdc++'
+))
+config.substitutions.append(('%{exec}',
+    '%{executor} --execdir %T -- '
+))
+
+import os, site
+site.addsitedir(os.path.join('@LIBCXX_SOURCE_DIR@', 'utils'))
+import libcxx.test.params, libcxx.test.config
+libcxx.test.config.configure(
+    libcxx.test.params.DEFAULT_PARAMETERS + LIBSTDCXX_PARAMETERS,
+    libcxx.test.features.DEFAULT_FEATURES,
+    config,
+    lit_config
+)
diff --git a/libcxx/test/configs/stdlib-native.cfg.in b/libcxx/test/configs/stdlib-native.cfg.in
new file mode 100644
index 0000000000000..3e25c1eedb3f8
--- /dev/null
+++ b/libcxx/test/configs/stdlib-native.cfg.in
@@ -0,0 +1,24 @@
+#
+# This testing configuration handles running the test suite against the
+# native C++ Standard Library, i.e. whatever standard library is used by
+# default when no special compiler flags are provided.
+#
+
+lit_config.load_config(config, '@CMAKE_CURRENT_BINARY_DIR@/cmake-bridge.cfg')
+
+config.substitutions.append(('%{flags}',
+    '-pthread' + (' -isysroot {}'.format('@CMAKE_OSX_SYSROOT@') if '@CMAKE_OSX_SYSROOT@' else '')
+))
+config.substitutions.append(('%{compile_flags}', '-I %{libcxx-dir}/test/support'))
+config.substitutions.append(('%{link_flags}', ''))
+config.substitutions.append(('%{exec}', '%{executor} --execdir %T -- '))
+
+import os, site
+site.addsitedir(os.path.join('@LIBCXX_SOURCE_DIR@', 'utils'))
+import libcxx.test.params, libcxx.test.config
+libcxx.test.config.configure(
+    libcxx.test.params.DEFAULT_PARAMETERS,
+    libcxx.test.features.DEFAULT_FEATURES,
+    config,
+    lit_config
+)

[ldionne]

@jwakely Is there a clean way to determine these version-specific and target-specific paths without hardcoding them?

[jwakely]

I just use --gcc-toolchain=<install-prefix> when I want to point clang to a custom libstdc++ installation. Would that work here?

I don't think there's a direct way to get the include search dirs, so you'd have to parse the output of a command like:

~/gcc/14/bin/g++ -v -x c++ /dev/null -fsyntax-only 2>&1 | sed '1,/^#include <...> search starts here:/d;/^End of search list./,$d'

For the library dirs there's:

g++ -print-search-dirs | sed -n 's/^libraries: //p'

but that's not ideal either.

[ldionne]

This is a start. It's not perfect but it works and we can build on top of that, so I'll merge this.