Skip to content

[libclc] Move conversion builtins to the CLC library #124727

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Feb 12, 2025

Conversation

frasercrmck
Copy link
Contributor

@frasercrmck frasercrmck commented Jan 28, 2025

This commit moves the implementations of conversion builtins to the CLC
library. It keeps the dichotomy of regular vs. clspv implementations of
the conversions. However, for the sake of a consistent interface all CLC
conversion routines are built, even the ones that clspv opts out of in
the user-facing OpenCL layer.

It simultaneously updates the python script to use f-strings for
formatting.

@frasercrmck frasercrmck added the libclc libclc OpenCL library label Jan 28, 2025
@frasercrmck frasercrmck requested a review from arsenm January 28, 2025 10:40
@frasercrmck
Copy link
Contributor Author

CC @rjodinchr - are you able to test this PR out for your needs, by any chance? It's quite intrusive so I'd like to be extra sure I haven't broken anything.

Copy link

github-actions bot commented Jan 28, 2025

✅ With the latest revision this PR passed the Python code formatter.

@rjodinchr
Copy link
Contributor

This is not correct for clspv. It breaks everything and it feels like it needs quite some work to fix. FYI, I'll be OOO from Feb 3 to Feb 21. Can it wait that long? I'm not sure I'll have enough time to figure it out before the end of the week.

@rjodinchr
Copy link
Contributor

I think for clspv, we would need to invert the condition to skip user-facing functions. By that I mean that clspv will need all the convert_* functions, and not the __clc_convert_*_rte. Then we will need to update clspv to support natively the function with the __clc_ prefix.

@frasercrmck
Copy link
Contributor Author

I think for clspv, we would need to invert the condition to skip user-facing functions. By that I mean that clspv will need all the convert_* functions, and not the __clc_convert_*_rte. Then we will need to update clspv to support natively the function with the __clc_ prefix.

Oh dear - thanks for trying out the PR so promptly.

I fear that if clspv doesn't provide (e.g.) __clc_convert_*_rte then no other CLC function would be able to call that for its own implementation. The CLC library should be internally self-sufficient. The idea here is that the CLC maths functions will need to make use of CLC conversion functions, and not call out to OpenCL. That's why the current user-facing skips are as they are. It seems that I've missed something fundamental here.

I think I may need to try out clspv for myself so I get a better idea of what's going wrong.

@rjodinchr
Copy link
Contributor

In fact my setup was busted. In the end it seems to work without change.

@frasercrmck
Copy link
Contributor Author

In fact my setup was busted. In the end it seems to work without change.

Thanks! Just for reference, is it just clspv's check-spirv you were running to test? Anything else I should be aware of?

@rjodinchr
Copy link
Contributor

clspv's check-spirv is what you want to test yes. But I was running within Google infra to try directly using the OpenCL-CTS.

rjodinchr added a commit to rjodinchr/clspv that referenced this pull request Jan 28, 2025
This is following llvm/llvm-project#124727,
but it can land before the llvm change.
rjodinchr added a commit to rjodinchr/clspv that referenced this pull request Jan 28, 2025
This is following llvm/llvm-project#124727,
but it can land before the llvm change.

Ref google#1447
rjodinchr added a commit to rjodinchr/clspv that referenced this pull request Jan 28, 2025
This is following llvm/llvm-project#124727,
but it can land before the llvm change.

Ref google#1447
rjodinchr added a commit to google/clspv that referenced this pull request Jan 29, 2025
This is following llvm/llvm-project#124727,
but it can land before the llvm change.

Ref #1447
This commit moves the implementations of conversion builtins to the CLC
library. It keeps the dichotomy of regular vs. clspv implementations of
the conversions. However, for the sake of a consistent interface all CLC
conversion routines are built, even the ones that clspv opts out of in
the user-facing OpenCL layer.

It simultaneously updates the python script to use f-strings for
formatting.
@frasercrmck frasercrmck force-pushed the libclc-clc-conversions branch from 3945d49 to 585b1f7 Compare February 11, 2025 11:20
@frasercrmck
Copy link
Contributor Author

I'm seeing several (82 out of 1045) conversions failures in the OpenCL-CTS. It's in saturating conversions from floating-point types to integers, and in regular float-to-float conversions. It isn't a result of this PR as they are already present on main.

I will merge this PR and investigate the failures next. They should be easier to fix on top of this PR. I wonder if the clspv-specific conversions are more correct.

@frasercrmck frasercrmck merged commit 25c0554 into llvm:main Feb 12, 2025
8 checks passed
@frasercrmck frasercrmck deleted the libclc-clc-conversions branch February 12, 2025 08:55
flovent pushed a commit to flovent/llvm-project that referenced this pull request Feb 13, 2025
This commit moves the implementations of conversion builtins to the CLC
library. It keeps the dichotomy of regular vs. clspv implementations of
the conversions. However, for the sake of a consistent interface all CLC
conversion routines are built, even the ones that clspv opts out of in
the user-facing OpenCL layer.

It simultaneously updates the python script to use f-strings for
formatting.
joaosaffran pushed a commit to joaosaffran/llvm-project that referenced this pull request Feb 14, 2025
This commit moves the implementations of conversion builtins to the CLC
library. It keeps the dichotomy of regular vs. clspv implementations of
the conversions. However, for the sake of a consistent interface all CLC
conversion routines are built, even the ones that clspv opts out of in
the user-facing OpenCL layer.

It simultaneously updates the python script to use f-strings for
formatting.
@mgorny
Copy link
Member

mgorny commented Feb 15, 2025

This change broke standalone libclc builds on Gentoo:

make -C /tmp/portage/llvm-core/libclc-21.0.0_pre20250215/work/libclc_build/gentoo_common_config.cmake -G Ninja -DCMAKE_INSTALL_PREFIX=/usr -DLIBCLC_TARGETS_TO_BUILD=spirv-mesa3d-;spirv64-mesa3d-;r600--;amdgcn--;amdgcn-mesa-mesa3d;amdgcn--amdhsa -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_TOOLCHAIN_FILE=/tmp/portage/llvm-core/libclc-21.0.0_pre20250215/work/libclc_build/gentoo_toolchain.cmake /tmp/portage/llvm-core/libclc-21.0.0_pre20250215/work/libclc
loading initial cache file /tmp/portage/llvm-core/libclc-21.0.0_pre20250215/work/libclc_build/gentoo_common_config.cmake
-- The CXX compiler identification is GNU 14.2.1
-- The C compiler identification is GNU 14.2.1
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/lib/ccache/bin/x86_64-pc-linux-gnu-g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/lib/ccache/bin/x86_64-pc-linux-gnu-gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Performing Test HAVE_FFI_CALL
-- Performing Test HAVE_FFI_CALL - Success
-- Found FFI: /usr/lib64/libffi.so
-- Looking for histedit.h
-- Looking for histedit.h - found
-- Found LibEdit: /usr/include (found version "2.11")
-- Found ZLIB: /usr/lib64/libz.so (found version "1.3.1")
-- Found zstd: /usr/lib64/libzstd.so
-- Found LibXml2: /usr/lib64/libxml2.so (found version "2.13.5")
-- Linker detection: GNU ld
-- libclc LLVM version: 21.0.0gitc30a7f45
-- Found Python3: /tmp/portage/llvm-core/libclc-21.0.0_pre20250215/temp/python3.13/bin/python3 (found version "3.13.2") found components: Interpreter
-- libclc target 'amdgcn--' is enabled
--   device: tahiti ( pitcairn;verde;oland;hainan;bonaire;kabini;kaveri;hawaii;mullins;tonga;tongapro;iceland;carrizo;fiji;stoney;polaris10;polaris11;gfx602;gfx705;gfx805;gfx900;gfx902;gfx904;gfx906;gfx908;gfx909;gfx90a;gfx90c;gfx940;gfx941;gfx942;gfx1010;gfx1011;gfx1012;gfx1013;gfx1030;gfx1031;gfx1032;gfx1033;gfx1034;gfx1035;gfx1036;gfx1100;gfx1101;gfx1102;gfx1103;gfx1150;gfx1151;gfx1152;gfx1153;gfx1200;gfx1201 )
-- libclc target 'amdgcn--amdhsa' is enabled
--   device: none (  )
-- libclc target 'amdgcn-mesa-mesa3d' is enabled
--   device: tahiti ( pitcairn;verde;oland;hainan;bonaire;kabini;kaveri;hawaii;mullins;tonga;tongapro;iceland;carrizo;fiji;stoney;polaris10;polaris11;gfx602;gfx705;gfx805;gfx900;gfx902;gfx904;gfx906;gfx908;gfx909;gfx90a;gfx90c;gfx940;gfx941;gfx942;gfx1010;gfx1011;gfx1012;gfx1013;gfx1030;gfx1031;gfx1032;gfx1033;gfx1034;gfx1035;gfx1036;gfx1100;gfx1101;gfx1102;gfx1103;gfx1150;gfx1151;gfx1152;gfx1153;gfx1200;gfx1201 )
-- libclc target 'r600--' is enabled
--   device: cedar ( palm;sumo;sumo2;redwood;juniper )
--   device: cypress ( hemlock )
--   device: barts ( turks;caicos )
--   device: cayman ( aruba )
-- libclc target 'spirv-mesa3d-' is enabled
--   device: none (  )
-- libclc target 'spirv64-mesa3d-' is enabled
--   device: none (  )
-- <<< Gentoo configuration >>>
Build type      RelWithDebInfo
Install path    /usr
Compiler flags:
C               -march=znver2 --param=l1-cache-size=32 --param=l1-cache-line-size=64 -O2 -pipe -frecord-gcc-switches
C++             -march=znver2 --param=l1-cache-size=32 --param=l1-cache-line-size=64 -O2 -pipe -frecord-gcc-switches
Linker flags:
Executable      -Wl,-O1 -Wl,--as-needed -Wl,--defsym=__gentoo_check_ldflags__=0
Module          -Wl,-O1 -Wl,--as-needed -Wl,--defsym=__gentoo_check_ldflags__=0
Shared          -Wl,-O1 -Wl,--as-needed -Wl,--defsym=__gentoo_check_ldflags__=0
- Configuring done (1.5s)
-- Generating done (2.5s)
-- Build files have been written to: /tmp/portage/llvm-core/libclc-21.0.0_pre20250215/work/libclc_build
>>> Source configured.
>>> Compiling source in /tmp/portage/llvm-core/libclc-21.0.0_pre20250215/work/libclc ...
 * Source directory (CMAKE_USE_DIR): "/tmp/portage/llvm-core/libclc-21.0.0_pre20250215/work/libclc"
 * Build directory  (BUILD_DIR):     "/tmp/portage/llvm-core/libclc-21.0.0_pre20250215/work/libclc_build"
ninja -v -j12 -l13
ninja: error: 'clspv-generate_convert.cl', needed by 'obj.libclc.dir/tahiti-amdgcn--/clc-convert.cl.bc', missing and no known rule to make it

@mgorny
Copy link
Member

mgorny commented Feb 15, 2025

The same failure also happens when SPIRV targets are disabled.

@mgorny
Copy link
Member

mgorny commented Feb 15, 2025

I'm guessing that both amdgcn and nvptx targets now require clspv-generate_convert.cl, but the targets for them are defined only when clspv targets are enabled. No clue whether it means that the targets need to be defined unconditionally, or clc-convert.cl skipped when clspv targets are disabled.

@mgorny
Copy link
Member

mgorny commented Feb 15, 2025

Ah, it's just incorrect dependency. Filed #127315 to fix it.

frasercrmck added a commit to frasercrmck/llvm-project that referenced this pull request Feb 17, 2025
In llvm#127378 it was reported that builds without clspv targets enabled
were failing after llvm#124727, as all targets had a dependency on a file
that only clspv targets generated.

A quick fix was merged in llvm#127315 which wasn't correct. It moved the
dependency on those generated files to the spirv targets, instead of
onto the clspv targets. This means a build with spirv targets and
without clspv targets would see the same problems as llvm#127378 reported.

I tried simply removing the requirement to explicitly add dependencies
to the custom command, relying instead on the file-level dependencies.
This didn't seem reliable enough; in some cases on a Makefiles build,
the clang command compiling (e.g.,) convert.cl would begin before the
file was fully written.

Instead, we keep the target-level dependency but automatically infer it
based on the generated file name, to avoid manual book-keeping of
pairs of files and targets.

This commit also fixes what looks like an unintended bug where, when
ENABLE_RUNTIME_SUBNORMAL was enabled, the OpenCL conversions weren't
being compiled.
frasercrmck added a commit that referenced this pull request Feb 17, 2025
In #127378 it was reported that builds without clspv targets enabled
were failing after #124727, as all targets had a dependency on a file
that only clspv targets generated.

A quick fix was merged in #127315 which wasn't correct. It moved the
dependency on those generated files to the spirv targets, instead of
onto the clspv targets. This means a build with spirv targets and
without clspv targets would see the same problems as #127378 reported.

I tried simply removing the requirement to explicitly add dependencies
to the custom command, relying instead on the file-level dependencies.
This didn't seem reliable enough; in some cases on a Makefiles build,
the clang command compiling (e.g.,) convert.cl would begin before the
file was fully written.

Instead, we keep the target-level dependency but automatically infer it
based on the generated file name, to avoid manual book-keeping of pairs
of files and targets.

This commit also fixes what looks like an unintended bug where, when
ENABLE_RUNTIME_SUBNORMAL was enabled, the OpenCL conversions weren't
being compiled.
sivan-shani pushed a commit to sivan-shani/llvm-project that referenced this pull request Feb 24, 2025
This commit moves the implementations of conversion builtins to the CLC
library. It keeps the dichotomy of regular vs. clspv implementations of
the conversions. However, for the sake of a consistent interface all CLC
conversion routines are built, even the ones that clspv opts out of in
the user-facing OpenCL layer.

It simultaneously updates the python script to use f-strings for
formatting.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
libclc libclc OpenCL library
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants