[Driver][SYCL][FPGA] Improve AOCX archive behaviors with -fsycl-link=image #11592

mdtoguchi · 2023-10-19T00:27:29Z

Use of -fsycl-link=image allows for FPGA users to create a full device binary and store it in an archive to be used later. This archive can then be picked up and linked in. The information stored in the archive was not sufficient, causing the device binary to not be 'visible' when linked in.

Improve the situation here by saving all of the information provided by the wrapping step (including symbols) in the archive, but also be sure that this entire final binary is saved as it would be if we were performing a full E2E compilation.

To accomodate the ability to pass around the needed symbols and properties that are used when wrapping the final binaries for any AOCR or AOCX type archives, update the way this information is passed around. Multiple binaries are added to the archive, including the full wrapped .bc file that is gathered and passed along to any subsequent wrapping sequence that occurs.

…image Use of -fsycl-link=image allows for FPGA users to create a full device binary and store it in an archive to be used later. This archive can then be picked up and linked in. The information stored in the archive was not sufficient, causing the device binary to not be 'visible' when linked in. Improve the situation here by saving all of the information provided by the wrapping step (including symbols) in the archive, but also be sure that this entire final binary is saved as it would be if we were performing a full E2E compilation. To accomodate the ability to pass around the needed symbols and properties that are used when wrapping the final binaries for any AOCR or AOCX type archives, update the way this information is passed around. Multiple binaries are added to the archive, including the full wrapped .bc file that is gathered and passed along to any subsequent wrapping sequence that occurs.

mdtoguchi · 2023-10-19T00:30:52Z

dependent on #10850

mdtoguchi · 2023-10-23T15:04:42Z

Also dependent on #11034

LU-JOHN · 2023-10-25T14:20:25Z

When we generate a new wrapped bc file with --sym-prop-bc-files, it is important that the order of the images provided through --sym-prop-bc-files matches the order of the images that is provided through the table. Otherwise the symbols and properties will be applied to mismatched images. Do the testcases check this?

Do we have E2E tests for this new functionality?

mdtoguchi · 2023-10-25T14:43:21Z

When we generate a new wrapped bc file with --sym-prop-bc-files, it is important that the order of the images provided through --sym-prop-bc-files matches the order of the images that is provided through the table. Otherwise the symbols and properties will be applied to mismatched images. Do the testcases check this?

Do we have E2E tests for this new functionality?

We should be able to test this via E2E with emulation. I don't think we have anything in place that performs AOCX and AOCR behaviors on our side.

LU-JOHN · 2023-10-25T21:23:25Z

We should be able to test this via E2E with emulation. I don't think we have anything in place that performs AOCX and AOCR behaviors on our side.

Will E2E tests be added to this PR or will they be done separately?

mdtoguchi · 2023-10-25T22:05:24Z

Will E2E tests be added to this PR or will they be done separately?

I think I can add here, add to the tests-e2e area.

…pdates

…case fails. Signed-off-by: Lu, John <[email protected]>

…ernels. Kernels provided by source (i.e. not a library). Splitting per-kernel. Signed-off-by: Lu, John <[email protected]>

Signed-off-by: Lu, John <[email protected]>

…pdates

…ga-fsycl-link-updates

hchilama

LGTM

mdtoguchi · 2023-11-04T00:38:45Z

@asudarsa, please take a look - thanks!

sycl/test-e2e/AOT/fpga-aoc-archive.cpp

clang/lib/Driver/ToolChains/Gnu.cpp

…pdates

mdtoguchi · 2023-11-14T02:31:35Z

@asudarsa, could you take another look? Thanks!

clang/lib/Driver/ToolChains/Clang.cpp

asudarsa

Overall changes look good to me. Minor nits. @LU-JOHN will need to approve as well.

Thanks

LU-JOHN

LGTM

…pdates

mdtoguchi · 2023-11-20T14:30:49Z

@intel/llvm-gatekeepers, can this be merged?

mdtoguchi requested review from asudarsa, LU-JOHN and hchilama October 19, 2023 00:28

mdtoguchi had a problem deploying to WindowsCILock October 19, 2023 00:55 — with GitHub Actions Failure

mdtoguchi temporarily deployed to WindowsCILock October 19, 2023 01:50 — with GitHub Actions Inactive

mdtoguchi mentioned this pull request Oct 23, 2023

[Driver][SYCL][FPGA] Improve AOCX behaviors with -fsycl-link=image #10499

Closed

mdtoguchi added 4 commits October 25, 2023 16:52

Merge remote-tracking branch 'intel_llvm/sycl' into fpga-fsycl-link-u…

66a988a

…pdates

Fix build problems from merge

ab53c01

Add E2E test that exercises -fsycl-link and FPGA AOT

e48b138

clang format

b10619f

mdtoguchi had a problem deploying to WindowsCILock October 26, 2023 00:51 — with GitHub Actions Failure

mdtoguchi temporarily deployed to WindowsCILock October 26, 2023 01:21 — with GitHub Actions Inactive

Update test, missing header file

a842591

mdtoguchi had a problem deploying to WindowsCILock October 26, 2023 19:15 — with GitHub Actions Failure

mdtoguchi temporarily deployed to WindowsCILock October 26, 2023 19:45 — with GitHub Actions Inactive

Remove any archives before creating new ones. Return non-zero if test…

255b394

…case fails. Signed-off-by: Lu, John <[email protected]>

LU-JOHN had a problem deploying to WindowsCILock October 30, 2023 19:04 — with GitHub Actions Failure

Lu, John added 2 commits October 30, 2023 12:43

Add more severe test case. Test more libraries. Libraries with more k…

6cce245

…ernels. Kernels provided by source (i.e. not a library). Splitting per-kernel. Signed-off-by: Lu, John <[email protected]>

Add XFAIL to split per-kernel test. Apply clang-format.

9455a61

Signed-off-by: Lu, John <[email protected]>

LU-JOHN had a problem deploying to WindowsCILock October 30, 2023 20:17 — with GitHub Actions Failure

LU-JOHN temporarily deployed to WindowsCILock October 30, 2023 21:02 — with GitHub Actions Inactive

mdtoguchi added 2 commits October 31, 2023 11:00

Merge remote-tracking branch 'intel_llvm/sycl' into fpga-fsycl-link-u…

55b2815

…pdates

Merge remote-tracking branch 'origin/fpga-fsycl-link-updates' into fp…

68e2988

…ga-fsycl-link-updates

hchilama approved these changes Nov 1, 2023

View reviewed changes

asudarsa reviewed Nov 8, 2023

View reviewed changes

sycl/test-e2e/AOT/fpga-aoc-archive.cpp Show resolved Hide resolved

asudarsa reviewed Nov 8, 2023

View reviewed changes

clang/lib/Driver/ToolChains/Gnu.cpp Show resolved Hide resolved

mdtoguchi added 3 commits November 8, 2023 12:58

Merge remote-tracking branch 'intel_llvm/sycl' into fpga-fsycl-link-u…

f493a46

…pdates

Update test to only retain the severe tests

6486280

update comment for TODO

953378b

mdtoguchi had a problem deploying to WindowsCILock November 8, 2023 21:10 — with GitHub Actions Failure

mdtoguchi requested a review from asudarsa November 8, 2023 22:13

mdtoguchi temporarily deployed to WindowsCILock November 8, 2023 23:58 — with GitHub Actions Inactive

mdtoguchi temporarily deployed to WindowsCILock November 9, 2023 01:50 — with GitHub Actions Inactive

mdtoguchi temporarily deployed to WindowsCILock November 9, 2023 02:34 — with GitHub Actions Inactive

asudarsa reviewed Nov 15, 2023

View reviewed changes

clang/lib/Driver/ToolChains/Clang.cpp Outdated Show resolved Hide resolved

asudarsa reviewed Nov 15, 2023

View reviewed changes

clang/lib/Driver/ToolChains/Clang.cpp Outdated Show resolved Hide resolved

asudarsa reviewed Nov 15, 2023

View reviewed changes

clang/lib/Driver/ToolChains/Clang.cpp Outdated Show resolved Hide resolved

asudarsa approved these changes Nov 15, 2023

View reviewed changes

LU-JOHN approved these changes Nov 15, 2023

View reviewed changes

mdtoguchi added 2 commits November 15, 2023 10:03

Merge remote-tracking branch 'intel_llvm/sycl' into fpga-fsycl-link-u…

7dfc826

…pdates

Address review comments (update comments)

e7769f2

mdtoguchi temporarily deployed to WindowsCILock November 15, 2023 18:12 — with GitHub Actions Inactive

mdtoguchi had a problem deploying to WindowsCILock November 15, 2023 19:05 — with GitHub Actions Failure

mdtoguchi temporarily deployed to WindowsCILock November 15, 2023 19:48 — with GitHub Actions Inactive

mdtoguchi had a problem deploying to WindowsCILock November 15, 2023 20:35 — with GitHub Actions Failure

mdtoguchi temporarily deployed to WindowsCILock November 17, 2023 01:31 — with GitHub Actions Inactive

mdtoguchi temporarily deployed to WindowsCILock November 17, 2023 08:56 — with GitHub Actions Inactive

asudarsa approved these changes Nov 17, 2023

View reviewed changes

dm-vodopyanov merged commit 219d4ef into intel:sycl Nov 20, 2023

KseniyaTikhomirova mentioned this pull request Nov 21, 2023

[SYCL][E2E] Reenable fpga_aocx.cpp test #11964

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Driver][SYCL][FPGA] Improve AOCX archive behaviors with -fsycl-link=image #11592

[Driver][SYCL][FPGA] Improve AOCX archive behaviors with -fsycl-link=image #11592

Uh oh!

mdtoguchi commented Oct 19, 2023

Uh oh!

mdtoguchi commented Oct 19, 2023

Uh oh!

mdtoguchi commented Oct 23, 2023

Uh oh!

LU-JOHN commented Oct 25, 2023

Uh oh!

mdtoguchi commented Oct 25, 2023

Uh oh!

LU-JOHN commented Oct 25, 2023

Uh oh!

mdtoguchi commented Oct 25, 2023

Uh oh!

hchilama left a comment

Uh oh!

mdtoguchi commented Nov 4, 2023

Uh oh!

Uh oh!

Uh oh!

mdtoguchi commented Nov 14, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

asudarsa left a comment

Uh oh!

LU-JOHN left a comment

Uh oh!

mdtoguchi commented Nov 20, 2023

Uh oh!

Uh oh!

[Driver][SYCL][FPGA] Improve AOCX archive behaviors with -fsycl-link=image #11592

[Driver][SYCL][FPGA] Improve AOCX archive behaviors with -fsycl-link=image #11592

Uh oh!

Conversation

mdtoguchi commented Oct 19, 2023

Uh oh!

mdtoguchi commented Oct 19, 2023

Uh oh!

mdtoguchi commented Oct 23, 2023

Uh oh!

LU-JOHN commented Oct 25, 2023

Uh oh!

mdtoguchi commented Oct 25, 2023

Uh oh!

LU-JOHN commented Oct 25, 2023

Uh oh!

mdtoguchi commented Oct 25, 2023

Uh oh!

hchilama left a comment

Choose a reason for hiding this comment

Uh oh!

mdtoguchi commented Nov 4, 2023

Uh oh!

Uh oh!

Uh oh!

mdtoguchi commented Nov 14, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

asudarsa left a comment

Choose a reason for hiding this comment

Uh oh!

LU-JOHN left a comment

Choose a reason for hiding this comment

Uh oh!

mdtoguchi commented Nov 20, 2023

Uh oh!

Uh oh!