Pass RewriteTypes=true to the GenXSPIRVWriterAdaptor pass

[NikitaRudenkoIntel]

This parameter enables encoding different VC attributes as
OpenCL types, which can be naturally translated from LLVM IR to SPIRV
and vice-versa.
Previously these attributes were transcoded using non-standard
spirv decorations, which are now deprecated and not supported.

[elizabethandrews]

FE changes look ok but I'm not familiar with metadata which needs to be generated here. So I cannot verify if the IR is correct. @bader I'm approving the change to unblock the review but someone who is more familiar with this probably needs to take a look.

[AlexeySachkov]

sycl-post-link changes are trivial, but I'm not familiar with all details of how ESIMD is implemented, so I can't verify the logic of those changes

[NikitaRudenkoIntel]

This PR is created to fix this issue: #3608

[NikitaRudenkoIntel]

Currently Jenkins fails on spec constants tests. I tried to debug it, but stuck on level_zero api.
The failure is happening inside build/tools/sycl/plugins/level_zero/level_zero/level_zero_loader/source/lib/ze_libapi.cpp here:

 821 ze_result_t ZE_APICALL
 822 zeContextCreate(
 823     ze_driver_handle_t hDriver,                     ///< [in] handle of the driver object
 824     const ze_context_desc_t* desc,                  ///< [in] pointer to context descriptor
 825     ze_context_handle_t* phContext                  ///< [out] pointer to handle of context object created
 826     )
 827 {
 828     auto pfnCreate = ze_lib::context->zeDdiTable.Context.pfnCreate;
 829     if( nullptr == pfnCreate )
 830         return ZE_RESULT_ERROR_UNSUPPORTED_VERSION;
 831
 832     return pfnCreate( hDriver, desc, phContext );
 833 }

pfnCreate returns PI_BUILD_PROGRAM_FAILURE without any diagnostic info. gdb cannot step into this func, so it is hard to understand what exactly level_zero does not like.

[bader]

This PR is created to fix this issue: #3608

Please, link them together.
Note, that you can use keywords in PR description for that.

[NikitaRudenkoIntel]

Ok, so I stepped into pfnCreate. The error is happening inside level_zero/core/source/module/module_imp.cpp in ModuleTranslationUnit::buildFromSpirV function. As I understand, it tries to get spec constants and for some reason it fails:

104     if (pConstants) {
105         NEO::SpecConstantInfo specConstInfo;
106         auto retVal = compilerInterface->getSpecConstantsInfo(*device->getNEODevice(), ArrayRef<const char>(input, inputSize), specConstInfo);
107         if (retVal != NEO::TranslationOutput::ErrorCode::Success) {
108             return false;
109         }
110         for (uint32_t i = 0; i < pConstants->numConstants; i++) {
111             uint64_t specConstantValue = 0;
112             uint32_t specConstantId = pConstants->pConstantIds[i];
113             auto atributeSize = 0u;
114             uint32_t j;
115             for (j = 0; j < specConstInfo.sizesBuffer->GetSize<uint32_t>(); j++) {
116                 if (specConstantId == specConstInfo.idsBuffer->GetMemory<uint32_t>()[j]) {
117                     atributeSize = specConstInfo.sizesBuffer->GetMemory<uint32_t>()[j];
118                     break;
119                 }
120             }
121             if (j == specConstInfo.sizesBuffer->GetSize<uint32_t>()) {
122                 return false;
123             }
124             memcpy_s(&specConstantValue, sizeof(uint64_t),
125                      const_cast<void *>(pConstants->pConstantValues[i]), atributeSize);
126             specConstantsValues[specConstantId] = specConstantValue;
127         }
128     }

So, it tries to find ID in idsBuffer, does not succeed and returns false on line 122. I cannot figure out why - this happens before even reading spirv kernel. It is not clear how reading spec constants in ZE is affected by the change of kernel arguments.

[NikitaRudenkoIntel]

Ok, so the problem is on IGC side. compilerInterface->getSpecConstantsInfo tries to determine number and sizes of spec constants by reading the spirv input. The problem is that this function calls the old SPIRV-LLVM-Translator from IGC, which does not support some features, including OpTypeBufferSurfaceINTEL. In turn, the latter is heavily used by the new scheme described in #3608
What happens here is that the old translator encounters OpTypeBufferSurfaceINTEL opcode and silently stops at this point (before actually reaching spec constants!) and tells level zero that there is no spec constants, which leads to further errors.
I will fix the problem on IGC side, and then return to this PR.

[kbobrovs]

LGTM.
I assume there are SPIRV translator tests elsewhere which verify that these new types are translated to SPIRV and back correctly.

[NikitaRudenkoIntel]

There are actually, here:
https://github.com/intel/vc-intrinsics/tree/master/GenXIntrinsics/test/Adaptors

[NikitaRudenkoIntel]

I fixed the problem on IGC side (see previous comments), so the question is how to update driver in Jenkins...

[bader]

I fixed the problem on IGC side (see previous comments), so the question is how to update driver in Jenkins...

@tfzhu and @DoyleLi can help with that. Basically, you need update driver version in https://github.com/intel/llvm/blob/sycl/buildbot/dependency.conf.
NOTE: the version must be publicly available.

[bader]

The content of the patch doesn't match the pull request description.
Please, address test failures.

[AGindinson]

@NikitaRudenkoIntel, are you planning on completing this PR? This would unblock #3462. Thanks!

[NikitaRudenkoIntel]

@NikitaRudenkoIntel, are you planning on completing this PR? This would unblock #3462. Thanks!

Well, changes in driver finally propagated to SYCL CI, and linux testing is passed. Now, have to investigate what is the deal with the windows part...

No objections from my side, but I think we need code owners to approve now. Thanks!

[NikitaRudenkoIntel]

@AGindinson, @AlexeySachkov, @kbobrovs after Windows driver got updated in CI, this finally passes all the checks.
Please take a look