[SYCL] WG-shared global variables must have external linkage

Currently hierarchical parallelism semantics is handled by SYCL specific
code generation and LowerWGScope pass. WG-shared global variables are
created for automatic variables in PFWG scope by CG and WG-shared shadow
variables are created by LowerWGScope pass to broadcast private value
from leader work item to other work items.

Currently these global variables are created with internal linkage which
is not correct. As a result wrong transformations are happening in the
LLVM middle end.

For example,

...
if (Leader work item)
    store %PrivateValue to @SharedGlobal -> leader shares the value
memory_barrier()
load %PrivateValue from @SharedGlobal -> all WIs load the shared value
...

Generated load/store operations are not supposed to be moved across
memory barrier but barrier intrinsics like @llvm.nvvm.barrier0() are
considered as regular functions in the LLVM middle end. As soon as
global has an interanl linkage it is considered as non-escaping and alias
analysis thinks that @llvm.nvvm.barrier0() cannot modify global variable
and only reads it. As a result the following transformation is performed
by GVN:

...
crit_edge:
  load %PrivateValue from @SharedGlobal -> all WIs load the shared value

if (Leader work item)
   store %PrivateValue to @SharedGlobal -> leader shares the value
memory_barrier()
...

That is why all WG-shared variables should have external linkage.

Signed-off-by: Artur Gainullin <[email protected]>

Author: againull

clang/lib/CodeGen/CGSYCLRuntime.cpp

@@ -82,7 +82,7 @@ void CGSYCLRuntime::emitWorkGroupLocalVarDecl(CodeGenFunction &CGF,
 #endif // NDEBUG
   // generate global variable in the address space selected by the clang CodeGen
   // (should be local)
-  CGF.EmitStaticVarDecl(D, llvm::GlobalValue::InternalLinkage);
+  CGF.EmitStaticVarDecl(D, llvm::GlobalValue::ExternalLinkage);
 }
 
 bool CGSYCLRuntime::actOnAutoVarEmit(CodeGenFunction &CGF, const VarDecl &D,

clang/test/CodeGenSYCL/wg_scope_var.cpp

@@ -1,7 +1,7 @@
 // RUN: %clang_cc1 -triple spir64-unknown-unknown-sycldevice -fsycl-is-device -disable-llvm-passes -I %S/Inputs -emit-llvm %s -o - | FileCheck %s
 
 // Checked that local variables declared by the user in PWFG scope are turned into globals in the local address space.
-// CHECK: @{{.*myLocal.*}} = internal addrspace(3) global i32 0
+// CHECK: @{{.*myLocal.*}} = addrspace(3) global i32 0
 
 #include "sycl.hpp"
 

llvm/lib/SYCLLowerIR/LowerWGScope.cpp

@@ -838,7 +838,7 @@ GlobalVariable *spirv::createWGLocalVariable(Module &M, Type *T,
       new GlobalVariable(M,                              // module
                          T,                              // type
                          false,                          // isConstant
-                         GlobalValue::InternalLinkage,   // Linkage
+                         GlobalValue::ExternalLinkage,   // Linkage
                          UndefValue::get(T),             // Initializer
                          Name,                           // Name
                          nullptr,                        // InsertBefore

llvm/test/SYCLLowerIR/byval_arg.ll

@@ -6,7 +6,7 @@
 
 %struct.baz = type { i64 }
 
-; CHECK: @[[SHADOW:[a-zA-Z0-9]+]] = internal unnamed_addr addrspace(3) global %struct.baz undef
+; CHECK: @[[SHADOW:[a-zA-Z0-9]+]] = unnamed_addr addrspace(3) global %struct.baz
 
 define internal spir_func void @wibble(%struct.baz* byval(%struct.baz) %arg1) !work_group_scope !0 {
 ; CHECK-LABEL: @wibble(

llvm/test/SYCLLowerIR/pfwg_and_pfwi.ll

@@ -13,9 +13,9 @@
 %struct.foo = type { %struct.barney }
 %struct.foo.0 = type { i8 }
 
-; CHECK: @[[PFWG_SHADOW:.*]] = internal unnamed_addr addrspace(3) global %struct.bar addrspace(4)*
-; CHECK: @[[PFWI_SHADOW:.*]] = internal unnamed_addr addrspace(3) global %struct.foo.0
-; CHECK: @[[GROUP_SHADOW:.*]] = internal unnamed_addr addrspace(3) global %struct.zot
+; CHECK: @[[PFWG_SHADOW:.*]] = unnamed_addr addrspace(3) global %struct.bar addrspace(4)*
+; CHECK: @[[PFWI_SHADOW:.*]] = unnamed_addr addrspace(3) global %struct.foo.0
+; CHECK: @[[GROUP_SHADOW:.*]] = unnamed_addr addrspace(3) global %struct.zot
 
 define internal spir_func void @wibble(%struct.bar addrspace(4)* %arg, %struct.zot* byval(%struct.zot) align 8 %arg1) align 2 !work_group_scope !0 {
 ; CHECK-LABEL: @wibble(

sycl/test/hier_par/hier_par_basic.cpp

@@ -12,9 +12,6 @@
 // RUN: %GPU_RUN_PLACEHOLDER %t.out
 // RUN: %ACC_RUN_PLACEHOLDER %t.out
 
-// TODO: ptxas fatal   : Unresolved extern function '__spirv_ControlBarrier'
-// XFAIL: cuda
-
 // This test checks hierarchical parallelism invocation APIs, but without any
 // data or code with side-effects between the work group and work item scopes.
 

sycl/test/hier_par/hier_par_wgscope.cpp

@@ -18,9 +18,6 @@
 // RUN: %GPU_RUN_PLACEHOLDER %t.out
 // RUN: %ACC_RUN_PLACEHOLDER %t.out
 
-// TODO: ptxas fatal   : Unresolved extern function '__spirv_ControlBarrier'
-// UNSUPPORTED: cuda
-
 // This test checks correctness of hierarchical kernel execution when there is
 // code and data in the work group scope.