[DirectX] introducing lowering for `bufferUpdateCounter` #115041

joaosaffran · 2024-11-05T18:19:12Z

Adding custom lowering for bufferUpdateCounter
introduces llvm intrinsic int_dx_updateCounter
adds tests

Closes #92147

github-actions · 2024-11-05T18:22:44Z

✅ With the latest revision this PR passed the C/C++ code formatter.

llvmbot · 2024-11-05T18:44:56Z

@llvm/pr-subscribers-llvm-ir

Author: None (joaosaffran)

Changes

Adding custom lowering for bufferUpdateCounter
introduces llvm intrinsic int_dx_updateCounter
adds tests

Closes #92147

Full diff: https://github.com/llvm/llvm-project/pull/115041.diff

4 Files Affected:

(modified) llvm/include/llvm/IR/IntrinsicsDirectX.td (+3)
(modified) llvm/lib/Target/DirectX/DXIL.td (+9)
(modified) llvm/lib/Target/DirectX/DXILOpLowering.cpp (+25)
(added) llvm/test/CodeGen/DirectX/updateCounter.ll (+31)

diff --git a/llvm/include/llvm/IR/IntrinsicsDirectX.td b/llvm/include/llvm/IR/IntrinsicsDirectX.td
index e30d37f69f781e..d460fe44e004d4 100644
--- a/llvm/include/llvm/IR/IntrinsicsDirectX.td
+++ b/llvm/include/llvm/IR/IntrinsicsDirectX.td
@@ -35,6 +35,9 @@ def int_dx_typedBufferLoad_checkbit
 def int_dx_typedBufferStore
     : DefaultAttrsIntrinsic<[], [llvm_any_ty, llvm_i32_ty, llvm_anyvector_ty]>;
 
+def int_dx_updateCounter
+    : DefaultAttrsIntrinsic<[], [llvm_any_ty, llvm_anyint_ty]>;
+    
 // Cast between target extension handle types and dxil-style opaque handles
 def int_dx_cast_handle : Intrinsic<[llvm_any_ty], [llvm_any_ty]>;
 
diff --git a/llvm/lib/Target/DirectX/DXIL.td b/llvm/lib/Target/DirectX/DXIL.td
index af12b74351058e..edfadd41d84216 100644
--- a/llvm/lib/Target/DirectX/DXIL.td
+++ b/llvm/lib/Target/DirectX/DXIL.td
@@ -730,6 +730,15 @@ def BufferStore : DXILOp<69, bufferStore> {
   let stages = [Stages<DXIL1_0, [all_stages]>];
 }
 
+def UpdateCounter : DXILOp<70, bufferUpdateCounter> {
+  let Doc = "increments/decrements a buffer counter";
+  let arguments = [
+    HandleTy, Int8Ty
+  ];
+  let result = VoidTy;
+  let stages = [Stages<DXIL1_0, [all_stages]>];
+}
+
 def CheckAccessFullyMapped : DXILOp<71, checkAccessFullyMapped> {
   let Doc = "checks whether a Sample, Gather, or Load operation "
             "accessed mapped tiles in a tiled resource";
diff --git a/llvm/lib/Target/DirectX/DXILOpLowering.cpp b/llvm/lib/Target/DirectX/DXILOpLowering.cpp
index 8acc9c1efa08c0..0dd3a8dc1ad4ce 100644
--- a/llvm/lib/Target/DirectX/DXILOpLowering.cpp
+++ b/llvm/lib/Target/DirectX/DXILOpLowering.cpp
@@ -463,6 +463,28 @@ class OpLowerer {
     });
   }
 
+  [[nodiscard]] bool lowerUpdateCounter(Function &F) {
+    IRBuilder<> &IRB = OpBuilder.getIRB();
+
+    return replaceFunction(F, [&](CallInst *CI) -> Error {
+      IRB.SetInsertPoint(CI);
+      Value *Handle =
+          createTmpHandleCast(CI->getArgOperand(0), OpBuilder.getHandleType());
+      Value *Op1 = CI->getArgOperand(1);
+
+      std::array<Value *, 2> Args{Handle, Op1};
+
+      Expected<CallInst *> OpCall =
+          OpBuilder.tryCreateOp(OpCode::UpdateCounter, Args, CI->getName());
+
+      if (Error E = OpCall.takeError())
+        return E;
+
+      CI->eraseFromParent();
+      return Error::success();
+    });
+  }
+
   [[nodiscard]] bool lowerTypedBufferStore(Function &F) {
     IRBuilder<> &IRB = OpBuilder.getIRB();
     Type *Int8Ty = IRB.getInt8Ty();
@@ -600,6 +622,9 @@ class OpLowerer {
       case Intrinsic::dx_typedBufferStore:
         HasErrors |= lowerTypedBufferStore(F);
         break;
+      case Intrinsic::dx_updateCounter:
+        HasErrors |= lowerUpdateCounter(F);
+        break;
       // TODO: this can be removed when
       // https://github.com/llvm/llvm-project/issues/113192 is fixed
       case Intrinsic::dx_splitdouble:
diff --git a/llvm/test/CodeGen/DirectX/updateCounter.ll b/llvm/test/CodeGen/DirectX/updateCounter.ll
new file mode 100644
index 00000000000000..459668c0a3cb51
--- /dev/null
+++ b/llvm/test/CodeGen/DirectX/updateCounter.ll
@@ -0,0 +1,31 @@
+; RUN: opt -S -dxil-op-lower %s | FileCheck %s
+
+
+target triple = "dxil-pc-shadermodel6.6-compute"
+
+define void @update_counter_decrement_vector() {
+  %buffer = call target("dx.TypedBuffer", <4 x float>, 0, 0, 0)
+      @llvm.dx.handle.fromBinding.tdx.TypedBuffer_v4f32_0_0_0(
+          i32 0, i32 0, i32 1, i32 0, i1 false)
+
+  call void @llvm.dx.updateCounter(target("dx.TypedBuffer", <4 x float>, 0, 0, 0) %buffer, i8 -1)
+  ret void
+}
+
+define void @update_counter_increment_vector() {
+  %buffer = call target("dx.TypedBuffer", <4 x float>, 0, 0, 0)
+      @llvm.dx.handle.fromBinding.tdx.TypedBuffer_v4f32_0_0_0(
+          i32 0, i32 0, i32 1, i32 0, i1 false)
+
+  call void @llvm.dx.updateCounter(target("dx.TypedBuffer", <4 x float>, 0, 0, 0) %buffer, i8 1)
+  ret void
+}
+
+define void @update_counter_decrement_scalar() {
+  %buffer = call target("dx.RawBuffer", i8, 0, 0)
+      @llvm.dx.handle.fromBinding.tdx.RawBuffer_i8_0_0t(
+          i32 1, i32 8, i32 1, i32 0, i1 false)
+
+  call void @llvm.dx.updateCounter(target("dx.RawBuffer", i8, 0, 0) %buffer, i8 -1)
+  ret void
+}

llvmbot · 2024-11-05T18:44:57Z

@llvm/pr-subscribers-backend-directx

Author: None (joaosaffran)

Changes

Adding custom lowering for bufferUpdateCounter
introduces llvm intrinsic int_dx_updateCounter
adds tests

Closes #92147

Full diff: https://github.com/llvm/llvm-project/pull/115041.diff

4 Files Affected:

(modified) llvm/include/llvm/IR/IntrinsicsDirectX.td (+3)
(modified) llvm/lib/Target/DirectX/DXIL.td (+9)
(modified) llvm/lib/Target/DirectX/DXILOpLowering.cpp (+25)
(added) llvm/test/CodeGen/DirectX/updateCounter.ll (+31)

diff --git a/llvm/include/llvm/IR/IntrinsicsDirectX.td b/llvm/include/llvm/IR/IntrinsicsDirectX.td
index e30d37f69f781e..d460fe44e004d4 100644
--- a/llvm/include/llvm/IR/IntrinsicsDirectX.td
+++ b/llvm/include/llvm/IR/IntrinsicsDirectX.td
@@ -35,6 +35,9 @@ def int_dx_typedBufferLoad_checkbit
 def int_dx_typedBufferStore
     : DefaultAttrsIntrinsic<[], [llvm_any_ty, llvm_i32_ty, llvm_anyvector_ty]>;
 
+def int_dx_updateCounter
+    : DefaultAttrsIntrinsic<[], [llvm_any_ty, llvm_anyint_ty]>;
+    
 // Cast between target extension handle types and dxil-style opaque handles
 def int_dx_cast_handle : Intrinsic<[llvm_any_ty], [llvm_any_ty]>;
 
diff --git a/llvm/lib/Target/DirectX/DXIL.td b/llvm/lib/Target/DirectX/DXIL.td
index af12b74351058e..edfadd41d84216 100644
--- a/llvm/lib/Target/DirectX/DXIL.td
+++ b/llvm/lib/Target/DirectX/DXIL.td
@@ -730,6 +730,15 @@ def BufferStore : DXILOp<69, bufferStore> {
   let stages = [Stages<DXIL1_0, [all_stages]>];
 }
 
+def UpdateCounter : DXILOp<70, bufferUpdateCounter> {
+  let Doc = "increments/decrements a buffer counter";
+  let arguments = [
+    HandleTy, Int8Ty
+  ];
+  let result = VoidTy;
+  let stages = [Stages<DXIL1_0, [all_stages]>];
+}
+
 def CheckAccessFullyMapped : DXILOp<71, checkAccessFullyMapped> {
   let Doc = "checks whether a Sample, Gather, or Load operation "
             "accessed mapped tiles in a tiled resource";
diff --git a/llvm/lib/Target/DirectX/DXILOpLowering.cpp b/llvm/lib/Target/DirectX/DXILOpLowering.cpp
index 8acc9c1efa08c0..0dd3a8dc1ad4ce 100644
--- a/llvm/lib/Target/DirectX/DXILOpLowering.cpp
+++ b/llvm/lib/Target/DirectX/DXILOpLowering.cpp
@@ -463,6 +463,28 @@ class OpLowerer {
     });
   }
 
+  [[nodiscard]] bool lowerUpdateCounter(Function &F) {
+    IRBuilder<> &IRB = OpBuilder.getIRB();
+
+    return replaceFunction(F, [&](CallInst *CI) -> Error {
+      IRB.SetInsertPoint(CI);
+      Value *Handle =
+          createTmpHandleCast(CI->getArgOperand(0), OpBuilder.getHandleType());
+      Value *Op1 = CI->getArgOperand(1);
+
+      std::array<Value *, 2> Args{Handle, Op1};
+
+      Expected<CallInst *> OpCall =
+          OpBuilder.tryCreateOp(OpCode::UpdateCounter, Args, CI->getName());
+
+      if (Error E = OpCall.takeError())
+        return E;
+
+      CI->eraseFromParent();
+      return Error::success();
+    });
+  }
+
   [[nodiscard]] bool lowerTypedBufferStore(Function &F) {
     IRBuilder<> &IRB = OpBuilder.getIRB();
     Type *Int8Ty = IRB.getInt8Ty();
@@ -600,6 +622,9 @@ class OpLowerer {
       case Intrinsic::dx_typedBufferStore:
         HasErrors |= lowerTypedBufferStore(F);
         break;
+      case Intrinsic::dx_updateCounter:
+        HasErrors |= lowerUpdateCounter(F);
+        break;
       // TODO: this can be removed when
       // https://github.com/llvm/llvm-project/issues/113192 is fixed
       case Intrinsic::dx_splitdouble:
diff --git a/llvm/test/CodeGen/DirectX/updateCounter.ll b/llvm/test/CodeGen/DirectX/updateCounter.ll
new file mode 100644
index 00000000000000..459668c0a3cb51
--- /dev/null
+++ b/llvm/test/CodeGen/DirectX/updateCounter.ll
@@ -0,0 +1,31 @@
+; RUN: opt -S -dxil-op-lower %s | FileCheck %s
+
+
+target triple = "dxil-pc-shadermodel6.6-compute"
+
+define void @update_counter_decrement_vector() {
+  %buffer = call target("dx.TypedBuffer", <4 x float>, 0, 0, 0)
+      @llvm.dx.handle.fromBinding.tdx.TypedBuffer_v4f32_0_0_0(
+          i32 0, i32 0, i32 1, i32 0, i1 false)
+
+  call void @llvm.dx.updateCounter(target("dx.TypedBuffer", <4 x float>, 0, 0, 0) %buffer, i8 -1)
+  ret void
+}
+
+define void @update_counter_increment_vector() {
+  %buffer = call target("dx.TypedBuffer", <4 x float>, 0, 0, 0)
+      @llvm.dx.handle.fromBinding.tdx.TypedBuffer_v4f32_0_0_0(
+          i32 0, i32 0, i32 1, i32 0, i1 false)
+
+  call void @llvm.dx.updateCounter(target("dx.TypedBuffer", <4 x float>, 0, 0, 0) %buffer, i8 1)
+  ret void
+}
+
+define void @update_counter_decrement_scalar() {
+  %buffer = call target("dx.RawBuffer", i8, 0, 0)
+      @llvm.dx.handle.fromBinding.tdx.RawBuffer_i8_0_0t(
+          i32 1, i32 8, i32 1, i32 0, i1 false)
+
+  call void @llvm.dx.updateCounter(target("dx.RawBuffer", i8, 0, 0) %buffer, i8 -1)
+  ret void
+}

llvm/test/CodeGen/DirectX/updateCounter.ll

llvm/lib/Target/DirectX/DXIL.td

llvm/include/llvm/IR/IntrinsicsDirectX.td

llvm/lib/Target/DirectX/DXILOpLowering.cpp

hekota · 2024-11-09T14:40:08Z

@joaosaffran - updateBufferCounter needs to return uint with the pre-incremented or pre-decremented counter value:

See https://godbolt.org/z/rGhWbcrY5
https://learn.microsoft.com/en-us/windows/win32/direct3dhlsl/sm5-object-rwstructuredbuffer-incrementcounter

joaosaffran · 2024-11-09T17:48:11Z

Will address this issue in a future PR @hekota, thanks for pointing it out.

- Adding custom lowering for `bufferUpdateCounter` - introduces llvm intrinsic `int_dx_updateCounter` - adds tests Closes llvm#92147 --------- Co-authored-by: Joao Saffran <[email protected]>

adding OPCode and Intrinsic

cfb0d93

joaosaffran force-pushed the intrinsics/bufferUpdateCounter branch from 79af814 to 682d101 Compare November 5, 2024 18:43

joaosaffran marked this pull request as ready for review November 5, 2024 18:44

llvmbot added backend:DirectX llvm:ir labels Nov 5, 2024

bogner reviewed Nov 6, 2024

View reviewed changes

llvm/test/CodeGen/DirectX/updateCounter.ll Show resolved Hide resolved

llvm/lib/Target/DirectX/DXIL.td Outdated Show resolved Hide resolved

joaosaffran force-pushed the intrinsics/bufferUpdateCounter branch from 682d101 to 5bc9d2d Compare November 6, 2024 22:21

adding test

0ee2596

joaosaffran force-pushed the intrinsics/bufferUpdateCounter branch from 5bc9d2d to 0ee2596 Compare November 6, 2024 22:22

bogner approved these changes Nov 7, 2024

View reviewed changes

spall reviewed Nov 7, 2024

View reviewed changes

llvm/include/llvm/IR/IntrinsicsDirectX.td Outdated Show resolved Hide resolved

addressing pr comments

02e8f67

inbelic reviewed Nov 7, 2024

View reviewed changes

llvm/lib/Target/DirectX/DXILOpLowering.cpp Show resolved Hide resolved

joaosaffran merged commit 1f25099 into llvm:main Nov 7, 2024
8 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[DirectX] introducing lowering for `bufferUpdateCounter` #115041

[DirectX] introducing lowering for `bufferUpdateCounter` #115041

Uh oh!

joaosaffran commented Nov 5, 2024

Uh oh!

github-actions bot commented Nov 5, 2024 •

edited

Loading

Uh oh!

llvmbot commented Nov 5, 2024

Uh oh!

llvmbot commented Nov 5, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hekota commented Nov 9, 2024

Uh oh!

joaosaffran commented Nov 9, 2024

Uh oh!

Uh oh!

[DirectX] introducing lowering for bufferUpdateCounter #115041

[DirectX] introducing lowering for bufferUpdateCounter #115041

Uh oh!

Conversation

joaosaffran commented Nov 5, 2024

Uh oh!

github-actions bot commented Nov 5, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

llvmbot commented Nov 5, 2024

Uh oh!

llvmbot commented Nov 5, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hekota commented Nov 9, 2024

Uh oh!

joaosaffran commented Nov 9, 2024

Uh oh!

Uh oh!

[DirectX] introducing lowering for `bufferUpdateCounter` #115041

[DirectX] introducing lowering for `bufferUpdateCounter` #115041

github-actions bot commented Nov 5, 2024 •

edited

Loading