[Clang][CodeGen] Emit `llvm.ptrmask` for `align_up` and `align_down` #71238

goldsteinn · 2023-11-03T21:48:19Z

Since PR's #69343 and #67166 we probably have enough support for
llvm.ptrmask to make it preferable to the GEP stategy.

llvmbot · 2023-11-03T21:48:47Z

@llvm/pr-subscribers-clang-codegen

@llvm/pr-subscribers-clang

Author: None (goldsteinn)

Changes

Since PR's #69343 and #67166 we probably have enough support for
llvm.ptrmask to make it preferable to the GEP stategy.

Full diff: https://github.com/llvm/llvm-project/pull/71238.diff

3 Files Affected:

(modified) clang/lib/CodeGen/CGBuiltin.cpp (+20-23)
(modified) clang/test/CodeGen/builtin-align-array.c (+12-24)
(modified) clang/test/CodeGen/builtin-align.c (+26-38)

diff --git a/clang/lib/CodeGen/CGBuiltin.cpp b/clang/lib/CodeGen/CGBuiltin.cpp
index 972aa1c708e5f65..e4d1b2c93e4b113 100644
--- a/clang/lib/CodeGen/CGBuiltin.cpp
+++ b/clang/lib/CodeGen/CGBuiltin.cpp
@@ -19671,41 +19671,38 @@ RValue CodeGenFunction::EmitBuiltinIsAligned(const CallExpr *E) {
 /// TODO: actually use ptrmask once most optimization passes know about it.
 RValue CodeGenFunction::EmitBuiltinAlignTo(const CallExpr *E, bool AlignUp) {
   BuiltinAlignArgs Args(E, *this);
-  llvm::Value *SrcAddr = Args.Src;
-  if (Args.Src->getType()->isPointerTy())
-    SrcAddr = Builder.CreatePtrToInt(Args.Src, Args.IntType, "intptr");
-  llvm::Value *SrcForMask = SrcAddr;
+  llvm::Value *SrcForMask = Args.Src;
   if (AlignUp) {
     // When aligning up we have to first add the mask to ensure we go over the
     // next alignment value and then align down to the next valid multiple.
     // By adding the mask, we ensure that align_up on an already aligned
     // value will not change the value.
-    SrcForMask = Builder.CreateAdd(SrcForMask, Args.Mask, "over_boundary");
+    if (Args.Src->getType()->isPointerTy()) {
+      if (getLangOpts().isSignedOverflowDefined())
+        SrcForMask =
+            Builder.CreateGEP(Int8Ty, SrcForMask, Args.Mask, "over_boundary");
+      else
+        SrcForMask = EmitCheckedInBoundsGEP(Int8Ty, SrcForMask, Args.Mask,
+                                            /*SignedIndices=*/true,
+                                            /*isSubtraction=*/false,
+                                            E->getExprLoc(), "over_boundary");
+    } else {
+      SrcForMask = Builder.CreateAdd(SrcForMask, Args.Mask, "over_boundary");
+    }
   }
   // Invert the mask to only clear the lower bits.
   llvm::Value *InvertedMask = Builder.CreateNot(Args.Mask, "inverted_mask");
-  llvm::Value *Result =
-      Builder.CreateAnd(SrcForMask, InvertedMask, "aligned_result");
+  llvm::Value *Result = nullptr;
   if (Args.Src->getType()->isPointerTy()) {
-    /// TODO: Use ptrmask instead of ptrtoint+gep once it is optimized well.
-    // Result = Builder.CreateIntrinsic(
-    //  Intrinsic::ptrmask, {Args.SrcType, SrcForMask->getType(), Args.IntType},
-    //  {SrcForMask, NegatedMask}, nullptr, "aligned_result");
-    Result->setName("aligned_intptr");
-    llvm::Value *Difference = Builder.CreateSub(Result, SrcAddr, "diff");
-    // The result must point to the same underlying allocation. This means we
-    // can use an inbounds GEP to enable better optimization.
-    if (getLangOpts().isSignedOverflowDefined())
-      Result =
-          Builder.CreateGEP(Int8Ty, Args.Src, Difference, "aligned_result");
-    else
-      Result = EmitCheckedInBoundsGEP(Int8Ty, Args.Src, Difference,
-                                      /*SignedIndices=*/true,
-                                      /*isSubtraction=*/!AlignUp,
-                                      E->getExprLoc(), "aligned_result");
+    Result = Builder.CreateIntrinsic(
+        Intrinsic::ptrmask, {Args.SrcType, Args.IntType},
+        {SrcForMask, InvertedMask}, nullptr, "aligned_result");
+
     // Emit an alignment assumption to ensure that the new alignment is
     // propagated to loads/stores, etc.
     emitAlignmentAssumption(Result, E, E->getExprLoc(), Args.Alignment);
+  } else {
+    Result = Builder.CreateAnd(SrcForMask, InvertedMask, "aligned_result");
   }
   assert(Result->getType() == Args.SrcType);
   return RValue::get(Result);
diff --git a/clang/test/CodeGen/builtin-align-array.c b/clang/test/CodeGen/builtin-align-array.c
index 5d1377b98d2814f..18a77b9a710db40 100644
--- a/clang/test/CodeGen/builtin-align-array.c
+++ b/clang/test/CodeGen/builtin-align-array.c
@@ -8,22 +8,16 @@ extern int func(char *c);
 // CHECK-NEXT:  entry:
 // CHECK-NEXT:    [[BUF:%.*]] = alloca [1024 x i8], align 16
 // CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds [1024 x i8], ptr [[BUF]], i64 0, i64 44
-// CHECK-NEXT:    [[INTPTR:%.*]] = ptrtoint ptr [[ARRAYIDX]] to i64
-// CHECK-NEXT:    [[ALIGNED_INTPTR:%.*]] = and i64 [[INTPTR]], -16
-// CHECK-NEXT:    [[DIFF:%.*]] = sub i64 [[ALIGNED_INTPTR]], [[INTPTR]]
-// CHECK-NEXT:    [[ALIGNED_RESULT:%.*]] = getelementptr inbounds i8, ptr [[ARRAYIDX]], i64 [[DIFF]]
+// CHECK-NEXT:    [[ALIGNED_RESULT:%.*]] = call ptr @llvm.ptrmask.p0.i64(ptr [[ARRAYIDX]], i64 -16)
 // CHECK-NEXT:    call void @llvm.assume(i1 true) [ "align"(ptr [[ALIGNED_RESULT]], i64 16) ]
 // CHECK-NEXT:    [[CALL:%.*]] = call i32 @func(ptr noundef [[ALIGNED_RESULT]])
 // CHECK-NEXT:    [[ARRAYIDX1:%.*]] = getelementptr inbounds [1024 x i8], ptr [[BUF]], i64 0, i64 22
-// CHECK-NEXT:    [[INTPTR2:%.*]] = ptrtoint ptr [[ARRAYIDX1]] to i64
-// CHECK-NEXT:    [[OVER_BOUNDARY:%.*]] = add i64 [[INTPTR2]], 31
-// CHECK-NEXT:    [[ALIGNED_INTPTR4:%.*]] = and i64 [[OVER_BOUNDARY]], -32
-// CHECK-NEXT:    [[DIFF5:%.*]] = sub i64 [[ALIGNED_INTPTR4]], [[INTPTR2]]
-// CHECK-NEXT:    [[ALIGNED_RESULT6:%.*]] = getelementptr inbounds i8, ptr [[ARRAYIDX1]], i64 [[DIFF5]]
-// CHECK-NEXT:    call void @llvm.assume(i1 true) [ "align"(ptr [[ALIGNED_RESULT6]], i64 32) ]
-// CHECK-NEXT:    [[CALL7:%.*]] = call i32 @func(ptr noundef [[ALIGNED_RESULT6]])
-// CHECK-NEXT:    [[ARRAYIDX8:%.*]] = getelementptr inbounds [1024 x i8], ptr [[BUF]], i64 0, i64 16
-// CHECK-NEXT:    [[SRC_ADDR:%.*]] = ptrtoint ptr [[ARRAYIDX8]] to i64
+// CHECK-NEXT:    [[OVER_BOUNDARY:%.*]] = getelementptr inbounds i8, ptr [[ARRAYIDX1]], i64 31
+// CHECK-NEXT:    [[ALIGNED_RESULT2:%.*]] = call ptr @llvm.ptrmask.p0.i64(ptr [[OVER_BOUNDARY]], i64 -32)
+// CHECK-NEXT:    call void @llvm.assume(i1 true) [ "align"(ptr [[ALIGNED_RESULT2]], i64 32) ]
+// CHECK-NEXT:    [[CALL3:%.*]] = call i32 @func(ptr noundef [[ALIGNED_RESULT2]])
+// CHECK-NEXT:    [[ARRAYIDX4:%.*]] = getelementptr inbounds [1024 x i8], ptr [[BUF]], i64 0, i64 16
+// CHECK-NEXT:    [[SRC_ADDR:%.*]] = ptrtoint ptr [[ARRAYIDX4]] to i64
 // CHECK-NEXT:    [[SET_BITS:%.*]] = and i64 [[SRC_ADDR]], 63
 // CHECK-NEXT:    [[IS_ALIGNED:%.*]] = icmp eq i64 [[SET_BITS]], 0
 // CHECK-NEXT:    [[CONV:%.*]] = zext i1 [[IS_ALIGNED]] to i32
@@ -40,20 +34,14 @@ int test_array(void) {
 // CHECK-NEXT:  entry:
 // CHECK-NEXT:    [[BUF:%.*]] = alloca [1024 x i8], align 32
 // CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds [1024 x i8], ptr [[BUF]], i64 0, i64 64
-// CHECK-NEXT:    [[INTPTR:%.*]] = ptrtoint ptr [[ARRAYIDX]] to i64
-// CHECK-NEXT:    [[ALIGNED_INTPTR:%.*]] = and i64 [[INTPTR]], -16
-// CHECK-NEXT:    [[DIFF:%.*]] = sub i64 [[ALIGNED_INTPTR]], [[INTPTR]]
-// CHECK-NEXT:    [[ALIGNED_RESULT:%.*]] = getelementptr inbounds i8, ptr [[ARRAYIDX]], i64 [[DIFF]]
+// CHECK-NEXT:    [[ALIGNED_RESULT:%.*]] = call ptr @llvm.ptrmask.p0.i64(ptr [[ARRAYIDX]], i64 -16)
 // CHECK-NEXT:    call void @llvm.assume(i1 true) [ "align"(ptr [[ALIGNED_RESULT]], i64 16) ]
 // CHECK-NEXT:    [[CALL:%.*]] = call i32 @func(ptr noundef [[ALIGNED_RESULT]])
 // CHECK-NEXT:    [[ARRAYIDX1:%.*]] = getelementptr inbounds [1024 x i8], ptr [[BUF]], i64 0, i64 32
-// CHECK-NEXT:    [[INTPTR2:%.*]] = ptrtoint ptr [[ARRAYIDX1]] to i64
-// CHECK-NEXT:    [[OVER_BOUNDARY:%.*]] = add i64 [[INTPTR2]], 31
-// CHECK-NEXT:    [[ALIGNED_INTPTR4:%.*]] = and i64 [[OVER_BOUNDARY]], -32
-// CHECK-NEXT:    [[DIFF5:%.*]] = sub i64 [[ALIGNED_INTPTR4]], [[INTPTR2]]
-// CHECK-NEXT:    [[ALIGNED_RESULT6:%.*]] = getelementptr inbounds i8, ptr [[ARRAYIDX1]], i64 [[DIFF5]]
-// CHECK-NEXT:    call void @llvm.assume(i1 true) [ "align"(ptr [[ALIGNED_RESULT6]], i64 32) ]
-// CHECK-NEXT:    [[CALL7:%.*]] = call i32 @func(ptr noundef [[ALIGNED_RESULT6]])
+// CHECK-NEXT:    [[OVER_BOUNDARY:%.*]] = getelementptr inbounds i8, ptr [[ARRAYIDX1]], i64 31
+// CHECK-NEXT:    [[ALIGNED_RESULT2:%.*]] = call ptr @llvm.ptrmask.p0.i64(ptr [[OVER_BOUNDARY]], i64 -32)
+// CHECK-NEXT:    call void @llvm.assume(i1 true) [ "align"(ptr [[ALIGNED_RESULT2]], i64 32) ]
+// CHECK-NEXT:    [[CALL3:%.*]] = call i32 @func(ptr noundef [[ALIGNED_RESULT2]])
 // CHECK-NEXT:    ret i32 1
 //
 int test_array_should_not_mask(void) {
diff --git a/clang/test/CodeGen/builtin-align.c b/clang/test/CodeGen/builtin-align.c
index 02661d19fa251de..b58d47078799eae 100644
--- a/clang/test/CodeGen/builtin-align.c
+++ b/clang/test/CodeGen/builtin-align.c
@@ -71,9 +71,9 @@ TYPE get_type(void) {
 // CHECK-VOID_PTR-LABEL: define {{[^@]+}}@is_aligned
 // CHECK-VOID_PTR-SAME: (ptr noundef [[PTR:%.*]], i32 noundef [[ALIGN:%.*]]) #0
 // CHECK-VOID_PTR-NEXT:  entry:
-// CHECK-VOID_PTR-NEXT:    [[ALIGNMENT:%.*]] = zext i32 [[ALIGN]] to i64
+// CHECK-VOID_PTR-NEXT:    [[ALIGNMENT:%.*]] = zext i32 [[ALIGN:%.*]] to i64
 // CHECK-VOID_PTR-NEXT:    [[MASK:%.*]] = sub i64 [[ALIGNMENT]], 1
-// CHECK-VOID_PTR-NEXT:    [[SRC_ADDR:%.*]] = ptrtoint ptr [[PTR]] to i64
+// CHECK-VOID_PTR-NEXT:    [[SRC_ADDR:%.*]] = ptrtoint ptr [[PTR:%.*]] to i64
 // CHECK-VOID_PTR-NEXT:    [[SET_BITS:%.*]] = and i64 [[SRC_ADDR]], [[MASK]]
 // CHECK-VOID_PTR-NEXT:    [[IS_ALIGNED:%.*]] = icmp eq i64 [[SET_BITS]], 0
 // CHECK-VOID_PTR-NEXT:    ret i1 [[IS_ALIGNED]]
@@ -81,9 +81,9 @@ TYPE get_type(void) {
 // CHECK-FLOAT_PTR-LABEL: define {{[^@]+}}@is_aligned
 // CHECK-FLOAT_PTR-SAME: (ptr noundef [[PTR:%.*]], i32 noundef [[ALIGN:%.*]]) #0
 // CHECK-FLOAT_PTR-NEXT:  entry:
-// CHECK-FLOAT_PTR-NEXT:    [[ALIGNMENT:%.*]] = zext i32 [[ALIGN]] to i64
+// CHECK-FLOAT_PTR-NEXT:    [[ALIGNMENT:%.*]] = zext i32 [[ALIGN:%.*]] to i64
 // CHECK-FLOAT_PTR-NEXT:    [[MASK:%.*]] = sub i64 [[ALIGNMENT]], 1
-// CHECK-FLOAT_PTR-NEXT:    [[SRC_ADDR:%.*]] = ptrtoint ptr [[PTR]] to i64
+// CHECK-FLOAT_PTR-NEXT:    [[SRC_ADDR:%.*]] = ptrtoint ptr [[PTR:%.*]] to i64
 // CHECK-FLOAT_PTR-NEXT:    [[SET_BITS:%.*]] = and i64 [[SRC_ADDR]], [[MASK]]
 // CHECK-FLOAT_PTR-NEXT:    [[IS_ALIGNED:%.*]] = icmp eq i64 [[SET_BITS]], 0
 // CHECK-FLOAT_PTR-NEXT:    ret i1 [[IS_ALIGNED]]
@@ -91,18 +91,18 @@ TYPE get_type(void) {
 // CHECK-LONG-LABEL: define {{[^@]+}}@is_aligned
 // CHECK-LONG-SAME: (i64 noundef [[PTR:%.*]], i32 noundef [[ALIGN:%.*]]) #0
 // CHECK-LONG-NEXT:  entry:
-// CHECK-LONG-NEXT:    [[ALIGNMENT:%.*]] = zext i32 [[ALIGN]] to i64
+// CHECK-LONG-NEXT:    [[ALIGNMENT:%.*]] = zext i32 [[ALIGN:%.*]] to i64
 // CHECK-LONG-NEXT:    [[MASK:%.*]] = sub i64 [[ALIGNMENT]], 1
-// CHECK-LONG-NEXT:    [[SET_BITS:%.*]] = and i64 [[PTR]], [[MASK]]
+// CHECK-LONG-NEXT:    [[SET_BITS:%.*]] = and i64 [[PTR:%.*]], [[MASK]]
 // CHECK-LONG-NEXT:    [[IS_ALIGNED:%.*]] = icmp eq i64 [[SET_BITS]], 0
 // CHECK-LONG-NEXT:    ret i1 [[IS_ALIGNED]]
 //
 // CHECK-USHORT-LABEL: define {{[^@]+}}@is_aligned
 // CHECK-USHORT-SAME: (i16 noundef zeroext [[PTR:%.*]], i32 noundef [[ALIGN:%.*]]) #0
 // CHECK-USHORT-NEXT:  entry:
-// CHECK-USHORT-NEXT:    [[ALIGNMENT:%.*]] = trunc i32 [[ALIGN]] to i16
+// CHECK-USHORT-NEXT:    [[ALIGNMENT:%.*]] = trunc i32 [[ALIGN:%.*]] to i16
 // CHECK-USHORT-NEXT:    [[MASK:%.*]] = sub i16 [[ALIGNMENT]], 1
-// CHECK-USHORT-NEXT:    [[SET_BITS:%.*]] = and i16 [[PTR]], [[MASK]]
+// CHECK-USHORT-NEXT:    [[SET_BITS:%.*]] = and i16 [[PTR:%.*]], [[MASK]]
 // CHECK-USHORT-NEXT:    [[IS_ALIGNED:%.*]] = icmp eq i16 [[SET_BITS]], 0
 // CHECK-USHORT-NEXT:    ret i1 [[IS_ALIGNED]]
 //
@@ -114,37 +114,31 @@ _Bool is_aligned(TYPE ptr, unsigned align) {
 // CHECK-VOID_PTR-LABEL: define {{[^@]+}}@align_up
 // CHECK-VOID_PTR-SAME: (ptr noundef [[PTR:%.*]], i32 noundef [[ALIGN:%.*]]) #0
 // CHECK-VOID_PTR-NEXT:  entry:
-// CHECK-VOID_PTR-NEXT:    [[ALIGNMENT:%.*]] = zext i32 [[ALIGN]] to i64
+// CHECK-VOID_PTR-NEXT:    [[ALIGNMENT:%.*]] = zext i32 [[ALIGN:%.*]] to i64
 // CHECK-VOID_PTR-NEXT:    [[MASK:%.*]] = sub i64 [[ALIGNMENT]], 1
-// CHECK-VOID_PTR-NEXT:    [[INTPTR:%.*]] = ptrtoint ptr [[PTR]] to i64
-// CHECK-VOID_PTR-NEXT:    [[OVER_BOUNDARY:%.*]] = add i64 [[INTPTR]], [[MASK]]
+// CHECK-VOID_PTR-NEXT:    [[OVER_BOUNDARY:%.*]] = getelementptr inbounds i8, ptr [[PTR:%.*]], i64 [[MASK]]
 // CHECK-VOID_PTR-NEXT:    [[INVERTED_MASK:%.*]] = xor i64 [[MASK]], -1
-// CHECK-VOID_PTR-NEXT:    [[ALIGNED_INTPTR:%.*]] = and i64 [[OVER_BOUNDARY]], [[INVERTED_MASK]]
-// CHECK-VOID_PTR-NEXT:    [[DIFF:%.*]] = sub i64 [[ALIGNED_INTPTR]], [[INTPTR]]
-// CHECK-VOID_PTR-NEXT:    [[ALIGNED_RESULT:%.*]] = getelementptr inbounds i8, ptr [[PTR]], i64 [[DIFF]]
+// CHECK-VOID_PTR-NEXT:    [[ALIGNED_RESULT:%.*]] = call ptr @llvm.ptrmask.p0.i64(ptr [[OVER_BOUNDARY]], i64 [[INVERTED_MASK]])
 // CHECK-VOID_PTR-NEXT:    call void @llvm.assume(i1 true) [ "align"(ptr [[ALIGNED_RESULT]], i64 [[ALIGNMENT]]) ]
 // CHECK-VOID_PTR-NEXT:    ret ptr [[ALIGNED_RESULT]]
 //
 // CHECK-FLOAT_PTR-LABEL: define {{[^@]+}}@align_up
 // CHECK-FLOAT_PTR-SAME: (ptr noundef [[PTR:%.*]], i32 noundef [[ALIGN:%.*]]) #0
 // CHECK-FLOAT_PTR-NEXT:  entry:
-// CHECK-FLOAT_PTR-NEXT:    [[ALIGNMENT:%.*]] = zext i32 [[ALIGN]] to i64
+// CHECK-FLOAT_PTR-NEXT:    [[ALIGNMENT:%.*]] = zext i32 [[ALIGN:%.*]] to i64
 // CHECK-FLOAT_PTR-NEXT:    [[MASK:%.*]] = sub i64 [[ALIGNMENT]], 1
-// CHECK-FLOAT_PTR-NEXT:    [[INTPTR:%.*]] = ptrtoint ptr [[PTR]] to i64
-// CHECK-FLOAT_PTR-NEXT:    [[OVER_BOUNDARY:%.*]] = add i64 [[INTPTR]], [[MASK]]
+// CHECK-FLOAT_PTR-NEXT:    [[OVER_BOUNDARY:%.*]] = getelementptr inbounds i8, ptr [[PTR:%.*]], i64 [[MASK]]
 // CHECK-FLOAT_PTR-NEXT:    [[INVERTED_MASK:%.*]] = xor i64 [[MASK]], -1
-// CHECK-FLOAT_PTR-NEXT:    [[ALIGNED_INTPTR:%.*]] = and i64 [[OVER_BOUNDARY]], [[INVERTED_MASK]]
-// CHECK-FLOAT_PTR-NEXT:    [[DIFF:%.*]] = sub i64 [[ALIGNED_INTPTR]], [[INTPTR]]
-// CHECK-FLOAT_PTR-NEXT:    [[ALIGNED_RESULT:%.*]] = getelementptr inbounds i8, ptr [[PTR]], i64 [[DIFF]]
+// CHECK-FLOAT_PTR-NEXT:    [[ALIGNED_RESULT:%.*]] = call ptr @llvm.ptrmask.p0.i64(ptr [[OVER_BOUNDARY]], i64 [[INVERTED_MASK]])
 // CHECK-FLOAT_PTR-NEXT:    call void @llvm.assume(i1 true) [ "align"(ptr [[ALIGNED_RESULT]], i64 [[ALIGNMENT]]) ]
 // CHECK-FLOAT_PTR-NEXT:    ret ptr [[ALIGNED_RESULT]]
 //
 // CHECK-LONG-LABEL: define {{[^@]+}}@align_up
 // CHECK-LONG-SAME: (i64 noundef [[PTR:%.*]], i32 noundef [[ALIGN:%.*]]) #0
 // CHECK-LONG-NEXT:  entry:
-// CHECK-LONG-NEXT:    [[ALIGNMENT:%.*]] = zext i32 [[ALIGN]] to i64
+// CHECK-LONG-NEXT:    [[ALIGNMENT:%.*]] = zext i32 [[ALIGN:%.*]] to i64
 // CHECK-LONG-NEXT:    [[MASK:%.*]] = sub i64 [[ALIGNMENT]], 1
-// CHECK-LONG-NEXT:    [[OVER_BOUNDARY:%.*]] = add i64 [[PTR]], [[MASK]]
+// CHECK-LONG-NEXT:    [[OVER_BOUNDARY:%.*]] = add i64 [[PTR:%.*]], [[MASK]]
 // CHECK-LONG-NEXT:    [[INVERTED_MASK:%.*]] = xor i64 [[MASK]], -1
 // CHECK-LONG-NEXT:    [[ALIGNED_RESULT:%.*]] = and i64 [[OVER_BOUNDARY]], [[INVERTED_MASK]]
 // CHECK-LONG-NEXT:    ret i64 [[ALIGNED_RESULT]]
@@ -152,9 +146,9 @@ _Bool is_aligned(TYPE ptr, unsigned align) {
 // CHECK-USHORT-LABEL: define {{[^@]+}}@align_up
 // CHECK-USHORT-SAME: (i16 noundef zeroext [[PTR:%.*]], i32 noundef [[ALIGN:%.*]]) #0
 // CHECK-USHORT-NEXT:  entry:
-// CHECK-USHORT-NEXT:    [[ALIGNMENT:%.*]] = trunc i32 [[ALIGN]] to i16
+// CHECK-USHORT-NEXT:    [[ALIGNMENT:%.*]] = trunc i32 [[ALIGN:%.*]] to i16
 // CHECK-USHORT-NEXT:    [[MASK:%.*]] = sub i16 [[ALIGNMENT]], 1
-// CHECK-USHORT-NEXT:    [[OVER_BOUNDARY:%.*]] = add i16 [[PTR]], [[MASK]]
+// CHECK-USHORT-NEXT:    [[OVER_BOUNDARY:%.*]] = add i16 [[PTR:%.*]], [[MASK]]
 // CHECK-USHORT-NEXT:    [[INVERTED_MASK:%.*]] = xor i16 [[MASK]], -1
 // CHECK-USHORT-NEXT:    [[ALIGNED_RESULT:%.*]] = and i16 [[OVER_BOUNDARY]], [[INVERTED_MASK]]
 // CHECK-USHORT-NEXT:    ret i16 [[ALIGNED_RESULT]]
@@ -167,45 +161,39 @@ TYPE align_up(TYPE ptr, unsigned align) {
 // CHECK-VOID_PTR-LABEL: define {{[^@]+}}@align_down
 // CHECK-VOID_PTR-SAME: (ptr noundef [[PTR:%.*]], i32 noundef [[ALIGN:%.*]]) #0
 // CHECK-VOID_PTR-NEXT:  entry:
-// CHECK-VOID_PTR-NEXT:    [[ALIGNMENT:%.*]] = zext i32 [[ALIGN]] to i64
+// CHECK-VOID_PTR-NEXT:    [[ALIGNMENT:%.*]] = zext i32 [[ALIGN:%.*]] to i64
 // CHECK-VOID_PTR-NEXT:    [[MASK:%.*]] = sub i64 [[ALIGNMENT]], 1
-// CHECK-VOID_PTR-NEXT:    [[INTPTR:%.*]] = ptrtoint ptr [[PTR]] to i64
 // CHECK-VOID_PTR-NEXT:    [[INVERTED_MASK:%.*]] = xor i64 [[MASK]], -1
-// CHECK-VOID_PTR-NEXT:    [[ALIGNED_INTPTR:%.*]] = and i64 [[INTPTR]], [[INVERTED_MASK]]
-// CHECK-VOID_PTR-NEXT:    [[DIFF:%.*]] = sub i64 [[ALIGNED_INTPTR]], [[INTPTR]]
-// CHECK-VOID_PTR-NEXT:    [[ALIGNED_RESULT:%.*]] = getelementptr inbounds i8, ptr [[PTR]], i64 [[DIFF]]
+// CHECK-VOID_PTR-NEXT:    [[ALIGNED_RESULT:%.*]] = call ptr @llvm.ptrmask.p0.i64(ptr [[PTR:%.*]], i64 [[INVERTED_MASK]])
 // CHECK-VOID_PTR-NEXT:    call void @llvm.assume(i1 true) [ "align"(ptr [[ALIGNED_RESULT]], i64 [[ALIGNMENT]]) ]
 // CHECK-VOID_PTR-NEXT:    ret ptr [[ALIGNED_RESULT]]
 //
 // CHECK-FLOAT_PTR-LABEL: define {{[^@]+}}@align_down
 // CHECK-FLOAT_PTR-SAME: (ptr noundef [[PTR:%.*]], i32 noundef [[ALIGN:%.*]]) #0
 // CHECK-FLOAT_PTR-NEXT:  entry:
-// CHECK-FLOAT_PTR-NEXT:    [[ALIGNMENT:%.*]] = zext i32 [[ALIGN]] to i64
+// CHECK-FLOAT_PTR-NEXT:    [[ALIGNMENT:%.*]] = zext i32 [[ALIGN:%.*]] to i64
 // CHECK-FLOAT_PTR-NEXT:    [[MASK:%.*]] = sub i64 [[ALIGNMENT]], 1
-// CHECK-FLOAT_PTR-NEXT:    [[INTPTR:%.*]] = ptrtoint ptr [[PTR]] to i64
 // CHECK-FLOAT_PTR-NEXT:    [[INVERTED_MASK:%.*]] = xor i64 [[MASK]], -1
-// CHECK-FLOAT_PTR-NEXT:    [[ALIGNED_INTPTR:%.*]] = and i64 [[INTPTR]], [[INVERTED_MASK]]
-// CHECK-FLOAT_PTR-NEXT:    [[DIFF:%.*]] = sub i64 [[ALIGNED_INTPTR]], [[INTPTR]]
-// CHECK-FLOAT_PTR-NEXT:    [[ALIGNED_RESULT:%.*]] = getelementptr inbounds i8, ptr [[PTR]], i64 [[DIFF]]
+// CHECK-FLOAT_PTR-NEXT:    [[ALIGNED_RESULT:%.*]] = call ptr @llvm.ptrmask.p0.i64(ptr [[PTR:%.*]], i64 [[INVERTED_MASK]])
 // CHECK-FLOAT_PTR-NEXT:    call void @llvm.assume(i1 true) [ "align"(ptr [[ALIGNED_RESULT]], i64 [[ALIGNMENT]]) ]
 // CHECK-FLOAT_PTR-NEXT:    ret ptr [[ALIGNED_RESULT]]
 //
 // CHECK-LONG-LABEL: define {{[^@]+}}@align_down
 // CHECK-LONG-SAME: (i64 noundef [[PTR:%.*]], i32 noundef [[ALIGN:%.*]]) #0
 // CHECK-LONG-NEXT:  entry:
-// CHECK-LONG-NEXT:    [[ALIGNMENT:%.*]] = zext i32 [[ALIGN]] to i64
+// CHECK-LONG-NEXT:    [[ALIGNMENT:%.*]] = zext i32 [[ALIGN:%.*]] to i64
 // CHECK-LONG-NEXT:    [[MASK:%.*]] = sub i64 [[ALIGNMENT]], 1
 // CHECK-LONG-NEXT:    [[INVERTED_MASK:%.*]] = xor i64 [[MASK]], -1
-// CHECK-LONG-NEXT:    [[ALIGNED_RESULT:%.*]] = and i64 [[PTR]], [[INVERTED_MASK]]
+// CHECK-LONG-NEXT:    [[ALIGNED_RESULT:%.*]] = and i64 [[PTR:%.*]], [[INVERTED_MASK]]
 // CHECK-LONG-NEXT:    ret i64 [[ALIGNED_RESULT]]
 //
 // CHECK-USHORT-LABEL: define {{[^@]+}}@align_down
 // CHECK-USHORT-SAME: (i16 noundef zeroext [[PTR:%.*]], i32 noundef [[ALIGN:%.*]]) #0
 // CHECK-USHORT-NEXT:  entry:
-// CHECK-USHORT-NEXT:    [[ALIGNMENT:%.*]] = trunc i32 [[ALIGN]] to i16
+// CHECK-USHORT-NEXT:    [[ALIGNMENT:%.*]] = trunc i32 [[ALIGN:%.*]] to i16
 // CHECK-USHORT-NEXT:    [[MASK:%.*]] = sub i16 [[ALIGNMENT]], 1
 // CHECK-USHORT-NEXT:    [[INVERTED_MASK:%.*]] = xor i16 [[MASK]], -1
-// CHECK-USHORT-NEXT:    [[ALIGNED_RESULT:%.*]] = and i16 [[PTR]], [[INVERTED_MASK]]
+// CHECK-USHORT-NEXT:    [[ALIGNED_RESULT:%.*]] = and i16 [[PTR:%.*]], [[INVERTED_MASK]]
 // CHECK-USHORT-NEXT:    ret i16 [[ALIGNED_RESULT]]
 //
 TYPE align_down(TYPE ptr, unsigned align) {

goldsteinn · 2023-11-03T21:50:43Z

If we get this in, any reason to add support for also directly canonicalizing the pattern (i.e: https://reviews.llvm.org/D154010) or is that gratuitous?

nikic

LGTM

As a possible followup, we can probably drop the alignment assumption now? I don't think it adds additional value if we're using ptrmask...

goldsteinn · 2023-11-03T23:22:51Z

LGTM

As a possible followup, we can probably drop the alignment assumption now? I don't think it adds additional value if we're using ptrmask...

Would think thats okay. Will post that after I push this.

zero9178 · 2023-11-04T09:11:11Z

clang/lib/CodeGen/CGBuiltin.cpp

@@ -19671,41 +19671,38 @@ RValue CodeGenFunction::EmitBuiltinIsAligned(const CallExpr *E) {
 /// TODO: actually use ptrmask once most optimization passes know about it.


Can this TODO be removed now?

Since PR's llvm#69343 and llvm#67166 we probably have enough support for `llvm.ptrmask` to make it preferable to the GEP stategy.

…wn}` Now that `align_{up,down}` use `llvm.ptrmask` (as of llvm#71238), the assume doesn't preserve any information that is not still easily re-computable.

…wn}` Now that `align_{up,down}` use `llvm.ptrmask` (as of #71238), the assume doesn't preserve any information that is not still easily re-computable. Closes #71295

llvmbot added clang Clang issues not falling into any other category clang:codegen IR generation bugs: mangling, exceptions, etc. labels Nov 3, 2023

goldsteinn requested review from nikic and arichardson November 3, 2023 21:49

nikic approved these changes Nov 3, 2023

View reviewed changes

zero9178 reviewed Nov 4, 2023

View reviewed changes

[Clang][CodeGen] Emit llvm.ptrmask for align_up and align_down

7df7db5

Since PR's llvm#69343 and llvm#67166 we probably have enough support for `llvm.ptrmask` to make it preferable to the GEP stategy.

goldsteinn force-pushed the goldsteinn/ptrmask-clang-codegen branch from fd00614 to 7df7db5 Compare November 4, 2023 18:40

goldsteinn closed this in 71be514 Nov 4, 2023

goldsteinn mentioned this pull request Nov 4, 2023

[Clang][CodeGen] Stoping emitting alignment assumes for align_{up,down} #71295

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Clang][CodeGen] Emit `llvm.ptrmask` for `align_up` and `align_down` #71238

[Clang][CodeGen] Emit `llvm.ptrmask` for `align_up` and `align_down` #71238

Uh oh!

goldsteinn commented Nov 3, 2023

Uh oh!

llvmbot commented Nov 3, 2023 •

edited

Loading

Uh oh!

goldsteinn commented Nov 3, 2023

Uh oh!

nikic left a comment

Uh oh!

goldsteinn commented Nov 3, 2023

Uh oh!

zero9178 Nov 4, 2023

Uh oh!

Uh oh!

		@@ -19671,41 +19671,38 @@ RValue CodeGenFunction::EmitBuiltinIsAligned(const CallExpr *E) {
		/// TODO: actually use ptrmask once most optimization passes know about it.

[Clang][CodeGen] Emit llvm.ptrmask for align_up and align_down #71238

[Clang][CodeGen] Emit llvm.ptrmask for align_up and align_down #71238

Uh oh!

Conversation

goldsteinn commented Nov 3, 2023

Uh oh!

llvmbot commented Nov 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

goldsteinn commented Nov 3, 2023

Uh oh!

nikic left a comment

Choose a reason for hiding this comment

Uh oh!

goldsteinn commented Nov 3, 2023

Uh oh!

zero9178 Nov 4, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

[Clang][CodeGen] Emit `llvm.ptrmask` for `align_up` and `align_down` #71238

[Clang][CodeGen] Emit `llvm.ptrmask` for `align_up` and `align_down` #71238

llvmbot commented Nov 3, 2023 •

edited

Loading