Skip to content

[SelectionDAG] Add space-optimized forms of OPC_CheckPredicate #73488

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 11, 2024

Conversation

wangpc-pp
Copy link
Contributor

We record the usage of each Predicate and sort them by usage.

For the top 8 Predicates, we will emit a PC_CheckPredicateN to
save one byte.

Overall this reduces the llc binary size with all in-tree targets by
about 61K.

This PR is stacked on #73310.

@llvmbot llvmbot added the llvm:SelectionDAG SelectionDAGISel as well label Nov 27, 2023
@wangpc-pp wangpc-pp requested review from ilovepi, jrtc27 and topperc and removed request for ilovepi November 27, 2023 08:31
@llvmbot
Copy link
Member

llvmbot commented Nov 27, 2023

@llvm/pr-subscribers-llvm-selectiondag

Author: Wang Pengcheng (wangpc-pp)

Changes

We record the usage of each Predicate and sort them by usage.

For the top 8 Predicates, we will emit a PC_CheckPredicateN to
save one byte.

Overall this reduces the llc binary size with all in-tree targets by
about 61K.

This PR is stacked on #73310.


Full diff: https://github.com/llvm/llvm-project/pull/73488.diff

9 Files Affected:

  • (modified) llvm/include/llvm/CodeGen/SelectionDAGISel.h (+16)
  • (modified) llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp (+33-3)
  • (modified) llvm/test/TableGen/address-space-patfrags.td (+2-2)
  • (modified) llvm/test/TableGen/dag-isel-complexpattern.td (+1-1)
  • (modified) llvm/test/TableGen/predicate-patfags.td (+2-2)
  • (modified) llvm/utils/TableGen/CodeGenDAGPatterns.h (+23)
  • (modified) llvm/utils/TableGen/DAGISelMatcher.h (+1-1)
  • (modified) llvm/utils/TableGen/DAGISelMatcherEmitter.cpp (+68-47)
  • (modified) llvm/utils/TableGen/DAGISelMatcherGen.cpp (+6-4)
diff --git a/llvm/include/llvm/CodeGen/SelectionDAGISel.h b/llvm/include/llvm/CodeGen/SelectionDAGISel.h
index e6513eb6abc8749..a88f4038f7de88f 100644
--- a/llvm/include/llvm/CodeGen/SelectionDAGISel.h
+++ b/llvm/include/llvm/CodeGen/SelectionDAGISel.h
@@ -152,6 +152,14 @@ class SelectionDAGISel : public MachineFunctionPass {
     OPC_CheckPatternPredicate,
     OPC_CheckPatternPredicate2,
     OPC_CheckPredicate,
+    OPC_CheckPredicate0,
+    OPC_CheckPredicate1,
+    OPC_CheckPredicate2,
+    OPC_CheckPredicate3,
+    OPC_CheckPredicate4,
+    OPC_CheckPredicate5,
+    OPC_CheckPredicate6,
+    OPC_CheckPredicate7,
     OPC_CheckPredicateWithOperands,
     OPC_CheckOpcode,
     OPC_SwitchOpcode,
@@ -176,6 +184,14 @@ class SelectionDAGISel : public MachineFunctionPass {
     OPC_CheckChild2CondCode,
     OPC_CheckValueType,
     OPC_CheckComplexPat,
+    OPC_CheckComplexPat0,
+    OPC_CheckComplexPat1,
+    OPC_CheckComplexPat2,
+    OPC_CheckComplexPat3,
+    OPC_CheckComplexPat4,
+    OPC_CheckComplexPat5,
+    OPC_CheckComplexPat6,
+    OPC_CheckComplexPat7,
     OPC_CheckAndImm,
     OPC_CheckOrImm,
     OPC_CheckImmAllOnesV,
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp
index 7d9bebdca127224..e6a68d8edd382ef 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp
@@ -2700,7 +2700,11 @@ CheckPatternPredicate(const unsigned char *MatcherTable, unsigned &MatcherIndex,
 LLVM_ATTRIBUTE_ALWAYS_INLINE static bool
 CheckNodePredicate(const unsigned char *MatcherTable, unsigned &MatcherIndex,
                    const SelectionDAGISel &SDISel, SDNode *N) {
-  return SDISel.CheckNodePredicate(N, MatcherTable[MatcherIndex++]);
+  unsigned Opcode = MatcherTable[MatcherIndex - 1];
+  unsigned PredNo = Opcode == SelectionDAGISel::OPC_CheckPredicate
+                        ? MatcherTable[MatcherIndex++]
+                        : Opcode - SelectionDAGISel::OPC_CheckPredicate0;
+  return SDISel.CheckNodePredicate(N, PredNo);
 }
 
 LLVM_ATTRIBUTE_ALWAYS_INLINE static bool
@@ -2847,6 +2851,14 @@ static unsigned IsPredicateKnownToFail(const unsigned char *Table,
         Table[Index - 1] == SelectionDAGISel::OPC_CheckPatternPredicate2);
     return Index;
   case SelectionDAGISel::OPC_CheckPredicate:
+  case SelectionDAGISel::OPC_CheckPredicate0:
+  case SelectionDAGISel::OPC_CheckPredicate1:
+  case SelectionDAGISel::OPC_CheckPredicate2:
+  case SelectionDAGISel::OPC_CheckPredicate3:
+  case SelectionDAGISel::OPC_CheckPredicate4:
+  case SelectionDAGISel::OPC_CheckPredicate5:
+  case SelectionDAGISel::OPC_CheckPredicate6:
+  case SelectionDAGISel::OPC_CheckPredicate7:
     Result = !::CheckNodePredicate(Table, Index, SDISel, N.getNode());
     return Index;
   case SelectionDAGISel::OPC_CheckOpcode:
@@ -3262,6 +3274,14 @@ void SelectionDAGISel::SelectCodeCommon(SDNode *NodeToMatch,
                                    Opcode == OPC_CheckPatternPredicate2))
         break;
       continue;
+    case SelectionDAGISel::OPC_CheckPredicate0:
+    case SelectionDAGISel::OPC_CheckPredicate1:
+    case SelectionDAGISel::OPC_CheckPredicate2:
+    case SelectionDAGISel::OPC_CheckPredicate3:
+    case SelectionDAGISel::OPC_CheckPredicate4:
+    case SelectionDAGISel::OPC_CheckPredicate5:
+    case SelectionDAGISel::OPC_CheckPredicate6:
+    case SelectionDAGISel::OPC_CheckPredicate7:
     case OPC_CheckPredicate:
       if (!::CheckNodePredicate(MatcherTable, MatcherIndex, *this,
                                 N.getNode()))
@@ -3279,8 +3299,18 @@ void SelectionDAGISel::SelectCodeCommon(SDNode *NodeToMatch,
         break;
       continue;
     }
-    case OPC_CheckComplexPat: {
-      unsigned CPNum = MatcherTable[MatcherIndex++];
+    case OPC_CheckComplexPat:
+    case OPC_CheckComplexPat0:
+    case OPC_CheckComplexPat1:
+    case OPC_CheckComplexPat2:
+    case OPC_CheckComplexPat3:
+    case OPC_CheckComplexPat4:
+    case OPC_CheckComplexPat5:
+    case OPC_CheckComplexPat6:
+    case OPC_CheckComplexPat7: {
+      unsigned CPNum = Opcode == OPC_CheckComplexPat
+                           ? MatcherTable[MatcherIndex++]
+                           : Opcode - OPC_CheckComplexPat0;
       unsigned RecNo = MatcherTable[MatcherIndex++];
       assert(RecNo < RecordedNodes.size() && "Invalid CheckComplexPat");
 
diff --git a/llvm/test/TableGen/address-space-patfrags.td b/llvm/test/TableGen/address-space-patfrags.td
index 8e92719e6520321..553448eb9a8488f 100644
--- a/llvm/test/TableGen/address-space-patfrags.td
+++ b/llvm/test/TableGen/address-space-patfrags.td
@@ -46,7 +46,7 @@ def inst_d : Instruction {
   let InOperandList = (ins GPR32:$src0, GPR32:$src1);
 }
 
-// SDAG: case 2: {
+// SDAG: case 4: {
 // SDAG-NEXT: // Predicate_pat_frag_b
 // SDAG-NEXT: // Predicate_truncstorei16_addrspace
 // SDAG-NEXT: SDNode *N = Node;
@@ -69,7 +69,7 @@ def : Pat <
 >;
 
 
-// SDAG: case 3: {
+// SDAG: case 7: {
 // SDAG: // Predicate_pat_frag_a
 // SDAG-NEXT: SDNode *N = Node;
 // SDAG-NEXT: (void)N;
diff --git a/llvm/test/TableGen/dag-isel-complexpattern.td b/llvm/test/TableGen/dag-isel-complexpattern.td
index 40fd03cc8839424..1bb473a9df5a954 100644
--- a/llvm/test/TableGen/dag-isel-complexpattern.td
+++ b/llvm/test/TableGen/dag-isel-complexpattern.td
@@ -22,7 +22,7 @@ def CP32 : ComplexPattern<i32, 0, "SelectCP32">;
 def INSTR : Instruction {
 // CHECK-LABEL: OPC_CheckOpcode, TARGET_VAL(ISD::STORE)
 // CHECK: OPC_CheckType, MVT::i32
-// CHECK: OPC_CheckComplexPat, /*CP*/0, /*#*/1, // SelectCP32:$
+// CHECK: OPC_CheckComplexPat0, /*#*/1, // SelectCP32:$
 // CHECK: Src: (st (add:{ *:[i32] } (CP32:{ *:[i32] }), (CP32:{ *:[i32] })), i64:{ *:[i64] }:$addr)
   let OutOperandList = (outs);
   let InOperandList = (ins GPR64:$addr);
diff --git a/llvm/test/TableGen/predicate-patfags.td b/llvm/test/TableGen/predicate-patfags.td
index 17cc74206b716b5..2155928116b53d2 100644
--- a/llvm/test/TableGen/predicate-patfags.td
+++ b/llvm/test/TableGen/predicate-patfags.td
@@ -39,10 +39,10 @@ def TGTmul24_oneuse : PatFrag<
 }
 
 // SDAG: OPC_CheckOpcode, TARGET_VAL(ISD::INTRINSIC_W_CHAIN),
-// SDAG: OPC_CheckPredicate, 0, // Predicate_TGTmul24_oneuse
+// SDAG: OPC_CheckPredicate0, // Predicate_TGTmul24_oneuse
 
 // SDAG: OPC_CheckOpcode, TARGET_VAL(TargetISD::MUL24),
-// SDAG: OPC_CheckPredicate, 0, // Predicate_TGTmul24_oneuse
+// SDAG: OPC_CheckPredicate0, // Predicate_TGTmul24_oneuse
 
 // GISEL: GIM_CheckOpcode, /*MI*/1, TargetOpcode::G_INTRINSIC_W_SIDE_EFFECTS,
 // GISEL: GIM_CheckIntrinsicID, /*MI*/1, /*Op*/1, Intrinsic::tgt_mul24,
diff --git a/llvm/utils/TableGen/CodeGenDAGPatterns.h b/llvm/utils/TableGen/CodeGenDAGPatterns.h
index 2611fe06f55ca53..df0842f5ac87e03 100644
--- a/llvm/utils/TableGen/CodeGenDAGPatterns.h
+++ b/llvm/utils/TableGen/CodeGenDAGPatterns.h
@@ -1117,6 +1117,12 @@ class CodeGenDAGPatterns {
   std::map<Record*, DAGDefaultOperand, LessRecordByID> DefaultOperands;
   std::map<Record*, DAGInstruction, LessRecordByID> Instructions;
 
+  /// ComplexPatternUsage - Record the usage of ComplexPattern.
+  std::map<const ComplexPattern *, unsigned> ComplexPatternUsage;
+
+  /// PredicateUsage - Record the usage of Predicate.
+  std::map<TreePattern *, unsigned> PredicateUsage;
+
   // Specific SDNode definitions:
   Record *intrinsic_void_sdnode;
   Record *intrinsic_w_chain_sdnode, *intrinsic_wo_chain_sdnode;
@@ -1163,6 +1169,23 @@ class CodeGenDAGPatterns {
     return F->second;
   }
 
+  const std::map<const ComplexPattern *, unsigned> &
+  getComplexPatternUsage() const {
+    return ComplexPatternUsage;
+  }
+
+  void increaseComplexPatternUsage(const ComplexPattern *CP) {
+    ComplexPatternUsage[CP]++;
+  }
+
+  const std::map<TreePattern *, unsigned> &getPredicateUsage() const {
+    return PredicateUsage;
+  }
+
+  void increasePredicateUsage(const TreePredicateFn &Predicate) {
+    PredicateUsage[Predicate.getOrigPatFragRecord()]++;
+  }
+
   const CodeGenIntrinsic &getIntrinsic(Record *R) const {
     for (unsigned i = 0, e = Intrinsics.size(); i != e; ++i)
       if (Intrinsics[i].TheDef == R) return Intrinsics[i];
diff --git a/llvm/utils/TableGen/DAGISelMatcher.h b/llvm/utils/TableGen/DAGISelMatcher.h
index e3cf847edd1273b..9e962b14285c395 100644
--- a/llvm/utils/TableGen/DAGISelMatcher.h
+++ b/llvm/utils/TableGen/DAGISelMatcher.h
@@ -34,7 +34,7 @@ namespace llvm {
   class TreePattern;
 
 Matcher *ConvertPatternToMatcher(const PatternToMatch &Pattern,unsigned Variant,
-                                 const CodeGenDAGPatterns &CGP);
+                                 CodeGenDAGPatterns &CGP);
 void OptimizeMatcher(std::unique_ptr<Matcher> &Matcher,
                      const CodeGenDAGPatterns &CGP);
 void EmitMatcherTable(Matcher *Matcher, const CodeGenDAGPatterns &CGP,
diff --git a/llvm/utils/TableGen/DAGISelMatcherEmitter.cpp b/llvm/utils/TableGen/DAGISelMatcherEmitter.cpp
index 4a11991036efc11..e4a63f948135d1e 100644
--- a/llvm/utils/TableGen/DAGISelMatcherEmitter.cpp
+++ b/llvm/utils/TableGen/DAGISelMatcherEmitter.cpp
@@ -52,9 +52,8 @@ class MatcherTableEmitter {
 
   SmallVector<unsigned, Matcher::HighestKind+1> OpcodeCounts;
 
-  DenseMap<TreePattern *, unsigned> NodePredicateMap;
-  std::vector<TreePredicateFn> NodePredicates;
-  std::vector<TreePredicateFn> NodePredicatesWithOperands;
+  std::vector<TreePattern *> NodePredicates;
+  std::vector<TreePattern *> NodePredicatesWithOperands;
 
   // We de-duplicate the predicates by code string, and use this map to track
   // all the patterns with "identical" predicates.
@@ -63,7 +62,6 @@ class MatcherTableEmitter {
   StringMap<unsigned> PatternPredicateMap;
   std::vector<std::string> PatternPredicates;
 
-  DenseMap<const ComplexPattern*, unsigned> ComplexPatternMap;
   std::vector<const ComplexPattern*> ComplexPatterns;
 
 
@@ -85,7 +83,45 @@ class MatcherTableEmitter {
 
 public:
   MatcherTableEmitter(const CodeGenDAGPatterns &cgp)
-      : CGP(cgp), OpcodeCounts(Matcher::HighestKind + 1, 0) {}
+      : CGP(cgp), OpcodeCounts(Matcher::HighestKind + 1, 0) {
+    // Sort ComplexPatterns by usage.
+    auto &Usage = cgp.getComplexPatternUsage();
+    std::vector<std::pair<const ComplexPattern *, unsigned>> PatternList(
+        Usage.begin(), Usage.end());
+    sort(PatternList, [](auto &A, auto &B) { return A.second > B.second; });
+    for (auto &Pattern : PatternList)
+      ComplexPatterns.push_back(Pattern.first);
+
+    // Sort Predicates by usage.
+    auto &PredicateUsage = cgp.getPredicateUsage();
+    // Merge predicates with same code.
+    for (auto &Usage : PredicateUsage) {
+      TreePattern *TP = Usage.first;
+      TreePredicateFn Pred(TP);
+      NodePredicatesByCodeToRun[Pred.getCodeToRunOnSDNode()].push_back(TP);
+    }
+
+    std::vector<std::pair<TreePattern *, unsigned>> PredicateList;
+    // Sum the usage.
+    for (auto &Predicate : NodePredicatesByCodeToRun) {
+      TinyPtrVector<TreePattern *> TPs = Predicate.second;
+      unsigned Uses = 0;
+      for (TreePattern *TP : TPs)
+        Uses += PredicateUsage.at(TP);
+
+      // We only add the first predicate here since they are with the same code.
+      PredicateList.push_back({TPs[0], Uses});
+    }
+
+    sort(PredicateList, [](auto &A, auto &B) { return A.second > B.second; });
+    for (auto &Predicate : PredicateList) {
+      TreePattern *TP = Predicate.first;
+      if (TreePredicateFn(TP).usesOperands())
+        NodePredicatesWithOperands.push_back(TP);
+      else
+        NodePredicates.push_back(TP);
+    }
+  }
 
   unsigned EmitMatcherList(const Matcher *N, const unsigned Indent,
                            unsigned StartIdx, raw_ostream &OS);
@@ -99,7 +135,7 @@ class MatcherTableEmitter {
   void EmitPatternMatchTable(raw_ostream &OS);
 
 private:
-  void EmitNodePredicatesFunction(const std::vector<TreePredicateFn> &Preds,
+  void EmitNodePredicatesFunction(const std::vector<TreePattern *> &Preds,
                                   StringRef Decl, raw_ostream &OS);
 
   unsigned SizeMatcher(Matcher *N, raw_ostream &OS);
@@ -108,33 +144,12 @@ class MatcherTableEmitter {
                        raw_ostream &OS);
 
   unsigned getNodePredicate(TreePredicateFn Pred) {
-    TreePattern *TP = Pred.getOrigPatFragRecord();
-    unsigned &Entry = NodePredicateMap[TP];
-    if (Entry == 0) {
-      TinyPtrVector<TreePattern *> &SameCodePreds =
-          NodePredicatesByCodeToRun[Pred.getCodeToRunOnSDNode()];
-      if (SameCodePreds.empty()) {
-        // We've never seen a predicate with the same code: allocate an entry.
-        if (Pred.usesOperands()) {
-          NodePredicatesWithOperands.push_back(Pred);
-          Entry = NodePredicatesWithOperands.size();
-        } else {
-          NodePredicates.push_back(Pred);
-          Entry = NodePredicates.size();
-        }
-      } else {
-        // We did see an identical predicate: re-use it.
-        Entry = NodePredicateMap[SameCodePreds.front()];
-        assert(Entry != 0);
-        assert(TreePredicateFn(SameCodePreds.front()).usesOperands() ==
-               Pred.usesOperands() &&
-               "PatFrags with some code must have same usesOperands setting");
-      }
-      // In both cases, we've never seen this particular predicate before, so
-      // mark it in the list of predicates sharing the same code.
-      SameCodePreds.push_back(TP);
-    }
-    return Entry-1;
+    TreePattern *PredPat =
+        NodePredicatesByCodeToRun[Pred.getCodeToRunOnSDNode()][0];
+    return Pred.usesOperands()
+               ? llvm::find(NodePredicatesWithOperands, PredPat) -
+                     NodePredicatesWithOperands.begin()
+               : llvm::find(NodePredicates, PredPat) - NodePredicates.begin();
   }
 
   unsigned getPatternPredicate(StringRef PredName) {
@@ -146,12 +161,7 @@ class MatcherTableEmitter {
     return Entry-1;
   }
   unsigned getComplexPat(const ComplexPattern &P) {
-    unsigned &Entry = ComplexPatternMap[&P];
-    if (Entry == 0) {
-      ComplexPatterns.push_back(&P);
-      Entry = ComplexPatterns.size();
-    }
-    return Entry-1;
+    return llvm::find(ComplexPatterns, &P) - ComplexPatterns.begin();
   }
 
   unsigned getNodeXFormID(Record *Rec) {
@@ -486,6 +496,7 @@ EmitMatcher(const Matcher *N, const unsigned Indent, unsigned CurrentIdx,
   case Matcher::CheckPredicate: {
     TreePredicateFn Pred = cast<CheckPredicateMatcher>(N)->getPredicate();
     unsigned OperandBytes = 0;
+    unsigned PredNo = getNodePredicate(Pred);
 
     if (Pred.usesOperands()) {
       unsigned NumOps = cast<CheckPredicateMatcher>(N)->getNumOperands();
@@ -494,10 +505,15 @@ EmitMatcher(const Matcher *N, const unsigned Indent, unsigned CurrentIdx,
         OS << cast<CheckPredicateMatcher>(N)->getOperandNo(i) << ", ";
       OperandBytes = 1 + NumOps;
     } else {
-      OS << "OPC_CheckPredicate, ";
+      if (PredNo < 8) {
+        OperandBytes = -1;
+        OS << "OPC_CheckPredicate" << PredNo << ", ";
+      } else
+        OS << "OPC_CheckPredicate, ";
     }
 
-    OS << getNodePredicate(Pred) << ',';
+    if (PredNo >= 8 || Pred.usesOperands())
+      OS << PredNo << ',';
     if (!OmitComments)
       OS << " // " << Pred.getFnName();
     OS << '\n';
@@ -625,8 +641,13 @@ EmitMatcher(const Matcher *N, const unsigned Indent, unsigned CurrentIdx,
   case Matcher::CheckComplexPat: {
     const CheckComplexPatMatcher *CCPM = cast<CheckComplexPatMatcher>(N);
     const ComplexPattern &Pattern = CCPM->getPattern();
-    OS << "OPC_CheckComplexPat, /*CP*/" << getComplexPat(Pattern) << ", /*#*/"
-       << CCPM->getMatchNumber() << ',';
+    unsigned PatternNo = getComplexPat(Pattern);
+    if (PatternNo < 8)
+      OS << "OPC_CheckComplexPat" << PatternNo << ", /*#*/"
+         << CCPM->getMatchNumber() << ',';
+    else
+      OS << "OPC_CheckComplexPat, /*CP*/" << PatternNo << ", /*#*/"
+         << CCPM->getMatchNumber() << ',';
 
     if (!OmitComments) {
       OS << " // " << Pattern.getSelectFunc();
@@ -638,7 +659,7 @@ EmitMatcher(const Matcher *N, const unsigned Indent, unsigned CurrentIdx,
         OS << " + chain result";
     }
     OS << '\n';
-    return 3;
+    return PatternNo < 8 ? 2 : 3;
   }
 
   case Matcher::CheckAndImm: {
@@ -891,8 +912,7 @@ EmitMatcherList(const Matcher *N, const unsigned Indent, unsigned CurrentIdx,
 }
 
 void MatcherTableEmitter::EmitNodePredicatesFunction(
-    const std::vector<TreePredicateFn> &Preds, StringRef Decl,
-    raw_ostream &OS) {
+    const std::vector<TreePattern *> &Preds, StringRef Decl, raw_ostream &OS) {
   if (Preds.empty())
     return;
 
@@ -902,7 +922,8 @@ void MatcherTableEmitter::EmitNodePredicatesFunction(
   OS << "  default: llvm_unreachable(\"Invalid predicate in table?\");\n";
   for (unsigned i = 0, e = Preds.size(); i != e; ++i) {
     // Emit the predicate code corresponding to this pattern.
-    const TreePredicateFn PredFn = Preds[i];
+    TreePattern *PredPat = Preds[i];
+    TreePredicateFn PredFn(PredPat);
     assert(!PredFn.isAlwaysTrue() && "No code in this predicate");
     std::string PredFnCodeStr = PredFn.getCodeToRunOnSDNode();
 
diff --git a/llvm/utils/TableGen/DAGISelMatcherGen.cpp b/llvm/utils/TableGen/DAGISelMatcherGen.cpp
index d08f57b84b95f08..d5e01af535c5b7c 100644
--- a/llvm/utils/TableGen/DAGISelMatcherGen.cpp
+++ b/llvm/utils/TableGen/DAGISelMatcherGen.cpp
@@ -56,7 +56,7 @@ static MVT::SimpleValueType getRegisterValueType(Record *R,
 namespace {
   class MatcherGen {
     const PatternToMatch &Pattern;
-    const CodeGenDAGPatterns &CGP;
+    CodeGenDAGPatterns &CGP;
 
     /// PatWithNoTypes - This is a clone of Pattern.getSrcPattern() that starts
     /// out with all of the types removed.  This allows us to insert type checks
@@ -102,7 +102,7 @@ namespace {
     /// which should have future checks stuck into its Next position.
     Matcher *CurPredicate;
   public:
-    MatcherGen(const PatternToMatch &pattern, const CodeGenDAGPatterns &cgp);
+    MatcherGen(const PatternToMatch &pattern, CodeGenDAGPatterns &cgp);
 
     bool EmitMatcherCode(unsigned Variant);
     void EmitResultCode();
@@ -146,7 +146,7 @@ namespace {
 } // end anonymous namespace
 
 MatcherGen::MatcherGen(const PatternToMatch &pattern,
-                       const CodeGenDAGPatterns &cgp)
+                       CodeGenDAGPatterns &cgp)
 : Pattern(pattern), CGP(cgp), NextRecordedOperandNo(0),
   TheMatcher(nullptr), CurPredicate(nullptr) {
   // We need to produce the matcher tree for the patterns source pattern.  To do
@@ -539,6 +539,7 @@ void MatcherGen::EmitMatchCode(const TreePatternNode *N,
         Operands.push_back(getNamedArgumentSlot(Name));
       }
     }
+    CGP.increasePredicateUsage(Pred.Fn);
     AddMatcher(new CheckPredicateMatcher(Pred.Fn, Operands));
   }
 
@@ -601,6 +602,7 @@ bool MatcherGen::EmitMatcherCode(unsigned Variant) {
 
     // Emit a CheckComplexPat operation, which does the match (aborting if it
     // fails) and pushes the matched operands onto the recorded nodes list.
+    CGP.increaseComplexPatternUsage(CP);
     AddMatcher(new CheckComplexPatMatcher(*CP, RecNodeEntry, N->getName(),
                                           NextRecordedOperandNo));
 
@@ -1081,7 +1083,7 @@ void MatcherGen::EmitResultCode() {
 /// the specified variant.  If the variant number is invalid, this returns null.
 Matcher *llvm::ConvertPatternToMatcher(const PatternToMatch &Pattern,
                                        unsigned Variant,
-                                       const CodeGenDAGPatterns &CGP) {
+                                       CodeGenDAGPatterns &CGP) {
   MatcherGen Gen(Pattern, CGP);
 
   // Generate the code for the matcher.

@wangpc-pp wangpc-pp requested review from asb and RKSimon November 27, 2023 08:32
Copy link

github-actions bot commented Nov 27, 2023

✅ With the latest revision this PR passed the C/C++ code formatter.

@wangpc-pp wangpc-pp requested a review from arsenm November 28, 2023 11:46
@wangpc-pp
Copy link
Contributor Author

Ping.

@wangpc-pp wangpc-pp force-pushed the main-matcher-table-check-predicate branch 2 times, most recently from 4d9332d to 166c0ec Compare December 12, 2023 12:39
@wangpc-pp
Copy link
Contributor Author

Ping.

@wangpc-pp wangpc-pp force-pushed the main-matcher-table-check-predicate branch from 1813334 to 1f4f5ed Compare January 8, 2024 08:03
We record the usage of each `Predicate` and sort them by usage.

For the top 8 `Predicate`s, we will emit a `PC_CheckPredicateN` to
save one byte.

Overall this reduces the llc binary size with all in-tree targets by
about 61K.

This PR is stacked on llvm#73310.
@wangpc-pp wangpc-pp force-pushed the main-matcher-table-check-predicate branch from 1f4f5ed to e8c1533 Compare January 11, 2024 07:43
@wangpc-pp wangpc-pp merged commit 1a57927 into llvm:main Jan 11, 2024
@metaflow
Copy link
Contributor

Test address-space-patfrags.td.test started to fire https://lab.llvm.org/buildbot/#/builders/104/builds/15012

metaflow added a commit that referenced this pull request Jan 11, 2024
@metaflow
Copy link
Contributor

I have reverted this for now

@wangpc-pp
Copy link
Contributor Author

I have reverted this for now

Thanks! I can't reproduce the failure, I will try to fix it later.

@RKSimon
Copy link
Collaborator

RKSimon commented Jan 11, 2024

@wangpc-pp This is on an expensive checks build (cmake -DLLVM_ENABLE_EXPENSIVE_CHECKS=true) - these builds do extra stable sorting which can sometimes cause check diffs

@MaskRay
Copy link
Member

MaskRay commented Jan 12, 2024

One of the recent TableGen changes (likely DAGISelMatcherEmitter.cpp) may make output *GenDAGISel.inc non-deterministic. Things like DenseMap iterating needs some attention.

for (const auto &Usage : PredicateUsage) {
TreePattern *TP = Usage.first;
TreePredicateFn Pred(TP);
NodePredicatesByCodeToRun[Pred.getCodeToRunOnSDNode()].push_back(TP);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Iterating over PredicateUsage and creating StringMap NodePredicatesByCodeToRun can cause non-determinism.

justinfargnoli pushed a commit to justinfargnoli/llvm-project that referenced this pull request Jan 28, 2024
…73488)

We record the usage of each `Predicate` and sort them by usage.

For the top 8 `Predicate`s, we will emit a `PC_CheckPredicateN` to
save one byte.

Overall this reduces the llc binary size with all in-tree targets by
about 61K.
justinfargnoli pushed a commit to justinfargnoli/llvm-project that referenced this pull request Jan 28, 2024
@wangpc-pp wangpc-pp deleted the main-matcher-table-check-predicate branch March 29, 2024 04:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
llvm:SelectionDAG SelectionDAGISel as well
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants