[clang][ExtractAPI] Add ability to create multiple symbol graphs #86676
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
llvmbot
added
clang
|
@llvm/pr-subscribers-clang-driver Author: Daniel Grumberg (daniel-grumberg) ChangesThis allows to segregate symbols that are extending symbols from other modules into a spearate graph. This also uses the opportunity to clean up the emit-symbol-graph interface. Patch is 528.30 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/86676.diff 77 Files Affected:
diff --git a/clang/include/clang/Basic/DiagnosticDriverKinds.td b/clang/include/clang/Basic/DiagnosticDriverKinds.td
index e33a1f4c45b949..5568a87dddf0e2 100644
--- a/clang/include/clang/Basic/DiagnosticDriverKinds.td
+++ b/clang/include/clang/Basic/DiagnosticDriverKinds.td
@@ -548,6 +548,12 @@ def err_drv_extract_api_wrong_kind : Error<
"header file '%0' input '%1' does not match the type of prior input "
"in api extraction; use '-x %2' to override">;
+def err_drv_missing_symbol_graph_dir: Error<
+ "Must provide a symbol graph output directory using --symbol-graph-dir=<directory>">;
+
+def err_drv_unexpected_symbol_graph_output : Error<
+ "Unexpected output symbol graph '%1'; please provide --symbol-graph-dir=<directory> instead">;
+
def warn_slash_u_filename : Warning<"'/U%0' treated as the '/U' option">,
InGroup<DiagGroup<"slash-u-filename">>;
def note_use_dashdash : Note<
diff --git a/clang/include/clang/Basic/DiagnosticFrontendKinds.td b/clang/include/clang/Basic/DiagnosticFrontendKinds.td
index ba23cf84c5e343..241544b2b8c6fa 100644
--- a/clang/include/clang/Basic/DiagnosticFrontendKinds.td
+++ b/clang/include/clang/Basic/DiagnosticFrontendKinds.td
@@ -366,4 +366,6 @@ def warn_profile_data_misexpect : Warning<
def err_extract_api_ignores_file_not_found :
Error<"file '%0' specified by '--extract-api-ignores=' not found">, DefaultFatal;
+def warn_missing_symbol_graph_dir : Warning<"Missing symbol graph output directory, defaulting to working directory">;
+
}
diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index 4a954258ce40b6..f410dbfae90871 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -1507,14 +1507,29 @@ def extract_api : Flag<["-"], "extract-api">,
def product_name_EQ: Joined<["--"], "product-name=">,
Visibility<[ClangOption, CC1Option]>,
MarshallingInfoString<FrontendOpts<"ProductName">>;
-def emit_symbol_graph_EQ: JoinedOrSeparate<["--"], "emit-symbol-graph=">,
+def emit_symbol_graph: Flag<["-"], "emit-symbol-graph">,
Visibility<[ClangOption, CC1Option]>,
- HelpText<"Generate Extract API information as a side effect of compilation.">,
- MarshallingInfoString<FrontendOpts<"SymbolGraphOutputDir">>;
+ HelpText<"Generate Extract API information as a side effect of compilation.">,
+ MarshallingInfoFlag<FrontendOpts<"EmitSymbolGraph">>;
+def emit_extension_symbol_graphs: Flag<["--"], "emit-extension-symbol-graphs">,
+ Visibility<[ClangOption, CC1Option]>,
+ HelpText<"Generate additional symbol graphs for extended modules.">,
+ MarshallingInfoFlag<FrontendOpts<"EmitExtensionSymbolGraphs">>;
def extract_api_ignores_EQ: CommaJoined<["--"], "extract-api-ignores=">,
Visibility<[ClangOption, CC1Option]>,
HelpText<"Comma separated list of files containing a new line separated list of API symbols to ignore when extracting API information.">,
MarshallingInfoStringVector<FrontendOpts<"ExtractAPIIgnoresFileList">>;
+def symbol_graph_dir_EQ: Joined<["--"], "symbol-graph-dir=">,
+ Visibility<[ClangOption, CC1Option]>,
+ HelpText<"Directory in which to emit symbol graphs.">,
+ MarshallingInfoString<FrontendOpts<"SymbolGraphOutputDir">>;
+def emit_pretty_sgf: Flag<["--"], "pretty-sgf">,
+ Visibility<[ClangOption, CC1Option]>,
+ HelpText<"Emit pretty printed symbol graphs">,
+ MarshallingInfoFlag<FrontendOpts<"EmitPrettySymbolGraphs">>;
+def emit_sgf_symbol_labels_for_testing: Flag<["--"], "emit-sgf-symbol-labels-for-testing">,
+ Visibility<[CC1Option]>,
+ MarshallingInfoFlag<FrontendOpts<"EmitSymbolGraphSymbolLabelsForTesting">>;
def e : Separate<["-"], "e">, Flags<[LinkerInput]>, Group<Link_Group>;
def fmax_tokens_EQ : Joined<["-"], "fmax-tokens=">, Group<f_Group>,
Visibility<[ClangOption, CC1Option]>,
diff --git a/clang/include/clang/ExtractAPI/API.h b/clang/include/clang/ExtractAPI/API.h
index b220db294101d8..90ee1e8dbb5062 100644
--- a/clang/include/clang/ExtractAPI/API.h
+++ b/clang/include/clang/ExtractAPI/API.h
@@ -20,17 +20,25 @@
#include "clang/AST/Availability.h"
#include "clang/AST/Decl.h"
+#include "clang/AST/DeclBase.h"
#include "clang/AST/DeclObjC.h"
#include "clang/AST/RawCommentList.h"
#include "clang/Basic/SourceLocation.h"
#include "clang/Basic/Specifiers.h"
#include "clang/ExtractAPI/DeclarationFragments.h"
+#include "llvm/ADT/ArrayRef.h"
#include "llvm/ADT/MapVector.h"
#include "llvm/ADT/StringRef.h"
#include "llvm/Support/Allocator.h"
#include "llvm/Support/Casting.h"
+#include "llvm/Support/Compiler.h"
+#include "llvm/Support/ErrorHandling.h"
+#include "llvm/Support/raw_ostream.h"
#include "llvm/TargetParser/Triple.h"
+#include <cstddef>
+#include <iterator>
#include <memory>
+#include <optional>
#include <type_traits>
namespace clang {
@@ -149,15 +157,57 @@ class Template {
/// \endcode
using DocComment = std::vector<RawComment::CommentLine>;
-// Classes deriving from APIRecord need to have USR be the first constructor
-// argument. This is so that they are compatible with `addTopLevelRecord`
-// defined in API.cpp
+struct APIRecord;
+
+// This represents a reference to another symbol that might come from external
+/// sources.
+struct SymbolReference {
+ StringRef Name;
+ StringRef USR;
+
+ /// The source project/module/product of the referred symbol.
+ StringRef Source;
+
+ // A Pointer to the APIRecord for this reference if known
+ const APIRecord *Record = nullptr;
+
+ SymbolReference() = default;
+ SymbolReference(StringRef Name, StringRef USR, StringRef Source = "")
+ : Name(Name), USR(USR), Source(Source) {}
+ SymbolReference(const APIRecord *R);
+
+ /// Determine if this SymbolReference is empty.
+ ///
+ /// \returns true if and only if all \c Name, \c USR, and \c Source is empty.
+ bool empty() const { return Name.empty() && USR.empty() && Source.empty(); }
+};
+
+class RecordContext;
+
+// Concrete classes deriving from APIRecord need to have a construct with first
+// arguments USR, and Name, in that order. This is so that they
+// are compatible with `APISet::createRecord`.
+// When adding a new kind of record don't forget to update APIRecords.inc!
/// The base representation of an API record. Holds common symbol information.
struct APIRecord {
/// Discriminator for LLVM-style RTTI (dyn_cast<> et al.)
enum RecordKind {
RK_Unknown,
+ // If adding a record context record kind here make sure to update
+ // RecordContext::classof if needed and add a RECORD_CONTEXT entry to
+ // APIRecords.inc RecordContext Kinds start
RK_Namespace,
+ RK_Enum,
+ RK_Struct,
+ RK_Union,
+ RK_ObjCInterface,
+ RK_ObjCCategory,
+ RK_ObjCProtocol,
+ RK_CXXClass,
+ RK_ClassTemplate,
+ RK_ClassTemplateSpecialization,
+ RK_ClassTemplatePartialSpecialization,
+ // RecordContexts Kinds end
RK_GlobalFunction,
RK_GlobalFunctionTemplate,
RK_GlobalFunctionTemplateSpecialization,
@@ -166,18 +216,11 @@ struct APIRecord {
RK_GlobalVariableTemplateSpecialization,
RK_GlobalVariableTemplatePartialSpecialization,
RK_EnumConstant,
- RK_Enum,
RK_StructField,
- RK_Struct,
RK_UnionField,
- RK_Union,
RK_StaticField,
RK_CXXField,
RK_CXXFieldTemplate,
- RK_CXXClass,
- RK_ClassTemplate,
- RK_ClassTemplateSpecialization,
- RK_ClassTemplatePartialSpecialization,
RK_Concept,
RK_CXXStaticMethod,
RK_CXXInstanceMethod,
@@ -190,40 +233,15 @@ struct APIRecord {
RK_ObjCIvar,
RK_ObjCClassMethod,
RK_ObjCInstanceMethod,
- RK_ObjCInterface,
- RK_ObjCCategory,
- RK_ObjCCategoryModule,
- RK_ObjCProtocol,
RK_MacroDefinition,
RK_Typedef,
};
- /// Stores information about the context of the declaration of this API.
- /// This is roughly analogous to the DeclContext hierarchy for an AST Node.
- struct HierarchyInformation {
- /// The USR of the parent API.
- StringRef ParentUSR;
- /// The name of the parent API.
- StringRef ParentName;
- /// The record kind of the parent API.
- RecordKind ParentKind = RK_Unknown;
- /// A pointer to the parent APIRecord if known.
- APIRecord *ParentRecord = nullptr;
-
- HierarchyInformation() = default;
- HierarchyInformation(StringRef ParentUSR, StringRef ParentName,
- RecordKind Kind, APIRecord *ParentRecord = nullptr)
- : ParentUSR(ParentUSR), ParentName(ParentName), ParentKind(Kind),
- ParentRecord(ParentRecord) {}
-
- bool empty() const {
- return ParentUSR.empty() && ParentName.empty() &&
- ParentKind == RK_Unknown && ParentRecord == nullptr;
- }
- };
-
StringRef USR;
StringRef Name;
+
+ SymbolReference Parent;
+
PresumedLoc Location;
AvailabilityInfo Availability;
LinkageInfo Linkage;
@@ -242,79 +260,168 @@ struct APIRecord {
/// Objective-C class/instance methods).
DeclarationFragments SubHeading;
- /// Information about the parent record of this record.
- HierarchyInformation ParentInformation;
-
/// Whether the symbol was defined in a system header.
bool IsFromSystemHeader;
+ AccessControl Access;
+
private:
const RecordKind Kind;
+ friend class RecordContext;
+ // Used to store the next child record in RecordContext. This works because
+ // APIRecords semantically only have one parent.
+ mutable APIRecord *NextInContex = nullptr;
public:
+ APIRecord *getNextInContex() const { return NextInContex; }
+
RecordKind getKind() const { return Kind; }
+ static APIRecord *castFromRecordContext(const RecordContext *Ctx);
+ static RecordContext *castToRecordContext(const APIRecord *Record);
+
APIRecord() = delete;
APIRecord(RecordKind Kind, StringRef USR, StringRef Name,
- PresumedLoc Location, AvailabilityInfo Availability,
- LinkageInfo Linkage, const DocComment &Comment,
- DeclarationFragments Declaration, DeclarationFragments SubHeading,
- bool IsFromSystemHeader)
- : USR(USR), Name(Name), Location(Location),
+ SymbolReference Parent, PresumedLoc Location,
+ AvailabilityInfo Availability, LinkageInfo Linkage,
+ const DocComment &Comment, DeclarationFragments Declaration,
+ DeclarationFragments SubHeading, bool IsFromSystemHeader,
+ AccessControl Access = AccessControl())
+ : USR(USR), Name(Name), Parent(std::move(Parent)), Location(Location),
Availability(std::move(Availability)), Linkage(Linkage),
Comment(Comment), Declaration(Declaration), SubHeading(SubHeading),
- IsFromSystemHeader(IsFromSystemHeader), Kind(Kind) {}
+ IsFromSystemHeader(IsFromSystemHeader), Access(std::move(Access)),
+ Kind(Kind) {}
APIRecord(RecordKind Kind, StringRef USR, StringRef Name)
: USR(USR), Name(Name), Kind(Kind) {}
// Pure virtual destructor to make APIRecord abstract
virtual ~APIRecord() = 0;
+ static bool classof(const APIRecord *Record) { return true; }
+ static bool classofKind(RecordKind K) { return true; }
+ static bool classof(const RecordContext *Ctx) { return true; }
+};
+
+/// Base class used for specific record types that have children records this is
+/// analogous to the DeclContext for the AST
+class RecordContext {
+public:
+ static bool classof(const APIRecord *Record) {
+ return classofKind(Record->getKind());
+ }
+ static bool classofKind(APIRecord::RecordKind K) {
+ return K >= APIRecord::RK_Namespace &&
+ K <= APIRecord::RK_ClassTemplatePartialSpecialization;
+ }
+
+ static bool classof(const RecordContext *Context) { return true; }
+
+ RecordContext(APIRecord::RecordKind Kind) : Kind(Kind) {}
+
+ APIRecord::RecordKind getKind() const { return Kind; }
+
+ struct record_iterator {
+ private:
+ APIRecord *Current = nullptr;
+
+ public:
+ using value_type = APIRecord *;
+ using reference = const value_type &;
+ using pointer = const value_type *;
+ using iterator_category = std::forward_iterator_tag;
+ using difference_type = std::ptrdiff_t;
+
+ record_iterator() = default;
+ explicit record_iterator(value_type R) : Current(R) {}
+ reference operator*() const { return Current; }
+ // This doesn't strictly meet the iterator requirements, but it's the
+ // behavior we want here.
+ value_type operator->() const { return Current; }
+ record_iterator &operator++() { Current = Current->getNextInContex();
+ return *this;
+ }
+ record_iterator operator++(int) {
+ record_iterator tmp(*this);
+ ++(*this);
+ return tmp;
+ }
+
+ friend bool operator==(record_iterator x, record_iterator y) {
+ return x.Current == y.Current;
+ }
+ friend bool operator!=(record_iterator x, record_iterator y) {
+ return x.Current != y.Current;
+ }
+ };
+
+ using record_range = llvm::iterator_range<record_iterator>;
+ record_range records() const {
+ return record_range(records_begin(), records_end());
+ }
+ record_iterator records_begin() const { return record_iterator(First); };
+ record_iterator records_end() const { return record_iterator(); }
+ bool records_empty() const { return First == nullptr; };
+
+private:
+ APIRecord::RecordKind Kind;
+ mutable APIRecord *First = nullptr;
+ mutable APIRecord *Last = nullptr;
+
+protected:
+ friend class APISet;
+ void addToRecordChain(APIRecord *) const;
};
-struct NamespaceRecord : APIRecord {
- NamespaceRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
- AvailabilityInfo Availability, LinkageInfo Linkage,
- const DocComment &Comment, DeclarationFragments Declaration,
+struct NamespaceRecord : APIRecord, RecordContext {
+ NamespaceRecord(StringRef USR, StringRef Name, SymbolReference Parent,
+ PresumedLoc Loc, AvailabilityInfo Availability,
+ LinkageInfo Linkage, const DocComment &Comment,
+ DeclarationFragments Declaration,
DeclarationFragments SubHeading, bool IsFromSystemHeader)
- : APIRecord(RK_Namespace, USR, Name, Loc, std::move(Availability),
+ : APIRecord(RK_Namespace, USR, Name, Parent, Loc, std::move(Availability),
Linkage, Comment, Declaration, SubHeading,
- IsFromSystemHeader) {}
+ IsFromSystemHeader),
+ RecordContext(RK_Namespace) {}
static bool classof(const APIRecord *Record) {
- return Record->getKind() == RK_Namespace;
+ return classofKind(Record->getKind());
}
+ static bool classofKind(RecordKind K) { return K == RK_Namespace; }
};
/// This holds information associated with global functions.
struct GlobalFunctionRecord : APIRecord {
FunctionSignature Signature;
- GlobalFunctionRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
- AvailabilityInfo Availability, LinkageInfo Linkage,
- const DocComment &Comment,
+ GlobalFunctionRecord(StringRef USR, StringRef Name, SymbolReference Parent,
+ PresumedLoc Loc, AvailabilityInfo Availability,
+ LinkageInfo Linkage, const DocComment &Comment,
DeclarationFragments Declaration,
DeclarationFragments SubHeading,
FunctionSignature Signature, bool IsFromSystemHeader)
- : APIRecord(RK_GlobalFunction, USR, Name, Loc, std::move(Availability),
- Linkage, Comment, Declaration, SubHeading,
- IsFromSystemHeader),
+ : APIRecord(RK_GlobalFunction, USR, Name, Parent, Loc,
+ std::move(Availability), Linkage, Comment, Declaration,
+ SubHeading, IsFromSystemHeader),
Signature(Signature) {}
GlobalFunctionRecord(RecordKind Kind, StringRef USR, StringRef Name,
- PresumedLoc Loc, AvailabilityInfo Availability,
- LinkageInfo Linkage, const DocComment &Comment,
+ SymbolReference Parent, PresumedLoc Loc,
+ AvailabilityInfo Availability, LinkageInfo Linkage,
+ const DocComment &Comment,
DeclarationFragments Declaration,
DeclarationFragments SubHeading,
FunctionSignature Signature, bool IsFromSystemHeader)
- : APIRecord(Kind, USR, Name, Loc, std::move(Availability), Linkage,
- Comment, Declaration, SubHeading, IsFromSystemHeader),
+ : APIRecord(Kind, USR, Name, Parent, Loc, std::move(Availability),
+ Linkage, Comment, Declaration, SubHeading,
+ IsFromSystemHeader),
Signature(Signature) {}
static bool classof(const APIRecord *Record) {
- return Record->getKind() == RK_GlobalFunction;
+ return classofKind(Record->getKind());
}
+ static bool classofKind(RecordKind K) { return K == RK_GlobalFunction; }
private:
virtual void anchor();
@@ -323,63 +430,74 @@ struct GlobalFunctionRecord : APIRecord {
struct GlobalFunctionTemplateRecord : GlobalFunctionRecord {
Template Templ;
- GlobalFunctionTemplateRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
+ GlobalFunctionTemplateRecord(StringRef USR, StringRef Name,
+ SymbolReference Parent, PresumedLoc Loc,
AvailabilityInfo Availability,
LinkageInfo Linkage, const DocComment &Comment,
DeclarationFragments Declaration,
DeclarationFragments SubHeading,
FunctionSignature Signature, Template Template,
bool IsFromSystemHeader)
- : GlobalFunctionRecord(RK_GlobalFunctionTemplate, USR, Name, Loc,
+ : GlobalFunctionRecord(RK_GlobalFunctionTemplate, USR, Name, Parent, Loc,
std::move(Availability), Linkage, Comment,
Declaration, SubHeading, Signature,
IsFromSystemHeader),
Templ(Template) {}
static bool classof(const APIRecord *Record) {
- return Record->getKind() == RK_GlobalFunctionTemplate;
+ return classofKind(Record->getKind());
+ }
+ static bool classofKind(RecordKind K) {
+ return K == RK_GlobalFunctionTemplate;
}
};
struct GlobalFunctionTemplateSpecializationRecord : GlobalFunctionRecord {
GlobalFunctionTemplateSpecializationRecord(
- StringRef USR, StringRef Name, PresumedLoc Loc,
+ StringRef USR, StringRef Name, SymbolReference Parent, PresumedLoc Loc,
AvailabilityInfo Availability, LinkageInfo Linkage,
const DocComment &Comment, DeclarationFragments Declaration,
DeclarationFragments SubHeading, FunctionSignature Signature,
bool IsFromSystemHeader)
: GlobalFunctionRecord(RK_GlobalFunctionTemplateSpecialization, USR, Name,
- Loc, std::move(Availability), Linkage, Comment,
- Declaration, SubHeading, Signature,
+ Parent, Loc, std::move(Availability), Linkage,
+ Comment, Declaration, SubHeading, Signature,
IsFromSystemHeader) {}
static bool classof(const APIRecord *Record) {
- return Record->getKind() == RK_GlobalFunctionTemplateSpecialization;
+ return classofKind(Record->getKind());
+ }
+ static bool classofKind(RecordKind K) {
+ return K == RK_GlobalFunctionTemplateSpecialization;
}
};
/// This holds information associated with global functions.
struct GlobalVariableRecord : APIRecord {
- GlobalVariableRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
- AvailabilityInfo Availability, LinkageInfo Linkage,
- const DocComment &Comment,
+ GlobalVariableRecord(StringRef USR, StringRef Name, SymbolReference Parent,
+ PresumedLoc Loc, AvailabilityInfo Availability,
+ LinkageInfo Linkage, const DocComment &Comment,
DeclarationFragments Declaration,
DeclarationFragments SubHeading, bool IsFromSystemHeader)
- : APIRecord(RK_GlobalVariable, USR, Name, Loc, std::move(Availability),
- Linkage, Comment, Declaration, SubHe...
[truncated]
|
|
@llvm/pr-subscribers-clang Author: Daniel Grumberg (daniel-grumberg) ChangesThis allows to segregate symbols that are extending symbols from other modules into a spearate graph. This also uses the opportunity to clean up the emit-symbol-graph interface. Patch is 528.30 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/86676.diff 77 Files Affected:
diff --git a/clang/include/clang/Basic/DiagnosticDriverKinds.td b/clang/include/clang/Basic/DiagnosticDriverKinds.td
index e33a1f4c45b949..5568a87dddf0e2 100644
--- a/clang/include/clang/Basic/DiagnosticDriverKinds.td
+++ b/clang/include/clang/Basic/DiagnosticDriverKinds.td
@@ -548,6 +548,12 @@ def err_drv_extract_api_wrong_kind : Error<
"header file '%0' input '%1' does not match the type of prior input "
"in api extraction; use '-x %2' to override">;
+def err_drv_missing_symbol_graph_dir: Error<
+ "Must provide a symbol graph output directory using --symbol-graph-dir=<directory>">;
+
+def err_drv_unexpected_symbol_graph_output : Error<
+ "Unexpected output symbol graph '%1'; please provide --symbol-graph-dir=<directory> instead">;
+
def warn_slash_u_filename : Warning<"'/U%0' treated as the '/U' option">,
InGroup<DiagGroup<"slash-u-filename">>;
def note_use_dashdash : Note<
diff --git a/clang/include/clang/Basic/DiagnosticFrontendKinds.td b/clang/include/clang/Basic/DiagnosticFrontendKinds.td
index ba23cf84c5e343..241544b2b8c6fa 100644
--- a/clang/include/clang/Basic/DiagnosticFrontendKinds.td
+++ b/clang/include/clang/Basic/DiagnosticFrontendKinds.td
@@ -366,4 +366,6 @@ def warn_profile_data_misexpect : Warning<
def err_extract_api_ignores_file_not_found :
Error<"file '%0' specified by '--extract-api-ignores=' not found">, DefaultFatal;
+def warn_missing_symbol_graph_dir : Warning<"Missing symbol graph output directory, defaulting to working directory">;
+
}
diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index 4a954258ce40b6..f410dbfae90871 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -1507,14 +1507,29 @@ def extract_api : Flag<["-"], "extract-api">,
def product_name_EQ: Joined<["--"], "product-name=">,
Visibility<[ClangOption, CC1Option]>,
MarshallingInfoString<FrontendOpts<"ProductName">>;
-def emit_symbol_graph_EQ: JoinedOrSeparate<["--"], "emit-symbol-graph=">,
+def emit_symbol_graph: Flag<["-"], "emit-symbol-graph">,
Visibility<[ClangOption, CC1Option]>,
- HelpText<"Generate Extract API information as a side effect of compilation.">,
- MarshallingInfoString<FrontendOpts<"SymbolGraphOutputDir">>;
+ HelpText<"Generate Extract API information as a side effect of compilation.">,
+ MarshallingInfoFlag<FrontendOpts<"EmitSymbolGraph">>;
+def emit_extension_symbol_graphs: Flag<["--"], "emit-extension-symbol-graphs">,
+ Visibility<[ClangOption, CC1Option]>,
+ HelpText<"Generate additional symbol graphs for extended modules.">,
+ MarshallingInfoFlag<FrontendOpts<"EmitExtensionSymbolGraphs">>;
def extract_api_ignores_EQ: CommaJoined<["--"], "extract-api-ignores=">,
Visibility<[ClangOption, CC1Option]>,
HelpText<"Comma separated list of files containing a new line separated list of API symbols to ignore when extracting API information.">,
MarshallingInfoStringVector<FrontendOpts<"ExtractAPIIgnoresFileList">>;
+def symbol_graph_dir_EQ: Joined<["--"], "symbol-graph-dir=">,
+ Visibility<[ClangOption, CC1Option]>,
+ HelpText<"Directory in which to emit symbol graphs.">,
+ MarshallingInfoString<FrontendOpts<"SymbolGraphOutputDir">>;
+def emit_pretty_sgf: Flag<["--"], "pretty-sgf">,
+ Visibility<[ClangOption, CC1Option]>,
+ HelpText<"Emit pretty printed symbol graphs">,
+ MarshallingInfoFlag<FrontendOpts<"EmitPrettySymbolGraphs">>;
+def emit_sgf_symbol_labels_for_testing: Flag<["--"], "emit-sgf-symbol-labels-for-testing">,
+ Visibility<[CC1Option]>,
+ MarshallingInfoFlag<FrontendOpts<"EmitSymbolGraphSymbolLabelsForTesting">>;
def e : Separate<["-"], "e">, Flags<[LinkerInput]>, Group<Link_Group>;
def fmax_tokens_EQ : Joined<["-"], "fmax-tokens=">, Group<f_Group>,
Visibility<[ClangOption, CC1Option]>,
diff --git a/clang/include/clang/ExtractAPI/API.h b/clang/include/clang/ExtractAPI/API.h
index b220db294101d8..90ee1e8dbb5062 100644
--- a/clang/include/clang/ExtractAPI/API.h
+++ b/clang/include/clang/ExtractAPI/API.h
@@ -20,17 +20,25 @@
#include "clang/AST/Availability.h"
#include "clang/AST/Decl.h"
+#include "clang/AST/DeclBase.h"
#include "clang/AST/DeclObjC.h"
#include "clang/AST/RawCommentList.h"
#include "clang/Basic/SourceLocation.h"
#include "clang/Basic/Specifiers.h"
#include "clang/ExtractAPI/DeclarationFragments.h"
+#include "llvm/ADT/ArrayRef.h"
#include "llvm/ADT/MapVector.h"
#include "llvm/ADT/StringRef.h"
#include "llvm/Support/Allocator.h"
#include "llvm/Support/Casting.h"
+#include "llvm/Support/Compiler.h"
+#include "llvm/Support/ErrorHandling.h"
+#include "llvm/Support/raw_ostream.h"
#include "llvm/TargetParser/Triple.h"
+#include <cstddef>
+#include <iterator>
#include <memory>
+#include <optional>
#include <type_traits>
namespace clang {
@@ -149,15 +157,57 @@ class Template {
/// \endcode
using DocComment = std::vector<RawComment::CommentLine>;
-// Classes deriving from APIRecord need to have USR be the first constructor
-// argument. This is so that they are compatible with `addTopLevelRecord`
-// defined in API.cpp
+struct APIRecord;
+
+// This represents a reference to another symbol that might come from external
+/// sources.
+struct SymbolReference {
+ StringRef Name;
+ StringRef USR;
+
+ /// The source project/module/product of the referred symbol.
+ StringRef Source;
+
+ // A Pointer to the APIRecord for this reference if known
+ const APIRecord *Record = nullptr;
+
+ SymbolReference() = default;
+ SymbolReference(StringRef Name, StringRef USR, StringRef Source = "")
+ : Name(Name), USR(USR), Source(Source) {}
+ SymbolReference(const APIRecord *R);
+
+ /// Determine if this SymbolReference is empty.
+ ///
+ /// \returns true if and only if all \c Name, \c USR, and \c Source is empty.
+ bool empty() const { return Name.empty() && USR.empty() && Source.empty(); }
+};
+
+class RecordContext;
+
+// Concrete classes deriving from APIRecord need to have a construct with first
+// arguments USR, and Name, in that order. This is so that they
+// are compatible with `APISet::createRecord`.
+// When adding a new kind of record don't forget to update APIRecords.inc!
/// The base representation of an API record. Holds common symbol information.
struct APIRecord {
/// Discriminator for LLVM-style RTTI (dyn_cast<> et al.)
enum RecordKind {
RK_Unknown,
+ // If adding a record context record kind here make sure to update
+ // RecordContext::classof if needed and add a RECORD_CONTEXT entry to
+ // APIRecords.inc RecordContext Kinds start
RK_Namespace,
+ RK_Enum,
+ RK_Struct,
+ RK_Union,
+ RK_ObjCInterface,
+ RK_ObjCCategory,
+ RK_ObjCProtocol,
+ RK_CXXClass,
+ RK_ClassTemplate,
+ RK_ClassTemplateSpecialization,
+ RK_ClassTemplatePartialSpecialization,
+ // RecordContexts Kinds end
RK_GlobalFunction,
RK_GlobalFunctionTemplate,
RK_GlobalFunctionTemplateSpecialization,
@@ -166,18 +216,11 @@ struct APIRecord {
RK_GlobalVariableTemplateSpecialization,
RK_GlobalVariableTemplatePartialSpecialization,
RK_EnumConstant,
- RK_Enum,
RK_StructField,
- RK_Struct,
RK_UnionField,
- RK_Union,
RK_StaticField,
RK_CXXField,
RK_CXXFieldTemplate,
- RK_CXXClass,
- RK_ClassTemplate,
- RK_ClassTemplateSpecialization,
- RK_ClassTemplatePartialSpecialization,
RK_Concept,
RK_CXXStaticMethod,
RK_CXXInstanceMethod,
@@ -190,40 +233,15 @@ struct APIRecord {
RK_ObjCIvar,
RK_ObjCClassMethod,
RK_ObjCInstanceMethod,
- RK_ObjCInterface,
- RK_ObjCCategory,
- RK_ObjCCategoryModule,
- RK_ObjCProtocol,
RK_MacroDefinition,
RK_Typedef,
};
- /// Stores information about the context of the declaration of this API.
- /// This is roughly analogous to the DeclContext hierarchy for an AST Node.
- struct HierarchyInformation {
- /// The USR of the parent API.
- StringRef ParentUSR;
- /// The name of the parent API.
- StringRef ParentName;
- /// The record kind of the parent API.
- RecordKind ParentKind = RK_Unknown;
- /// A pointer to the parent APIRecord if known.
- APIRecord *ParentRecord = nullptr;
-
- HierarchyInformation() = default;
- HierarchyInformation(StringRef ParentUSR, StringRef ParentName,
- RecordKind Kind, APIRecord *ParentRecord = nullptr)
- : ParentUSR(ParentUSR), ParentName(ParentName), ParentKind(Kind),
- ParentRecord(ParentRecord) {}
-
- bool empty() const {
- return ParentUSR.empty() && ParentName.empty() &&
- ParentKind == RK_Unknown && ParentRecord == nullptr;
- }
- };
-
StringRef USR;
StringRef Name;
+
+ SymbolReference Parent;
+
PresumedLoc Location;
AvailabilityInfo Availability;
LinkageInfo Linkage;
@@ -242,79 +260,168 @@ struct APIRecord {
/// Objective-C class/instance methods).
DeclarationFragments SubHeading;
- /// Information about the parent record of this record.
- HierarchyInformation ParentInformation;
-
/// Whether the symbol was defined in a system header.
bool IsFromSystemHeader;
+ AccessControl Access;
+
private:
const RecordKind Kind;
+ friend class RecordContext;
+ // Used to store the next child record in RecordContext. This works because
+ // APIRecords semantically only have one parent.
+ mutable APIRecord *NextInContex = nullptr;
public:
+ APIRecord *getNextInContex() const { return NextInContex; }
+
RecordKind getKind() const { return Kind; }
+ static APIRecord *castFromRecordContext(const RecordContext *Ctx);
+ static RecordContext *castToRecordContext(const APIRecord *Record);
+
APIRecord() = delete;
APIRecord(RecordKind Kind, StringRef USR, StringRef Name,
- PresumedLoc Location, AvailabilityInfo Availability,
- LinkageInfo Linkage, const DocComment &Comment,
- DeclarationFragments Declaration, DeclarationFragments SubHeading,
- bool IsFromSystemHeader)
- : USR(USR), Name(Name), Location(Location),
+ SymbolReference Parent, PresumedLoc Location,
+ AvailabilityInfo Availability, LinkageInfo Linkage,
+ const DocComment &Comment, DeclarationFragments Declaration,
+ DeclarationFragments SubHeading, bool IsFromSystemHeader,
+ AccessControl Access = AccessControl())
+ : USR(USR), Name(Name), Parent(std::move(Parent)), Location(Location),
Availability(std::move(Availability)), Linkage(Linkage),
Comment(Comment), Declaration(Declaration), SubHeading(SubHeading),
- IsFromSystemHeader(IsFromSystemHeader), Kind(Kind) {}
+ IsFromSystemHeader(IsFromSystemHeader), Access(std::move(Access)),
+ Kind(Kind) {}
APIRecord(RecordKind Kind, StringRef USR, StringRef Name)
: USR(USR), Name(Name), Kind(Kind) {}
// Pure virtual destructor to make APIRecord abstract
virtual ~APIRecord() = 0;
+ static bool classof(const APIRecord *Record) { return true; }
+ static bool classofKind(RecordKind K) { return true; }
+ static bool classof(const RecordContext *Ctx) { return true; }
+};
+
+/// Base class used for specific record types that have children records this is
+/// analogous to the DeclContext for the AST
+class RecordContext {
+public:
+ static bool classof(const APIRecord *Record) {
+ return classofKind(Record->getKind());
+ }
+ static bool classofKind(APIRecord::RecordKind K) {
+ return K >= APIRecord::RK_Namespace &&
+ K <= APIRecord::RK_ClassTemplatePartialSpecialization;
+ }
+
+ static bool classof(const RecordContext *Context) { return true; }
+
+ RecordContext(APIRecord::RecordKind Kind) : Kind(Kind) {}
+
+ APIRecord::RecordKind getKind() const { return Kind; }
+
+ struct record_iterator {
+ private:
+ APIRecord *Current = nullptr;
+
+ public:
+ using value_type = APIRecord *;
+ using reference = const value_type &;
+ using pointer = const value_type *;
+ using iterator_category = std::forward_iterator_tag;
+ using difference_type = std::ptrdiff_t;
+
+ record_iterator() = default;
+ explicit record_iterator(value_type R) : Current(R) {}
+ reference operator*() const { return Current; }
+ // This doesn't strictly meet the iterator requirements, but it's the
+ // behavior we want here.
+ value_type operator->() const { return Current; }
+ record_iterator &operator++() { Current = Current->getNextInContex();
+ return *this;
+ }
+ record_iterator operator++(int) {
+ record_iterator tmp(*this);
+ ++(*this);
+ return tmp;
+ }
+
+ friend bool operator==(record_iterator x, record_iterator y) {
+ return x.Current == y.Current;
+ }
+ friend bool operator!=(record_iterator x, record_iterator y) {
+ return x.Current != y.Current;
+ }
+ };
+
+ using record_range = llvm::iterator_range<record_iterator>;
+ record_range records() const {
+ return record_range(records_begin(), records_end());
+ }
+ record_iterator records_begin() const { return record_iterator(First); };
+ record_iterator records_end() const { return record_iterator(); }
+ bool records_empty() const { return First == nullptr; };
+
+private:
+ APIRecord::RecordKind Kind;
+ mutable APIRecord *First = nullptr;
+ mutable APIRecord *Last = nullptr;
+
+protected:
+ friend class APISet;
+ void addToRecordChain(APIRecord *) const;
};
-struct NamespaceRecord : APIRecord {
- NamespaceRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
- AvailabilityInfo Availability, LinkageInfo Linkage,
- const DocComment &Comment, DeclarationFragments Declaration,
+struct NamespaceRecord : APIRecord, RecordContext {
+ NamespaceRecord(StringRef USR, StringRef Name, SymbolReference Parent,
+ PresumedLoc Loc, AvailabilityInfo Availability,
+ LinkageInfo Linkage, const DocComment &Comment,
+ DeclarationFragments Declaration,
DeclarationFragments SubHeading, bool IsFromSystemHeader)
- : APIRecord(RK_Namespace, USR, Name, Loc, std::move(Availability),
+ : APIRecord(RK_Namespace, USR, Name, Parent, Loc, std::move(Availability),
Linkage, Comment, Declaration, SubHeading,
- IsFromSystemHeader) {}
+ IsFromSystemHeader),
+ RecordContext(RK_Namespace) {}
static bool classof(const APIRecord *Record) {
- return Record->getKind() == RK_Namespace;
+ return classofKind(Record->getKind());
}
+ static bool classofKind(RecordKind K) { return K == RK_Namespace; }
};
/// This holds information associated with global functions.
struct GlobalFunctionRecord : APIRecord {
FunctionSignature Signature;
- GlobalFunctionRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
- AvailabilityInfo Availability, LinkageInfo Linkage,
- const DocComment &Comment,
+ GlobalFunctionRecord(StringRef USR, StringRef Name, SymbolReference Parent,
+ PresumedLoc Loc, AvailabilityInfo Availability,
+ LinkageInfo Linkage, const DocComment &Comment,
DeclarationFragments Declaration,
DeclarationFragments SubHeading,
FunctionSignature Signature, bool IsFromSystemHeader)
- : APIRecord(RK_GlobalFunction, USR, Name, Loc, std::move(Availability),
- Linkage, Comment, Declaration, SubHeading,
- IsFromSystemHeader),
+ : APIRecord(RK_GlobalFunction, USR, Name, Parent, Loc,
+ std::move(Availability), Linkage, Comment, Declaration,
+ SubHeading, IsFromSystemHeader),
Signature(Signature) {}
GlobalFunctionRecord(RecordKind Kind, StringRef USR, StringRef Name,
- PresumedLoc Loc, AvailabilityInfo Availability,
- LinkageInfo Linkage, const DocComment &Comment,
+ SymbolReference Parent, PresumedLoc Loc,
+ AvailabilityInfo Availability, LinkageInfo Linkage,
+ const DocComment &Comment,
DeclarationFragments Declaration,
DeclarationFragments SubHeading,
FunctionSignature Signature, bool IsFromSystemHeader)
- : APIRecord(Kind, USR, Name, Loc, std::move(Availability), Linkage,
- Comment, Declaration, SubHeading, IsFromSystemHeader),
+ : APIRecord(Kind, USR, Name, Parent, Loc, std::move(Availability),
+ Linkage, Comment, Declaration, SubHeading,
+ IsFromSystemHeader),
Signature(Signature) {}
static bool classof(const APIRecord *Record) {
- return Record->getKind() == RK_GlobalFunction;
+ return classofKind(Record->getKind());
}
+ static bool classofKind(RecordKind K) { return K == RK_GlobalFunction; }
private:
virtual void anchor();
@@ -323,63 +430,74 @@ struct GlobalFunctionRecord : APIRecord {
struct GlobalFunctionTemplateRecord : GlobalFunctionRecord {
Template Templ;
- GlobalFunctionTemplateRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
+ GlobalFunctionTemplateRecord(StringRef USR, StringRef Name,
+ SymbolReference Parent, PresumedLoc Loc,
AvailabilityInfo Availability,
LinkageInfo Linkage, const DocComment &Comment,
DeclarationFragments Declaration,
DeclarationFragments SubHeading,
FunctionSignature Signature, Template Template,
bool IsFromSystemHeader)
- : GlobalFunctionRecord(RK_GlobalFunctionTemplate, USR, Name, Loc,
+ : GlobalFunctionRecord(RK_GlobalFunctionTemplate, USR, Name, Parent, Loc,
std::move(Availability), Linkage, Comment,
Declaration, SubHeading, Signature,
IsFromSystemHeader),
Templ(Template) {}
static bool classof(const APIRecord *Record) {
- return Record->getKind() == RK_GlobalFunctionTemplate;
+ return classofKind(Record->getKind());
+ }
+ static bool classofKind(RecordKind K) {
+ return K == RK_GlobalFunctionTemplate;
}
};
struct GlobalFunctionTemplateSpecializationRecord : GlobalFunctionRecord {
GlobalFunctionTemplateSpecializationRecord(
- StringRef USR, StringRef Name, PresumedLoc Loc,
+ StringRef USR, StringRef Name, SymbolReference Parent, PresumedLoc Loc,
AvailabilityInfo Availability, LinkageInfo Linkage,
const DocComment &Comment, DeclarationFragments Declaration,
DeclarationFragments SubHeading, FunctionSignature Signature,
bool IsFromSystemHeader)
: GlobalFunctionRecord(RK_GlobalFunctionTemplateSpecialization, USR, Name,
- Loc, std::move(Availability), Linkage, Comment,
- Declaration, SubHeading, Signature,
+ Parent, Loc, std::move(Availability), Linkage,
+ Comment, Declaration, SubHeading, Signature,
IsFromSystemHeader) {}
static bool classof(const APIRecord *Record) {
- return Record->getKind() == RK_GlobalFunctionTemplateSpecialization;
+ return classofKind(Record->getKind());
+ }
+ static bool classofKind(RecordKind K) {
+ return K == RK_GlobalFunctionTemplateSpecialization;
}
};
/// This holds information associated with global functions.
struct GlobalVariableRecord : APIRecord {
- GlobalVariableRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
- AvailabilityInfo Availability, LinkageInfo Linkage,
- const DocComment &Comment,
+ GlobalVariableRecord(StringRef USR, StringRef Name, SymbolReference Parent,
+ PresumedLoc Loc, AvailabilityInfo Availability,
+ LinkageInfo Linkage, const DocComment &Comment,
DeclarationFragments Declaration,
DeclarationFragments SubHeading, bool IsFromSystemHeader)
- : APIRecord(RK_GlobalVariable, USR, Name, Loc, std::move(Availability),
- Linkage, Comment, Declaration, SubHe...
[truncated]
|
|
✅ With the latest revision this PR passed the C/C++ code formatter. |
clang/include/clang/ExtractAPI/API.h
Outdated
Comment on lines
273
to
278
| mutable APIRecord *NextInContex = nullptr; | ||
|
|
||
| public: | ||
| APIRecord *getNextInContex() const { return NextInContex; } | ||
|
|
There was a problem hiding this comment.
Looks like NextInContext is typo'd here.
clang/include/clang/ExtractAPI/API.h
Outdated
Comment on lines
314
to
318
| static bool classofKind(APIRecord::RecordKind K) { | ||
| return K >= APIRecord::RK_Namespace && | ||
| K <= APIRecord::RK_ClassTemplatePartialSpecialization; | ||
| } |
There was a problem hiding this comment.
Would it be possible to add tombstones for FirstRecordContext/LastRecordContext to make this easier to maintain?
Comment on lines
38
to
39
| /// 3. given a (record, class) combination where 'class' is some base class of | ||
| /// the dynamic type of 'node', call a user-overridable function to actually |
There was a problem hiding this comment.
Should this 'node' here actually say 'record'?
| if (!Obj) | ||
| return; | ||
| ExtendedModule &Module = getModuleForCurrentSymbol(); | ||
| // If the hierarchy has at leas one parent and child. |
There was a problem hiding this comment.
Typo:
Suggested change
| // If the hierarchy has at leas one parent and child. | |
| // If the hierarchy has at least one parent and child. |
daniel-grumberg
commented
Mar 26, 2024
| } else { | ||
| clang::index::generateUSRForType(Type, Context, TypeUSR); | ||
| } | ||
|
|
||
| return {API.copyString(TypeName), API.copyString(TypeUSR)}; | ||
| return {API.copyString(TypeName), API.copyString(TypeUSR), OwningModuleName}; |
There was a problem hiding this comment.
TODO: Can this be expressed as APISet.createSymbolReference
clang/test/ExtractAPI/objc_block.m
Outdated
| @interface Foo | ||
| // RUN: Filecheck %s --input-file %t/output.symbols.json --check-prefix NOPARAM |
There was a problem hiding this comment.
Is this Filecheck (not FileCheck) valid? I'm not sure if this will work properly on all platforms where these tests are run.
| header "ExternalModule.h" | ||
| } | ||
|
|
||
| // RUN: Filecheck %s --input-file %t/symbols/Module.symbols.json --check-prefix MOD |
There was a problem hiding this comment.
Same Filecheck capitalization issue.
daniel-grumberg
commented
Mar 26, 2024
| recordObjCInstanceVariables(ObjCInterfaceRecord, Decl->ivars()); | ||
|
|
||
| return true; | ||
| return VisitObjCInterfaceDecl(Decl->getClassInterface()); |
There was a problem hiding this comment.
Restore logic to grab symbols defined only in the implementation. First visit the interface and then add the things only defined in the implementation block to the interface record.
QuietMisdreavus
requested changes
Mar 26, 2024
This allows to segregate symbols that are extending symbols from other modules into a spearate graph. This also uses the opportunity to clean up the emit-symbol-graph interface.
daniel-grumberg
force-pushed
the
categories-external-types
branch
from
cc55423 to
e7a1d27
Compare
QuietMisdreavus
approved these changes
Mar 28, 2024
daniel-grumberg
added a commit
to daniel-grumberg/llvm-project
that referenced
this pull request
Apr 2, 2024
…mbol graphs (llvm#86676)"" This reverts commit 209a1e8.
|
Just reverted this due to buildbot failures in |
daniel-grumberg
added a commit
to daniel-grumberg/llvm-project
that referenced
this pull request
Apr 3, 2024
…m#86676) This extends ExtractAPI to take into account symbols defined in categories to types defined in an external module. This introduces 2 new command line flags, `--symbol-graph-dir=DIR` and `--emit-extension-symbol-graphs`, when used together this generates additional symbol graph files at `DIR/[email protected]` for each external module that is extended in this way. Additionally this makes some cleanups to tests to make them more resilient and cleans up the `APISet` data structure.
daniel-grumberg
added a commit
to daniel-grumberg/llvm-project
that referenced
this pull request
Apr 3, 2024
…phs (llvm#86676)" This failed the test suite due to missing DiagGroup for a new warning. This reverts commit b31414b.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This allows to segregate symbols that are extending symbols from other modules into a spearate graph. This also uses the opportunity to clean up the emit-symbol-graph interface.