Skip to content

[clang-doc] add namespaces to JSON generator #143209

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 10, 2025
Merged

[clang-doc] add namespaces to JSON generator #143209

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions clang-tools-extra/clang-doc/JSONGenerator.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -440,6 +440,39 @@ static void serializeInfo(const RecordInfo &I, json::Object &Obj,
serializeCommonChildren(I.Children, Obj, RepositoryUrl);
}

static void serializeInfo(const NamespaceInfo &I, json::Object &Obj,
std::optional<StringRef> RepositoryUrl) {
serializeCommonAttributes(I, Obj, RepositoryUrl);

if (!I.Children.Namespaces.empty()) {
json::Value NamespacesArray = Array();
auto &NamespacesArrayRef = *NamespacesArray.getAsArray();
NamespacesArrayRef.reserve(I.Children.Namespaces.size());
for (auto &Namespace : I.Children.Namespaces) {
json::Value NamespaceVal = Object();
auto &NamespaceObj = *NamespaceVal.getAsObject();
serializeReference(Namespace, NamespaceObj);
NamespacesArrayRef.push_back(NamespaceVal);
}
Obj["Namespaces"] = NamespacesArray;
}

if (!I.Children.Functions.empty()) {
json::Value FunctionsArray = Array();
auto &FunctionsArrayRef = *FunctionsArray.getAsArray();
FunctionsArrayRef.reserve(I.Children.Functions.size());
for (const auto &Function : I.Children.Functions) {
json::Value FunctionVal = Object();
auto &FunctionObj = *FunctionVal.getAsObject();
serializeInfo(Function, FunctionObj, RepositoryUrl);
FunctionsArrayRef.push_back(FunctionVal);
}
Obj["Functions"] = FunctionsArray;
}

serializeCommonChildren(I.Children, Obj, RepositoryUrl);
}

Error JSONGenerator::generateDocs(
StringRef RootDir, llvm::StringMap<std::unique_ptr<doc::Info>> Infos,
const ClangDocContext &CDCtx) {
Expand Down Expand Up @@ -482,6 +515,7 @@ Error JSONGenerator::generateDocForInfo(Info *I, raw_ostream &OS,

switch (I->IT) {
case InfoType::IT_namespace:
serializeInfo(*static_cast<NamespaceInfo *>(I), Obj, CDCtx.RepositoryUrl);
break;
case InfoType::IT_record:
serializeInfo(*static_cast<RecordInfo *>(I), Obj, CDCtx.RepositoryUrl);
Expand Down
1 change: 1 addition & 0 deletions clang-tools-extra/clang-doc/Serialize.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -742,6 +742,7 @@ static void populateFunctionInfo(FunctionInfo &I, const FunctionDecl *D,
I.ReturnType = getTypeInfoForType(D->getReturnType(), LO);
I.Prototype = getFunctionPrototype(D);
parseParameters(I, D);
I.IsStatic = D->isStatic();

populateTemplateParameters(I.Template, D);

Expand Down
25 changes: 25 additions & 0 deletions clang-tools-extra/test/clang-doc/json/function-specifiers.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
// RUN: rm -rf %t && mkdir -p %t
// RUN: clang-doc --output=%t --format=json --executor=standalone %s
// RUN: FileCheck %s < %t/GlobalNamespace/index.json

static void myFunction() {}

void noExceptFunction() noexcept {}

inline void inlineFunction() {}

extern void externFunction() {}

constexpr void constexprFunction() {}

// CHECK: "Functions": [
// CHECK-NEXT: {
// CHECK: "IsStatic": true,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't you need something like CHECK-LABEL to make sure you're matching the correct part of the output? If the name is later in the output, maybe we should move it to make testing easier? Otherwise, I'd guess we would still need to check that we're matching the right property w/ the right entity...

Most of this test is just precommit for the FIXMEs, so it isn't a big deal right now, but I can imagine as the test grows, it may become harder to maintain.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah this was motivated by convenience. The functions are serialized in order, so I know that the first entry to be checked will be the static function. Hence, the "IsStatic": true check. The contents of an object are serialized alphabetically, so I haven't figured how to check the name first, since that comes after all the Is fields.

That would be useful. I can investigate this more.

Copy link
Contributor

@ilovepi ilovepi Jun 10, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Matching the property and the name after should be OK, though not ideal. IIRC the JSON we emit from llvm-readobj isn't sorted by key, so maybe there is a setting in the JSON builder? Or maybe it's because it uses a JSON stream? I can't quite recall, but it may be worth looking at other subprojects to see how they deal with the issue.

// COM: FIXME: Emit ExceptionSpecificationType
// CHECK-NOT: "ExceptionSpecifcation" : "noexcept",
// COM: FIXME: Emit inline
// CHECK-NOT: "IsInline": true,
// COM: FIXME: Emit extern
// CHECK-NOT: "IsExtern": true,
// COM: FIXME: Emit constexpr
// CHECK-NOT: "IsConstexpr": true,
107 changes: 107 additions & 0 deletions clang-tools-extra/test/clang-doc/json/namespace.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
// RUN: rm -rf %t && mkdir -p %t
// RUN: clang-doc --output=%t --format=json --executor=standalone %s
// RUN: FileCheck %s < %t/GlobalNamespace/index.json

class MyClass {};

void myFunction(int Param);

namespace NestedNamespace {
} // namespace NestedNamespace

// FIXME: Global variables are not mapped or serialized.
static int Global;

enum Color {
RED,
GREEN,
BLUE = 5
};

typedef int MyTypedef;

// CHECK: {
// CHECK-NEXT: "Enums": [
// CHECK-NEXT: {
// CHECK-NEXT: "Location": {
// CHECK-NEXT: "Filename": "{{.*}}namespace.cpp",
// CHECK-NEXT: "LineNumber": 15
// CHECK-NEXT: },
// CHECK-NEXT: "Members": [
// CHECK-NEXT: {
// CHECK-NEXT: "Name": "RED",
// CHECK-NEXT: "Value": "0"
// CHECK-NEXT: },
// CHECK-NEXT: {
// CHECK-NEXT: "Name": "GREEN",
// CHECK-NEXT: "Value": "1"
// CHECK-NEXT: },
// CHECK-NEXT: {
// CHECK-NEXT: "Name": "BLUE",
// CHECK-NEXT: "ValueExpr": "5"
// CHECK-NEXT: }
// CHECK-NEXT: ],
// CHECK-NEXT: "Name": "Color",
// CHECK-NEXT: "Scoped": false,
// CHECK-NEXT: "USR": "{{[0-9A-F]*}}"
// CHECK-NEXT: }
// CHECK-NEXT: ],
// CHECK-NEXT: "Functions": [
// CHECK-NEXT: {
// CHECK-NEXT: "IsStatic": false,
// CHECK-NEXT: "Name": "myFunction",
// CHECK-NEXT: "Params": [
// CHECK-NEXT: {
// CHECK-NEXT: "Name": "Param",
// CHECK-NEXT: "Type": "int"
// CHECK-NEXT: }
// CHECK-NEXT: ],
// CHECK-NEXT: "ReturnType": {
// CHECK-NEXT: "IsBuiltIn": false,
// CHECK-NEXT: "IsTemplate": false,
// CHECK-NEXT: "Name": "void",
// CHECK-NEXT: "QualName": "void",
// CHECK-NEXT: "USR": "0000000000000000000000000000000000000000"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do builtin types like void and int get fixed USRs? Also, shouldn't IsBuiltIn be true here? The default value is false, but we should set it in Serialize.cpp, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes those USRs are fixed.

IsBuiltIn is set in Serialize.cpp and I just checked in gdb. IsBuiltIn is true for void but it gets emitted to its default value. I'm not sure how it works but I have a feeling the bitcode reader doesn't properly serialize everything.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, we're probably updating the wrong data structure/copy. Booleans should serialize fine. It could also be that we're doing something silly in a copy constructor or using a bad initializer list.

Copy link
Member Author

@evelez7 evelez7 Jun 11, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fairly certain the reason we are seeing lots of default values is that these values aren't defined in BitecodeWriter.cpp. Every Info field has a corresponding enum thing.

llvm-project/clang-tools-extra/clang-doc/BitcodeWriter.cpp

Lines 196 to 201 in d75e284

{REFERENCE_USR, {"USR", &genSymbolIdAbbrev}},
{REFERENCE_NAME, {"Name", &genStringAbbrev}},
{REFERENCE_QUAL_NAME, {"QualName", &genStringAbbrev}},
{REFERENCE_TYPE, {"RefType", &genIntAbbrev}},
{REFERENCE_PATH, {"Path", &genStringAbbrev}},
{REFERENCE_FIELD, {"Field", &genIntAbbrev}},

And then corresponding values in a couple other places + BitcodeReader.cpp

// CHECK-NEXT: },
// CHECK-NEXT: "USR": "{{[0-9A-F]*}}"
// CHECK-NEXT: }
// CHECK-NEXT: ],
// CHECK-NEXT: "Name": "",
// CHECK-NEXT: "Namespaces": [
// CHECK-NEXT: {
// CHECK-NEXT: "Name": "NestedNamespace",
// CHECK-NEXT: "Path": "",
// CHECK-NEXT: "QualName": "NestedNamespace",
// CHECK-NEXT: "USR": "{{[0-9A-F]*}}"
// CHECK-NEXT: }
// CHECK-NEXT: ],
// CHECK-NEXT: "Records": [
// CHECK-NEXT: {
// CHECK-NEXT: "Name": "MyClass",
// CHECK-NEXT: "Path": "GlobalNamespace",
// CHECK-NEXT: "QualName": "MyClass",
// CHECK-NEXT: "USR": "{{[0-9A-F]*}}"
// CHECK-NEXT: }
// CHECK-NEXT: ],
// CHECK-NEXT: "Typedefs": [
// CHECK-NEXT: {
// CHECK-NEXT: "IsUsing": false,
// CHECK-NEXT: "Location": {
// CHECK-NEXT: "Filename": "{{.*}}namespace.cpp",
// CHECK-NEXT: "LineNumber": 21
// CHECK-NEXT: },
// CHECK-NEXT: "Name": "MyTypedef",
// CHECK-NEXT: "TypeDeclaration": "",
// CHECK-NEXT: "USR": "{{[0-9A-F]*}}",
// CHECK-NEXT: "Underlying": {
// CHECK-NEXT: "IsBuiltIn": false,
// CHECK-NEXT: "IsTemplate": false,
// CHECK-NEXT: "Name": "int",
// CHECK-NEXT: "QualName": "int",
// CHECK-NEXT: "USR": "0000000000000000000000000000000000000000"
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: ],
// CHECK-NEXT: "USR": "0000000000000000000000000000000000000000"
// CHECK-NOT: "Variables": [
// CHECK-NEXT: }
72 changes: 72 additions & 0 deletions clang-tools-extra/unittests/clang-doc/JSONGeneratorTest.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -171,5 +171,77 @@ TEST(JSONGeneratorTest, emitRecordJSON) {
})raw";
EXPECT_EQ(Expected, Actual.str());
}

TEST(JSONGeneratorTest, emitNamespaceJSON) {
NamespaceInfo I;
I.Name = "Namespace";
I.Path = "path/to/A";
I.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);

I.Children.Namespaces.emplace_back(
EmptySID, "ChildNamespace", InfoType::IT_namespace,
"path::to::A::Namespace::ChildNamespace", "path/to/A/Namespace");
I.Children.Records.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
"path::to::A::Namespace::ChildStruct",
"path/to/A/Namespace");
I.Children.Functions.emplace_back();
I.Children.Functions.back().Name = "OneFunction";
I.Children.Functions.back().Access = AccessSpecifier::AS_none;
I.Children.Enums.emplace_back();
I.Children.Enums.back().Name = "OneEnum";

auto G = getJSONGenerator();
assert(G);
std::string Buffer;
llvm::raw_string_ostream Actual(Buffer);
auto Err = G->generateDocForInfo(&I, Actual, ClangDocContext());
assert(!Err);
std::string Expected = R"raw({
"Enums": [
{
"Name": "OneEnum",
"Scoped": false,
"USR": "0000000000000000000000000000000000000000"
}
],
"Functions": [
{
"IsStatic": false,
"Name": "OneFunction",
"ReturnType": {
"IsBuiltIn": false,
"IsTemplate": false,
"Name": "",
"QualName": "",
"USR": "0000000000000000000000000000000000000000"
},
"USR": "0000000000000000000000000000000000000000"
}
],
"Name": "Namespace",
"Namespace": [
"A"
],
"Namespaces": [
{
"Name": "ChildNamespace",
"Path": "path/to/A/Namespace",
"QualName": "path::to::A::Namespace::ChildNamespace",
"USR": "0000000000000000000000000000000000000000"
}
],
"Path": "path/to/A",
"Records": [
{
"Name": "ChildStruct",
"Path": "path/to/A/Namespace",
"QualName": "path::to::A::Namespace::ChildStruct",
"USR": "0000000000000000000000000000000000000000"
}
],
"USR": "0000000000000000000000000000000000000000"
})raw";
EXPECT_EQ(Expected, Actual.str());
}
} // namespace doc
} // namespace clang