Skip to content

[clang][modules] Allow module maps with textual headers to be non-affecting #89441

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Apr 24, 2024
Merged

[clang][modules] Allow module maps with textual headers to be non-affecting #89441

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
9165d60
[clang][modules] Allow module map files with textual header be non-af…
jansvoboda11 Apr 19, 2024
17b1860
[clang][modules] Mark 'use'd modules as affecting
jansvoboda11 Apr 22, 2024
9eeea3f
[clang][modules] Do consider module maps of included textual headers …
jansvoboda11 Apr 23, 2024
ddc6bf9
Reject all non-included headers from that are !isCompilingModuleHeade…
jansvoboda11 Apr 23, 2024
2993a63
Don't add modules for textual headers to `ModulesToProcess`
jansvoboda11 Apr 23, 2024
b03c34f
**Locally** included
jansvoboda11 Apr 23, 2024
1228f5c
Fix test on Windows
jansvoboda11 Apr 24, 2024
14ac904
Split out HFI conditions
jansvoboda11 Apr 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions clang/include/clang/Lex/HeaderSearch.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,12 @@ class TargetInfo;
/// The preprocessor keeps track of this information for each
/// file that is \#included.
struct HeaderFileInfo {
// TODO: Whether the file was included is not a property of the file itself.
// It's a preprocessor state, move it there.
/// True if this file has been included (or imported) **locally**.
LLVM_PREFERRED_TYPE(bool)
unsigned IsLocallyIncluded : 1;

// TODO: Whether the file was imported is not a property of the file itself.
// It's a preprocessor state, move it there.
/// True if this is a \#import'd file.
Expand Down Expand Up @@ -135,10 +141,10 @@ struct HeaderFileInfo {
StringRef Framework;

HeaderFileInfo()
: isImport(false), isPragmaOnce(false), DirInfo(SrcMgr::C_User),
External(false), isModuleHeader(false), isTextualModuleHeader(false),
isCompilingModuleHeader(false), Resolved(false),
IndexHeaderMapHeader(false), IsValid(false) {}
: IsLocallyIncluded(false), isImport(false), isPragmaOnce(false),
DirInfo(SrcMgr::C_User), External(false), isModuleHeader(false),
isTextualModuleHeader(false), isCompilingModuleHeader(false),
Resolved(false), IndexHeaderMapHeader(false), IsValid(false) {}

/// Retrieve the controlling macro for this header file, if
/// any.
Expand Down
1 change: 1 addition & 0 deletions clang/lib/Lex/HeaderSearch.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1574,6 +1574,7 @@ bool HeaderSearch::ShouldEnterIncludeFile(Preprocessor &PP,
}
}

FileInfo.IsLocallyIncluded = true;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd consider placing this at the end of HandleHeaderIncludeOrImport rather than here:

  • it looks like there are cases where this function returns true and we don't enter the file textually (or at least, it's not obvious that there are no such cases)
  • having a load-bearing side-effect of a "should we do X" query seems unexpected and liable to break during refactoring or in edge cases

Concretely I don't see a case where this doesn't work though, up to you.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, I wanted to colocate this with call to Preprocessor::markIncluded(). If we find a better way of doing things, we can move both of them.

IsFirstIncludeOfFile = PP.markIncluded(File);
return true;
}
Expand Down
78 changes: 44 additions & 34 deletions clang/lib/Serialization/ASTWriter.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -171,34 +171,9 @@ GetAffectingModuleMaps(const Preprocessor &PP, Module *RootModule) {
.ModulesPruneNonAffectingModuleMaps)
return std::nullopt;

SmallVector<const Module *> ModulesToProcess{RootModule};

const HeaderSearch &HS = PP.getHeaderSearchInfo();

SmallVector<OptionalFileEntryRef, 16> FilesByUID;
HS.getFileMgr().GetUniqueIDMapping(FilesByUID);

if (FilesByUID.size() > HS.header_file_size())
FilesByUID.resize(HS.header_file_size());

for (unsigned UID = 0, LastUID = FilesByUID.size(); UID != LastUID; ++UID) {
OptionalFileEntryRef File = FilesByUID[UID];
if (!File)
continue;

const HeaderFileInfo *HFI = HS.getExistingLocalFileInfo(*File);
if (!HFI || (HFI->isModuleHeader && !HFI->isCompilingModuleHeader))
continue;

for (const auto &KH : HS.findResolvedModulesForHeader(*File)) {
if (!KH.getModule())
continue;
ModulesToProcess.push_back(KH.getModule());
}
}

const ModuleMap &MM = HS.getModuleMap();
SourceManager &SourceMgr = PP.getSourceManager();
const SourceManager &SourceMgr = PP.getSourceManager();

std::set<const FileEntry *> ModuleMaps;
auto CollectIncludingModuleMaps = [&](FileID FID, FileEntryRef F) {
Expand Down Expand Up @@ -233,12 +208,48 @@ GetAffectingModuleMaps(const Preprocessor &PP, Module *RootModule) {
}
};

for (const Module *CurrentModule : ModulesToProcess) {
// Handle all the affecting modules referenced from the root module.

std::queue<const Module *> Q;
Q.push(RootModule);
while (!Q.empty()) {
const Module *CurrentModule = Q.front();
Q.pop();

CollectIncludingMapsFromAncestors(CurrentModule);
for (const Module *ImportedModule : CurrentModule->Imports)
CollectIncludingMapsFromAncestors(ImportedModule);
for (const Module *UndeclaredModule : CurrentModule->UndeclaredUses)
CollectIncludingMapsFromAncestors(UndeclaredModule);

for (auto *M : CurrentModule->submodules())
Q.push(M);
}

// Handle textually-included headers that belong to other modules.

SmallVector<OptionalFileEntryRef, 16> FilesByUID;
HS.getFileMgr().GetUniqueIDMapping(FilesByUID);

if (FilesByUID.size() > HS.header_file_size())
FilesByUID.resize(HS.header_file_size());

for (unsigned UID = 0, LastUID = FilesByUID.size(); UID != LastUID; ++UID) {
OptionalFileEntryRef File = FilesByUID[UID];
if (!File)
continue;

const HeaderFileInfo *HFI = HS.getExistingLocalFileInfo(*File);
if (!HFI)
continue; // We have no information on this being a header file.
if (!HFI->isCompilingModuleHeader && HFI->isModuleHeader)
continue; // Modular header, handled in the above module-based loop.
if (!HFI->isCompilingModuleHeader && !HFI->IsLocallyIncluded)
continue; // Non-modular header not included locally is not affecting.

for (const auto &KH : HS.findResolvedModulesForHeader(*File))
if (const Module *M = KH.getModule())
CollectIncludingMapsFromAncestors(M);
}

return ModuleMaps;
Expand Down Expand Up @@ -2053,14 +2064,13 @@ void ASTWriter::WriteHeaderSearch(const HeaderSearch &HS) {
if (!File)
continue;

// Get the file info. Skip emitting this file if we have no information on
// it as a header file (in which case HFI will be null) or if it hasn't
// changed since it was loaded. Also skip it if it's for a modular header
// from a different module; in that case, we rely on the module(s)
// containing the header to provide this information.
const HeaderFileInfo *HFI = HS.getExistingLocalFileInfo(*File);
if (!HFI || (HFI->isModuleHeader && !HFI->isCompilingModuleHeader))
continue;
if (!HFI)
continue; // We have no information on this being a header file.
if (!HFI->isCompilingModuleHeader && HFI->isModuleHeader)
continue; // Header file info is tracked by the owning module file.
if (!HFI->isCompilingModuleHeader && !PP->alreadyIncluded(*File))
continue; // Non-modular header not included is not needed.

// Massage the file path into an appropriate form.
StringRef Filename = File->getName();
Expand Down
46 changes: 46 additions & 0 deletions clang/test/Modules/prune-non-affecting-module-map-files-textual.c
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
// This test checks that a module map with a textual header can be marked as
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a useful test, I think there are a couple of other affecting-ness properties that are important to test:

  • that a textual header that is included means its module is affecting (i.e. the include matters)
  • that this textual-affecting-ness doesn't leak outside the module (a module depending on B isn't affected by A.modulemap)

I tried to cover these in https://github.com/llvm/llvm-project/pull/89729/files#diff-2eed05f9a85bc5a79b9651ee0f23e5f1494e94a2f32e093847aa6dae5ce5d839 - it might be nice to add these as a second test here (I can do it as a followup if you prefer)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This patch unfortunately doesn't pass that test (on line 44) due to this: when computing affecting module maps for B, "A.h" makes it through the HFI filter, module A is added to ModulesToProcess and the module map for its import X is considered affecting.

I tried splitting up that logic so that we consider imports and undeclared uses only of the current module and its submodules. That doesn't work either, because PP.alreadyIncluded("X.h") returns true, since it checks the global (cross-module) state, not the local state. This might be possible with #71117. Until then, I think we need some "has this been included locally" state that's local to the current TU. Maybe that can live in HeaderFileInfo as per your patch, or as a Preprocessor state.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should work with b03c34f.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(And thanks for the extended test case!)

// non-affecting.

// RUN: rm -rf %t && mkdir %t
// RUN: split-file %s %t

//--- X.modulemap
module X { textual header "X.h" }
//--- X.h
typedef int X_int;

//--- Y.modulemap
module Y { textual header "Y.h" }
//--- Y.h
typedef int Y_int;

//--- A.modulemap
module A { header "A.h" export * }
//--- A.h
#include "X.h"

// RUN: %clang_cc1 -fmodules -emit-module %t/A.modulemap -fmodule-name=A -o %t/A0.pcm \
// RUN: -fmodule-map-file=%t/X.modulemap
// RUN: %clang_cc1 -fsyntax-only -module-file-info %t/A0.pcm | FileCheck %s --check-prefix=A0 --implicit-check-not=Y.modulemap
// A0: Input file: {{.*}}X.modulemap

// RUN: %clang_cc1 -fmodules -emit-module %t/A.modulemap -fmodule-name=A -o %t/A1.pcm \
// RUN: -fmodule-map-file=%t/X.modulemap -fmodule-map-file=%t/Y.modulemap
// RUN: %clang_cc1 -fsyntax-only -module-file-info %t/A0.pcm | FileCheck %s --check-prefix=A1 \
// RUN: --implicit-check-not=Y.modulemap
// A1: Input file: {{.*}}X.modulemap

// RUN: diff %t/A0.pcm %t/A1.pcm

//--- B.modulemap
module B { header "B.h" export * }
//--- B.h
#include "A.h"
typedef X_int B_int;

// RUN: %clang_cc1 -fmodules -emit-module %t/B.modulemap -fmodule-name=B -o %t/B.pcm \
// RUN: -fmodule-file=A=%t/A0.pcm \
// RUN: -fmodule-map-file=%t/A.modulemap -fmodule-map-file=%t/X.modulemap -fmodule-map-file=%t/Y.modulemap
// RUN: %clang_cc1 -fsyntax-only -module-file-info %t/B.pcm | FileCheck %s --check-prefix=B \
// RUN: --implicit-check-not=X.modulemap --implicit-check-not=Y.modulemap
// B: Input file: {{.*}}B.modulemap