Reapply "[Clang] Implement P3034R1 Module Declarations Shouldn’t be Macros" #102135
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: yronglin <[email protected]>
Signed-off-by: yronglin <[email protected]>
… pass this flag as argument Signed-off-by: yronglin <[email protected]>
Signed-off-by: yronglin <[email protected]>
|
✅ With the latest revision this PR passed the C/C++ code formatter. |
yronglin
changed the title
|
I'm not sure if this is the right approach, what do you think? |
|
is this ready for review? |
cor3ntin
reviewed
Aug 14, 2024
Comment on lines
137
to
+139
| bool IsAtPhysicalStartOfLine; | ||
|
|
||
| bool IsCurrentLexingTokAtPhysicalStartOfLine; |
There was a problem hiding this comment.
we have IsAtStartOfLine and IsAtPhysicalStartOfLine - Why do we need a third one?
There was a problem hiding this comment.
Thanks for your review! This change introduced by 829b1c1. It's used to replace the bool TokAtPhysicalStartOfLine agurments in
llvm-project/clang/lib/Lex/Lexer.cpp
Line 3715 in fcefe95
Lexer::IsAtPhysicalStartOfLine , It represents the current token starts at the physical line. We may skip the '\n' or comment before the real token starts. with this change, we can avoid pass this value as a function argument.
cor3ntin
changed the title
Signed-off-by: yronglin <[email protected]>
Yeah, it's ready now. |
llvmbot
added
clang
|
@llvm/pr-subscribers-clang-modules @llvm/pr-subscribers-clang Author: None (yronglin) ChangesThis PR reapply [Clang] Implement P3034R1 Module Declarations Shouldn’t be Macros, and partially implement P1857R3 Modules Dependency Discovery. The original PR introduced the The 1st commit is the original PR, the others is the partial implementation of P1857R3. Patch is 87.30 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/102135.diff 27 Files Affected:
diff --git a/clang/docs/ReleaseNotes.rst b/clang/docs/ReleaseNotes.rst
index 4c7bd099420abf..a5953a3641fff7 100644
--- a/clang/docs/ReleaseNotes.rst
+++ b/clang/docs/ReleaseNotes.rst
@@ -89,6 +89,8 @@ C++2c Feature Support
- Add ``__builtin_is_virtual_base_of`` intrinsic, which supports
`P2985R0 A type trait for detecting virtual base classes <https://wg21.link/p2985r0>`_
+- Implemented `P3034R1 Module Declarations Shouldn’t be Macros <https://wg21.link/P3034R1>`_.
+
Resolutions to C++ Defect Reports
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
diff --git a/clang/include/clang/Basic/DiagnosticLexKinds.td b/clang/include/clang/Basic/DiagnosticLexKinds.td
index 12d7b8c0205ee9..08ece01009387d 100644
--- a/clang/include/clang/Basic/DiagnosticLexKinds.td
+++ b/clang/include/clang/Basic/DiagnosticLexKinds.td
@@ -952,6 +952,11 @@ def warn_module_conflict : Warning<
InGroup<ModuleConflict>;
// C++20 modules
+def err_module_decl_cannot_be_macros : Error<
+ "the module name in a module%select{| partition}0 declaration cannot contain "
+ "an object-like macro %1">;
+def err_unxepected_paren_in_module_decl : Error<
+ "unexpected '(' after the module name in a module%select{| partition}0 declaration">;
def err_header_import_semi_in_macro : Error<
"semicolon terminating header import declaration cannot be produced "
"by a macro">;
diff --git a/clang/include/clang/Basic/IdentifierTable.h b/clang/include/clang/Basic/IdentifierTable.h
index ae9ebd9f59154e..e8b9e381a5b4c3 100644
--- a/clang/include/clang/Basic/IdentifierTable.h
+++ b/clang/include/clang/Basic/IdentifierTable.h
@@ -180,6 +180,10 @@ class alignas(IdentifierInfoAlignment) IdentifierInfo {
LLVM_PREFERRED_TYPE(bool)
unsigned IsModulesImport : 1;
+ // True if this is the 'module' contextual keyword.
+ LLVM_PREFERRED_TYPE(bool)
+ unsigned IsModulesDecl : 1;
+
// True if this is a mangled OpenMP variant name.
LLVM_PREFERRED_TYPE(bool)
unsigned IsMangledOpenMPVariantName : 1;
@@ -196,7 +200,7 @@ class alignas(IdentifierInfoAlignment) IdentifierInfo {
LLVM_PREFERRED_TYPE(bool)
unsigned IsFinal : 1;
- // 22 bits left in a 64-bit word.
+ // 21 bits left in a 64-bit word.
// Managed by the language front-end.
void *FETokenInfo = nullptr;
@@ -212,8 +216,8 @@ class alignas(IdentifierInfoAlignment) IdentifierInfo {
IsCPPOperatorKeyword(false), NeedsHandleIdentifier(false),
IsFromAST(false), ChangedAfterLoad(false), FEChangedAfterLoad(false),
RevertedTokenID(false), OutOfDate(false), IsModulesImport(false),
- IsMangledOpenMPVariantName(false), IsDeprecatedMacro(false),
- IsRestrictExpansion(false), IsFinal(false) {}
+ IsModulesDecl(false), IsMangledOpenMPVariantName(false),
+ IsDeprecatedMacro(false), IsRestrictExpansion(false), IsFinal(false) {}
public:
IdentifierInfo(const IdentifierInfo &) = delete;
@@ -520,6 +524,18 @@ class alignas(IdentifierInfoAlignment) IdentifierInfo {
RecomputeNeedsHandleIdentifier();
}
+ /// Determine whether this is the contextual keyword \c module.
+ bool isModulesDeclaration() const { return IsModulesDecl; }
+
+ /// Set whether this identifier is the contextual keyword \c module.
+ void setModulesDeclaration(bool I) {
+ IsModulesDecl = I;
+ if (I)
+ NeedsHandleIdentifier = true;
+ else
+ RecomputeNeedsHandleIdentifier();
+ }
+
/// Determine whether this is the mangled name of an OpenMP variant.
bool isMangledOpenMPVariantName() const { return IsMangledOpenMPVariantName; }
@@ -569,7 +585,8 @@ class alignas(IdentifierInfoAlignment) IdentifierInfo {
void RecomputeNeedsHandleIdentifier() {
NeedsHandleIdentifier = isPoisoned() || hasMacroDefinition() ||
isExtensionToken() || isFutureCompatKeyword() ||
- isOutOfDate() || isModulesImport();
+ isOutOfDate() || isModulesImport() ||
+ isModulesDeclaration();
}
};
@@ -740,6 +757,8 @@ class IdentifierTable {
// If this is the 'import' contextual keyword, mark it as such.
if (Name == "import")
II->setModulesImport(true);
+ else if (Name == "module")
+ II->setModulesDeclaration(true);
return *II;
}
diff --git a/clang/include/clang/Basic/TokenKinds.def b/clang/include/clang/Basic/TokenKinds.def
index 7e638dc1ddcdba..bea46f617e690d 100644
--- a/clang/include/clang/Basic/TokenKinds.def
+++ b/clang/include/clang/Basic/TokenKinds.def
@@ -1006,6 +1006,9 @@ ANNOTATION(module_include)
ANNOTATION(module_begin)
ANNOTATION(module_end)
+// Annotations for C++, Clang and Objective-C named modules.
+ANNOTATION(module_name)
+
// Annotation for a header_name token that has been looked up and transformed
// into the name of a header unit.
ANNOTATION(header_unit)
diff --git a/clang/include/clang/Lex/Lexer.h b/clang/include/clang/Lex/Lexer.h
index b6ecc7e5ded9e2..142f3f05381298 100644
--- a/clang/include/clang/Lex/Lexer.h
+++ b/clang/include/clang/Lex/Lexer.h
@@ -124,7 +124,7 @@ class Lexer : public PreprocessorLexer {
//===--------------------------------------------------------------------===//
// Context that changes as the file is lexed.
// NOTE: any state that mutates when in raw mode must have save/restore code
- // in Lexer::isNextPPTokenLParen.
+ // in Lexer::peekNextPPToken.
// BufferPtr - Current pointer into the buffer. This is the next character
// to be lexed.
@@ -136,6 +136,8 @@ class Lexer : public PreprocessorLexer {
bool IsAtPhysicalStartOfLine;
+ bool IsCurrentLexingTokAtPhysicalStartOfLine;
+
bool HasLeadingSpace;
bool HasLeadingEmptyMacro;
@@ -609,7 +611,7 @@ class Lexer : public PreprocessorLexer {
/// LexTokenInternal - Internal interface to lex a preprocessing token. Called
/// by Lex.
///
- bool LexTokenInternal(Token &Result, bool TokAtPhysicalStartOfLine);
+ bool LexTokenInternal(Token &Result);
bool CheckUnicodeWhitespace(Token &Result, uint32_t C, const char *CurPtr);
@@ -629,10 +631,10 @@ class Lexer : public PreprocessorLexer {
BufferPtr = TokEnd;
}
- /// isNextPPTokenLParen - Return 1 if the next unexpanded token will return a
- /// tok::l_paren token, 0 if it is something else and 2 if there are no more
- /// tokens in the buffer controlled by this lexer.
- unsigned isNextPPTokenLParen();
+ /// peekNextPPToken - Return std::nullopt if there are no more tokens in the
+ /// buffer controlled by this lexer, otherwise return the next unexpanded
+ /// token.
+ std::optional<Token> peekNextPPToken();
//===--------------------------------------------------------------------===//
// Lexer character reading interfaces.
@@ -749,12 +751,9 @@ class Lexer : public PreprocessorLexer {
bool LexCharConstant (Token &Result, const char *CurPtr,
tok::TokenKind Kind);
bool LexEndOfFile (Token &Result, const char *CurPtr);
- bool SkipWhitespace (Token &Result, const char *CurPtr,
- bool &TokAtPhysicalStartOfLine);
- bool SkipLineComment (Token &Result, const char *CurPtr,
- bool &TokAtPhysicalStartOfLine);
- bool SkipBlockComment (Token &Result, const char *CurPtr,
- bool &TokAtPhysicalStartOfLine);
+ bool SkipWhitespace (Token &Result, const char *CurPtr);
+ bool SkipLineComment (Token &Result, const char *CurPtr);
+ bool SkipBlockComment (Token &Result, const char *CurPtr);
bool SaveLineComment (Token &Result, const char *CurPtr);
bool IsStartOfConflictMarker(const char *CurPtr);
diff --git a/clang/include/clang/Lex/Preprocessor.h b/clang/include/clang/Lex/Preprocessor.h
index 623f868ca1e648..c6f1709aa874eb 100644
--- a/clang/include/clang/Lex/Preprocessor.h
+++ b/clang/include/clang/Lex/Preprocessor.h
@@ -109,6 +109,77 @@ class TokenValue {
}
};
+/// Represents module or partition name token sequance.
+///
+/// module-name:
+/// module-name-qualifier[opt] identifier
+///
+/// partition-name: [C++20]
+/// : module-name-qualifier[opt] identifier
+///
+/// module-name-qualifier
+/// module-name-qualifier[opt] identifier .
+///
+/// This class can only be created by the preprocessor and guarantees that the
+/// two source array being contiguous in memory and only contains 3 kind of
+/// tokens (identifier, '.' and ':'). And only available when the preprocessor
+/// returns annot_module_name token.
+///
+/// For exmaple:
+///
+/// export module m.n:c.d
+///
+/// The module name array has 3 tokens ['m', '.', 'n'].
+/// The partition name array has 4 tokens [':', 'c', '.', 'd'].
+///
+/// When import a partition in a named module fragment (Eg. import :part1;),
+/// the module name array will be empty, and the partition name array has 2
+/// tokens.
+///
+/// When we meet a private-module-fragment (Eg. module :private;), preprocessor
+/// will not return a annot_module_name token, but will return 2 separate tokens
+/// [':', 'kw_private'].
+
+class ModuleNameInfo {
+ friend class Preprocessor;
+ ArrayRef<Token> ModuleName;
+ ArrayRef<Token> PartitionName;
+
+ ModuleNameInfo(ArrayRef<Token> AnnotToks, std::optional<unsigned> ColonIndex);
+
+public:
+ /// Return the contiguous token array.
+ ArrayRef<Token> getTokens() const {
+ if (ModuleName.empty())
+ return PartitionName;
+ if (PartitionName.empty())
+ return ModuleName;
+ return ArrayRef(ModuleName.begin(), PartitionName.end());
+ }
+ bool hasModuleName() const { return !ModuleName.empty(); }
+ bool hasPartitionName() const { return !PartitionName.empty(); }
+ ArrayRef<Token> getModuleName() const { return ModuleName; }
+ ArrayRef<Token> getPartitionName() const { return PartitionName; }
+ Token getColonToken() const {
+ assert(hasPartitionName() && "Do not have a partition name");
+ return getPartitionName().front();
+ }
+
+ /// Under the standard C++ Modules, the dot is just part of the module name,
+ /// and not a real hierarchy separator. Flatten such module names now.
+ std::string getFlatName() const;
+
+ /// Build a module id path from the contiguous token array, both include
+ /// module name and partition name.
+ void getModuleIdPath(
+ SmallVectorImpl<std::pair<IdentifierInfo *, SourceLocation>> &Path) const;
+
+ /// Build a module id path from \param ModuleName.
+ static void getModuleIdPath(
+ ArrayRef<Token> ModuleName,
+ SmallVectorImpl<std::pair<IdentifierInfo *, SourceLocation>> &Path);
+};
+
/// Context in which macro name is used.
enum MacroUse {
// other than #define or #undef
@@ -337,6 +408,9 @@ class Preprocessor {
/// Whether the last token we lexed was an '@'.
bool LastTokenWasAt = false;
+ /// Whether the last token we lexed was an 'export' keyword.
+ std::optional<Token> LastTokenWasExportKeyword = std::nullopt;
+
/// A position within a C++20 import-seq.
class StdCXXImportSeq {
public:
@@ -540,24 +614,12 @@ class Preprocessor {
reset();
}
- void handleIdentifier(IdentifierInfo *Identifier) {
- if (isModuleCandidate() && Identifier)
- Name += Identifier->getName().str();
- else if (!isNamedModule())
- reset();
- }
-
- void handleColon() {
- if (isModuleCandidate())
- Name += ":";
- else if (!isNamedModule())
- reset();
- }
-
- void handlePeriod() {
- if (isModuleCandidate())
- Name += ".";
- else if (!isNamedModule())
+ void handleModuleName(Token ModuleName) {
+ assert(ModuleName.is(tok::annot_module_name) && "Expect a module name");
+ if (isModuleCandidate()) {
+ Name =
+ ModuleName.getAnnotationValueAs<ModuleNameInfo *>()->getFlatName();
+ } else if (!isNamedModule())
reset();
}
@@ -615,10 +677,6 @@ class Preprocessor {
ModuleDeclSeq ModuleDeclState;
- /// Whether the module import expects an identifier next. Otherwise,
- /// it expects a '.' or ';'.
- bool ModuleImportExpectsIdentifier = false;
-
/// The identifier and source location of the currently-active
/// \#pragma clang arc_cf_code_audited begin.
std::pair<IdentifierInfo *, SourceLocation> PragmaARCCFCodeAuditedInfo;
@@ -1763,11 +1821,14 @@ class Preprocessor {
/// Lex a token, forming a header-name token if possible.
bool LexHeaderName(Token &Result, bool AllowMacroExpansion = true);
+ /// Lex a module name or a partition name.
+ bool LexModuleName(Token &Result, bool IsImport);
+
/// Lex the parameters for an #embed directive, returns nullopt on error.
std::optional<LexEmbedParametersResult> LexEmbedParameters(Token &Current,
bool ForHasEmbed);
-
bool LexAfterModuleImport(Token &Result);
+ bool LexAfterModuleDecl(Token &Result);
void CollectPpImportSuffix(SmallVectorImpl<Token> &Toks);
void makeModuleVisible(Module *M, SourceLocation Loc);
@@ -2329,6 +2390,8 @@ class Preprocessor {
/// token stream.
bool HandleEndOfTokenLexer(Token &Result);
+ bool HandleModuleContextualKeyword(Token &Result);
+
/// Callback invoked when the lexer sees a # token at the start of a
/// line.
///
@@ -2650,10 +2713,16 @@ class Preprocessor {
void removeCachedMacroExpandedTokensOfLastLexer();
+ /// Peek the next token. If so, return the token, if not, this
+ /// method should have no observable side-effect on the lexed tokens.
+ std::optional<Token> peekNextPPToken();
+
/// Determine whether the next preprocessor token to be
/// lexed is a '('. If so, consume the token and return true, if not, this
/// method should have no observable side-effect on the lexed tokens.
- bool isNextPPTokenLParen();
+ bool isNextPPTokenLParen() {
+ return peekNextPPToken().value_or(Token{}).is(tok::l_paren);
+ }
/// After reading "MACRO(", this method is invoked to read all of the formal
/// arguments specified for the macro invocation. Returns null on error.
@@ -3059,6 +3128,9 @@ class Preprocessor {
static bool CLK_LexAfterModuleImport(Preprocessor &P, Token &Result) {
return P.LexAfterModuleImport(Result);
}
+ static bool CLK_LexAfterModuleDecl(Preprocessor &P, Token &Result) {
+ return P.LexAfterModuleDecl(Result);
+ }
};
/// Abstract base class that describes a handler that will receive
@@ -3090,7 +3162,6 @@ struct EmbedAnnotationData {
/// Registry of pragma handlers added by plugins
using PragmaHandlerRegistry = llvm::Registry<PragmaHandler>;
-
} // namespace clang
#endif // LLVM_CLANG_LEX_PREPROCESSOR_H
diff --git a/clang/include/clang/Lex/Token.h b/clang/include/clang/Lex/Token.h
index 4f29fb7d114159..8400ab7ed07e2a 100644
--- a/clang/include/clang/Lex/Token.h
+++ b/clang/include/clang/Lex/Token.h
@@ -235,6 +235,9 @@ class Token {
assert(isAnnotation() && "Used AnnotVal on non-annotation token");
return PtrData;
}
+ template <class T> T getAnnotationValueAs() const {
+ return static_cast<T>(getAnnotationValue());
+ }
void setAnnotationValue(void *val) {
assert(isAnnotation() && "Used AnnotVal on non-annotation token");
PtrData = val;
@@ -289,6 +292,10 @@ class Token {
/// Return the ObjC keyword kind.
tok::ObjCKeywordKind getObjCKeywordID() const;
+ /// Return true if we have an C++20 Modules contextual keyword(export, import
+ /// or module).
+ bool isModuleContextualKeyword() const;
+
bool isSimpleTypeSpecifier(const LangOptions &LangOpts) const;
/// Return true if this token has trigraphs or escaped newlines in it.
diff --git a/clang/include/clang/Lex/TokenLexer.h b/clang/include/clang/Lex/TokenLexer.h
index 4d229ae6106743..777b4e6266c714 100644
--- a/clang/include/clang/Lex/TokenLexer.h
+++ b/clang/include/clang/Lex/TokenLexer.h
@@ -139,10 +139,9 @@ class TokenLexer {
void Init(const Token *TokArray, unsigned NumToks, bool DisableMacroExpansion,
bool OwnsTokens, bool IsReinject);
- /// If the next token lexed will pop this macro off the
- /// expansion stack, return 2. If the next unexpanded token is a '(', return
- /// 1, otherwise return 0.
- unsigned isNextTokenLParen() const;
+ /// If the next token lexed will pop this macro off the expansion stack,
+ /// return std::nullopt, otherwise return the next unexpanded token.
+ std::optional<Token> peekNextPPToken() const;
/// Lex and return a token from this macro stream.
bool Lex(Token &Tok);
diff --git a/clang/include/clang/Parse/Parser.h b/clang/include/clang/Parse/Parser.h
index ba7d6866ebacd8..7c9c569b9d2e46 100644
--- a/clang/include/clang/Parse/Parser.h
+++ b/clang/include/clang/Parse/Parser.h
@@ -165,10 +165,6 @@ class Parser : public CodeCompletionHandler {
mutable IdentifierInfo *Ident_GNU_final;
mutable IdentifierInfo *Ident_override;
- // C++2a contextual keywords.
- mutable IdentifierInfo *Ident_import;
- mutable IdentifierInfo *Ident_module;
-
// C++ type trait keywords that can be reverted to identifiers and still be
// used as type traits.
llvm::SmallDenseMap<IdentifierInfo *, tok::TokenKind> RevertibleTypeTraits;
@@ -3878,7 +3874,7 @@ class Parser : public CodeCompletionHandler {
}
bool ParseModuleName(
- SourceLocation UseLoc,
+ SourceLocation UseLoc, ArrayRef<Token> ModuleName,
SmallVectorImpl<std::pair<IdentifierInfo *, SourceLocation>> &Path,
bool IsImport);
diff --git a/clang/lib/Basic/IdentifierTable.cpp b/clang/lib/Basic/IdentifierTable.cpp
index 4f7ccaf4021d63..97d830214f8900 100644
--- a/clang/lib/Basic/IdentifierTable.cpp
+++ b/clang/lib/Basic/IdentifierTable.cpp
@@ -322,8 +322,9 @@ void IdentifierTable::AddKeywords(const LangOptions &LangOpts) {
if (LangOpts.IEEE128)
AddKeyword("__ieee128", tok::kw___float128, KEYALL, LangOpts, *this);
- // Add the 'import' contextual keyword.
+ // Add the 'import' and 'module' contextual keyword.
get("import").setModulesImport(true);
+ get("module").setModulesDeclaration(true);
}
/// Checks if the specified token kind represents a keyword in the
diff --git a/clang/lib/Frontend/PrintPreprocessedOutput.cpp b/clang/lib/Frontend/PrintPreprocessedOutput.cpp
index 135dca0e6a1775..e21f2b945b86ad 100644
--- a/clang/lib/Frontend/PrintPreprocessedOutput.cpp
+++ b/clang/lib/Frontend/PrintPreprocessedOutput.cpp
@@ -758,9 +758,10 @@ void PrintPPOutputPPCallbacks::HandleWhitespaceBeforeTok(const Token &Tok,
// These tokens are not expanded to anything and don't need whitespace before
// them.
if (Tok.is(tok::eof) ||
- (Tok.isAnnotation() && !Tok.is(tok::annot_header_unit) &&
- !Tok.is(tok::annot_module_begin) && !Tok.is(tok::annot_module_end) &&
- !Tok.is(tok::annot_repl_input_end) && !Tok.is(tok::annot_embed)))
+ (Tok.isAnnotation() && Tok.isNot(tok::annot_header_unit) &&
+ Tok.isNot(tok::annot_module_begin) && Tok.isNot(tok::annot_module_end) &&
+ Tok.isNot(tok::annot_module_name) &&
+ Tok.isNot(tok::annot_repl_input_end) && Tok.isNot(tok::annot_embed)))
return;
// EmittedDirectiveOnThisLine takes priority over RequireSameLine.
@@ -951,6 +952,11 @@ static void PrintPreprocessedTokens(Preprocessor &PP, Token &Tok,
PP.Lex(Tok);
IsStartOfLine = true;
continue;
+ } else if (Tok.is(tok::annot_module_name)) {
+ auto *Info = static_cast<ModuleNameInfo *>(Tok.getAnnotationValue());
+ *Callbacks->OS << Info->getFlatName();
+ PP.Lex(Tok);
+ continue;
} else if (Tok.is(tok::annot_header_unit)) {
// This is a header-name that has been (effectively) converted into a
// module-name.
diff --git a/clang/lib/Lex/Lexer.cpp b/clang/lib/Lex/Lexer.cpp
index ef1e1f4bd9aeb4..ffb82fa46984a6 100644
--- a/clang/lib/Lex/Lexer.cpp
+++ b/clang/lib/Lex/Lexer.cpp
@@ -74,6 +74,17 @@ tok::ObjCKeywordKind Token::getObjCKeywordID() const {
return specId ? specId->getObjCKeywordID() : tok::objc_not_keyword;
}
+/// Return true if we have an C++20 Modules contextual keyword(export, import
+/// or module).
+bool Token::isModuleContextualKeyword() const {
+ if (is(tok::kw_export))
+ return true;
+ if (isNot(tok::identifier))
+ return false;
+ const auto *II = getIdentifierInfo();
+ return II->isModulesImport() || II->isModulesDeclaration();
+}
+
/// Determine whether the token kind star...
[truncated]
|
Signed-off-by: yronglin <[email protected]>
|
This PR is starting to get quite large. Can Modules Dependency Discover be split in a separate PR? Edit: ie, maybe we can implement P1857R3 first ? |
This sounds good to me. Let me give a try. IIUC, we should implement P1857R3(include new tok::annot_module_name) firstly? P3034R1 has very little content. Most of this PR is dealing with tok::annot_module_name. WDYT? Edit: It might be easier to implement P1857R3 and tok::annot_module_name in one PR. |
shafik
reviewed
Sep 5, 2024
| @@ -4558,6 +4574,12 @@ bool Lexer::LexDependencyDirectiveToken(Token &Result) { | |||
| Result.setRawIdentifierData(TokPtr); | |||
| if (!isLexingRawMode()) { | |||
| const IdentifierInfo *II = PP->LookUpIdentifierInfo(Result); | |||
| if (Result.isModuleContextualKeyword()) { | |||
| // Result.setFlagValue(Token::StartOfLine, | |||
| else | ||
| goto import_decl; | ||
| } | ||
| // IdentifierInfo *II = NextToken().getIdentifierInfo(); |
There was a problem hiding this comment.
Yes, it's dead code.
|
@yronglin What is the status of that? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR reapply [Clang] Implement P3034R1 Module Declarations Shouldn’t be Macros, and partially implement P1857R3 Modules Dependency Discovery.
The original PR introduced the
annot_module_nametoken, and in some cases, the PR incorrectly treated the contextual keyword as a keyword, causing syntax parsing errors. Without the partial features of P1857R3, we need to unpack the unexpectedannot_module_nametoken into a token sequence such as "A.B.C:D" in the Parser. Likes playing whack-a-mole, it is difficult to completely solve this issue. The partial features introduced in P1857R3 allow us to determine whether the contextual keyword is amodule/importdirective introducer through a simple check.The 1st commit is the original PR, the others is the partial implementation of P1857R3.