Skip to content

[clang-format] Do not update cursor pos if no includes replacement #77456

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

NorthBlue333
Copy link
Contributor

@NorthBlue333 NorthBlue333 commented Jan 9, 2024

Fix #77450.

Copy link

github-actions bot commented Jan 9, 2024

Thank you for submitting a Pull Request (PR) to the LLVM Project!

This PR will be automatically labeled and the relevant teams will be
notified.

If you wish to, you can add reviewers by using the "Reviewers" section on this page.

If this is not working for you, it is probably because you do not have write
permissions for the repository. In which case you can instead tag reviewers by
name in a comment by using @ followed by their GitHub username.

If you have received no comments on your PR for a week, you can request a review
by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate
is once a week. Please remember that you are asking for valuable time from other developers.

If you have further questions, they may be answered by the LLVM GitHub User Guide.

You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums.

@llvmbot
Copy link
Member

llvmbot commented Jan 9, 2024

@llvm/pr-subscribers-clang-format

Author: None (NorthBlue333)

Changes

Hello,
This PR should fix #77450.
I have not added tests though.

Also, a similar fix might be required on sortJavaImports.


Full diff: https://github.com/llvm/llvm-project/pull/77456.diff

1 Files Affected:

  • (modified) clang/lib/Format/Format.cpp (+10-4)
diff --git a/clang/lib/Format/Format.cpp b/clang/lib/Format/Format.cpp
index f798d555bf9929..a91f6a639fb00b 100644
--- a/clang/lib/Format/Format.cpp
+++ b/clang/lib/Format/Format.cpp
@@ -3119,6 +3119,7 @@ static void sortCppIncludes(const FormatStyle &Style,
     return;
   }
 
+  unsigned NewCursor = UINT_MAX;
   std::string result;
   for (unsigned Index : Indices) {
     if (!result.empty()) {
@@ -3131,13 +3132,10 @@ static void sortCppIncludes(const FormatStyle &Style,
     }
     result += Includes[Index].Text;
     if (Cursor && CursorIndex == Index)
-      *Cursor = IncludesBeginOffset + result.size() - CursorToEOLOffset;
+      NewCursor = IncludesBeginOffset + result.size() - CursorToEOLOffset;
     CurrentCategory = Includes[Index].Category;
   }
 
-  if (Cursor && *Cursor >= IncludesEndOffset)
-    *Cursor += result.size() - IncludesBlockSize;
-
   // If the #includes are out of order, we generate a single replacement fixing
   // the entire range of blocks. Otherwise, no replacement is generated.
   if (replaceCRLF(result) == replaceCRLF(std::string(Code.substr(
@@ -3145,6 +3143,14 @@ static void sortCppIncludes(const FormatStyle &Style,
     return;
   }
 
+  if (Cursor) {
+    if (UINT_MAX != NewCursor) {
+      *Cursor = NewCursor;
+    } else if (*Cursor >= IncludesEndOffset) {
+      *Cursor += result.size() - IncludesBlockSize;
+    }
+  }
+
   auto Err = Replaces.add(tooling::Replacement(
       FileName, Includes.front().Offset, IncludesBlockSize, result));
   // FIXME: better error handling. For now, just skip the replacement for the

@NorthBlue333 NorthBlue333 force-pushed the clang-format-crlf-regroup-includes-cursor branch 2 times, most recently from e21cf67 to 4890743 Compare January 9, 2024 13:01
@NorthBlue333 NorthBlue333 changed the title fix: no update cursor pos in sort includes if no replacement [clang-format] Do not update cursor pos if no includes replacement Jan 9, 2024
@NorthBlue333 NorthBlue333 force-pushed the clang-format-crlf-regroup-includes-cursor branch 2 times, most recently from 897c90c to 263996d Compare January 9, 2024 13:22
@NorthBlue333 NorthBlue333 force-pushed the clang-format-crlf-regroup-includes-cursor branch 2 times, most recently from 8081b93 to 04258b4 Compare January 10, 2024 09:15
@NorthBlue333
Copy link
Contributor Author

The change is done 👍
Is the code formatting check broken?

@NorthBlue333 NorthBlue333 force-pushed the clang-format-crlf-regroup-includes-cursor branch from 04258b4 to 18163c8 Compare January 10, 2024 09:34
@owenca
Copy link
Contributor

owenca commented Jan 10, 2024

The change is done 👍 Is the code formatting check broken?

See here.

@NorthBlue333
Copy link
Contributor Author

Currently fixing the tests.

@NorthBlue333 NorthBlue333 force-pushed the clang-format-crlf-regroup-includes-cursor branch 2 times, most recently from 490343e to 7ee61d5 Compare January 19, 2024 18:43
Copy link

github-actions bot commented Jan 19, 2024

✅ With the latest revision this PR passed the C/C++ code formatter.

@NorthBlue333 NorthBlue333 force-pushed the clang-format-crlf-regroup-includes-cursor branch from 7ee61d5 to 1c794e5 Compare January 19, 2024 18:47
@NorthBlue333 NorthBlue333 force-pushed the clang-format-crlf-regroup-includes-cursor branch from 1c794e5 to 4dab4e9 Compare January 20, 2024 09:08
@NorthBlue333
Copy link
Contributor Author

NorthBlue333 commented Jan 20, 2024

Oh, sorry @HazardyKnusperkeks I missed this in your previous comment.

I've updated the PR.

As a side note, I tried adding these two tests but they fail, meaning the cursor is still incorrectly computed with CRLF when replacing lines.

This is due to adding lines between the include groups (with Regroup option), as this new line is added with only the character \n (while the line break between include lines is right because the include line itself contains a trailing \r).
I am not sure how to fix this correctly because I do not want to break things by trimming the include line and I do not think sortCppIncludes should manage whitespaces (WhitespaceManager seems to be the correct class to handle this?). Maybe the cursor should be updated when replacing these whitespaces?

TEST_F(
       SortIncludesTest,
       CalculatesCorrectCursorPositionWhenNewLineReplacementsWithRegroupingAndCRLF) {
  Style.IncludeBlocks	  = Style.IBS_Regroup;
  FmtStyle.LineEnding	  = FormatStyle::LE_CRLF;
  Style.IncludeCategories = {
      {"^\"a\"", 0, 0, false}, {"^\"b\"", 1, 1, false}, {".*", 2, 2, false}};
  std::string Code = "#include \"a\"\r\n" // Start of line: 0
                     "#include \"b\"\r\n" // Start of line: 14
                     "#include \"c\"\r\n" // Start of line: 28
                     "\r\n"               // Start of line: 42
                     "int i;";            // Start of line: 44
  std::string Expected = "#include \"a\"\r\n" // Start of line: 0
                         "\r\n"               // Start of line: 14
                         "#include \"b\"\r\n" // Start of line: 16
                         "\r\n"               // Start of line: 30
                         "#include \"c\"\r\n" // Start of line: 32
                         "\r\n"               // Start of line: 46
                         "int i;";            // Start of line: 48
  EXPECT_EQ(Expected, sort(Code));
  EXPECT_EQ(0u, newCursor(Code, 0));
  EXPECT_EQ(15u, newCursor(Code, 14)); // FIXME: should expect 16, caused by \r
  EXPECT_EQ(30u, newCursor(Code, 28)); // FIXME: should expect 32, caused by \r
  EXPECT_EQ(44u, newCursor(Code, 42)); // FIXME: should expect 46, caused by \r
  EXPECT_EQ(46u, newCursor(Code, 44)); // FIXME: should expect 48, caused by \r
}

TEST_F(
       SortIncludesTest,
       CalculatesCorrectCursorPositionWhenNoNewLineReplacementsWithRegroupingAndCRLF) {
  Style.IncludeBlocks = Style.IBS_Regroup;
  FmtStyle.LineEnding = FormatStyle::LE_CRLF;
  Style.IncludeCategories = {
      {"^\"a\"", 0, 0, false}, {"^\"b\"", 1, 1, false}, {".*", 2, 2, false}};
  std::string Code = "#include \"a\"\r\n" // Start of line: 0
                     "\r\n"               // Start of line: 14
                     "#include \"c\"\r\n" // Start of line: 16
                     "\r\n"               // Start of line: 30
                     "#include \"b\"\r\n" // Start of line: 32
                     "\r\n"               // Start of line: 46
                     "int i;";            // Start of line: 48
  std::string Expected = "#include \"a\"\r\n" // Start of line: 0
                         "\r\n"               // Start of line: 14
                         "#include \"b\"\r\n" // Start of line: 16
                         "\r\n"               // Start of line: 30
                         "#include \"c\"\r\n" // Start of line: 32
                         "\r\n"               // Start of line: 46
                         "int i;";            // Start of line: 48
  EXPECT_EQ(Expected, sort(Code));
  EXPECT_EQ(0u, newCursor(Code, 0));
  EXPECT_EQ(14u, newCursor(Code, 14));
  EXPECT_EQ(30u, newCursor(Code, 16)); // FIXME: should expect 32, caused by \r
  EXPECT_EQ(30u, newCursor(Code, 30));
  EXPECT_EQ(15u, newCursor(Code, 32)); // FIXME: should expect 15, caused by \r
  EXPECT_EQ(44u, newCursor(Code, 46)); // FIXME: should expect 46, caused by \r
  EXPECT_EQ(46u, newCursor(Code, 48)); // FIXME: should expect 48, caused by \r
}

@NorthBlue333 NorthBlue333 force-pushed the clang-format-crlf-regroup-includes-cursor branch 3 times, most recently from c3fa004 to ac5dccf Compare January 20, 2024 09:22
@HazardyKnusperkeks
Copy link
Contributor

Oh, sorry @HazardyKnusperkeks I missed this in your previous comment.

I've updated the PR.

As a side note, I tried adding these two tests but they fail, meaning the cursor is still incorrectly computed with CRLF when replacing lines.

This is due to adding lines between the include groups (with Regroup option), as this new line is added with only the character \n (while the line break between include lines is right because the include line itself contains a trailing \r). I am not sure how to fix this correctly because I do not want to break things by trimming the include line and I do not think sortCppIncludes should manage whitespaces (WhitespaceManager seems to be the correct class to handle this?). Maybe the cursor should be updated when replacing these whitespaces?

TEST_F(
       SortIncludesTest,
       CalculatesCorrectCursorPositionWhenNewLineReplacementsWithRegroupingAndCRLF) {
  Style.IncludeBlocks	  = Style.IBS_Regroup;
  FmtStyle.LineEnding	  = FormatStyle::LE_CRLF;
  Style.IncludeCategories = {
      {"^\"a\"", 0, 0, false}, {"^\"b\"", 1, 1, false}, {".*", 2, 2, false}};
  std::string Code = "#include \"a\"\r\n" // Start of line: 0
                     "#include \"b\"\r\n" // Start of line: 14
                     "#include \"c\"\r\n" // Start of line: 28
                     "\r\n"               // Start of line: 42
                     "int i;";            // Start of line: 44
  std::string Expected = "#include \"a\"\r\n" // Start of line: 0
                         "\r\n"               // Start of line: 14
                         "#include \"b\"\r\n" // Start of line: 16
                         "\r\n"               // Start of line: 30
                         "#include \"c\"\r\n" // Start of line: 32
                         "\r\n"               // Start of line: 46
                         "int i;";            // Start of line: 48
  EXPECT_EQ(Expected, sort(Code));
  EXPECT_EQ(0u, newCursor(Code, 0));
  EXPECT_EQ(15u, newCursor(Code, 14)); // FIXME: should expect 16, caused by \r
  EXPECT_EQ(30u, newCursor(Code, 28)); // FIXME: should expect 32, caused by \r
  EXPECT_EQ(44u, newCursor(Code, 42)); // FIXME: should expect 46, caused by \r
  EXPECT_EQ(46u, newCursor(Code, 44)); // FIXME: should expect 48, caused by \r
}

TEST_F(
       SortIncludesTest,
       CalculatesCorrectCursorPositionWhenNoNewLineReplacementsWithRegroupingAndCRLF) {
  Style.IncludeBlocks = Style.IBS_Regroup;
  FmtStyle.LineEnding = FormatStyle::LE_CRLF;
  Style.IncludeCategories = {
      {"^\"a\"", 0, 0, false}, {"^\"b\"", 1, 1, false}, {".*", 2, 2, false}};
  std::string Code = "#include \"a\"\r\n" // Start of line: 0
                     "\r\n"               // Start of line: 14
                     "#include \"c\"\r\n" // Start of line: 16
                     "\r\n"               // Start of line: 30
                     "#include \"b\"\r\n" // Start of line: 32
                     "\r\n"               // Start of line: 46
                     "int i;";            // Start of line: 48
  std::string Expected = "#include \"a\"\r\n" // Start of line: 0
                         "\r\n"               // Start of line: 14
                         "#include \"b\"\r\n" // Start of line: 16
                         "\r\n"               // Start of line: 30
                         "#include \"c\"\r\n" // Start of line: 32
                         "\r\n"               // Start of line: 46
                         "int i;";            // Start of line: 48
  EXPECT_EQ(Expected, sort(Code));
  EXPECT_EQ(0u, newCursor(Code, 0));
  EXPECT_EQ(14u, newCursor(Code, 14));
  EXPECT_EQ(30u, newCursor(Code, 16)); // FIXME: should expect 32, caused by \r
  EXPECT_EQ(30u, newCursor(Code, 30));
  EXPECT_EQ(15u, newCursor(Code, 32)); // FIXME: should expect 15, caused by \r
  EXPECT_EQ(44u, newCursor(Code, 46)); // FIXME: should expect 46, caused by \r
  EXPECT_EQ(46u, newCursor(Code, 48)); // FIXME: should expect 48, caused by \r
}

That's outside my comfortable zone and I'll refer to @owenca or @mydeveloperday.

@NorthBlue333
Copy link
Contributor Author

NorthBlue333 commented Jan 22, 2024

clang-check builds in local.

Updating the PR with the latest commits of main branch.

@NorthBlue333 NorthBlue333 force-pushed the clang-format-crlf-regroup-includes-cursor branch from ac5dccf to a3ddae1 Compare January 22, 2024 08:56
@NorthBlue333
Copy link
Contributor Author

Hi there,
Any update on this?
Friendly ping @owenca or @mydeveloperday.
I think we could maybe divide the issue in two @HazardyKnusperkeks?

Copy link
Contributor

@owenca owenca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

git diff Format.cpp output:

--- a/clang/lib/Format/Format.cpp
+++ b/clang/lib/Format/Format.cpp
@@ -3116,6 +3116,7 @@ static void sortCppIncludes(const FormatStyle &Style,
     return;
   }
 
+  const auto OldCursor = Cursor ? *Cursor : 0;
   std::string result;
   for (unsigned Index : Indices) {
     if (!result.empty()) {
@@ -3139,6 +3140,8 @@ static void sortCppIncludes(const FormatStyle &Style,
   // the entire range of blocks. Otherwise, no replacement is generated.
   if (replaceCRLF(result) == replaceCRLF(std::string(Code.substr(
                                  IncludesBeginOffset, IncludesBlockSize)))) {
+    if (Cursor)
+      *Cursor = OldCursor;
     return;
   }
 

@owenca
Copy link
Contributor

owenca commented Feb 22, 2024

As a side note, I tried adding these two tests but they fail, meaning the cursor is still incorrectly computed with CRLF when replacing lines.

This is due to adding lines between the include groups (with Regroup option), as this new line is added with only the character \n (while the line break between include lines is right because the include line itself contains a trailing \r). I am not sure how to fix this correctly because I do not want to break things by trimming the include line and I do not think sortCppIncludes should manage whitespaces (WhitespaceManager seems to be the correct class to handle this?). Maybe the cursor should be updated when replacing these whitespaces?

TEST_F(
       SortIncludesTest,
       CalculatesCorrectCursorPositionWhenNewLineReplacementsWithRegroupingAndCRLF) {
  Style.IncludeBlocks	  = Style.IBS_Regroup;
  FmtStyle.LineEnding	  = FormatStyle::LE_CRLF;
  Style.IncludeCategories = {
      {"^\"a\"", 0, 0, false}, {"^\"b\"", 1, 1, false}, {".*", 2, 2, false}};
  std::string Code = "#include \"a\"\r\n" // Start of line: 0
                     "#include \"b\"\r\n" // Start of line: 14
                     "#include \"c\"\r\n" // Start of line: 28
                     "\r\n"               // Start of line: 42
                     "int i;";            // Start of line: 44
  std::string Expected = "#include \"a\"\r\n" // Start of line: 0
                         "\r\n"               // Start of line: 14
                         "#include \"b\"\r\n" // Start of line: 16
                         "\r\n"               // Start of line: 30
                         "#include \"c\"\r\n" // Start of line: 32
                         "\r\n"               // Start of line: 46
                         "int i;";            // Start of line: 48
  EXPECT_EQ(Expected, sort(Code));
  EXPECT_EQ(0u, newCursor(Code, 0));
  EXPECT_EQ(15u, newCursor(Code, 14)); // FIXME: should expect 16, caused by \r
  EXPECT_EQ(30u, newCursor(Code, 28)); // FIXME: should expect 32, caused by \r
  EXPECT_EQ(44u, newCursor(Code, 42)); // FIXME: should expect 46, caused by \r
  EXPECT_EQ(46u, newCursor(Code, 44)); // FIXME: should expect 48, caused by \r
}

TEST_F(
       SortIncludesTest,
       CalculatesCorrectCursorPositionWhenNoNewLineReplacementsWithRegroupingAndCRLF) {
  Style.IncludeBlocks = Style.IBS_Regroup;
  FmtStyle.LineEnding = FormatStyle::LE_CRLF;
  Style.IncludeCategories = {
      {"^\"a\"", 0, 0, false}, {"^\"b\"", 1, 1, false}, {".*", 2, 2, false}};
  std::string Code = "#include \"a\"\r\n" // Start of line: 0
                     "\r\n"               // Start of line: 14
                     "#include \"c\"\r\n" // Start of line: 16
                     "\r\n"               // Start of line: 30
                     "#include \"b\"\r\n" // Start of line: 32
                     "\r\n"               // Start of line: 46
                     "int i;";            // Start of line: 48
  std::string Expected = "#include \"a\"\r\n" // Start of line: 0
                         "\r\n"               // Start of line: 14
                         "#include \"b\"\r\n" // Start of line: 16
                         "\r\n"               // Start of line: 30
                         "#include \"c\"\r\n" // Start of line: 32
                         "\r\n"               // Start of line: 46
                         "int i;";            // Start of line: 48
  EXPECT_EQ(Expected, sort(Code));
  EXPECT_EQ(0u, newCursor(Code, 0));
  EXPECT_EQ(14u, newCursor(Code, 14));
  EXPECT_EQ(30u, newCursor(Code, 16)); // FIXME: should expect 32, caused by \r
  EXPECT_EQ(30u, newCursor(Code, 30));
  EXPECT_EQ(15u, newCursor(Code, 32)); // FIXME: should expect 15, caused by \r
  EXPECT_EQ(44u, newCursor(Code, 46)); // FIXME: should expect 46, caused by \r
  EXPECT_EQ(46u, newCursor(Code, 48)); // FIXME: should expect 48, caused by \r
}

With my suggested fix, these tests should also pass.

@NorthBlue333 NorthBlue333 force-pushed the clang-format-crlf-regroup-includes-cursor branch 2 times, most recently from e3e725b to 93d5711 Compare March 26, 2024 11:08
@NorthBlue333
Copy link
Contributor Author

NorthBlue333 commented Mar 26, 2024

Woops, sorry, I missed the update here.

I've done all your changes in a second commit just to be able to keep track of my previous changes; but I am actually not sure how your suggested modifications are any different from mine? Well, I admit it looks a bit better, but it works the same? Or am I missing something? You are just storing the old value of Cursor and resetting it when no changes, while I was doing it the other way around: storing the new value and updating it only if no changes.

It does not fix the tests I mentioned.

@NorthBlue333
Copy link
Contributor Author

Friendly ping again here, as the status on this has been asked again by my team! :)
@owenca, @mydeveloperday, and maybe @HazardyKnusperkeks (about splitting the issue in 2).

@owenca
Copy link
Contributor

owenca commented Apr 19, 2024

I've done all your changes in a second commit just to be able to keep track of my previous changes; but I am actually not sure how your suggested modifications are any different from mine? Well, I admit it looks a bit better, but it works the same? Or am I missing something? You are just storing the old value of Cursor and resetting it when no changes, while I was doing it the other way around: storing the new value and updating it only if no changes.

It's been a while, but IIRC your original fix was not equivalent to my suggestion, which IMO is simpler and cleaner.

It does not fix the tests I mentioned.

My bad. I copy-pasted the two tests you said would fail and ran FormatTests with my suggested fix, and they passed. Of course, I missed the comments in your snippet, e.g.:

  EXPECT_EQ(15u, newCursor(Code, 14)); // FIXME: should expect 16, caused by \r

@NorthBlue333 NorthBlue333 force-pushed the clang-format-crlf-regroup-includes-cursor branch from aecbb12 to 6c184f9 Compare April 19, 2024 10:14
@NorthBlue333
Copy link
Contributor Author

NorthBlue333 commented Apr 19, 2024

Sure, that was was not very clear from my side I agree.

Yes, your implementation is cleaner (I don't know why I did not do that from the beginning, I think I was struggling to fix the other tests and ended up with something more complicated than necessary).

I have squashed the commits in only one.
Note that I have left the failing tests in the commit, I am not sure if I should remove them or not.

Thanks for your time!

@owenca
Copy link
Contributor

owenca commented Apr 25, 2024

I have squashed the commits in only one. Note that I have left the failing tests in the commit, I am not sure if I should remove them or not.

Unfortunately, this wiped out my updates that fixed a formatting error and added #if 0 for the failing tests.

@owenca owenca merged commit 2de0bed into llvm:main Apr 26, 2024
@NorthBlue333 NorthBlue333 deleted the clang-format-crlf-regroup-includes-cursor branch June 17, 2024 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[clang-format] Incorrect cursor position with IncludeBlocksStyle Regroup and LineEnding CRLF
4 participants