[CursorInfo] Add Clang documentation to SymbolGraph output #42268
Conversation
This currently doesn't check for inherited docs, ie. either the imported declaration has docs or it doesn't. There's also a few odd cases with mixed doc types and when each line is prefixed with '*', but it's good enough for an initial implementation. Moves UTF8 sanitisation out of ASTPrinter.h and into Unicode.h so that it can be used here as well. Resolves rdar://91388603.
|
@swift-ci please test |
| // CHECK-ART-NEXT: "text": " last" | ||
| // CHECK-ART-NEXT: }, | ||
| // CHECK-ART-NEXT: { | ||
| // CHECK-ART-NEXT: "text": " " |
There was a problem hiding this comment.
The awkward newline and whitespace handling surrounding block doc comments makes me a little concerned. While this can be easily handled in a Markdown parser after the fact, the raw text coming out of Clang here is a bit confusing, especially this last line here. It's probably not something we should fix in this PR, but part of me would like to look into it at some point.
There was a problem hiding this comment.
Oh definitely, I mentioned that in the commit message but maybe that wasn't clear enough:
There's also a few odd
cases with mixed doc types and when each line is prefixed with '*', but
it's good enough for an initial implementation.
Mostly I was hoping that the new getFormattedLines handles these cases properly already, so I didn't look into it too much. We should use that when we can + fix it if it still has these weird cases. @zixu-w any idea if these are fixed?
There was a problem hiding this comment.
Thanks for the clarification! I'm not aware of getFormattedLines, but if it's something we can use to fix this in the future, then i'm all for it.
There was a problem hiding this comment.
See https://reviews.llvm.org/D119479. Though from a very quick glance through it looks like it mostly just adds the locations, so probably doesn't fix. Anyway, something we should fix on the LLVM side regardless.
There was a problem hiding this comment.
Actually I did try something in the new getFormattedLines to rectify this 😅
See https://github.com/llvm/llvm-project/blob/e9c8d0ff71baba2702765d6cd173421d54883fee/clang/lib/AST/RawCommentList.cpp#L400-L408 and additional logics with PreviousLine in that method.
I just gave it a try and apparently it didn't fully fix the excessive leading/trailing whitespace issue... The only diff of output using getFormattedLines is that there's only one space character in the last line, instead of two here.
It would definitely require more work to fix this as the lexer handles these block comments a bit weird IMO. On top of my head I think just a simple post-process to remove any leading/trailing whitespace lines would work great and easy.
There was a problem hiding this comment.
Oh cool, thanks @zixu-w! Seems like that would handle most of these. There's still the ASCII art blocks, where they contain the initial space. Naively we could include the star+whitespace in the calculation for the indent level, but then that gets tricky with things like:
/** Not sure if we care about this case, but here's some dot points:
* this
* is actually
* a list of dot point
*/
Pretty odd though, so 🤷
|
@swift-ci please test macOS platform |
This currently doesn't check for inherited docs, ie. either the
imported declaration has docs or it doesn't. There's also a few odd
cases with mixed doc types and when each line is prefixed with '*', but
it's good enough for an initial implementation.
Moves UTF8 sanitisation out of ASTPrinter.h and into Unicode.h so that
it can be used here as well.
Resolves rdar://91388603.