[Parse] 2nd attempt at multiline attribute messages (SE-200)

[johnno1962]

This is a follow up on #19170 and the subsequent revert #19206 by @rintaro of some dubious code by me which attempted to support modified escaping and multiline literals for attribute messages and caused ASAN failures. Reworked to try to at least get multiline strings working which might be useful as messages for attributes (for example a detailed “unavailable” annotation) minus the code which read off the start of the StringRef buffer.

Please ASAN test before merging.

Resolves SR-8724.

[jrose-apple]

I was looking at this too, because this isn't the only place getEncodedStringSegment is being called with something that's not a direct source literal. I decided I'd rather copy the string in those cases than try to figure out all the places where getEncodedStringSegment does a one-byte lookahead (there's a bunch more besides the one you fixed).

[jrose-apple]

I'll have my PR up soon (rebuilding and running tests locally first) and I'll CC you both there.

[johnno1962]

It’s a tricky one. At the moment multiline are allowed but processed incorrectly in attributes. getEncodedStringSegment() should respect its bounds now.

[jrose-apple]

It still doesn't here:

https://github.com/apple/swift/blob/2032969ee2bc0901a929b47f1f3ebe3e50ece89c/lib/Parse/Lexer.cpp#L2119-L2122

or here:

https://github.com/apple/swift/blob/2032969ee2bc0901a929b47f1f3ebe3e50ece89c/lib/Parse/Lexer.cpp#L2131-L2142

or here:

https://github.com/apple/swift/blob/2032969ee2bc0901a929b47f1f3ebe3e50ece89c/lib/Parse/Lexer.cpp#L2164-L2169

or here:

https://github.com/apple/swift/blob/2032969ee2bc0901a929b47f1f3ebe3e50ece89c/lib/Parse/Lexer.cpp#L1196-L1208

or here:

https://github.com/apple/swift/blob/2032969ee2bc0901a929b47f1f3ebe3e50ece89c/lib/Parse/Lexer.cpp#L1166-L1176

and we'd have to do an investigating of the effect on parsing performance if we wanted to fix it.

[johnno1962]

So we can’t assume this?

  // Note that it is always safe to read one over the end of "Bytes" because
  // we know that there is a terminating " character.  Use BytesPtr to avoid a
  // range check subscripting on the StringRef.

[jrose-apple]

On my end I considered removing that assumption vs. just making a copy when the function is used outside of a normal Lexer context, and I decided the latter was safer and didn't require me to go off and test performance.

[johnno1962]

I’ve tidied up the code a little on this PR. Can somebody perform an ASAN check on it for me please?

[jrose-apple]

ASan is currently failing on master anyway, so we'll have to wait (https://ci.swift.org/job/oss-swift-incremental-ASAN-RA-osx/).

[jrose-apple]

@swift-ci Please asan test

[jrose-apple]

@swift-ci Please test

[swift-ci]

Build failed
Swift Test OS X Platform
Git Sha - f430e0189cd1c6379887826310191138a7b129d4

[swift-ci]

Build failed
Swift Test Linux Platform
Git Sha - f430e0189cd1c6379887826310191138a7b129d4

[jrose-apple]

@swift-ci Please asan test

[beccadax]

@swift-ci please asan test

[beccadax]

@swift-ci please asan test

[johnno1962]

Rebased and ready to merge. The Asan checks seem to abort today though there was one successful run yesterday https://ci.swift.org/job/swift-PR-osx-ASAN-test/51/ before the rebase which only contained a minor header fix due to #19230.

[jrose-apple]

@swift-ci Please test

[rintaro]

@swift-ci Please smoke test

[rintaro]

@swift-ci Please ASAN test

[rintaro]

LGTM. Thanks!

[johnno1962]

Thanks everyone. All this trouble to get multiline attribute messages working!