[stable/20221013] Cherry-pick CAS changes #7143
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Allows us to use CLANG_CACHE_TEST_DETERMINISTIC_OUTPUTS, CLANG_CACHE_REDACT_TIME_MACROS, and CLANG_CACHE_CHECK_REPRODUCIBLE_CACHING_ISSUES without going through clang-cache. For deterministic outputs this calls cc1_main twice. rdar://108161760 (cherry picked from commit a71d588)
* Allow ingesting a symlink to a non-existent file. While this may be unexpected, it is not a filesystem error and we should not assume it will not occur in practice in llvm-cas. * Avoid walking into directories repeatedly when there are symlinks, which could cause stack overflow from recursion or a filesystem error about a path with too many symlinks. (cherry picked from commit 23a1a10)
…source processing is done Introduce `FrontendOptions.MayEmitDiagnosticsAfterProcessingSourceFiles` to indicate that `CompilerInstance::ExecuteAction` should not "finish" the diagnostic client. This is set for caching compilations. rdar://108014441 (cherry picked from commit ff889ff)
… import We only want to make PCH imports visible once for the the TU, not repeatedly after every subsequent import. This causes some incorrect behaviour with submodule visibility, and causes us to get extra module dependencies in the scanner. So far I have only seen obviously incorrect behaviour when building with -fmodule-name to cause a submodule to be textually included when using the PCH, though the old behaviour seems wrong regardless. rdar://107449644 Differential Revision: https://reviews.llvm.org/D148176 (cherry picked from commit e06a91c)
Allows configuring to use the plugin for all the tools that use `createCASFromIdentifier`. (cherry picked from commit 1fbcf60)
(cherry picked from commit be12a51)
…for convience This matches the options for `clang` and `clang-cas-test`. (cherry picked from commit d9c488f)
We want to avoid spurious cache misses due to module context hash including the -fcas-path, -fcas-plugin-path, -fcas-plugin-option options. These cannot cause collisions since we only have a single CAS for the dependency scanner, and they cannot collide with non-cas invocations, because the include-tree and cas-fs-working-dir options are still present. rdar://108341513 (cherry picked from commit 64bbbc0)
Currently we are failing to find dependency directives for prefix-mapped paths in the scanner, such as -include options. Match how ivfsoverlay marks remapped paths. We want to use the external (remapped) path when we lookup directives; currently the way to do that is getName() combined with ExposesExternalVFSPath; if in the future we add explicit API for this we can drop the status flag. (cherry picked from commit 492c997)
When we generate <module-includes> for a system module that uses builtin clang headers, the clang headers show up with an absolute path since they are not relative to the module root directory. Prefix-map those paths so that we can get cache hits across different clang resource directory paths. rdar://108381336 (cherry picked from commit 6c3651b)
(cherry picked from commit ab601d8)
(cherry picked from commit b36bc6f)
…header This ensures we get the correct FileCharacteristic during scanning. In a yet-to-be-upstreamed branch this fixes observable failures, but it's also good to handle this on principle: the FileCharacteristic is a property of the file that is observable in the scanner, so there is nothing preventing us from depending on it. rdar://108627403 Differential Revision: https://reviews.llvm.org/D149777 (cherry picked from commit 7b492d1)
This verifies include-tree matches uncached build diagnostics when using `#pragma clang system_header`, which applies to transitive includes. rdar://108627403 (cherry picked from commit 0bfdf3b)
While we cannot handle `_Pragma` used inside macros, we can handle
this at the top level, and it some projects use the `_Pragma("once")`
spelling like that, which was causing spurious failures in the scanner.
Limitations
* Cannot handle #define ONCE _Pragma("once"), same issue as using
@import in a macro -- ideally we should diagnose this in obvious cases
* Our LangOpts are currently fixed, so we are not handling u"" strings
or R"()" strings that require C11/C++11.
rdar://108629982
Differential Revision: https://reviews.llvm.org/D149884
(cherry picked from commit ee8ed0b)
Avoid inferring new submodules for headers in ASTWriter's collection of affecting modulemap files, since we don't want to pick up dependencies that didn't actually exist during parsing. rdar://109112624 Differential Revision: https://reviews.llvm.org/D150151 (cherry picked from commit 5984ea2)
…heResult::getOutputKindName` (cherry picked from commit 995d22a)
…caching When caching is disabled, we don't want to run the compile twice. First, because it is wasteful when we won't detect any issues. Second, because it works around errors with -E writing twice to stdout in scripts. Technically, we could have caching enabled with -E or -o - and see the same issue, but it's non-trivial to fix that without introducing maintenance issues in cc1_main until we upstream this code. rdar://109163869 (cherry picked from commit 99a1a13)
Avoid emitting DIAGNOSTIC_OPTIONS block when caching. In cached builds, there is no benefit to emitting diagnostic options into the PCM/PCH, since we know it can never mismatch. Remove this block, because it can contain volatile options such as fmessage-length that should not have any impact on the output when using cached diagnostics. Also add a test for -ferror-limit: while it is okay to clear this option it's important to test it since it relies on the fact we are not caching compilation failures. Unlike the other "formatting" options, this affects the set of diagnostics passed to the diagnostic consumer. rdar://109285136 (cherry picked from commit e939ada)
This test has been silently broken for a while due to using -serialize-diagnostics-file, which interferes with -verify (the verify consumer runs, but does not cause the compilation to fail). Fix the checks and use -fallow-editor-placeholders instead to enable the correct diagnostic mode. (cherry picked from commit 8ffa4d4)
…uirements When outputting fixits for missing methods, we had two issues * If we found an optional method we could incorrectly hide the fixit for a requiremed method with the same selector from another protocol * The fixit order wasn't deterministic when two protocols declared the same required selector Fix by choosing the best candidate to store when walking methods. rdar://109411920 (cherry picked from commit 635f5a6)
Otherwise there are cache misses just from changing the name of a path, even though the input modules did not change. rdar://109672225 Differential Revision: https://reviews.llvm.org/D151165 (cherry picked from commit bacb45e)
…rnNoWerror/ShowInSystemHeader/ShowInSystemMacro This allows the warning to be visible during a caching invocation by default, without getting affected by -Werror. (cherry picked from commit f31ea45)
…ause loading of regular files `HeaderSearch::loadSubdirectoryModuleMaps` `stat`s all the files in a directory which causes the dependency scanning service to load and cache their contents. This is problematic because a file may be in the process of being generated and could be cached by the dep-scan service while it is still incomplete. To address this change `loadSubdirectoryModuleMaps` to ignore regular files. Differential Revision: https://reviews.llvm.org/D153670 (cherry picked from commit 03a0f4b)
…stic order When writing a pcm, we serialize diagnostic mappings in order to accurately reproduce the diagnostic environment inside any headers from that module. However, the diagnostic state mapping table contains entries for every diagnostic ID ever accessed, while we only want to serialize the ones that are actually modified from their default value. Futher, we need to serialize them in a deterministic order. rdar://111477511 Differential Revision: https://reviews.llvm.org/D154016 (cherry picked from commit 1ede7b4)
We have no use for debug info for the scanner modules, and writing raw ast files speeds up scanning ~15% in some cases. Note that the compile commands produced by the scanner will still build the obj format (if requested), and the scanner can *read* obj format pcms, e.g. from a PCH. rdar://108807592 Differential Revision: https://reviews.llvm.org/D149693 (cherry picked from commit 8fe8d69)
|
@swift-ci please test |
|
LGTM. That |
|
@swift-ci please test Windows |
1 similar comment
|
@swift-ci please test Windows |
|
@swift-ci please test Windows |
|
@swift-ci please test macOS |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.