Skip to content

[clang dependency scanning] C APIs for Current Working Directory Optimization #10146

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Mar 11, 2025

Conversation

qiongsiwu
Copy link

@qiongsiwu qiongsiwu commented Mar 1, 2025

This PR implements two new C APIs so the build system can communicate with the dependency scanner about current working directory optimization.

Two new functions are added:

  1. void clang_experimental_DependencyScannerServiceOptions_setCWDOptimization(CXDependencyScannerServiceOptions Opts, int): the caller can use this function to set Opts to indicate that it can support current working directory optimization.
  2. int clang_experimental_DepGraphModule_isCWDIgnored(CXDepGraphModule): this function can retrieve the module info from the scanner, indicating if the current working directory is ignored for this module's context hash.

As an example usage, the user of the APIs (e.g. a build system) can use them to set reasonable debug working directories for each pcms.

rdar://145860213

@qiongsiwu
Copy link
Author

@swift-ci please test

@qiongsiwu
Copy link
Author

@swift-ci please test llvm

@qiongsiwu
Copy link
Author

@swift-ci please test

@qiongsiwu
Copy link
Author

@swift-ci please test llvm

2 similar comments
@qiongsiwu
Copy link
Author

@swift-ci please test llvm

@qiongsiwu
Copy link
Author

@swift-ci please test llvm

Copy link

@Bigcheese Bigcheese left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good with the name changed.

@qiongsiwu
Copy link
Author

@swift-ci please test llvm

@benlangmuir
Copy link

I'm concerned about making the change to default optimizations in this downstream branch. Can you say more about why we think this optimization is okay by default for upstream but not here? I can see how informing the build system that the CWD can be canonicalized or ignored would be helpful to the build system, but I'm not clear why it's necessary to have this handshake about whether the optimization is enabled or not.

@Bigcheese
Copy link

I think upstream it should be disabled by default too. Most build systems care about the working directory, and so the build system needs to know to look at that field. The non-libclang parts of this should also go upstream.

It's necessary because if the dep scanner collides names that the build system thinks are different due to WD, it can create problems. Although exactly what happens depends on the build system and how it's used. Best case it's just non-deterministic which WD gets used, but you can also get multiple different jobs targeting the same file path, or the build system may just error out.

@qiongsiwu
Copy link
Author

Thanks for the comment @benlangmuir and @Bigcheese.

llvm#129809 is posted to change the default behavior upstream. I will modify this PR to exclude the default behavior change.

@qiongsiwu
Copy link
Author

llvm#129809 has landed. I will land this PR once llvm#129809 is in next.

* Otherwise returns 0.
*/
CINDEX_LINKAGE
int clang_experimental_DepGraphModule_isCWDIgnored(CXDepGraphModule);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there precedent for "CWD" in the C API layer? Maybe we should spell it out as "WorkingDirectory".

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried doing this but clang_experimental_DependencyScannerServiceOptions_setCurrentWorkingDirOptimization got too long (83 chars), and I don't want to rename one API but not the other. In this case I think I slightly prefer keeping the names as they are. But I am all ears for better/consistent names.

@qiongsiwu qiongsiwu self-assigned this Mar 6, 2025
@qiongsiwu
Copy link
Author

@swift-ci please test llvm

@qiongsiwu qiongsiwu merged commit b9c8668 into swiftlang:next Mar 11, 2025
2 checks passed
@qiongsiwu qiongsiwu deleted the eng_145860213 branch March 11, 2025 17:01
qiongsiwu added a commit that referenced this pull request Mar 18, 2025
…mization (#10146)

This PR implements two new C APIs so the build system can communicate with the dependency scanner about current working directory optimization.

Two new functions are added:

1. `void clang_experimental_DependencyScannerServiceOptions_setCWDOptimization(CXDependencyScannerServiceOptions Opts, int)`: the caller can use this function to set `Opts` to indicate that it can support current working directory optimization.
2. `int clang_experimental_DepGraphModule_isCWDIgnored(CXDepGraphModule)`: this function can retrieve the module info from the scanner, indicating if the current working directory is ignored for this module's context hash.

As an example usage, the user of the APIs (e.g. a build system) can use them to set reasonable debug working directories for each `pcm`s.

rdar://145860213
(cherry picked from commit b9c8668)

 Conflicts:
	clang/test/ClangScanDeps/cas-fs-multiple-commands.c
	clang/test/ClangScanDeps/include-tree-multiple-commands.c
	clang/tools/libclang/libclang.map
owenv added a commit to swiftlang/swift-build that referenced this pull request Mar 28, 2025
Adopt new libclang API which the scanner uses to notify the build system when it is ok to compile a module
using an arbitrary current working directory to facilitate increased module sharing. The relevant API
was introduced in swiftlang/llvm-project#10146.

rdar://145991369
owenv added a commit to swiftlang/swift-build that referenced this pull request Mar 28, 2025
Adopt new libclang API which the scanner uses to notify the build system when it is ok to compile a module
using an arbitrary current working directory to facilitate increased module sharing. The relevant API
was introduced in swiftlang/llvm-project#10146.

rdar://145991369
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants