Skip to content

Fix response file support for batch jobs. #24008

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 16, 2019

Conversation

allevato
Copy link
Member

This fixes an issue where batch jobs don't recompute whether they need response files due to command line length limits, because BatchJobs are added directly to the task queue and don't go through the same code path in constructJob that checks the length (the comment in constructBatchJob was incorrect).

I've verified that the test added in this PR fails when this patch is not present (the batch frontend job doesn't use a response file) and succeeds when it is.

@jrose-apple jrose-apple requested a review from davidungar April 13, 2019 04:28
Copy link
Contributor

@davidungar davidungar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a worthy goal. Let me know about my suggestions.

@@ -62,6 +62,27 @@ ToolChain::JobContext::getTemporaryFilePath(const llvm::Twine &name,
return C.getArgs().MakeArgString(buffer.str());
}

ToolChain::ResponseFileInfo ToolChain::getResponseFileInfoIfNeeded(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not return an Optional? Wherever ResponseFileInfo is used and the pointers might be null, you could use the optional type. That would ensure that the next maintainer doesn't forget the null checks.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. I originally did that, but it didn't seem worth it because I didn't also change the Job constructor signature to use it, so the call sites actually became more complicated. After passing the optional throughout Job/BatchJob as well, it came out much cleaner.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool!

struct ResponseFileInfo {
/// The path to the response file that a job should use, or null if no
/// response file is needed.
const char *responseFilePath;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about this, but would std:string work better for these?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Addressed in the other comment about std::string.)

@allevato allevato force-pushed the batch-mode-response-files branch from ac19ccc to 6d72230 Compare April 13, 2019 17:07
@allevato
Copy link
Member Author

Is there anything else to be done here? Could someone kick off CI please?

@allevato
Copy link
Member Author

allevato commented May 9, 2019

@davidungar Friendly ping

@jrose-apple
Copy link
Contributor

Getting the CI, and I'll check in with David today.

@swift-ci Please test

@davidungar
Copy link
Contributor

@allevato Looks fine! Sorry I missed your ping earlier. @jrose-apple Thanks for the ping!

@jrose-apple jrose-apple merged commit 5191b03 into swiftlang:master May 16, 2019
@jrose-apple
Copy link
Contributor

Please cherry-pick this to the 5.1 branch as well. (I'll help merge it through.)

@allevato allevato deleted the batch-mode-response-files branch May 16, 2019 16:16
allevato pushed a commit to allevato/swift that referenced this pull request May 16, 2019
…files

Fix response file support for batch jobs.
jrose-apple pushed a commit that referenced this pull request May 16, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants