[Batch Mode] Pass PrimarySpecificPaths through compiler. (4) #14230

davidungar · 2018-01-29T04:18:04Z

Batch mode requires that supplementary output paths be specific to the primary input being compiled.
This PR adds arguments and replaces existing output file arguments with an instance of PrimarySpecificPaths in order to make all outputs, not just the main output, dependent on the primary.
It goes along with an lldb PR. Must be merged with apple/swift-lldb#290

Two pieces of state have been removed from IRGenOptions, since they might vary from primary to primary. Because parallel IR generation still needs all of the main output filenames, a parameter has been added, parallelOutputFilenames. I don't have a better solution, but it is a bit hokey since it has to be passed down several layers and only used for the parallel case.

davidungar · 2018-01-29T04:21:36Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please smoke test os x platform

davidungar · 2018-01-29T04:34:56Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please smoke test os x platform

davidungar · 2018-01-29T04:58:40Z

[Batch Mode] WIP: Pass PrimarySpecificPaths to createEmptyModule and IRGenModule() #290

davidungar · 2018-01-29T05:00:16Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please smoke test linux platform

davidungar · 2018-01-29T05:05:14Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please smoke test os x platform

davidungar · 2018-01-29T05:14:35Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please smoke test os x platform

davidungar · 2018-01-29T05:25:29Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please smoke test os x platform

davidungar · 2018-01-29T05:43:56Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please smoke test os x platform

davidungar · 2018-01-29T05:48:23Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please smoke test os x platform

davidungar · 2018-01-29T20:56:58Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please smoke test os x platform

davidungar · 2018-02-08T03:32:24Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please smoke test os x platform

davidungar · 2018-02-08T04:16:45Z

Please test with the following PR:
apple/swift-lldb#290

davidungar · 2018-02-08T04:56:21Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please smoke test os x platform

davidungar · 2018-02-08T05:06:15Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please smoke test os x platform

davidungar · 2018-02-08T06:01:53Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please smoke test os x platform

davidungar · 2018-02-08T06:08:19Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please smoke test os x platform

davidungar · 2018-02-08T06:12:38Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please smoke test os x platform

davidungar · 2018-02-15T22:30:52Z

After more thought, I decided it was appropriate to use the supplemental outputs for only the first parallel-genererated input.

Get rid of IRGenOpts attributes that won’t work for batch mode and also remove fakeNamesStub.

davidungar · 2018-02-15T22:40:34Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please smoke test

davidungar · 2018-02-15T22:41:54Z

@jrose-apple At this point, I think all issues have been addressed. Will see if it passes the tests.

jrose-apple · 2018-02-15T22:46:19Z

After more thought, I decided it was appropriate to use the supplemental outputs for only the first parallel-genererated input.

I'm not sure I like this. Which input is first is essentially arbitrary, and whatever output we would get from such a thing wouldn't have all the data. I'd really rather not pretend we can handle this when we can't.

@eeckstein, thoughts? We're talking about if IRGen were to have outputs in addition to its main output, like if it could output a .ll file but still go on to output a .o. In the case of parallel WMO, we've set everything else up so that there's only one of each supplemental output for the whole process, but that breaks down once we hit IRGen.

davidungar · 2018-02-15T22:59:55Z

Fair point. I'll look at NOT passing a PrimarySpecificPaths into the IRGenModule constructor. If it doesn't emit any supplementaries, then it should only receive an output filename, and a main input filename (for debug info).

davidungar · 2018-02-15T23:41:08Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please smoke test

davidungar · 2018-02-15T23:50:20Z

@jrose-apple OK, this version should be cleaner. (Thanks for your patience.) Since IRGenModule does nothing with the supplementaries, I think it may make sense to not pass in supplementary output names at all to its constructor. I've also tried to reformat in the latest version to minimize the diffs.
If it looks good and you would like me to streamline the commits, I can squash some. Thanks.

davidungar · 2018-02-15T23:50:29Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please smoke test

davidungar · 2018-02-15T23:55:44Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please smoke test os x platform

davidungar · 2018-02-15T23:59:48Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please smoke test os x platform

davidungar · 2018-02-16T00:16:27Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please smoke test os x platform

davidungar · 2018-02-16T00:21:28Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please clean smoke test os x platform

davidungar · 2018-02-16T01:19:12Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please clean smoke test os x platform

davidungar · 2018-02-16T01:49:05Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please clean smoke test os x platform

jrose-apple

I think this is it!

jrose-apple · 2018-02-16T02:17:39Z

tools/sil-opt/SILOpt.cpp

@@ -426,8 +426,10 @@ int main(int argc, char **argv) {
    runSILLoweringPasses(*CI.getSILModule());
  } else {
    auto *SILMod = CI.getSILModule();
-    {
-      auto T = irgen::createIRGenModule(SILMod, getGlobalLLVMContext());
+    { const auto PSPs = CI.getPrimarySpecificPathsForAtMostOnePrimary();


Missing newline here!

Fixed! Great to have worked together and figured out how to do this right. Thanks!

… IRGenModule constructor.

davidungar · 2018-02-16T03:56:19Z

Please test with the following PR:
apple/swift-lldb#290

@swift-ci Please clean smoke test

davidungar mentioned this pull request Jan 29, 2018

[Batch Mode] Pass PrimarySpecificPaths to createEmptyModule and IRGenModule() apple/swift-lldb#290

Merged

davidungar changed the title ~~[Batch Mode] Pass PrimarySpecificPaths through compiler.~~ [Batch Mode] Pass PrimarySpecificPaths through compiler. (4) Jan 29, 2018

davidungar force-pushed the PR-18-5-lldb-interface branch from 1e4cdbb to b3e8ed2 Compare January 29, 2018 05:43

davidungar force-pushed the PR-18-5-lldb-interface branch from b3e8ed2 to 07914b2 Compare January 29, 2018 05:48

davidungar force-pushed the PR-18-5-lldb-interface branch from 07914b2 to 1850e88 Compare January 29, 2018 20:56

davidungar force-pushed the PR-18-5-lldb-interface branch from 1850e88 to 11e6dd2 Compare January 30, 2018 00:34

davidungar force-pushed the PR-18-5-lldb-interface branch 2 times, most recently from e8dd1eb to 8c1e064 Compare February 8, 2018 03:31

davidungar force-pushed the PR-18-5-lldb-interface branch from 8c1e064 to c633e32 Compare February 8, 2018 04:15

davidungar force-pushed the PR-18-5-lldb-interface branch from c633e32 to a49aa03 Compare February 8, 2018 05:06

davidungar force-pushed the PR-18-5-lldb-interface branch from a49aa03 to 8934888 Compare February 8, 2018 06:00

davidungar force-pushed the PR-18-5-lldb-interface branch from 8934888 to aeef46d Compare February 8, 2018 06:07

davidungar force-pushed the PR-18-5-lldb-interface branch from aeef46d to 7ced740 Compare February 8, 2018 06:12

davidungar added 6 commits February 15, 2018 15:32

Move SupplementaryOutputPaths to Basic.

1e32ff3

Pass around arguments for primary-specific filenames.

1f9a4f3

Get rid of IRGenOpts attributes that won’t work for batch mode and also remove fakeNamesStub.

reformatted

fec411e

Added explanation for emitIndexDataIfNeeded call.

a2863a6

Avoid copying strings and PrimarySpecificPaths.

e7a120f

Only use supplementary outputs for first parallel-generated input.

dec7890

davidungar force-pushed the PR-18-5-lldb-interface branch from 1d30537 to dec7890 Compare February 15, 2018 22:39

davidungar force-pushed the PR-18-5-lldb-interface branch from 4b247d7 to abe1f18 Compare February 15, 2018 23:49

davidungar force-pushed the PR-18-5-lldb-interface branch from abe1f18 to 9fd7f0a Compare February 15, 2018 23:55

davidungar force-pushed the PR-18-5-lldb-interface branch from 9fd7f0a to 8262e12 Compare February 15, 2018 23:59

jrose-apple approved these changes Feb 16, 2018

View reviewed changes

Only pass output filename and main input file name for debugging into…

026b850

… IRGenModule constructor.

davidungar force-pushed the PR-18-5-lldb-interface branch from 8262e12 to 026b850 Compare February 16, 2018 03:52

davidungar merged commit e7ba87f into swiftlang:master Feb 16, 2018

davidungar deleted the PR-18-5-lldb-interface branch May 16, 2018 16:59

[Batch Mode] Pass PrimarySpecificPaths through compiler. (4) #14230

[Batch Mode] Pass PrimarySpecificPaths through compiler. (4) #14230

Uh oh!

Conversation

davidungar commented Jan 29, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

davidungar commented Jan 29, 2018

Uh oh!

davidungar commented Jan 29, 2018

Uh oh!

davidungar commented Jan 29, 2018

Uh oh!

davidungar commented Jan 29, 2018

Uh oh!

davidungar commented Jan 29, 2018

Uh oh!

davidungar commented Jan 29, 2018

Uh oh!

davidungar commented Jan 29, 2018

Uh oh!

davidungar commented Jan 29, 2018

Uh oh!

davidungar commented Jan 29, 2018

Uh oh!

davidungar commented Jan 29, 2018

Uh oh!

davidungar commented Feb 8, 2018

Uh oh!

davidungar commented Feb 8, 2018

Uh oh!

davidungar commented Feb 8, 2018

Uh oh!

davidungar commented Feb 8, 2018

Uh oh!

davidungar commented Feb 8, 2018

Uh oh!

davidungar commented Feb 8, 2018

Uh oh!

davidungar commented Feb 8, 2018

Uh oh!

davidungar commented Feb 15, 2018

Uh oh!

davidungar commented Feb 15, 2018

Uh oh!

davidungar commented Feb 15, 2018

Uh oh!

jrose-apple commented Feb 15, 2018

Uh oh!

davidungar commented Feb 15, 2018

Uh oh!

davidungar commented Feb 15, 2018

Uh oh!

davidungar commented Feb 15, 2018

Uh oh!

davidungar commented Feb 15, 2018

Uh oh!

davidungar commented Feb 15, 2018

Uh oh!

davidungar commented Feb 15, 2018

Uh oh!

davidungar commented Feb 16, 2018

Uh oh!

davidungar commented Feb 16, 2018

Uh oh!

davidungar commented Feb 16, 2018

Uh oh!

davidungar commented Feb 16, 2018

Uh oh!

jrose-apple left a comment

Choose a reason for hiding this comment

Uh oh!

jrose-apple Feb 16, 2018

Choose a reason for hiding this comment

Uh oh!

davidungar Feb 16, 2018

Choose a reason for hiding this comment

Uh oh!

davidungar commented Feb 16, 2018

Uh oh!

davidungar commented Jan 29, 2018 •

edited

Loading