[Incremental] Skip post-compile jobs only if no compiles ran, and all the outputs are up-to-date.

[davidungar]

If the merge-modules step fails or is cancelled, all the compile steps may be skipped, yet the post-compile jobs still must run.
Improve the logic that was skipping the post-compile jobs whenever there are no compile jobs.
When the compile jobs would otherwise be skipped, check their outputs against the time the previous build finished.
In order to perform the check, enhance the build record to store the end time as well as the start time of the prior build.

rdar://76127929

[davidungar]

@swift-ci please test

[davidungar]

@swift-ci please test

[davidungar]

FYI, I plan to add a regression test before landing this PR.

[davidungar]

@cltnschlosser Thank you for such a quick review! I'll look at your concerns today.

[davidungar]

@swift-ci please test

[davidungar]

@swift-ci please test

[CodaFi]

Err, can we change the format of the build record? If the user needs to fall back/upgrade to or from the old and new drivers, will this break anything?

[davidungar]

Err, can we change the format of the build record? If the user needs to fall back/upgrade to or from the old and new drivers, will this break anything?

Thanks for taking a look at this PR. I'm not sure I understand. The new build end time has a new section name, and the code should accept either the old name for the old time or the new name for the old time. At least, that's the code I thought I wrote.
If the user upgrades and there are files to be compiled, it should just work as before, writing out the more complete build record.
If the user upgrades and there are no files to be compiled, the post-compile jobs might run one time, even though they are not needed.
If the user downgrades, the read of the build record start date will fail, and everything will be recompiled on the first post-downgrade compile.
The tradeoff here is between using a clearer name for the start date in the build record, and avoiding the recompilation-when-downgrading penalty. I had chosen the former, but did you mean to argue for the latter? If so, what is the rationale? I'm open to it.

[CodaFi]

The question is more base than that: This new format appears to replace a key from the legacy driver’s build record format outright. So, if I use the new driver and need to downgrade then the old driver will be missing information. Is the legacy driver tolerant of that use case? Similarly if I upgrade, the new driver will not have these new keys to read.

If we are unsure of the answer to this question, I would prefer we make only additive changes to the build record format.

[davidungar]

The question is more base than that: This new format appears to replace a key from the legacy driver’s build record format outright. So, if I use the new driver and need to downgrade then the old driver will be missing information. Is the legacy driver tolerant of that use case? Similarly if I upgrade, the new driver will not have these new keys to read.

If we are unsure of the answer to this question, I would prefer we make only additive changes to the build record format.

Thanks for explaining. This sort of issue is exactly why I like to reviews! I just performed a few experiments: Tried an upgrade from a main swift-driver to this one, a downgrade from this one to the main, and also a downgrade from this one to the legacy driver. I tried these transitions in one of two scenarios, with "build_time" changed to "build_start_time" and with "build_time" left alone.

Surprisingly, it turned out that there was no benefit to the latter scenario. Both the legacy driver and the main (status quo) swift driver reject the build record when there is a new field (in this case "build_end_time") added to it. I conclude that the better option is to go with the former scenario. We might as well rename the field since the added field will cause rejection anyway.

To summarize the results:

Upgrading from the main swift driver to this PR:

• Works fine

Downgrading from this PR to a main swift driver:

• "Incremental compilation has been disabled due to malformed build record file Unexpected key 'build_start_time'"
• Subseqenent compilations work fine

Downgrading from this PR or from main swift driver to legacy driver:

• "Incremental compilation has been disabled, because different arguments were passed to the compiler."

Thanks for asking the question. I don't see any better solution than this PR as it stands. Do you?

[davidungar]

@swift-ci please test

[davidungar]

Also, cleaned up the commit history