Switch from .sid files to using FILE_DEPENDENCY records in INPUT_BLOCK #20170

graydon · 2018-10-31T00:21:59Z

This backs off previous approach I made in b4a96bb that used separate .sid files to store dependency info. Instead, automatically-generated .swiftmodules will now contain some new INPUT_BLOCK records called FILE_DEPENDENCY, which encode the same data but don't make as much of a mess (no new filetype).

I'll expand this next to track file sizes too, and will redo the dependency test to be test meaningful scenarios; I just figured I'd get this much tidied up and reviewed since it seems to kinda work and deleting the .sid code ASAP seemed like a good move.

graydon · 2018-10-31T00:26:12Z

@swift-ci please test

lib/Serialization/ModuleFile.cpp

include/swift/Serialization/Validation.h

include/swift/Serialization/SerializationOptions.h

graydon · 2018-10-31T07:39:22Z

@swift-ci please test

graydon · 2018-10-31T07:39:39Z

@jrose-apple updated with size and time and exact matching.

swift-ci · 2018-10-31T07:41:22Z

Build failed
Swift Test OS X Platform
Git Sha - 359edaef6b1e99c9441ed1c3ec4eb4372032b4a2

swift-ci · 2018-10-31T07:41:31Z

Build failed
Swift Test Linux Platform
Git Sha - 359edaef6b1e99c9441ed1c3ec4eb4372032b4a2

graydon · 2018-11-01T01:08:33Z

@swift-ci please test

swift-ci · 2018-11-01T01:15:42Z

Build failed
Swift Test OS X Platform
Git Sha - ccbfd1a6990e77e455a642e6d7c08995bbf6b39b

swift-ci · 2018-11-01T01:39:32Z

Build failed
Swift Test Linux Platform
Git Sha - ccbfd1a6990e77e455a642e6d7c08995bbf6b39b

…_BLOCK.

…ion.

…logic.

graydon · 2018-11-01T17:52:21Z

@swift-ci please test

swift-ci · 2018-11-01T17:54:16Z

Build failed
Swift Test Linux Platform
Git Sha - a5abdb335511a3a90b96105a2b6ae95e72a5eee8

swift-ci · 2018-11-01T17:54:23Z

Build failed
Swift Test OS X Platform
Git Sha - a5abdb335511a3a90b96105a2b6ae95e72a5eee8

test/ParseableInterface/client.swift

lib/Frontend/ParseableInterfaceSupport.cpp

graydon · 2018-11-02T04:19:03Z

@jrose-apple pushed update with review comments addressed (I hope!)

graydon · 2018-11-02T04:19:10Z

@swift-ci please test

swift-ci · 2018-11-02T04:19:58Z

Build failed
Swift Test OS X Platform
Git Sha - a0de5b3

swift-ci · 2018-11-02T04:21:16Z

Build failed
Swift Test Linux Platform
Git Sha - a0de5b3

graydon · 2018-11-02T18:52:48Z

@swift-ci please test

swift-ci · 2018-11-02T18:54:41Z

Build failed
Swift Test OS X Platform
Git Sha - 0a3bd3e

swift-ci · 2018-11-02T18:54:55Z

Build failed
Swift Test Linux Platform
Git Sha - 0a3bd3e

graydon · 2018-11-05T18:40:07Z

@swift-ci please test OSX platform

jrose-apple

Looks good, only minor comments. I do still think we need the follow-up where dependencies propagate out (so that my app depends on Foundation.swiftinterface and not just Foundation.swiftmodule), but that can and probably should be a separate PR.

lib/Frontend/ParseableInterfaceSupport.cpp

test/ParseableInterface/module-cache-init.swift

include/swift/Serialization/ModuleFormat.h

graydon · 2018-11-05T22:52:05Z

@swift-ci please test OSX platform

graydon · 2018-11-05T22:53:31Z

@swift-ci please test OSX platform

graydon · 2018-11-05T22:53:45Z

@swift-ci please test macos platform

graydon · 2018-11-05T23:04:59Z

@swift-ci please test

swift-ci · 2018-11-06T00:31:19Z

Build failed
Swift Test OS X Platform
Git Sha - c82e76b

graydon · 2018-11-06T01:56:40Z

@swift-ci Please Python lint

graydon · 2018-11-06T01:58:17Z

@swift-ci please test and merge

graydon · 2018-11-06T04:51:55Z

@swift-ci please test and merge

graydon · 2018-11-06T07:08:21Z

@swift-ci please test linux platform

graydon · 2018-11-06T07:08:43Z

@swift-ci please test Linux platform

graydon requested a review from jrose-apple October 31, 2018 00:21

jrose-apple reviewed Oct 31, 2018

View reviewed changes

lib/Serialization/ModuleFile.cpp Show resolved Hide resolved

include/swift/Serialization/Validation.h Outdated Show resolved Hide resolved

include/swift/Serialization/SerializationOptions.h Outdated Show resolved Hide resolved

graydon force-pushed the text-interface-to-module branch from ccbfd1a to a5abdb3 Compare November 1, 2018 00:58

graydon added 10 commits November 1, 2018 10:51

[ModuleInterface] Add FILE_DEPENDENCY record to INPUT_BLOCK.

9fceeb4

[ModuleInterface] Remove SwiftParseableInterfaceDeps files, use INPUT…

4938b00

…_BLOCK.

[ModuleInterface] Address review comments.

fd226ae

[ModuleInterface] Factor out clang module cache path extraction funct…

1a4ba38

…ion.

[ModuleInterface] Add PrintingDiagnosticConsumer to sub-instance.

11f3c94

[ModuleInterface] Pass clang module cache path to sub-instance.

8d6f6e2

[ModuleInterface] Set EnableParseableModuleInterface in sub-instance.

6cb7b46

[ModuleInterface] Add more debug logging.

9b25819

[ModuleInterface] Make up-to-date check recursive, since it runs first.

fc4510e

[ModuleInterface] Redo testcase to exercise (recursive) invalidation …

a0de5b3

…logic.

graydon force-pushed the text-interface-to-module branch from a5abdb3 to a0de5b3 Compare November 1, 2018 17:52

jrose-apple reviewed Nov 1, 2018

View reviewed changes

test/ParseableInterface/client.swift Outdated Show resolved Hide resolved

test/ParseableInterface/client.swift Outdated Show resolved Hide resolved

lib/Frontend/ParseableInterfaceSupport.cpp Outdated Show resolved Hide resolved

[ModuleInterface] Address more review comments.

0a3bd3e

[ModuleInterface] Overhaul module-cache tests for clarity, independence.

c82e76b

jrose-apple approved these changes Nov 5, 2018

View reviewed changes

lib/Frontend/ParseableInterfaceSupport.cpp Show resolved Hide resolved

test/ParseableInterface/module-cache-init.swift Show resolved Hide resolved

include/swift/Serialization/ModuleFormat.h Show resolved Hide resolved

[ModuleInterface] Fix python flake8 lints.

edb7ccb

graydon merged commit e272ece into swiftlang:master Nov 6, 2018

Switch from .sid files to using FILE_DEPENDENCY records in INPUT_BLOCK #20170

Switch from .sid files to using FILE_DEPENDENCY records in INPUT_BLOCK #20170

Uh oh!

Conversation

graydon commented Oct 31, 2018

Uh oh!

graydon commented Oct 31, 2018

Uh oh!

Uh oh!

Uh oh!

Uh oh!

graydon commented Oct 31, 2018

Uh oh!

graydon commented Oct 31, 2018

Uh oh!

swift-ci commented Oct 31, 2018

Uh oh!

swift-ci commented Oct 31, 2018

Uh oh!

graydon commented Nov 1, 2018

Uh oh!

swift-ci commented Nov 1, 2018

Uh oh!

swift-ci commented Nov 1, 2018

Uh oh!

graydon commented Nov 1, 2018

Uh oh!

swift-ci commented Nov 1, 2018

Uh oh!

swift-ci commented Nov 1, 2018

Uh oh!

Uh oh!

Uh oh!

Uh oh!

graydon commented Nov 2, 2018

Uh oh!

graydon commented Nov 2, 2018

Uh oh!

swift-ci commented Nov 2, 2018

Uh oh!

swift-ci commented Nov 2, 2018

Uh oh!

graydon commented Nov 2, 2018

Uh oh!

swift-ci commented Nov 2, 2018

Uh oh!

swift-ci commented Nov 2, 2018

Uh oh!

graydon commented Nov 5, 2018

Uh oh!

jrose-apple left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

graydon commented Nov 5, 2018

Uh oh!

graydon commented Nov 5, 2018

Uh oh!

graydon commented Nov 5, 2018

Uh oh!

graydon commented Nov 5, 2018

Uh oh!

swift-ci commented Nov 6, 2018

Uh oh!

graydon commented Nov 6, 2018

Uh oh!

graydon commented Nov 6, 2018

Uh oh!

graydon commented Nov 6, 2018

Uh oh!

graydon commented Nov 6, 2018

Uh oh!

graydon commented Nov 6, 2018

Uh oh!

Uh oh!