[stdlib][Part 1] De-gyb almost everything

[Azoy]

See here: https://forums.swift.org/t/struggling-to-find-things-in-the-new-stdlib-for-5-0-e-g-where-is-addingreportingoverflow-implemented/18612/7

This is part 1 in a 4 part series.

This removes most .gyb files from the stdlib. Commits titled de-gyb ... are the product of copying and pasting the de-gybbed version over. I have a post de-gyb formatting commit that cleans up some of the messy formatting from gyb.

cc: @airspeedswift @stephentyrone @lorentey @xwu

[stephentyrone]

@airspeedswift and I just talked about this briefly. We'd like to do this, but want to be a bit more gradual about it. In particular, Ben wants to keep the .gyb files around (because they're better for editing--at least once you're used to them), but add a test that validates that the result of expanding the gyb matches the non-gyb source.

In cases where there isn't too much duplication, removing gyb entirely makes sense, but we should do those one file-per-commit, I think.

[Azoy]

Understandable. I read "all in" from Ben and I was ready to go "all in" 😅 I imagine we might one day get around to de-gybbing IntegerTypes which might allow us to break up the integers into separate files, Int8.swift, Int16.swift, etc. How would we go about testing this split in files with IntegerTypes.swift.gyb? (Assuming we want to do something like that instead of just testing against a IntegerTypes.swift with everything in it)

[stephentyrone]

@Azoy let's leave IntegerTypes and FloatingPointTypes alone for the time being, since they're a bit more involved than the others (but: we would simply run the test multiple times with different arguments to validate).

I think the other thing worth doing here is to more clearly document how to generate the .swift files from the .swift.gyb (probably via a comment line at the top the .gyb file that has exactly the required invocation(s)).

[airspeedswift]

which might allow us to break up the integers into separate files, Int8.swift, Int16.swift

I'd rather leave them all in one big file than clutter the folder with lots of near-identical files. The goal of degybbing is to improve search ability of the source code (the "where on earth is xxx declared" question on the forum that led to this idea), to allow stepping through the std lib, allow an LSP server to parse the whole std lib etc, all of which works from the single file approach. These files are probably rarely going to be touched in future so making them easier to navigate/read for editing purposes isn't that important.

[Azoy]

I only bring up splitting them up because the gyb file itself is almost 2k LoC, and a single ungybbed IntegerTypes.swift would be about 20k LoC. Some editors/IDEs/github even would have some trouble handling a buffer that large which might decrease ability to efficiently search for source code.

[stephentyrone]

@Azoy @airspeedswift Also worth noting that as the generics story and compiler optimizations get better, we can move progressively more and more integer stuff out of gyb and into default implementations, so the gyb piece will gradually get smaller.

It should eventually be pretty much just the wrappers around Builtins, which should ~never need to be edited or debugged unless something is catastrophically wrong.

[airspeedswift]

We talked about this some more and I am won over to the idea of breaking up the files (though re the 20k, I am guessing that's mostly docs and maybe some day we'll have doc-deduplication built into the swift doc capabilities). But it'd be good if we can do it by putting them all in an IntegerTypes folder.

[Azoy]

While looking for gyb comments in ExistentialCollection.swift, I made some more minor formatting changes which might need a quick review.

[CodaFi]

@Azoy Could you rebase this one more time?

[Azoy]

@CodaFi I've successfully rebased a years worth of commits! @airspeedswift @stephentyrone I know it's been a WHILE, but I would love another look to make sure this is good to go!

[CodaFi]

@swift-ci please test

[swift-ci]

Build failed
Swift Test OS X Platform
Git Sha - 861e0f79a63884bce7b5e4f96eb81153cfdeec34

[swift-ci]

Build failed
Swift Test Linux Platform
Git Sha - 861e0f79a63884bce7b5e4f96eb81153cfdeec34

[Azoy]

@CodaFi sigh sorry bout that one... it should be good now.

[CodaFi]

It happens. Let’s give it another go.

[CodaFi]

@swift-ci please test

[swift-ci]

Build failed
Swift Test OS X Platform
Git Sha - c9966192a45f01871072e2576571f754fef27a2d

[CodaFi]

Ignore the macOS result. There’s an infrastructure problem here.

[Azoy]

@CodaFi I've rebased and now this should be ready.

[CodaFi]

@swift-ci please test

[swift-ci]

Build failed
Swift Test OS X Platform
Git Sha - 0a1c30e9655d9d67738f4e65c0a0835831b8e174

[Azoy]

macOS failure unrelated?

[CodaFi]

Ugh, infrastructure failure. Let's try again

@swift-ci please test macOS platform

[swift-ci]

Build failed
Swift Test OS X Platform
Git Sha - 0a1c30e9655d9d67738f4e65c0a0835831b8e174

[Azoy]

Another infrastructure problem again? x)

[CodaFi]

I think this one’s an actual failure

api-digester/dump-module.swift

Suggests you might have changed the ABI surface of stdlib.

[benrimmington]

The conformances of Int (to MirrorPath, CVarArg, and CustomReflectable) in the cake test module have been reordered, possibly due to your CMakeLists.txt changes.

There's a comment in dump-module.swift which might be relevant:

// The input JSON files need to be modified when standard library declarations
// are reordered. This is expected behavior and we simply shouldn't run the test
// when automatically evolving the standard library.

[benrimmington]

This won't fix the test failure, but you could move ExistentialCollection.swift to the SWIFTLIB_SOURCES list (at line 210) if it isn't an "essential" source; and also change the copyright year of CMakeLists.txt (from 2018 to 2019).

[Azoy]

@benrimmington thanks so much for the help.
@CodaFi I've tested locally and this test passes now. We should be good to go.

[theblixguy]

@swift-ci please test

[swift-ci]

Build failed
Swift Test OS X Platform
Git Sha - 0a1c30e9655d9d67738f4e65c0a0835831b8e174

[swift-ci]

Build failed
Swift Test Linux Platform
Git Sha - 0a1c30e9655d9d67738f4e65c0a0835831b8e174

[Azoy]

@CodaFi all tests have passed!

[CodaFi]

⛵️