Skip to content

[build-script] Transform IndexStoreDB and SorcekitLSP to use ProductBuilder #24854

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

drodriguez
Copy link
Contributor

Create a new helper builder for IndexStoreDB and SourceKitLSP called
BuildScriptHelperBuilder (because the invoked script is called
build-script-helper.py). Move most of the code that existed in a
function in indexstore.db into that helper class.

Create small subtypes of the helper paramtrized for IndexStoreDB and
SourceKitLSP.

Modify the main build-script to invoke the new build instead of the
Product class directly. This way, when all the products use builders
to be build, all of them will use the same Python invocations, but
some of them will use build-script-impl, and others will use
build-script-helper.py.

@drodriguez
Copy link
Contributor Author

@swift-ci please test

@drodriguez drodriguez force-pushed the build-script-indexstoredb-sourcekitlsp-builders branch from 82c6995 to db26d74 Compare May 17, 2019 22:12
@drodriguez drodriguez requested a review from yln May 17, 2019 22:12
@drodriguez
Copy link
Contributor Author

Rebased on top of master to adapt to the changes introduced in #24330. Ideally TSanLibDispatch should use a future CMakeBuilder (like the one than can be seen WIP in drodriguez@6807ef6#diff-ac29258457eb63cf4ac9e0f42ce3ba6d), but that code is slightly blocked by #23303 at the moment. I can do the change when the right pieces fall into place.

@swift-ci please test

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 82c699591b9a0340f569a364e262fd8db2b58730

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 82c699591b9a0340f569a364e262fd8db2b58730

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - 82c699591b9a0340f569a364e262fd8db2b58730

@drodriguez
Copy link
Contributor Author

@swift-ci please test linux platform

@benlangmuir
Copy link
Contributor

Looks like the non-smoke macOS tests are not building sourcekit-lsp (fix is #24916).

@benlangmuir
Copy link
Contributor

@swift-ci please smoke test macOS

@drodriguez
Copy link
Contributor Author

Sorry about that. I was expecting the non-smoke tests to be all that smoke test do, and then some. Thanks for noticing that.

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - db26d7483ae7468a4cbc5e96ecb8f6e563e38384

@benlangmuir
Copy link
Contributor

@swift-ci please test Linux

@@ -961,20 +961,16 @@ class BuildScriptInvocation(object):

# Non-build-script-impl products...
# Note: currently only supports building for the host.
for host_target in [self.args.host_target]:
for host_target in [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why leave the list if its a single element?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are asking why have a for ... in for a known one element list, right? I think the reason is future-proofing, because the comment says that the products only support building for the host, but they might some day support building for more targets, and that change will be easy having this in place. Ben might know the real reason, since he wrote the original code.

def __init__(self, product_class, args, toolchain, workspace, host):
self.__source_dir = workspace.source_dir(
product_class.product_source_name())
self.__build_dir = workspace.build_dir(host.name,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why double underscores?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Python performs name mangling when you use two underscores as a prefix. It is useful for private variables that cannot be accessed from other classes (including subclasses of this one).

https://docs.python.org/2/tutorial/classes.html#private-variables-and-class-local-references

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - db26d7483ae7468a4cbc5e96ecb8f6e563e38384

@drodriguez
Copy link
Contributor Author

That Linux error is on me. I will have a look tomorrow. Benchmarks wasn’t there when I started these changes, and I notice the new toolchain needing product, but I didn’t realize benchmarks was there.

@drodriguez drodriguez force-pushed the build-script-indexstoredb-sourcekitlsp-builders branch from db26d74 to 6277c99 Compare May 21, 2019 21:02
@drodriguez
Copy link
Contributor Author

@swift-ci please test

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - db26d7483ae7468a4cbc5e96ecb8f6e563e38384

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - db26d7483ae7468a4cbc5e96ecb8f6e563e38384

…uilder.

Create a new helper builder for IndexStoreDB and SourceKitLSP called
BuildScriptHelperBuilder (because the invoked script is called
build-script-helper.py). Move most of the code that existed in a
function in indexstore.db into that helper class.

Create small subtypes of the helper paramtrized for IndexStoreDB and
SourceKitLSP.

Modify the main build-script to invoke the new build instead of the
Product class directly. This way, when all the products use builders
to be build, all of them will use the same Python invocations, but
some of them will use build-script-impl, and others will use
build-script-helper.py.
@drodriguez drodriguez force-pushed the build-script-indexstoredb-sourcekitlsp-builders branch from 6277c99 to 6d0c1e7 Compare May 21, 2019 22:37
@drodriguez
Copy link
Contributor Author

Fix the linter problem.

@swift-ci please test

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 6277c994c92318c391c098a70cde6c887a275013

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - 6277c994c92318c391c098a70cde6c887a275013

@drodriguez
Copy link
Contributor Author

Abandoning these efforts. It might still be useful for better Windows builds for most of the community, but I will not have time to keep improving this and it doesn't seem to be a lot of interest in improving the build system in this direction.

@drodriguez drodriguez closed this Jan 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants