Skip to content

[SourceKit] In-Proc implementation of sourcekitd API functions #3026

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 24, 2016

Conversation

briancroom
Copy link
Contributor

What's in this pull request?

The current sourcekitd API implementation is only available on Darwin because it relies on XPC services. Darwin uses this for both the standard and in-process build variations, but this alternative implementation is needed for Linux where XPC services are not available.

This implementation is derived from the existing XPC one but replaces XPC services' object model with a basic class hierarchy providing reference-counted objects for the value types and collections that compose the sourcekitd API's request and response types. It uses polymorphism to implement the RTTI required by the heterogenous collections.

Note that additional effort is required before SourceKit can build on Linux, but this is a significant piece of the puzzle. I'll trigger CI to smoke-test the changes, but the new code isn't touched by the CI build yet.

This supersedes #2769.

cc @modocache @gribozavr @akyrtzi

Resolved bug number: (SR-1639)


Before merging this pull request to apple/swift repository:

  • Test pull request on Swift continuous integration.

Triggering Swift CI

The swift-ci is triggered by writing a comment on this PR addressed to the GitHub user @swift-ci. Different tests will run depending on the specific comment that you use. The currently available comments are:

Smoke Testing

Platform Comment
All supported platforms @swift-ci Please smoke test
All supported platforms @swift-ci Please smoke test and merge
OS X platform @swift-ci Please smoke test OS X platform
Linux platform @swift-ci Please smoke test Linux platform

Validation Testing

Platform Comment
All supported platforms @swift-ci Please test
All supported platforms @swift-ci Please test and merge
OS X platform @swift-ci Please test OS X platform
OS X platform @swift-ci Please benchmark
Linux platform @swift-ci Please test Linux platform

Lint Testing

Language Comment
Python @swift-ci Please Python lint

Note: Only members of the Apple organization can trigger swift-ci.

@briancroom
Copy link
Contributor Author

@swift-ci please test

@briancroom briancroom force-pushed the sourcekit-inproc-API branch from 81eb4b1 to 93f525d Compare June 20, 2016 14:40
@akyrtzi
Copy link
Contributor

akyrtzi commented Jun 21, 2016

The "Move some functions into sourcekitdAPI-Common" commit LGTM on its own, could you have it as a separate pull request so we can test & merge it as is ?
We can continue refining the "in-proc implementation" commit separately, from here.

@briancroom
Copy link
Contributor Author

Sounds good: #3096. Once that goes in, I'll get this rebased.

I've not written much C++ lately so I'm sure there's much to be improved here! I'm looking forward to specific feedback when you get a chance to take a look.

--RetainCount;
}
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

llvm has helpers to define and use ref-counted objects, see "llvm/ADT/IntrusiveRefCntPtr.h" and "swift/Basic/ThreadSafeRefCounted.h" that introduces ThreadSafeRefCountedBaseVPTR.
Essentially you can do this:

class SKDObject : public ThreadSafeRefCountedBaseVPTR {

This will give you an object (with a virtual destructor) that can be passed around as ref-counted pointer using llvm::IntrusiveRefCntPtr<T>.

@briancroom briancroom force-pushed the sourcekit-inproc-API branch from 93f525d to c645205 Compare June 22, 2016 16:23
@briancroom
Copy link
Contributor Author

Thanks for the schooling in LLVM patterns @akyrtzi. I've pushed a first attempt at reworking the code using LLVM-style refcounting and RTTI. Notes:

  • I tried to use IntrusiveRefCntPtr<SKDObject> where possible, however the RequestDict and ResponseBuilder types both reference the underlying object as a void *, so the implementations of those class' methods still end up working with SKDObject * a fair bit.
  • I implemented the LLVM RTTI, and used it to replace the asDictionary() etc. methods from the first version, however I didn't remove the get* methods, because it seemed like doing so would make the interface quite a bit harder to use, because clients would have to first do a dyn_cast and then follow it up with a NULL check before asking the subclass for its underlying value (unless there's a pattern for chaining that I'm not aware of?) What do you think?

using llvm::ArrayRef;
using llvm::StringRef;
using llvm::raw_ostream;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These using declarations shouldn't be necessary since "SourceKit/Core/LLVM.h" is included.

@akyrtzi
Copy link
Contributor

akyrtzi commented Jun 22, 2016

LGTM, with exception of the nitpicks and suggestion to see if SKDObjectVisitor, SKDObjectPrinter, and sourcekitd::printRequestObject, can be shared between the implementations.

@briancroom
Copy link
Contributor Author

Thanks for the thorough review and great feedback! This is very helpful.

@briancroom
Copy link
Contributor Author

Updated to reflect the comments, except for the duplication around SKDObjectPrinter. I'd like to wait on merging this until #3163 is resolved and I can update this to use the shared code.

@briancroom briancroom force-pushed the sourcekit-inproc-API branch from f533cc5 to 080fd2d Compare June 24, 2016 14:35
@briancroom
Copy link
Contributor Author

@swift-ci please test

@briancroom
Copy link
Contributor Author

Ok, updated again to take advantage of the RequestResponsePrinterBase.

@akyrtzi do you have any more feedback before this goes in?

virtual size_t getCount() const { return 0; }

virtual sourcekitd_uid_t getUID() const { return nullptr; }
virtual Optional<int64_t> getInt64() const { return Optional<int64_t>(); }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idiom to use when wanting to pass an Optional without value is to use llvm::None:

virtual Optional<int64_t> getInt64() const { return None; }

There are other instances like this in the rest of the file.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahhh nice. I was looking for something like that but couldn't track it down before.

@akyrtzi
Copy link
Contributor

akyrtzi commented Jun 24, 2016

Other than a couple of extra nitpicks, LGTM!

Darwin can use the XPC implementation of this even for the in-proc library
build, this alternative implementation is need for Linux where XPC services
are not available.

This implementation is derived from the existing XPC one but replaces XPC
services' object model with a basic class hierarchy providing
reference-counted objects for the value types and collections that
compose the sourcekitd API's request and response types.
@briancroom briancroom force-pushed the sourcekit-inproc-API branch from 080fd2d to 2c4eb84 Compare June 24, 2016 21:44
@briancroom
Copy link
Contributor Author

Excellent! I'm going to go ahead and merge right away now after having made those last updates since CI doesn't actually build this file yet anyway.

Thanks again for your support on this @akyrtzi

@briancroom briancroom merged commit 73d0e86 into swiftlang:master Jun 24, 2016
@briancroom briancroom deleted the sourcekit-inproc-API branch June 24, 2016 21:49
@akyrtzi
Copy link
Contributor

akyrtzi commented Jun 24, 2016

Thanks for working on this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants