Skip to content

[Swift-ObjC-Interop] Speculative cyclic lookup fix #66177

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 30, 2023

Conversation

NuriAmari
Copy link
Contributor

The lookup to resolve an ObjC forward declaration
to its potential native Swift definition within
a mixed module is becoming cyclic somehow.

The old uncached lookup still works, so it seems
this is an issue with cache invalidation somehow.
Until the root of the issue is resolved, never mark the cache as populated.
This is the same behaviour as before the cached lookup was introduced, but
we can easily turn the caching back on later.

@NuriAmari
Copy link
Contributor Author

@swift-ci Please test

@NuriAmari NuriAmari force-pushed the cyclic-lookup-fix branch from 2098108 to ab14d8f Compare May 28, 2023 17:53
@NuriAmari
Copy link
Contributor Author

@swift-ci Please test

@NuriAmari NuriAmari force-pushed the cyclic-lookup-fix branch from ab14d8f to d80b4f4 Compare May 28, 2023 18:09
The lookup to resolve an ObjC forward declaration
to its potential native Swift definition within
a mixed module is becoming cyclic somehow.

The old uncached lookup still works, so it seems
this is an issue with cache invalidation somehow.
Until the route of the issue, use the old uncached
lookup method.
@NuriAmari NuriAmari force-pushed the cyclic-lookup-fix branch from d80b4f4 to f676b75 Compare May 28, 2023 18:46
@NuriAmari
Copy link
Contributor Author

@swift-ci Please test

@NuriAmari NuriAmari marked this pull request as ready for review May 28, 2023 18:49
@slavapestov
Copy link
Contributor

Can we do this by writing out a lookup table instead of calling getTopLevelDeclsWhereAttributesMatch(), which performs an O(n) walk and a lot of unnecessary deserialization?

@NuriAmari
Copy link
Contributor Author

Can we do this by writing out a lookup table instead of calling getTopLevelDeclsWhereAttributesMatch(), which performs an O(n) walk and a lot of unnecessary deserialization?

In some sense that was the old behavior, but apparently it caused infinite recursion when compiling some Apple framework. I suspect that would work, but we apparently need to re-populate the lookup table at some point, but I don't have a repro for the issue so I can't figure that out.

For what its worth, this change is a partial revert of #61606, so the lookup strategy restored by this patch has been in use for quite some time.

@slavapestov
Copy link
Contributor

slavapestov commented May 30, 2023

In some sense that was the old behavior, but apparently it caused infinite recursion when compiling some Apple framework. I suspect that would work, but we apparently need to re-populate the lookup table at some point, but I don't have a repro for the issue so I can't figure that out.

I debugged and diagnosed the request cycle a few weeks ago. It was caused by excessive deserialization. The problem is not the caching, it's the fact that ultimately this code path was triggering deserialization of large numbers of top-level decls in the cache miss case.

You need to find a way of doing this without calling getTopLevelDeclsWhereAttributesMatch(). What I'm suggesting is to serialize an on-disk lookup table mapping identifiers to declarations, so that you can avoid the O(n) walk that triggers deserialization of all top-level declarations when you want to find a single identifier.

@slavapestov
Copy link
Contributor

Oh I see, getTopLevelDeclsWhereAttributesMatch() does skip some work if the predicate fails, and the predicate is narrower with this PR. My apologies. We should still find a way of doing this that doesn't involve getTopLevelDeclsWhereAttributesMatch(), but maybe this fix will address the issue.

@NuriAmari
Copy link
Contributor Author

Oh I see, getTopLevelDeclsWhereAttributesMatch() does skip some work if the predicate fails, and the predicate is narrower with this PR. My apologies. We should still find a way of doing this that doesn't involve getTopLevelDeclsWhereAttributesMatch(), but maybe this fix will address the issue.

Right. It worked this way up until #61606, landed (including Swift5.8), so I imagine it should continue to work.

Can you also approve the pick into Swift5.9: #66195? I will work on the proposed serialized lookup table, but in the immediate future I am trying to as best possible make sure the change remains in Swift5.9.

@NuriAmari NuriAmari merged commit ea4646f into swiftlang:main May 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants