Skip to content

[lld] check cache before real_path in loadDylib #140791

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 29, 2025
Merged

[lld] check cache before real_path in loadDylib #140791

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 20 additions & 6 deletions lld/MachO/DriverUtils.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -229,19 +229,31 @@ static DenseMap<CachedHashStringRef, DylibFile *> loadedDylibs;

DylibFile *macho::loadDylib(MemoryBufferRef mbref, DylibFile *umbrella,
bool isBundleLoader, bool explicitlyLinked) {
// Frameworks can be found from different symlink paths, so resolve
// symlinks before looking up in the dylib cache.
SmallString<128> realPath;
std::error_code err = fs::real_path(mbref.getBufferIdentifier(), realPath);
CachedHashStringRef path(!err ? uniqueSaver().save(StringRef(realPath))
: mbref.getBufferIdentifier());
CachedHashStringRef path(mbref.getBufferIdentifier());
DylibFile *&file = loadedDylibs[path];
if (file) {
if (explicitlyLinked)
file->setExplicitlyLinked();
return file;
}

// Frameworks can be found from different symlink paths, so resolve
// symlinks and look up in the dylib cache.
DylibFile *&realfile = file;
SmallString<128> realPath;
std::error_code err = fs::real_path(mbref.getBufferIdentifier(), realPath);
if (!err) {
CachedHashStringRef resolvedPath(uniqueSaver().save(StringRef(realPath)));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
CachedHashStringRef resolvedPath(uniqueSaver().save(StringRef(realPath)));
CachedHashStringRef resolvedPath(uniqueSaver().save(realPath.str()));

realfile = loadedDylibs[resolvedPath];
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a subtle use-after-invalidation here. Both file and realfile are references to loadedDylibs[Path]. The right-hand-side may grow the map, invalidating realfile (and file), so when performing the assignment we get use-after-free.

See https://crbug.com/422206408 for a reproducer.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the repro and revert, looks like this is going to need a better approach.

if (realfile) {
if (explicitlyLinked)
realfile->setExplicitlyLinked();

file = realfile;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is redundant since both file and realfile are references to loadedDylibs[Path].

As are all realfile = file assignments below.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review.

At this point, in my head, file = loadedDylibs[Path], but realfile = loadedDylibs[resolvedPath]. Originally there was no DylibFile *&realfile = file; (it was directly assigned from loadedDylibs[resolvedPath]), which might change things, but I am not sure if it does.

The realfile = file; below was supposed to also make sure that the entry for Path was assigned the result assigned to resolvedPath as well, so both refer to the same actual DylibFile *.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

C++ references are sneaky. Once they're bound, they cannot be re-bound. I.e., after DylibFile *&realfile = file; any assignment to realfile will be like assigning to file.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not have added that line then.

I found this about references to unordered_map being invalidated: https://eel.is/c++draft/unord.req#general-9 (emphasis mine)

Rehashing invalidates iterators, changes ordering between elements, and changes which buckets elements appear in, but does not invalidate pointers or references to elements

I am not sure if llvm::DenseMap follows all the rules of std::unordered_map, but that little bit might be something they forgot to keep compatible.

return realfile;
}
}

DylibFile *newFile;
file_magic magic = identify_magic(mbref.getBuffer());
if (magic == file_magic::tapi_file) {
Expand All @@ -253,6 +265,7 @@ DylibFile *macho::loadDylib(MemoryBufferRef mbref, DylibFile *umbrella,
}
file =
make<DylibFile>(**result, umbrella, isBundleLoader, explicitlyLinked);
realfile = file;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the common case we have

DylibFile *&file = loadedDylibs[path];
DylibFile *&realfile = file;

Then what happens on this line? It does a load and a store to the same address? I'm sure it does the right thing it just looks funny to me.

Also, if we set file later in the future, how can we make sure to not forget to also set realfile? Is there a better way of doing this?

Copy link
Contributor Author

@rmaz rmaz May 22, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then what happens on this line? It does a load and a store to the same address?

Thats what I thought would happen, yes.

Also, if we set file later in the future, how can we make sure to not forget to also set realfile? Is there a better way of doing this?

What would you suggest? Ultimately we have 2 cache pointers, that may or may not point to the same thing, and we need to update them.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if we just use realfile everywhere? In the case when the pointers are the same, everything works as expected. But when they are different, the first cache lookup is basically always a miss and will rely on the second resolved path lookup to find the result. In theory, it shouldn't be a huge hit, but it's still a minor regression from the current implementation. But, it makes things easier to reason about without needing to set both file and realfile?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would mean that we would never cache symlinks, only their resolved paths, which is the unhappy path. That would be a significant regression as a large number of load commands are symlinks (eg Foundation.framework/Foundation -> Foundation.framework/Versions/A/Foundation).

I also disagree with the idea of regressing the performance to reduce the amount of code by 2 lines.

The only alternative I can see is to do have realfile default to a nullptr and have the two setters change to:

if (realfile)
  realfile = file;

Can't say I prefer it though.

Copy link
Contributor

@drodriguez drodriguez May 23, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would something like llvm::make_scope_exit([&]{ realfile = file; }) be OK to keep things organized? Would that work?

Edit: or add a single line after this if {} else {} block. I am scared that it will not be obvious there if something changes in the if {} else {} block, though.


// parseReexports() can recursively call loadDylib(). That's fine since
// we wrote the DylibFile we just loaded to the loadDylib cache via the
Expand All @@ -268,6 +281,7 @@ DylibFile *macho::loadDylib(MemoryBufferRef mbref, DylibFile *umbrella,
magic == file_magic::macho_executable ||
magic == file_magic::macho_bundle);
file = make<DylibFile>(mbref, umbrella, isBundleLoader, explicitlyLinked);
realfile = file;

// parseLoadCommands() can also recursively call loadDylib(). See comment
// in previous block for why this means we must copy `file` here.
Expand Down