Skip to content

[Runtime] Fix memory ordering and TSan compatibility in MetadataAllocator. #80021

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 20 additions & 1 deletion stdlib/public/runtime/Metadata.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@
#include "swift/Runtime/Portability.h"
#include "swift/Strings.h"
#include "swift/Threading/Mutex.h"
#include "swift/Threading/ThreadSanitizer.h"
#include "llvm/ADT/StringExtras.h"
#include <algorithm>
#include <cctype>
Expand Down Expand Up @@ -7977,10 +7978,28 @@ void *MetadataAllocator::Allocate(size_t size, size_t alignment) {
Tag);
}

// If we allocated a new page, then we need to do a store-release to ensure
// the initialization writes are properly ordered when viewed from other
// threads that read from the new page. If we did not allocate a new page,
// then we need a load-consume to cover the other side of that.
std::memory_order successOrder = allocatedNewPage
? std::memory_order_release
: SWIFT_MEMORY_ORDER_CONSUME;

// Swap in the new state.
if (AllocationPool.compare_exchange_weak(curState, newState,
std::memory_order_relaxed,
successOrder,
std::memory_order_relaxed)) {
// If the program is using Thread Sanitizer, it can't see our memory
// ordering, so inform it manually. TSan will track the consume ordering
// in __swift_instantiateConcreteTypeFromMangledName so we register the
// correct ordering with threads that get a metadata pointer from a cache
// variable too.
if (allocatedNewPage)
swift::tsan::release(&AllocationPool);
else
swift::tsan::acquire(&AllocationPool);

// If that succeeded, we've successfully allocated.
__msan_allocated_memory(allocation, sizeWithHeader);
__asan_unpoison_memory_region(allocation, sizeWithHeader);
Expand Down