Skip to content

[Runtime] Fix incorrect memory ordering in ConcurrentReadableArray/HashMap #35348

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 12, 2021
Merged

[Runtime] Fix incorrect memory ordering in ConcurrentReadableArray/HashMap #35348

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 16 additions & 3 deletions include/swift/Runtime/Concurrent.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -527,7 +527,12 @@ template <class ElemTy> struct ConcurrentReadableArray {

storage = newStorage;
Capacity = newCapacity;
Elements.store(storage, std::memory_order_release);

// Use seq_cst here to ensure that the subsequent load of ReaderCount is
// ordered after this store. If ReaderCount is loaded first, then a new
// reader could come in between that load and this store, and then we
// could end up freeing the old storage pointer while it's still in use.
Elements.store(storage, std::memory_order_seq_cst);
Copy link
Contributor

@davezarzycki davezarzycki Jan 12, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the fix is correct but the reasoning doesn't feel right to me at all. The problem isn't that the readers are picking up the old value but that the writer isn't reasoning correctly about whether there are readers or not. After all, this proposed fix is on the writer side of the design.

On x86, the only C++11 memory model hint that matters "sequential consistency" and even then, it only matters for stores. There are two reasons for this: 1) atomic read-modify-write x86 instructions (i.e. "LOCK prefixed") don't support anything other than sequential consistency and 2) the only reordering otherwise allowed by x86's "total store order" memory model is newer loads being moved ahead of older stores.

By switching Elements.store(...) to sequential consistency, we're forcing ReaderCount.load() to happen after the store.

I think if one is going to use std::memory_order_seq_cst, then one really ought to document at least one example in a comment of an actual subsequent load in the same thread that must come after the given store.

Copy link
Contributor

@davezarzycki davezarzycki Jan 12, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is the ASCII art of what I think is happening:

Effective program order due to out-of-order loads.

Writer                 | Reader
-----------------------+--------------------
............ ReaderCount == 0 ..............
                       |
 -> ReaderCount.load() |
/                      |
|                      | incrementReaders()
|                      | storage = Elements.load()
|                      |
| Elements.store()     |
\                      |
 *ReaderCount.load()   |

Now the writer wrongly thinks there are zero readers of the old data.

}

new(&storage->data()[count]) ElemTy(elem);
Expand Down Expand Up @@ -866,7 +871,11 @@ struct ConcurrentReadableHashMap {
FreeListNode::add(&FreeList, elements);
}

Elements.store(newElements, std::memory_order_release);
// Use seq_cst here to ensure that the subsequent load of ReaderCount is
// ordered after this store. If ReaderCount is loaded first, then a new
// reader could come in between that load and this store, and then we
// could end up freeing the old elements pointer while it's still in use.
Elements.store(newElements, std::memory_order_seq_cst);
return newElements;
}

Expand Down Expand Up @@ -900,7 +909,11 @@ struct ConcurrentReadableHashMap {
newIndices.storeIndexAt(nullptr, index, newI, std::memory_order_relaxed);
}

Indices.store(newIndices.Value, std::memory_order_release);
// Use seq_cst here to ensure that the subsequent load of ReaderCount is
// ordered after this store. If ReaderCount is loaded first, then a new
// reader could come in between that load and this store, and then we
// could end up freeing the old indices pointer while it's still in use.
Indices.store(newIndices.Value, std::memory_order_seq_cst);

if (auto *ptr = indices.pointer())
FreeListNode::add(&FreeList, ptr);
Expand Down
Loading