[OpenMP] Fix node destruction race in __kmpc_omp_taskwait_deps_51 #86130
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The __kmpc_omp_taskwait_deps_51 allocates a kmp_depnode_t node on its stack, and there is currently a race condition where another thread might still be accessing that node after the function has returned and its stack frame was released. While the function does wait until the node's npredecessors count has reached zero before exiting, there is still a window where the function that last decremented the npredecessors count assumes the node is still accessible. For heap-allocated kmp_depnode_t nodes, this normally works via a separate ndeps count that only reaches zero at the point where no accesses to the node are expected at all; in fact, at this point the heap allocation will be freed. For this case of a stack-allocated kmp_depnode_t node, it therefore makes sense to similarly respect the ndeps count; we need to wait until this reaches 1 (not 0, because it is not heap-allocated so there's always one extra count to prevent it from being freed), before we can safely deallocate our stack frame. As this is expected to be a short race window of only a few instructions, it should be fine to just use a busy wait loop checking the ndeps count. Fixes: llvm#85963
shiltian
reviewed
Mar 27, 2024
| // Wait until the last __kmp_release_deps is finished before we free the | ||
| // current stack frame holding the "node" variable; once its nrefs count | ||
| // reaches 1, we're sure nobody else can try to reference it again. | ||
| while (node.dn.nrefs > 1) |
There was a problem hiding this comment.
I'm not sure this is the right way to do that, especially using task yield. @jprotze might have a better idea.
There was a problem hiding this comment.
An alternative solution would be to just not use a stack-allocated node, but allocate it on the heap like everywhere else. Then there'd still be a race which thread would be the one to free it, but that race would be harmless ... Let me know if you'd like me to try this (or any other) approach.
|
This fix makes sense to me. Looking at the code, I realized that the current implementation is still stupid. The function should really create an empty task to be inserted into the DAG, if has_no_wait is set. |
jprotze
approved these changes
Mar 28, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The __kmpc_omp_taskwait_deps_51 allocates a kmp_depnode_t node on its stack, and there is currently a race condition where another thread might still be accessing that node after the function has returned and its stack frame was released.
While the function does wait until the node's npredecessors count has reached zero before exiting, there is still a window where the function that last decremented the npredecessors count assumes the node is still accessible.
For heap-allocated kmp_depnode_t nodes, this normally works via a separate ndeps count that only reaches zero at the point where no accesses to the node are expected at all; in fact, at this point the heap allocation will be freed.
For this case of a stack-allocated kmp_depnode_t node, it therefore makes sense to similarly respect the ndeps count; we need to wait until this reaches 1 (not 0, because it is not heap-allocated so there's always one extra count to prevent it from being freed), before we can safely deallocate our stack frame.
As this is expected to be a short race window of only a few instructions, it should be fine to just use a busy wait loop checking the ndeps count.
Fixes: #85963