Skip to content

Commit d0c0064

Browse files
ubizjakPeter Zijlstra
authored andcommitted
jump_label: Use atomic_try_cmpxchg() in static_key_slow_inc_cpuslocked()
Use atomic_try_cmpxchg() instead of atomic_cmpxchg (*ptr, old, new) == old in static_key_slow_inc_cpuslocked(). x86 CMPXCHG instruction returns success in ZF flag, so this change saves a compare after cmpxchg (and related move instruction in front of cmpxchg). Also, atomic_try_cmpxchg() implicitly assigns old *ptr value to "old" when cmpxchg fails, enabling further code simplifications. No functional change intended. Signed-off-by: Uros Bizjak <[email protected]> Signed-off-by: Peter Zijlstra (Intel) <[email protected]> Link: https://lkml.kernel.org/r/[email protected]
1 parent 247f34f commit d0c0064

File tree

1 file changed

+2
-6
lines changed

1 file changed

+2
-6
lines changed

kernel/jump_label.c

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -115,8 +115,6 @@ EXPORT_SYMBOL_GPL(static_key_count);
115115

116116
void static_key_slow_inc_cpuslocked(struct static_key *key)
117117
{
118-
int v, v1;
119-
120118
STATIC_KEY_CHECK_USE(key);
121119
lockdep_assert_cpus_held();
122120

@@ -132,11 +130,9 @@ void static_key_slow_inc_cpuslocked(struct static_key *key)
132130
* so it counts as "enabled" in jump_label_update(). Note that
133131
* atomic_inc_unless_negative() checks >= 0, so roll our own.
134132
*/
135-
for (v = atomic_read(&key->enabled); v > 0; v = v1) {
136-
v1 = atomic_cmpxchg(&key->enabled, v, v + 1);
137-
if (likely(v1 == v))
133+
for (int v = atomic_read(&key->enabled); v > 0; )
134+
if (likely(atomic_try_cmpxchg(&key->enabled, &v, v + 1)))
138135
return;
139-
}
140136

141137
jump_label_lock();
142138
if (atomic_read(&key->enabled) == 0) {

0 commit comments

Comments
 (0)