Skip to content

Commit a11d678

Browse files
arighihtejun
authored andcommitted
sched_ext: Fix missing rq lock in scx_bpf_cpuperf_set()
scx_bpf_cpuperf_set() can be used to set a performance target level on any CPU. However, it doesn't correctly acquire the corresponding rq lock, which may lead to unsafe behavior and trigger the following warning, due to the lockdep_assert_rq_held() check: [ 51.713737] WARNING: CPU: 3 PID: 3899 at kernel/sched/sched.h:1512 scx_bpf_cpuperf_set+0x1a0/0x1e0 ... [ 51.713836] Call trace: [ 51.713837] scx_bpf_cpuperf_set+0x1a0/0x1e0 (P) [ 51.713839] bpf_prog_62d35beb9301601f_bpfland_init+0x168/0x440 [ 51.713841] bpf__sched_ext_ops_init+0x54/0x8c [ 51.713843] scx_ops_enable.constprop.0+0x2c0/0x10f0 [ 51.713845] bpf_scx_reg+0x18/0x30 [ 51.713847] bpf_struct_ops_link_create+0x154/0x1b0 [ 51.713849] __sys_bpf+0x1934/0x22a0 Fix by properly acquiring the rq lock when possible or raising an error if we try to operate on a CPU that is not the one currently locked. Fixes: d86adb4 ("sched_ext: Add cpuperf support") Signed-off-by: Andrea Righi <[email protected]> Acked-by: Changwoo Min <[email protected]> Signed-off-by: Tejun Heo <[email protected]>
1 parent 18853ba commit a11d678

File tree

1 file changed

+23
-4
lines changed

1 file changed

+23
-4
lines changed

kernel/sched/ext.c

Lines changed: 23 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7113,13 +7113,32 @@ __bpf_kfunc void scx_bpf_cpuperf_set(s32 cpu, u32 perf)
71137113
}
71147114

71157115
if (ops_cpu_valid(cpu, NULL)) {
7116-
struct rq *rq = cpu_rq(cpu);
7116+
struct rq *rq = cpu_rq(cpu), *locked_rq = scx_locked_rq();
7117+
struct rq_flags rf;
7118+
7119+
/*
7120+
* When called with an rq lock held, restrict the operation
7121+
* to the corresponding CPU to prevent ABBA deadlocks.
7122+
*/
7123+
if (locked_rq && rq != locked_rq) {
7124+
scx_ops_error("Invalid target CPU %d", cpu);
7125+
return;
7126+
}
7127+
7128+
/*
7129+
* If no rq lock is held, allow to operate on any CPU by
7130+
* acquiring the corresponding rq lock.
7131+
*/
7132+
if (!locked_rq) {
7133+
rq_lock_irqsave(rq, &rf);
7134+
update_rq_clock(rq);
7135+
}
71177136

71187137
rq->scx.cpuperf_target = perf;
7138+
cpufreq_update_util(rq, 0);
71197139

7120-
rcu_read_lock_sched_notrace();
7121-
cpufreq_update_util(cpu_rq(cpu), 0);
7122-
rcu_read_unlock_sched_notrace();
7140+
if (!locked_rq)
7141+
rq_unlock_irqrestore(rq, &rf);
71237142
}
71247143
}
71257144

0 commit comments

Comments
 (0)