mirror of
https://github.com/torvalds/linux.git
synced 2026-03-08 04:04:43 +01:00
b3d99f43c7
It turns out that zero_vruntime tracking is broken when there is but a single
task running. Current update paths are through __{en,de}queue_entity(), and
when there is but a single task, pick_next_task() will always return that one
task, and put_prev_set_next_task() will end up in neither function.
This can cause entity_key() to grow indefinitely large and cause overflows,
leading to much pain and suffering.
Furtermore, doing update_zero_vruntime() from __{de,en}queue_entity(), which
are called from {set_next,put_prev}_entity() has problems because:
- set_next_entity() calls __dequeue_entity() before it does cfs_rq->curr = se.
This means the avg_vruntime() will see the removal but not current, missing
the entity for accounting.
- put_prev_entity() calls __enqueue_entity() before it does cfs_rq->curr =
NULL. This means the avg_vruntime() will see the addition *and* current,
leading to double accounting.
Both cases are incorrect/inconsistent.
Noting that avg_vruntime is already called on each {en,de}queue, remove the
explicit avg_vruntime() calls (which removes an extra 64bit division for each
{en,de}queue) and have avg_vruntime() update zero_vruntime itself.
Additionally, have the tick call avg_vruntime() -- discarding the result, but
for the side-effect of updating zero_vruntime.
While there, optimize avg_vruntime() by noting that the average of one value is
rather trivial to compute.
Test case:
# taskset -c -p 1 $$
# taskset -c 2 bash -c 'while :; do :; done&'
# cat /sys/kernel/debug/sched/debug | awk '/^cpu#/ {P=0} /^cpu#2,/ {P=1} {if (P) print $0}' | grep -e zero_vruntime -e "^>"
PRE:
.zero_vruntime : 31316.407903
>R bash 487 50787.345112 E 50789.145972 2.800000 50780.298364 16 120 0.000000 0.000000 0.000000 /
.zero_vruntime : 382548.253179
>R bash 487 427275.204288 E 427276.003584 2.800000 427268.157540 23 120 0.000000 0.000000 0.000000 /
POST:
.zero_vruntime : 17259.709467
>R bash 526 17259.709467 E 17262.509467 2.800000 16915.031624 9 120 0.000000 0.000000 0.000000 /
.zero_vruntime : 18702.723356
>R bash 526 18702.723356 E 18705.523356 2.800000 18358.045513 9 120 0.000000 0.000000 0.000000 /
Fixes:
|
||
|---|---|---|
| .. | ||
| autogroup.c | ||
| autogroup.h | ||
| build_policy.c | ||
| build_utility.c | ||
| clock.c | ||
| completion.c | ||
| core.c | ||
| core_sched.c | ||
| cpuacct.c | ||
| cpudeadline.c | ||
| cpudeadline.h | ||
| cpufreq.c | ||
| cpufreq_schedutil.c | ||
| cpupri.c | ||
| cpupri.h | ||
| cputime.c | ||
| deadline.c | ||
| debug.c | ||
| ext.c | ||
| ext.h | ||
| ext_idle.c | ||
| ext_idle.h | ||
| ext_internal.h | ||
| fair.c | ||
| features.h | ||
| idle.c | ||
| isolation.c | ||
| loadavg.c | ||
| Makefile | ||
| membarrier.c | ||
| pelt.c | ||
| pelt.h | ||
| psi.c | ||
| rq-offsets.c | ||
| rt.c | ||
| sched-pelt.h | ||
| sched.h | ||
| smp.h | ||
| stats.c | ||
| stats.h | ||
| stop_task.c | ||
| swait.c | ||
| syscalls.c | ||
| topology.c | ||
| wait.c | ||
| wait_bit.c | ||