Linux kernel source tree
Find a file
Peter Zijlstra 570c8efd5e sched/psi: Optimize psi_group_change() cpu_clock() usage
Dietmar reported that commit 3840cbe24c ("sched: psi: fix bogus
pressure spikes from aggregation race") caused a regression for him on
a high context switch rate benchmark (schbench) due to the now
repeating cpu_clock() calls.

In particular the problem is that get_recent_times() will extrapolate
the current state to 'now'. But if an update uses a timestamp from
before the start of the update, it is possible to get two reads
with inconsistent results. It is effectively back-dating an update.

(note that this all hard-relies on the clock being synchronized across
CPUs -- if this is not the case, all bets are off).

Combine this problem with the fact that there are per-group-per-cpu
seqcounts, the commit in question pushed the clock read into the group
iteration, causing tree-depth cpu_clock() calls. On architectures
where cpu_clock() has appreciable overhead, this hurts.

Instead move to a per-cpu seqcount, which allows us to have a single
clock read for all group updates, increasing internal consistency and
lowering update overhead. This comes at the cost of a longer update
side (proportional to the tree depth) which can cause the read side to
retry more often.

Fixes: 3840cbe24c ("sched: psi: fix bogus pressure spikes from aggregation race")
Reported-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Tested-by: Dietmar Eggemann <dietmar.eggemann@arm.com>,
Link: https://lkml.kernel.org/20250522084844.GC31726@noisy.programming.kicks-ass.net
2025-07-09 13:40:21 +02:00
arch The delayed from_timer() API cleanup: 2025-06-08 11:33:00 -07:00
block treewide, timers: Rename from_timer() to timer_container_of() 2025-06-08 09:07:37 +02:00
certs sign-file,extract-cert: use pkcs11 provider for OPENSSL MAJOR >= 3 2024-09-20 19:52:48 +03:00
crypto EFI updates for v6.16 2025-05-30 12:42:57 -07:00
Documentation fourteen smb3 client fixes, most smbdirect related 2025-06-08 10:20:21 -07:00
drivers The delayed from_timer() API cleanup: 2025-06-08 11:33:00 -07:00
fs The delayed from_timer() API cleanup: 2025-06-08 11:33:00 -07:00
include sched/psi: Optimize psi_group_change() cpu_clock() usage 2025-07-09 13:40:21 +02:00
init rust: Introduce file_from_location() 2025-06-24 15:53:46 -07:00
io_uring io_uring-6.16-20250606 2025-06-06 13:09:03 -07:00
ipc - The 3 patch series "hung_task: extend blocking task stacktrace dump to 2025-05-31 19:12:53 -07:00
kernel sched/psi: Optimize psi_group_change() cpu_clock() usage 2025-07-09 13:40:21 +02:00
lib 13 hotfixes. 6 are cc:stable and the remainder address post-6.15 issues 2025-06-06 21:45:45 -07:00
LICENSES LICENSES: add CC0-1.0 license text 2025-05-21 14:54:17 +02:00
mm treewide, timers: Rename from_timer() to timer_container_of() 2025-06-08 09:07:37 +02:00
net treewide, timers: Rename from_timer() to timer_container_of() 2025-06-08 09:07:37 +02:00
rust rust: task: Add Rust version of might_sleep() 2025-06-24 15:53:50 -07:00
samples - The 3 patch series "hung_task: extend blocking task stacktrace dump to 2025-05-31 19:12:53 -07:00
scripts Kbuild updates for v6.16 2025-06-07 10:05:35 -07:00
security require gcc-8 and binutils-2.30 2025-05-31 08:16:52 -07:00
sound treewide, timers: Rename from_timer() to timer_container_of() 2025-06-08 09:07:37 +02:00
tools turbostat v2025.06.08 2025-06-08 11:44:41 -07:00
usr usr/include: openrisc: don't HDRTEST bpf_perf_event.h 2025-05-12 15:03:17 +09:00
virt Merge branch 'kvm-lockdep-common' into HEAD 2025-05-28 06:29:17 -04:00
.clang-format Linux 6.15-rc5 2025-05-06 16:39:25 +10:00
.clippy.toml rust: clean Rust 1.88.0's warning about clippy::disallowed_macros configuration 2025-05-07 00:11:47 +02:00
.cocciconfig scripts: add Linux .cocciconfig for coccinelle 2016-07-22 12:13:39 +02:00
.editorconfig .editorconfig: remove trim_trailing_whitespace option 2024-06-13 16:47:52 +02:00
.get_maintainer.ignore MAINTAINERS: Retire Ralf Baechle 2024-11-12 15:48:59 +01:00
.gitattributes .gitattributes: set diff driver for Rust source code files 2023-05-31 17:48:25 +02:00
.gitignore .gitignore: ignore Python compiled bytecode 2025-04-24 10:12:46 -06:00
.mailmap fourteen smb3 client fixes, most smbdirect related 2025-06-08 10:20:21 -07:00
.pylintrc docs: add a .pylintrc file with sys path for docs scripts 2025-04-09 12:10:33 -06:00
.rustfmt.toml rust: add .rustfmt.toml 2022-09-28 09:02:20 +02:00
COPYING COPYING: state that all contributions really are covered by this file 2020-02-10 13:32:20 -08:00
CREDITS Update Christoph's Email address and make it consistent 2025-05-12 23:50:31 -07:00
Kbuild drm: ensure drm headers are self-contained and pass kernel-doc 2025-02-12 10:44:43 +02:00
Kconfig io_uring: Rename KConfig to Kconfig 2025-02-19 14:53:27 -07:00
MAINTAINERS fourteen smb3 client fixes, most smbdirect related 2025-06-08 10:20:21 -07:00
Makefile Linux 6.16-rc1 2025-06-08 13:44:43 -07:00
README README: Fix spelling 2024-03-18 03:36:32 -06:00

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the reStructuredText markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.