HomeFreeBSD

sched_ule: Use explicit atomic accesses for tdq fields

Description

sched_ule: Use explicit atomic accesses for tdq fields

Different fields in the tdq have different synchronization protocols.
Some are constant, some are accessed only while holding the tdq lock,
some are modified with the lock held but accessed without the lock, some
are accessed only on the tdq's CPU, and some are not synchronized by the
lock at all.

Convert ULE to stop using volatile and instead use atomic_load_* and
atomic_store_* to provide the desired semantics for lockless accesses.
This makes the intent of the code more explicit, gives more freedom to
the compiler when accesses do not need to be qualified, and lets KCSAN
intercept unlocked accesses.

Thus:

  • Introduce macros to provide unlocked accessors for certain fields.
  • Use atomic_load/store for all accesses of tdq_cpu_idle, which is not synchronized by the mutex.
  • Use atomic_load/store for accesses of the switch count, which is updated by sched_clock().
  • Add some comments to fields of struct tdq describing how accesses are synchronized.

No functional change intended.

Reviewed by: mav, kib
Sponsored by: The FreeBSD Foundation

(cherry picked from commit 11484ad8a2b01049b3e4f25c0fae6041c2060629)

Details

Provenance
markjAuthored on Jul 14 2022, 2:43 PM
Parents
rG84a0d34d10ac: x86: Add a required store-load barrier in cpu_idle()
Branches
Unknown
Tags
Unknown