sched_ule: Use explicit atomic accesses for tdq fields
Different fields in the tdq have different synchronization protocols.
Some are constant, some are accessed only while holding the tdq lock,
some are modified with the lock held but accessed without the lock, some
are accessed only on the tdq's CPU, and some are not synchronized by the
lock at all.
Convert ULE to stop using volatile and instead use atomic_load_* and
atomic_store_* to provide the desired semantics for lockless accesses.
This makes the intent of the code more explicit, gives more freedom to
the compiler when accesses do not need to be qualified, and lets KCSAN
intercept unlocked accesses.
Thus:
- Introduce macros to provide unlocked accessors for certain fields.
- Use atomic_load/store for all accesses of tdq_cpu_idle, which is not synchronized by the mutex.
- Use atomic_load/store for accesses of the switch count, which is updated by sched_clock().
- Add some comments to fields of struct tdq describing how accesses are synchronized.
No functional change intended.
Reviewed by: mav, kib
Sponsored by: The FreeBSD Foundation
(cherry picked from commit 11484ad8a2b01049b3e4f25c0fae6041c2060629)