For discussion. There are two parts in the change: kernel and userland.
The kernel part is not functionally necessary, but a proof of concept of using SSE2 which may or may not be more performant than the current CMPXCHG8B implementation, but doesn't have the drawback of faulting on read-only mappings. It has been tested in a VM.
The userland part is more straightforward (for reasons indicated in the commit message). The produced assembly code has been verified correct. I didn't create an atomic_load_acq_64() wrapper yet, since there is no pure userland implementation possible for processors before the Pentium/i586. The intent would be to actually rename this userland atomic_load_acq_64_sse2() to atomic_load_acq_64() once support i386 is dropped from the kernel, meaning we are guaranteed to run on at least SSE2-capable processors.
Current commit message:
The current (kernel) implementation, for i586 processors or higher, uses
CMPXCHG8B (with LOCK prefix) which also may write to the destination
(the same value as was read). While this write is invisible to the
C abstract machine, it causes a #GP(0) on read-only mappings, which is
very surprising for an atomic load operation (no other architecture has
this peculiarity).
For i586 processors, an alternative could be using FILD, but that
requires FPU state saving. Given that we are very likely to stop
supporting the i386 architecture in the kernel in 15.0, and that uses in
the kernel have been adapted to avoid the above problem, we didn't
change atomic_load_acq_64_i586().
We still propose an alternative for processors supporting SSE2, using
MOVQ with XMM registers, as the new atomic_load_acq_64_sse2(). It needs
to clear CR0_TS but avoids saving the full FPU/SSE/AVX state by
temporarily storing into memory the original content of the XMM scratch
register it requires. Provided fiddling with CR0 is not too costly,
this variant, especially on older processors, should be visibly faster
than using CMPXCHG8B with LOCK prefix.
Plug atomic_load_acq_64_sse2() into atomic_load_acq_64 when CPUID_SSE2
is set. Add 'npx.h' to headers to copy from i386 on amd64 to avoid
a compilation error in 32-bit procstat's ZFS support (which needs to
include headers with _KERNEL).
As we are going to continue supporting i386 userland on amd64 and
because all amd64 processors support SSE2, it is now possible to provide
a userland implementation for atomic_load_acq_64()/atomic_load_64(). It
is much simpler than the kernel variant as saving the FPU/SSE/AVX state
will be taken care of by the kernel as needed, but also likely slower if
the corresponding thread does not already use the FPU/SSE/AVX (or just
SSE, if XSAVEOPT can track modification for just that state) instruction
sets.