HomeFreeBSD

Make nvme(4) driver some more NUMA aware.

Description

Make nvme(4) driver some more NUMA aware.

  • For each queue pair precalculate CPU and domain it is bound to.

If queue pairs are not per-CPU, then use the domain of the device.

  • Allocate most of queue pair memory from the domain it is bound to.
  • Bind callouts to the same CPUs as queue pair to avoid migrations.
  • Do not assign queue pairs to each SMT thread. It just wasted

resources and increased lock congestions.

  • Remove fixed multiplier of CPUs per queue pair, spread them even.

This allows to use more queue pairs in some hardware configurations.

  • If queue pair serves multiple CPUs, bind different NVMe devices to

different CPUs.

MFC after: 1 month
Sponsored by: iXsystems, Inc.

Details

Provenance
mavAuthored on Sep 23 2019, 5:53 PM
Parents
rG9093dd9a6611: Implement x86 dtrace_invop_(un)init() in C.
Branches
Unknown
Tags
Unknown

Event Timeline