epair: Simplify the transmit path and address lost wakeups
epairs currently shuttle all transmitted packets through a single global
taskqueue thread. To hand packets over to the taskqueue thread, each
epair maintains a pair of ring buffers and a lockless scheme for
notifying the thread of pending work. The implementation can lead to
lost wakeups, causing to-be-transmitted packets to end up stuck in the
queue.
Rather than extending the existing scheme, simply replace it with a
linked list protected by a mutex, and use the mutex to synchronize
wakeups of the taskqueue thread. This appears to give equivalent or
better throughput with >= 16 producer threads and eliminates the lost
wakeups.
Reviewed by: kp
MFC after: 1 week
Sponsored by: Klara, Inc.
Sponsored by: Modirum MDPay
Differential Revision: https://reviews.freebsd.org/D38843