HomeFreeBSD

AF_UNIX: make unix socket locking finer grained

Description

AF_UNIX: make unix socket locking finer grained

This change moves to using a reference count across lock drop / reacquire
to guarantee liveness.

Currently sends on unix sockets contend heavily on read locking the list lock.
unix1_processes in will-it-scale peaks at 6 processes and then declines.

With this change I get a substantial improvement in number of operations per second
with 96 processes:

x before
+ after

N           Min           Max        Median           Avg        Stddev

x 11 1688420 1696389 1693578 1692766.3 2971.1702
+ 10 63417955 71030114 70662504 69576423 2374684.6
Difference at 95.0% confidence

6.78837e+07 +/- 1.49463e+06
4010.22% +/- 88.4246%
(Student's t, pooled s = 1.63437e+06)

And even for 2 processes shows a ~18% improvement.
"Small" iron changes (1, 2, and 4 processes):

x before1
+ after1.2
+------------------------------------------------------------------------+

+
x +
x +
x +
x ++
xx ++
x x xx ++
__________________A_____M_____AM____

+------------------------------------------------------------------------+

N           Min           Max        Median           Avg        Stddev

x 10 1131648 1197750 1197138.5 1190369.3 20651.839
+ 10 1203840 1205056 1204919 1204827.9 353.27404
Difference at 95.0% confidence

14458.6 +/- 13723
1.21463% +/- 1.16683%
(Student's t, pooled s = 14605.2)

x before2
+ after2.2
+------------------------------------------------------------------------+

+
+
+
+
+
+
x +
x +
x xx +
x xxxx +
___AM_A

+------------------------------------------------------------------------+

N           Min           Max        Median           Avg        Stddev

x 10 1972843 2045866 2038186.5 2030443.8 21367.694
+ 10 2400853 2402196 2401043.5 2401172.7 385.40024
Difference at 95.0% confidence

370729 +/- 14198.9
18.2585% +/- 0.826943%
(Student's t, pooled s = 15111.7)

x before4
+ after4.2

N           Min           Max        Median           Avg        Stddev

x 10 3986994 3991728 3990137.5 3989985.2 1300.0164
+ 10 4799990 4806664 4806116.5 4805194 1990.6625
Difference at 95.0% confidence

815209 +/- 1579.64
20.4314% +/- 0.0421713%
(Student's t, pooled s = 1681.19)

Tested by: pho
Reported by: mjg
Approved by: sbruno
Sponsored by: Limelight Networks
Differential Revision: https://reviews.freebsd.org/D15430

Details

Provenance
mmacyAuthored on May 17 2018, 5:59 PM
Parents
rG087522b8400b: Fix build if USB_DEBUG is defined.
Branches
Unknown
Tags
Unknown