pf: Stop using net_epoch to synchronize access to eth rules
Commit 20c4899a8eea4 modified pf_test_eth_rule() to not acquire the
rules read lock, so pf_commit_eth() was changed to wait until the
now-inactive rules are no longer in use before freeing them. In
particular, it uses the net_epoch to schedule callbacks once the
inactive rules are no longer visible to packet processing threads.
However, since commit 812839e5aaaf4, pf_test_eth_rule() acquires the
rules read lock, so this deferred action is unneeded. This patch
reverts a portion of 20c4899a8eea4 such that we avoid using deferred
callbacks to free inactive rules.
The main motivation is performance: epoch_drain_callbacks() is quite
slow, especially on busy systems, and its use in the DIOCXBEGIN handler
in particular causes long stalls in relayd when reloading configuration.
Reviewed by: kp
MFC after: 2 weeks
Sponsored by: Klara, Inc.
Sponsored by: Modirum MDPay
Differential Revision: https://reviews.freebsd.org/D48822