pf: do not block new Ethernet rules for in-progress transactions

Make Ethernet rule addition behave just like L3 rules, in that we now
allow ongoing transaction to be interrupted, rather than rejecting a new
one.

The result of that is that we can no longer end up in a state where a
transaction failed, but was not rolled back, blocking us from setting
new rules.

It's safe to assume there's no pending epoch callback for cleanup here,
because we've explicitly called it before hitting pf_begin_eth().

Sponsored by: Rubicon Communications, LLC ("Netgate")