Virtualise if_vxlan

[zlei]

Summary

This is the last driver which lacks VNET support. With this change it will be practical to do tests via test framework. This should also fix the long standing PR 255882.

Test Plan

1. 1. stress test with iperf3, with different vxlan interface in different jails which connected with epair(4).
2. test vmove, aka ifconfig vxlan0 vnet foo, ifconfig vxlan0 -vnet foo.
3. test vxlan hardware offload ( no hardware available right now )
4. create and config vxlan in vnet jail. Destroy the jail ( jail -R foo ).

[zlei]

Current know problem, the fourth test,

create and config vxlan in vnet jail. Destroy the jail ( jail -R foo ).

may panic, or block the jail from destroying, or the jail keep in dying state ( jls -d ).

[afedorov]

It won't work becaus, if_vxlan(4) exports various statistics and tables via sysctl. But for VNET we can only change the values, and not the tree itself.

The problem is somewhat broader.

[afedorov]

sys/net/if_vxlan.c

3582

We can’t move vxlan to the jail., because vxl_unit is the same.

[zlei]

In D48646#1109329, @afedorov wrote:

It won't work becaus, if_vxlan(4) exports various statistics and tables via sysctl. But for VNET we can only change the values, and not the tree itself.

The problem is somewhat broader.

Yeah, this is another known issue, I forgot to mention this. I'm proposing to have a global unique number for those dynamically added sysctl nodes, but that apparently may leak sensitive info , such as forwarding table entries. Or we should apply restrictions for that, but I have no idea how to restrict that.

[zlei]

sys/net/if_vxlan.c

3582

ifnet vmove is another known problem. For vxlan(4), it appears less useful to move an interface to another vnet, given the cloner has been already virtualized and you can attache to that jail / vnet to clone create a new one.

I once have the idea to introduce a cloner flag, say IFC_F_NO_VMOVE to forbid some clone create interfaces to be moved. Known good candidates are if_pflog, if_pfsync, and if_lo. Tunnel drivers such as if_gif / if_gre / if_vxlan may also have this restriction, so we no longer need if_reassign handler.

[zlei]

sys/net/if_vxlan.c

3582

We can’t move vxlan to the jail., because vxl_unit is the same.

See vxlan_reassign(), the interface to be vmoved was reset to factory default status, aka no vni, vxlan local or vxlan remote .