socket: Move SO_SETFIB handling to protocol layers

[markj]

Summary

In particular, we store a FIB number in both struct socket and in struct
inpcb. When updating the FIB number with setsockopt(SO_SETFIB), make
the update atomic. This is required to support the new bind_all_fibs
mode, since in that mode changing the FIB of a bound socket is not
permitted.

This requires a bit more code, but avoids a layering violation in
sosetopt(), where we hard-code the list of protocol families that
implement SO_SETFIB.

Sponsored by: Klara, Inc.
Sponsored by: Stormshield

[glebius]

The change definitely improves, but some questions remain :) Why do we need to store fib both in socket and inpcb? I believe it belongs only to the inpcb, cause many sockets (unix, netlink, ...) doesn't have a notion of forwarding. The rtsock(4) doesn't have notion of forwarding, but we use it configure other protocols forwarding, hence it needs fib. But it has its own pcb that of course can store fib.

[markj]

In D48666#1109935, @glebius wrote:

The change definitely improves, but some questions remain :) Why do we need to store fib both in socket and inpcb? I believe it belongs only to the inpcb, cause many sockets (unix, netlink, ...) doesn't have a notion of forwarding. The rtsock(4) doesn't have notion of forwarding, but we use it configure other protocols forwarding, hence it needs fib. But it has its own pcb that of course can store fib.

From what I can see, there is no really good reason to have it in the socket as well. I agree that we should (eventually) remove so_fibnum, so that there's only a single source of truth. I will try adding a mechanism to get the current FIB as well (i.e., a new sockopt).

[markj]

Update raw socket option handlers as well.

Also fix a small bug: rip(6)_ctloutput() wasn't checking whether it was
handling a SOPT_GET or an SOPT_SET in the SO_SETFIB handler.