What is netlink?
Netlinks is a communication protocol currently used in Linux kernel to modify, read and subscribe for nearly all networking state. Interface state, addresses, routes, firewall, rules, fibs, etc are controlled via netlink.
It is async, TLV-based protocol, providing 1-1 and 1-many communications.
Why netlink is important for FreeBSD?
POSIX defined API for base functions/system calls. There is no such standard for plethora of various protocol/device-level/subsystem-level ioctls. Each subsystem/driver invents its own protocol, handling format and compatibility.
Netlink changes that by providing standard communication layer and basic extendable message formatting. It can serve as a "broker", automatically combining requested data from different sources in a single request (example: interface state dump).
For example, devd can be easily switch to use netlink, retiring one-off protocol. Tools like jail, pfilctl, can be converted to use netlink instead of a bunch of private ioctls. It will be easier for app developers to interact with our network stack.
Immediate drivers for netlink
Nexthop and nexthop-group-related changes in the routing stack opened the way for more effective and feature-rich route-related interaction between userland and kernel. Extending our existing protocol, rtsock(4) is not easy - to provide efficient multipath signalling, one need to introduce a new type of messages, other that RTM_ADD/RTM_DEL for the purposes of signalling route changes. I did a test implementation with extended rtsock and net/bird. De-facto it ended up being an own TLV-based protocol, sharing nothing with rtsock except the socket and base message header. Pushing out a new protocol, which is not even shared by other BSDs doesn't look promising.
Instead, netlink was chosen as a transport.
Implementation overview
Initial implementation was written in GSoC 2021, based on Luigi's work in 2015. It was not possible to write a full-featured netlink implementation in the time allocated for GSoC, especially given the fact that it was shorter than the previous GSoCs. As a result, initial implementation delivered some working code core code, allowing to use netlink sockets in kernel and perform route table manipulations. The code in this diff derives from this implementation, but has largely been rewritten to address large netlink message support, large dumps support, locking, sockets/vnet specifics and so on.
Netlink is implemented via loadable/unloadable kernel module, not touching many kernel parts.
To support async operation handling such as interface creation, dedicated tasqueue is created for each netlink socket. All message processing is handled within these task queues.
Handling messages to/from Linux processes requires their modification (address families and rtableid rewrites), which is performed by the transparent intercept layer. Its functionality allows to rewrite messages, including full message reconstruction.
What works
- Dumps:
- routes
- nexhtops / nexthop groups
- interfaces
- interface addresses
- neighbours (arp/ndp)
- genetlink families & ops
- Notifications
- interface arrival/departure
- interface address arrival/departure
- route addition/deletion (from kernel, netlink and rtsock)
- Modifications
- adding/deleting routes
- adding/deleting nexthops/nexthop groups
- adding/deleting neighbours (arp/ndp)
- adding/deleting interfaces (only basic version, no properties/updates supported)
Next steps
To add nhop/nhg supportTo add interface change notificationsTo add rtsock -> netlink notificationsTo add netlink mpath route creation w/o user nexthop groupsTo add generic netlinkTo add ext ack supportSimplify netlink<>rtsock bridgeNeighbour notifications- Ifaddr change notifications
- Genetlink dummy module
- Tests
- Manual pages