cxgbe(4): Allow a T6 adapter to switch between TOE and NIC TLS mode.
The hw.cxgbe.kern_tls tunable was used for this in the past and if it
was set then all T6 adapters would be configured for NIC TLS operation
and could not be reconfigured for TOE without a reload. With this
change ifconfig can be used to manipulate toe and txtls caps like any
other caps. hw.cxgbe.kern_tls continues to work as usual but its
effects are not permanent any more.
- Enable nic_ktls_ofld in the default configuration file and use the firmware instead of direct register manipulation to apply/rollback NIC TLS configuration. This allows the driver to switch the hardware between TOE and NIC TLS mode in a safe manner. Note that the configuration is adapter-wide and not per-port.
- Remove the kern_tls config file as it works with 100G T6 cards only and leads to firmware crashes with 25G cards. The configurations included with the driver (with the exception of the FPGA configs) are supposed to work with all adapters.
Reported by: Veeresh U.K. at Chelsio
Sponsored by: Chelsio Communications
Reviewed by: jhb@
Differential Revision: https://reviews.freebsd.org/D29291
(cherry picked from commit 15f33555678300953858f6ed98dfc72c399a9139)