Page MenuHomeFreeBSD

ctld: Add support for NVMe over Fabrics
Needs ReviewPublic

Authored by jhb on Fri, Jan 31, 8:16 PM.
Tags
None
Referenced Files
F109356852: D48773.id150295.diff
Tue, Feb 4, 12:11 AM
F109350113: D48773.diff
Mon, Feb 3, 10:03 PM
F109347426: D48773.id150294.diff
Mon, Feb 3, 9:16 PM
F109338274: D48773.id.diff
Mon, Feb 3, 6:36 PM
Subscribers

Details

Reviewers
asomers
mav
trasz
Summary

While the overall structure is similar for NVMeoF controllers and
iSCSI targets, there are a sufficient differences that NVMe support
uses alternate configuration syntax.

  • In authentication groups, permitted NVMeoF hosts can be allowed by names (NQNs) via "host-nqn" values (similar to "initiator-name" for iSCSI). Similarly, "host-address" accepts permitted host addresses similar to "initiator-portal" for iSCSI.
  • A new "transport-group" context enumerates transports that can be used by a group of NVMeoF controllers similar to the "portal-group" context for iSCSI. In this section, the "listen" keyword accepts a transport as well as an address to permit other types of transports besides TCP in the future. The "foreign", "offload", and "redirect" keywords are also not meaningful and thus not supported.
  • A new "controller" context describes an NVMeoF I/O controller similar to the "target" context for iSCSI. One key difference here is that "lun" objects are replaced by "namespace" objects. However, a "namespace" can reference a named global lun permitting LUNs to be shared between iSCSI targets and NVMeoF controllers.

NB: Authentication via CHAP is not implemented for NVMeoF.

Sponsored by: Chelsio Communications

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 62179
Build 59063: arc lint + arc unit

Event Timeline

jhb requested review of this revision.Fri, Jan 31, 8:16 PM

Update manpage for change to default TCP port

As an example, this is the config file I used for testing. It exports the same LUNs via both iSCSI and NVMe (and I can connect to both at the same time as you can see from my ctladm output over in D48776.

lun zvol {
	path /dev/zvol/bhyve/iscsi
	size 4G
}

lun ramdisk {
	backend ramdisk
	size 1G
	option capacity 1G
}

portal-group pg0 {
	discovery-auth-group no-authentication
	listen 0.0.0.0
	listen [::]
	offload cxgbei
}

target iqn.2001-03.com.chelsio:frodo0 {
	auth-group no-authentication
	portal-group pg0
	lun 0 zvol
	lun 1 ramdisk
}

controller nqn.2001-03.com.chelsio:frodo0 {
	auth-group no-authentication
	namespace 1 zvol
	namespace 2 ramdisk
}

In this case, the portal group is only used to enable the cxgbei offload.

usr.sbin/ctld/nvme.c
219

Perhaps I should be adding a ctld_transport_group_name key similar to the ctld_portal_group_name used for iSCSI.