ibcore: Kernel space update based on Linux 5.7-rc1

[• hselasky]

Summary

Overview:

This is the first stage of a RDMA stack upgrade introducing kernel
changes only based on Linux 5.7-rc1.

This patch is based on about four main areas of work:

• Update of the IB uobjects system:
  • The memory holding so-called AH, CQ, PD, SRQ and UCONTEXT objects is now managed by ibcore. This also require some changes in the kernel verbs API. The updated verbs changes are typically about initialize and deinitialize objects, and remove allocation and free of memory.

• Update of the uverbs IOCTL framework:
  • The parsing and handling of user-space commands has been completely refactored to integrate with the updated IB uobjects system.

• Various changes and updates to the generic uverbs interfaces in device drivers including the new uAPI surface.

• The mlx5_ib_devx.c in mlx5ib and related mlx5 core changes.

Dependencies:

• The mlx4ib driver code has been updated with the minimum changes

needed.

• The mlx5ib driver code has been updated with the minimum changes

needed including DV support.

Compatibility:

• All user-space facing APIs are backwards compatible after this change.

• All kernel-space facing RDMA APIs are backwards compatible after this change, with exception of ib_create_ah() and ib_destroy_ah() which takes a new flag.

• The "ib_device_ops" structure exist, but only contains the driver ID and some structure sizes.

Differences from Linux:

• Infiniband drivers must use the INIT_IB_DEVICE_OPS() macro to set the sizes needed for allocating various IB objects, when adding IB device instances.

Security:

• PRIV_NET_RAW is needed to use raw ethernet transmit features.
• PRIV_DRIVER is needed to use other privileged operations.

Based on upstream Linux, Torvalds (5.7-rc1):
8632e9b5645bbc2331d21d892b0d6961c1a08429

MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking

Test Plan

Navdeep: You need to test the driver, especially the mmap part of the CXGBE driver.

QLNXR has no maintainer? Who can test?

[• hselasky]

@np: I'd like to push this by end of week. Please respond!

[np]

In D31149#701525, @hselasky wrote:

@np: I'd like to push this by end of week. Please respond!

I'm planning to try this patch sometime this week.

Do the iw_cxgbe changes come from upstream iw_cxgb4 in k.org or is this new code? I'm trying to figure out whether I should review them myself or get one of the iw_cxgb4 maintainers to take a look.

[• hselasky]

Do the iw_cxgbe changes come from upstream iw_cxgb4 in k.org or is this new code?

Yes, the come from the Linux version I've referred to, but not a specific commit.

[• hselasky]

@np: Feel free to ask questions, if you have any.

[• hselasky]

sys/dev/cxgbe/iw_cxgbe/provider.c

419

This part is to avoid too many changes to the driver.
In the future the device ops structure will be implemented.

[• hselasky]

Hi Navdeep,

NVidia is planning to merge the infiniband update* on Thursday 22nd of July somewhere after 10:00 CEST.

Please let us know your status, and if you have any patches that we should merge as part of this update.

[np]

sys/dev/cxgbe/iw_cxgbe/cq.c

898–899

No need to check for udata before calling rdma_udata_to_drv_context.

924–925

No need to check for udata before calling rdma_udata_to_drv_context.

sys/dev/cxgbe/iw_cxgbe/provider.c

254–256

Fix KTR build.

[• hselasky]

@np: Is is possible you could send the changes as an incremental diff to: [e-mail] ?

[np]

In D31149#703220, @hselasky wrote:

@np: Is is possible you could send the changes as an incremental diff to: [e-mail] ?

sure, I'll send email.

[• hselasky]

Update patch.

[• hselasky]

@np: I've included your patches now. Do you have more? I know at least the mmap support needs a check.

[erj]

Did this ever get MFCed to stable/13? (and is it planned to get MFCed to stable/12?)

[• hselasky]

It was not supposed to get MFC'ed, because it breaks a lot of APIs.