net/kafka: update to v3.6.1
ChangeLog: https://downloads.apache.org/kafka/3.6.1/RELEASE_NOTES.html
Improvement
- In Java-client, backoff should be skipped for retried producer-batch to a new leader
- Upgrade ZooKeeper to 3.8.3
Bug
- block-cache-capacity metrics worth twice as much as normal
- Gradle build fails with missing commitId after git gc
- Concurrency bug in RemoteIndexCache leads to IOException
- RackId doesn't exist error while running WordCountDemo
- Handle large keystores in SslEngineValidator
- Duplicate Producer ID blocks during ZK migration
- StateRestoreListener#onRestoreSuspended is never called because wrapper DelegatingStateRestoreListener doesn't implement onRestoreSuspended
- Breaking change in 3.4.0 ByteBufferSerializer
- Topics marked for deletion in ZK are incorrectly migrated to KRaft
- Possible NPE is thrown in MirrorCheckpointTask
- Fix CVE-2023-4586 in netty:handler
- NPE in ChunkedByteStream
- Zookeeper.jar | CVE-2023-44981
- Partition-Count is not getting updated Correctly in the Incremental Co-operative Rebalancing(ICR) Mode of Rebalancing
- Disabling scheduled rebalance delay in Connect can lead to indefinitely unassigned connectors and tasks
- LeaveGroupResponse v0-v2 should handle no members
- ProduceRequest#partitionSizes() is not an atomic operation
- ZK brokers incorrectly handle KRaft metadata snapshots
- Malformed connect source offsets corrupt other partitions with DataException
- Trying to access uncopied segments metadata on listOffsets
- KRaft controller writes empty state to ZK after migration
Task
- Add 3.5.0 to broker/client and streams upgrade/compatibility tests
- Rolling upgrade system tests are failing
- Remote log segments should be considered once for retention breach
- Add 3.4.0 streams upgrade/compatibility tests
Test
- Add tests for RemoteIndexCache
- Flaky test ZkMigrationIntegrationTest.testMigrateTopicDeletions
PR: 275611
Reported by: timp87@gmail.com
MFH: 2023Q4 (security fix)
Security: CVE-2023-4586
(cherry picked from commit 452f0014ea7e0a1495dbbd29ae30955ca7d7f1f0)