HomeFreeBSD

Always validate checksums for Direct I/O reads

Description

Always validate checksums for Direct I/O reads

This fixes an oversight in the Direct I/O PR. There is nothing that
stops a process from manipulating the contents of a buffer for a
Direct I/O read while the I/O is in flight. This can lead checksum
verify failures. However, the disk contents are still correct, and this
would lead to false reporting of checksum validation failures.

To remedy this, all Direct I/O reads that have a checksum verification
failure are treated as suspicious. In the event a checksum validation
failure occurs for a Direct I/O read, then the I/O request will be
reissued though the ARC. This allows for actual validation to happen and
removes any possibility of the buffer being manipulated after the I/O
has been issued.

Just as with Direct I/O write checksum validation failures, Direct I/O
read checksum validation failures are reported though zpool status -d in
the DIO column. Also the zevent has been updated to have both:

  1. dio_verify_wr -> Checksum verification failure for writes
  2. dio_verify_rd -> Checksum verification failure for reads.

This allows for determining what I/O operation was the culprit for the
checksum verification failure. All DIO errors are reported only on the
top-level VDEV.

Even though FreeBSD can write protect pages (stable pages) it still has
the same issue as Linux with Direct I/O reads.

This commit updates the following:

  1. Propogates checksum failures for reads all the way up to the top-level VDEV.
  2. Reports errors through zpool status -d as DIO.
  3. Has two zevents for checksum verify errors with Direct I/O. One for read and one for write.
  4. Updates FreeBSD ABD code to also check for ABD_FLAG_FROM_PAGES and handle ABD buffer contents validation the same as Linux.
  5. Updated manipulate_user_buffer.c to also manipulate a buffer while a Direct I/O read is taking place.
  6. Adds a new ZTS test case dio_read_verify that stress tests the new code.
  7. Updated man pages.
  8. Added an IMPLY statement to zio_checksum_verify() to make sure that Direct I/O reads are not issued as speculative.
  9. Removed self healing through mirror, raidz, and dRAID VDEVs for Direct I/O reads.

This issue was first observed when installing a Windows 11 VM on a ZFS
dataset with the dataset property direct set to always. The zpool
devices would report checksum failures, but running a subsequent zpool
scrub would not repair any data and report no errors.

Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Signed-off-by: Brian Atkinson <batkinson@lanl.gov>
Closes #16598

Details

Provenance
Brian Atkinson <batkinson@lanl.gov>Authored on Oct 9 2024, 7:28 PM
GitHub <noreply@github.com>Committed on Oct 9 2024, 7:28 PM
Parents
rGefeb60b86a22: FreeBSD: ignore some includes when not building kernel
Branches
Unknown
Tags
Unknown