nvme: Use shared timeout rather than timeout per transaction

[imp]

Summary

Keep track of the approximate time commands are 'due' and the next
deadline for a command. twice a second, wake up to see if any commands
have entered timeout. If so, quiessce and then enter a recovery mode
half the timeout further in the future to allow the ISR to
complete. Once we exit recovery mode, send a command with no negative
effect to the drive to ensure that any commands that completed after
we last poll are handled in a timely manner.

[imp]

oldest is always front of queue, use this observation to make deadline more robust.

[rpokala]

to ensure that any commands that completed after we last poll are handled in a timely manner.

How does sending a NOP ensure that result?

sys/dev/nvme/nvme_private.h

170

nvme_recovery_state would be clearer, IMHO.

173

/* This phase of recovery, initiate controller reset */ scans better to me.

sys/dev/nvme/nvme_qpair.c

1076

The first operation will set deadline to non-SBT_MAX, and then outstanding_tr would become empty when all in-flight operations complete, right? Wouldn't that mean that this would get printed all the time on an idle system?

[imp]

An updated patch was uploaded as D28583, so abandoning this in favor of that.