mpi3mr: poll reply queue and add MPI3MR_DEV_REMOVE_HS_COMPLETED flag

[chandrakanth.patil_broadcom.com]

Summary

An outstanding IO counter per target check has been added before deleting
the target from the OS which will poll the reply queue if there are any
outstanding IOs are found.

A new flag, named "MPI3MR_DEV_REMOVE_HS_COMPLETED," is added. If a remove event
for a target occurs and before the deletion of the target resource if the add event
for another target arrives reusing the same target ID then this flag will prevent
the removal of the target reference. This flag ensures synchronization between the interrupt
top and bottom half during target removal and addition events.

[sumit.saxena_broadcom.com]

Need to adjust patch description a bit to keep code changes and description better aligned.

[imp]

Fine as it is if the same sort of redundancy is done elsewhere

sys/dev/mpi3mr/mpi3mr_cam.c

1871

This assignment is redundant for a local variable exiting scope.

[chandrakanth.patil_broadcom.com]

In D44423#1013689, @sumit.saxena_broadcom.com wrote:

Need to adjust patch description a bit to keep code changes and description better aligned.

Agreed. Will update the patch description.

[chandrakanth.patil_broadcom.com]

In D44423#1013705, @imp wrote:

Fine as it is if the same sort of redundancy is done elsewhere

Ok. We will keep this now. however, we are going to submit a separate cleanup patch where we are going to address this and other similar issues.

[mav]

sys/dev/mpi3mr/mpi3mr.c

3615–3616

This looks like an extra tab. Not neat. I saw the same extra tabs in other places too.

sys/dev/mpi3mr/mpi3mr_cam.c

1826–1832

Are the spaces missing here between %d and the strings continuations next line?

Another cosmetic issue is that you are reading the counter separately for condition and printing. I suppose it may report 0

1844

Do we really need to wait for outstanding commands with 1 second granularity, while burning CPU time in a tight loop? Could we sleep in this context and/or do it in smaller increments to not wait longer than necessary?

[mav]

sys/dev/mpi3mr/mpi3mr_cam.c

1844

Ah, I see you are removing it in next commit.

[chandrakanth.patil_broadcom.com]

sys/dev/mpi3mr/mpi3mr.c

3615–3616

It will be taken care of in the debug/cleanup patch.

sys/dev/mpi3mr/mpi3mr_cam.c

1826–1832

Agreed. The space is missing and the outstanding counter could be zero by the time it gets printed. I will re-submit the patch correcting it.

[chandrakanth.patil_broadcom.com]

sys/dev/mpi3mr/mpi3mr_cam.c

1826–1832

Hi Mavi,

Thanks for the feedback. I have uploaded the new patch: https://reviews.freebsd.org/D44494 addressing this issue.