mpi3mr: Honor the dma mask from IOCFacts

[imp]

Summary

The number of signficant bits that are decoded are returned in the flags
field of the IOCFacts structure from the device. Rather than assume the
worst with a pessimal 32-bit maximum, look at this value and pass it
along to all the dma map creation requests.

A lof of those creations are repeditive and could just inherit from the
base tag if we moved to the templated interface. This is called out as
desireable future work not done at this time.

In addition, due to a check and an egg problem, we have to allocate some
of the maps with a 32-bit loaddr. These are the ones we need to read
iocfacts. And they are fine to be so restircted: they are little used
after startup, and when they are used, bouncing is fine.

Sponsored by: Netflix

Test Plan

The test plan is to run it on a huge memory system and do a sysctl hw.busdma to see if we see the huge numbers of deferals and page bounces.

[jhb]

s/repeditive/repetitive/, s/check/chicken/, s/restircted/restricted/ in the commit log.

I also generally still prefer double spaces for sentence breaks in monospace text (commit logs and source comments).

sys/dev/mpi3mr/mpi3mr.c

986

DMA is an acronym, so normally capitalized

1446

1683

1686

I think this last sentence is a bit banal. By definition BUS_SPACE_MAXADDR has to fit in a bus_addr_t.

1694

I don't think you really need dma_hiaddr. It is always going to be BUS_SPACE_MAXADDR. I would just add dma_loaddr.

sys/dev/mpi3mr/mpi3mr_pci.c

277

282

[imp]

OK. fixed the commit message, made most of these changes. will make the actual code change in another review. I have two such things I need to do and retest with.

sys/dev/mpi3mr/mpi3mr.c

1686

fair point. I think I snagged the notion for it from somewhere else, but I can't find that now with a quick grep. I'll just drop.

1694

Yea. Not in this review. I'll do a followup review since I'm trying to keep code changes out of this review since I've already tested it and the loop back is easier with a new rev. If it's just comments, then I'll fold those back since I don't need to re-test them (and I can make a catch-up commit that I can test with md5).