Add a VM flag to prevent reclaim on a failed contig allocation

[rstone]

Summary

If a M_WAITOK contig alloc fails, the VM subsystem will try to
reclaim contiguous memory twice before actually failing the
request. On a system with 64GB of RAM I've observed this take
400-500ms before it finally gives up, and I believe that this
will only be worse on systems with even more memory.

In certain contexts this delay is extremely harmful, so add a flag
that will skip reclaim for allocation requests to allow those
paths to opt-out of doing an expensive reclaim.

Sponsored by: Dell Inc

[kib]

sys/sys/malloc.h

55

I think it is time to use 8 digits for M_ constants. Or use the flags from low byte, they are available I believe.

sys/vm/vm_page.h

552

Doesn't the value conflict with VM_ALLOC_COUNT() ?

[rstone]

sys/vm/vm_page.h

552

Oh jeez, I didn't even notice. Should I take 0x04 as bit 2 is supposed to be free or do we need to do something else here?

[kib]

sys/vm/vm_page.h

552

I would take 0x80 or 0x800.

[rstone]

Fix overflap between new flag and count field

[rstone]

Change the value of M_NORECLAIM to avoid the style issue