Illumos 5408 - managing ZFS cache devices requires lots of RAM

5408 managing ZFS cache devices requires lots of RAM
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Don Brady <dev.fs.zfs@gmail.com>
Reviewed by: Josef 'Jeff' Sipek <josef.sipek@nexenta.com>
Approved by: Garrett D'Amore <garrett@damore.org>

Porting notes:

Due to the restructuring of the ARC-related structures, this
patch conflicts with at least the following existing ZoL commits:

6e1d7276c94cbd7c2e19f9232f6ba4bafa62dbe0
Fix inaccurate arcstat_l2_hdr_size calculations

    The ARC_SPACE_HDRS constant no longer exists and has been
    somewhat equivalently replaced by HDR_L2ONLY_SIZE.

e0b0ca983d6897bcddf05af2c0e5d01ff66f90db
Add visibility in to cached dbufs

    The new layering of l{1,2}arc_buf_hdr_t within the arc_buf_hdr
    struct requires additional structure member names to be used
    when referencing the inner items.  Also, the presence of L1 or L2
    inner member is indicated by flags using the new HDR_HAS_L{1,2}HDR
    macros.

Ported by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>