Include scatter_chunk_waste in arc_size

The ARC caches data in scatter ABD's, which are collections of pages,
which are typically 4K. Therefore, the space used to cache each block
is rounded up to a multiple of 4K. The ABD subsystem tracks this wasted
memory in the scatter_chunk_waste kstat. However, the ARC's size is
not aware of the memory used by this round-up, it only accounts for the
size that it requested from the ABD subsystem.

Therefore, the ARC is effectively using more memory than it is aware of,
due to the scatter_chunk_waste. This impacts observability, e.g.
arcstat will show that the ARC is using less memory than it
effectively is. It also impacts how the ARC responds to memory
pressure. As the amount of scatter_chunk_waste changes, it appears to
the ARC as memory pressure, so it needs to resize arc_c.

If the sector size (1<<ashift) is the same as the page size (or
larger), there won't be any waste. If the (compressed) block size is
relatively large compared to the page size, the amount of
scatter_chunk_waste will be small, so the problematic effects are
minimal.

However, if using 512B sectors (ashift=9), and the (compressed) block
size is small (e.g. compression=on with the default volblocksize=8k
or a decreased recordsize), the amount of scatter_chunk_waste can be
very large. On a production system, with arc_size at a constant 50%
of memory, scatter_chunk_waste has been been observed to be 10-30% of
memory.

This commit adds scatter_chunk_waste to arc_size, and adds a new
waste field to arcstat. As a result, the ARC's memory usage is more
observable, and arc_c does not need to be adjusted as frequently.

Reviewed-by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Wilson <gwilson@delphix.com>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
Closes #10701