Clean up reassignbuf() and buf_vlist_remove() a bit.

• Convert panic() calls to INVARIANTS-only assertions. The PCTRIE code provides some of the same protection since it will panic upon an attempt to remove a non-resident buffer.
• Update the comment above reassignbuf() to reflect reality.

Reviewed by: cem, kib, mjg
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D25965