improvements to cap_sysctl.3

• Correct some function prototypes which were documented with the wrong pointer type.
• Clarify return values and requirements for freeing the limit handle.

[skip ci]

Sponsored by: Axcient
Reviewed by: oshogbo
Differential Revision: https://reviews.freebsd.org/D37586

(cherry picked from commit 6c93a2d0bc37f0c912e402f3f94c3c01350dca26)