libpfctl: ensure we return useful error codes

Return errno rather than -1 on error. This allows pfctl to report much
more useful errors.

Reported by: Alexander Leidinger <Alexander@Leidinger.net>
MFC after: 1 week

(cherry picked from commit 93e96359c980ccf318fe089b30b863f7c910b622)