bhyve: Fix linking with ld.bfd after libcasper changes.

[jhb]

Summary

ld.bfd wants to ensure that unresolved symbols in libraries are all
resolved when linking a binary. It will follow DT_NEEDED entries in
libraries to find dependencies, but it then needs a way to find the
dependencies. Since DT_NEEDED are resolved names
(e.g. libcap_pwd.so.1 instead of libcap_pwd.so), ld.bfd needs to find
the real libraries, not the symlinks in /usr/lib. The casper
libraries are tucked away in /lib/casper, so use -Wl,-rpath-link to
tell the linker to look there to find the missing dependencies.

I've no idea why lld doesn't require this. Maybe it doesn't validate
unresolved symbols in shared libraries pulled in, only unresolved
symbols in the objects being linked?

Alternatively, perhaps lib9p should have an -rpath of /lib/casper
along with other things that link against casper helpers. This would
also let us remove /lib/casper from rtld's default path which seems
a bit odd.

Fixes: cff482380436 bhyve: Move libcasper dependecy to lib9p

[jhb]

I kind of suspect that using -rpath of /lib/casper for things using libcap_* is the better long-term solution overall, but am open to other suggestions. Even if we want to do that, we might want this commit still as a workaround until the larger change is made to use -rpath.

[markj]

Alternatively, perhaps lib9p should have an -rpath of /lib/casper
along with other things that link against casper helpers. This would
also let us remove /lib/casper from rtld's default path which seems
a bit odd.

That seems like a better solution? If we're going to fix this in bhyve for now, it'd be best to add a comment explaining that this is for lib9p's benefit.

[jhb]

Add comment

[jhb]

Not sure if I should trim some of the commit log?

[markj]

This is ok with me, though I prefer the other revision.