fdescfs: Fix a file ref leak

[markj]

Summary

On github, CTurt points out that vn_vget_ino_gen() may fail without
invoking the callback, in which case the ref on fp is leaked. Moreover,
we cannot safely drop the ref while the dvp is locked.

So:

• Use a flag variable to indicate whether the ref is dropped.
• Reorganize things to handle the leak.

[mjg]

freaking mess

[kib]

sys/fs/fdescfs/fdesc_vnops.c

339

I think that the comment should be updated

374

I do not think that error can be zero on this path

[markj]

sys/fs/fdescfs/fdesc_vnops.c

374

It will be zero when VTOFDESC(dvp)->fd_ix == FD_DESC + fd.

[markj]

Adjust a comment.

[kib]

sys/fs/fdescfs/fdesc_vnops.c

341

I suspect that this check is not enough, for instance the fdescfs root dir can be opened more then once. Perhaps fdescfs vnodes locks should allow recursion.

[markj]

sys/fs/fdescfs/fdesc_vnops.c

341

But the root dir always has fd_ix == FD_ROOT. In fact, I cannot see how this check is ever true.

[kib]

sys/fs/fdescfs/fdesc_vnops.c

341

Open /dev/fd with the result file descriptor N, and look up /dev/fd/N.

[markj]

sys/fs/fdescfs/fdesc_vnops.c

341

Then the root dir has fd_ix = FD_ROOT, which is never equal to FD_DESC + N.

[kib]

sys/fs/fdescfs/fdesc_vnops.c

341

Exactly. Then it hits the vn_vget_ino() path.

[markj]

sys/fs/fdescfs/fdesc_vnops.c

341

But what exactly is the problem? fdesc_lookup() doesn't lock both vnodes at the same time.

(I did try this experiment, with and without -o nodup.)

[kib]

sys/fs/fdescfs/fdesc_vnops.c

341

Most likely it was a vnode lock recursion, I just proposed a review. See 60af8a6a7a37d8b02657e45577f019cd13144f19 (I think)