Commit 1:
Generalize the VV_CROSSLOCK logic in vfs_lookup()
When VV_CROSSLOCK is present, the lock for the vnode at the current
stage of lookup must be held across the VFS_ROOT() call for the
filesystem mounted at the vnode. Since VV_CROSSLOCK implies that
the root vnode reuses the already-held lock, the possibility for
recursion should be made clear in the flags passed to VFS_ROOT().
For cases in which the lock is held exclusive, this means passing
LK_CANRECURSE. For cases in which the lock is held shared, it
means clearing LK_NODDLKTREAT to allow VFS_ROOT() to potentially
recurse on the shared lock even in the presence of an exclusive
waiter.
That the existing code works for unionfs is due to a coincidence
of the current unionfs implementation.
Commit 2:
unionfs: allow recursion on covered vnode lock during mount/unmount
When taking the covered vnode lock during mount and unmount operations,
specify LK_CANRECURSE as the existing lock state of the covered vnode
is not guaranteed (AFAIK) either by assertion or documentation for
these code paths.
For the mount path, this is done only for completeness as the covered
vnode lock is not currently held when VFS_MOUNT() is called.
For the unmount path, the covered vnode is currently held across
VFS_UNMOUNT(), and the existing code only happens to work when unionfs
is mounted atop FFS because FFS sets LO_RECURSABLE on its vnode locks.
This of course doesn't cover a hypothetical case in which the covered
vnode may be held shared, but for the mount and unmount paths such a
scenario seems unlikely to materialize.
Commit 3:
nullfs: adopt VV_CROSSLOCK
When the lower filesystem directory hierarchy is the same as the nullfs
mount point (admittedly not likely to be a useful situation in
practice), nullfs is subject to the exact deadlock between the busy
count drain and the covered vnode lock that VV_CROSSLOCK is intended
to address.