HomeFreeBSD

qsort.c: prevent undefined behavior

Description

qsort.c: prevent undefined behavior

Mark Milliard has detected a case of undefined behavior with the LLVM
UBSAN. The mandoc program called qsort with a==NULL and n==0, which is
allowed by the POSIX standard. The qsort() in FreeBSD did not attempt
to perform any accesses using the passed pointer for n==0, but it did
add an offset to the pointer value, which is undefined behavior in
case of a NULL pointer. This operation has no adverse effects on any
achitecture supported by FreeBSD, but could be caught in more strict
environments.

After some discussion in the freebsd-current mail list, it was
concluded that the case of a==NULL and n!=0 should still be caught by
UBSAN (or cause a program abort due to an illegal access) in order to
not hide errors in programs incorrectly invoking qsort().

Only the the case of a==NULL and n==0 should be fixed to not perform
the undefined operation on a NULL pointer.

This commit makes qsort() exit before reaching the point of
potentially undefined behvior for the case n==0, but does not test
the value of a, since the result will not depend on whether this
pointer is NULL or an actual pointer to an array if n==0.

The issue found by Mark Milliard in the whatis command has been
reported to the upstream (OpenBSD) and has already been patched
there.

(cherry picked from commit d106f982a54cd299671ccad58bc456138a22ae7b)

Details

Provenance
seAuthored on Jan 13 2022, 10:09 AM
Parents
rG63cc3d693de7: bitset.9: add description of BIT_ISSET
Branches
Unknown
Tags
Unknown