arm64: Check TDP_NOFAULTING in a data abort

[andrew]

Summary

As with other architectures when the TDP_NOFAULTING flag is set we
shouldn't panic when td_critnest is non-zero or if the witness check
fails.

The EFI runtime service functions will soon set this flag to handle
exceptions in the services.

Sponsored by: Arm Ltd

[kib]

This change is specifically for efirt fault handling, am I right?

If yes, I suggest to add a different pcb onfault pointer, to not weak the assumptions in case of normal userspace access. E.g. if we take an interrupt while in copyin/out, kernel fault must not result in silent retry.

[andrew]

Is the concern about pcb->pcb_onfault being set when interrupts are enabled so an interrupt handler could trash it calling into an EFI runtime service?

[kib]

In D46815#1068288, @andrew wrote:

Is the concern about pcb->pcb_onfault being set when interrupts are enabled so an interrupt handler could trash it calling into an EFI runtime service?

No. Suppose a thread executes copyout. The interrupts are enabled, and cpu could receive one with pcb_onfault set to copyout fault handler. Then suppose that the interrupt handler faulted. The patched code would see intr_nesting_level != 0 and pcb_onfault != 0, erronously passing control to copyout fault handler. The result would be further state corruption, perhaps garbled x0/x1 for the handler, and _not_ handled fault.

[andrew]

If intr_nesting_level != 0 then the intr_nesting_level == 0 check will fail and the kernel will fall through to the existing panic calls.

[kib]

In D46815#1068297, @andrew wrote:

If intr_nesting_level != 0 then the intr_nesting_level == 0 check will fail and the kernel will fall through to the existing panic calls.

Ok, same argument for code that took a spinlock but not incremented td_intr_nesting_level.

Let me explain why this patch triggered me: pcb_onfault, while formally MD, has the same semantic on all arches. Having arm64 being an outlier IMO is not worth it, and the cost of not doing that is easy by adding dedicated pcb_onefifault (name is only an example).

[andrew]

Switch to __predict_true((td->td_pflags & TDP_NOFAULTING) == 0) as on other archs. We can then call vm_fault_disable_pagefaults in efi_arch_enter.

(will update commit message later)

[markj]

sys/arm64/arm64/trap.c

343

It's a bit weird to have a __predict_true annotation in a condition that we expect to be false. Perhaps it'd make more sense to check the unlikely cases first.

[andrew]

Remove __predict_true