amd64: do not leak pcpu pages

[kib]

Summary

Do not preallocate pcpu area backing pages on early startup, only
allocate enough of KVA for pcpu[MAXCPU] and the page for BSP.  Other
pages are allocated after we know the number of cpus and their
assignments to the domains.

PCPUs are not accessed until they are initialized, which happens on AP
startup.

amd64: initialize APs kpmap_store in init_secondary()

The APs pcpu area is zeroed in init_secondary() by pcpu_init(), so the
early initialization in pmap_bootstrap() is nop.

Fixes:  42f722e721cd010ae5759a4b0d3b7b93c2b9cad2

[markj]

sys/amd64/amd64/mp_machdep.c

312

The domain could be empty. The code needs to try to allocate from other domains before giving up.

sys/amd64/amd64/pmap.c

1998–2004

Perhaps add a comment pointing to amd64_mp_alloc_pcpu()?

[kib]

Add comment.
Retry if allocation in domain failed.

[markj]

sys/amd64/amd64/mp_machdep.c

306

This should be ifdef, at least.

It seems cleanest to do the fallback only if vm_ndomains > 1.

[kib]

Fix boot on pcid configs.
Change page allocation retry logic.
Zero pcpu_ptes for APs in pmap_bootstrap().

[kib]

Upload the correct diff

[kib]

Fix alloc fallback logic for the numa case.

[markj]

sys/amd64/amd64/mp_machdep.c

310

Should the allocations specify VM_ALLOC_SYSTEM?

sys/amd64/amd64/pmap.c

1999

2049

Can't you use PCPU_SET just like the code immediately below?

[kib]

sys/amd64/amd64/mp_machdep.c

310

I think it does not matter. Or, if it matter, the system is already hosed, since we are dipping into low memory reserve.

[kib]

Grammar.
Use PCPU_SET.