jemalloc: set LG_VADDR to 64 on amd64

[kib]

Summary

The current value 48 causes immediate breakage on la57 kernels. In principle, the value can be increased to 57, but I do not see a reason.

With this change, and the series of fixes to the trampoline, I can boot la57 on real hardware.

PS. Arm64 would need a similar fix.

[andrew]

I don't think you'll see any difference between 57 and 64.

It looks like LG_VADDR is used to decide which bits in an internal pointer can be reused. On arm64 if the value is at most 56 then RTREE_LEAF_COMPACT is defined so parts of the pointer are reused, if it's 57 then it's not and struct rtree_leaf_elm_s pulls the index and slab bit out.

On arm64 the value could be 52 for when FEAT_LPA2 is implemented.

Do you see any performance or memory usage change with this?

[kib]

In D46686#1064294, @andrew wrote:

I don't think you'll see any difference between 57 and 64.

It looks like LG_VADDR is used to decide which bits in an internal pointer can be reused. On arm64 if the value is at most 56 then RTREE_LEAF_COMPACT is defined so parts of the pointer are reused, if it's 57 then it's not and struct rtree_leaf_elm_s pulls the index and slab bit out.

On arm64 the value could be 52 for when FEAT_LPA2 is implemented.

Do you see any performance or memory usage change with this?

Machine which I have is unusable for any perf work (BIOS is too broken). Perhaps I should add an option for small/large jemalloc VADDR. default to large?

[kib]

Add build option to revert back to 48 bit wide.

[alc]

tools/build/options/WITH_LG_VADDR_WIDE

3 ↗

(On Diff #143473)

[kib]

Fix grammar in the option' description.

[emaste]

share/mk/src.opts.mk

366–369

I don't think we generally have options that have a default on one arch and do not exist on other archs, ping @imp

What does make makeman think of this?

This will presumably be coming to at least arm64 as well it seems, and I imagine we should just have a MI on-by-default option for it.

[kib]

Rename option to JEMALLOC_LG_ADDR_WIDE
Make it MI, with the only effect on amd64.

[kib]

Ping? This is the last bit to restore LA57.

[emaste]

If JEMALLOC_LG_VADDR_WIDE is default YES we need a WITHOUT_JEMALLOC_LG_VADDR_WIDE description - src.conf documents the non-default setting. Otherwise LGTM.

[kib]

WITH->WITHOUT, edit the description.

[emaste]

LGTM.