x86: Adjust base addr for PCI MCFG regions

[cperciva]

Summary

Each bus gets 1 MB of address space; the actual base address for an
MCFG bus range is the address from the table plus the starting bus
number times 1 MB.

Fixes: f54a3890b1f4 ("x86: Support multiple PCI MCFG regions")

[cperciva]

I haven't read the PCI spec, but this works on the m7i.2xlarge instance I was testing with, and seems to match what Linux does:

https://github.com/torvalds/linux/blob/fbafc3e621c3f4ded43720fdb1d6ce1728ec664e/drivers/acpi/pci_mcfg.c#L260

The i386 change is completely untested; but I think the previous commit was also untested on i386 so maybe that's ok?

[jhb]

The spec language here simply isn't clear. I had misread the Linux code yesterday thinking it was only applying the adjustment in the _CBA case. Here is the relevant language from the spec:

The MCFG table format allows for more than one memory mapped base address entry. Each entry (memory mapped configuration space base address allocation structure) corresponds to a unique PCI Segment Group consisting of 256 PCI buses. However, multiple entries could be used described a single PCI Segment Group in the case of multiple host bridge implementations. In such cases, multiple entries that belong to the same segment is allowed as long as the bus ranges described by the entries do not overlap.
• The PCI Segment Group Number field denotes the PCI Segment Group corresponding to the base address field in the structure. For systems supporting multiple PCI Segment Groups, this field should correspond to the value returned by _SEG object in ACPI name space for the applicable host bridge device. If the system only contains a single (default) PCI Segment Group, namely, PCI Segment Group 0, no corresponding _SEG object is required.
• The base address field provides the 64-bit physical address of the base of the memory mapped configuration space associated with the PCI Segment Group. It is the responsibility of the provider of the table to ensure that the base address reported is consistent with the requirementsforthehardwareimplementation. ForPCI-XandPCIExpressplatforms utilizing the enhanced configuration access method, the base address of the memory mapped configuration space (corresponding to start bus number 0) must comply with alignment requirements of the corresponding local bus specification. The unsupported upper bits of the physical address must be set to 0.
Note that there is no alignment requirement for additional base address entries in a given PCI Segment Group provided the start bus number is not zero for such an entry.

In the last bullet point, the first sentence seems like it might imply that all entries for a given segment ("Segment group") should use the same base address. However, the last sentence seems to counteract that. Specifically, it doesn't make sense to relax the alignment requirements for an entry with a min bus number != 0 unless those entries can have a different base address from the entries with 0. It's also not-intuitive at all that the base address for a table entry always starts at bus 0 and not for the "minbus". That said, matching Linux is probably going to work out in practice. This is all quite odd, and the PCI folks could really have done a better job here when writing this spec.

I barely tested the earlier commit on i386 (booted a bhyve VM, but bhyve uses a simple MCFG table with a single entry).