Differential D31768
Add support for gicv2m as a child of gicv3 Authored by andrew on Sep 1 2021, 9:48 AM. Tags None Referenced Files
Subscribers
Details
On some systems, e.g. Parallels set to host a Linux VM under an M1 Mac, PR: 258136
Diff Detail
Event TimelineComment Actions Would be nice if someone could test this on ACPI system, without FDT (I don't have a FreeBSD build environment atm). If someone is interested, here is how it works in Parallels v17:
Comment Actions I've tested this (applied on top of https://reviews.freebsd.org/D31767) and it fixes the problem, thank you :-) Just in case, the difference in dmesg between "Other" and "Other Linux", is: --- other.txt 2021-09-01 16:24:24.720243000 +0000 +++ linux.txt 2021-09-01 16:21:58.590526000 +0000 @@ -34,8 +34,7 @@ kbd0 at kbdmux0 ofwbus0: <Open Firmware Device Tree> clk_fixed0: <Fixed clock> on ofwbus0 psci0: <ARM Power State Co-ordination Interface Driver> on ofwbus0 -gic0: <ARM Generic Interrupt Controller> mem 0x2010000-0x201ffff,0x2000000-0x2002fff on ofwbus0 -gic0: pn 0x0, arch 0x2, rev 0x0, implementer 0x0 irqs 128 +gic0: <ARM Generic Interrupt Controller v3.0> mem 0x2010000-0x201ffff,0x2500000-0x2cfffff on ofwbus0 gicv2m0: <ARM Generic Interrupt Controller MSI/MSIX> mem 0x2250000-0x2250fff on gic0 generic_timer0: <ARMv8 Generic Timer> irq 0,1,2,3 on ofwbus0 Timecounter "ARM MPCore Timecounter" frequency 24000000 Hz quality 1000 @@ -59,8 +58,9 @@ ahcich2: <AHCI channel> at channel 2 on ahci0 ahcich3: <AHCI channel> at channel 3 on ahci0 ahcich4: <AHCI channel> at channel 4 on ahci0 ahcich5: <AHCI channel> at channel 5 on ahci0 -virtio_mmio0: <VirtIO MMIO adapter> mem 0x21c0000-0x21c3fff irq 10 on ofwbus0 -vtnet0: <VirtIO Networking Adapter> on virtio_mmio0 +virtio_mmio0: <VirtIO MMIO adapter> mem 0x21a0000-0x21a3fff irq 10 on ofwbus0 +virtio_mmio1: <VirtIO MMIO adapter> mem 0x21c0000-0x21c3fff irq 11 on ofwbus0 +vtnet0: <VirtIO Networking Adapter> on virtio_mmio1 vtnet0: Ethernet address: 00:1c:42:aa:d3:02 pcib0: <Generic PCI host controller> mem 0x2300000-0x23fffff on ofwbus0 pci0: <OFW PCI bus> on pcib0 @@ -105,9 +105,9 @@ WARNING: WITNESS option enabled, expect reduced perfor ugen0.1: <Intel EHCI root HUB> at usbus0 ugen1.1: <0x1033 XHCI root HUB> at usbus1 uhub0 on usbus0 +uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0 uhub1 on usbus1 uhub1: <0x1033 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus1 -uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0 ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: <v3 F.79KQEG> ATA8-ACS SATA 2.x device ada0: Serial Number 2JM3AK5JVEDMB4BN7SZE @@ -132,6 +132,7 @@ Root mount waiting for: usbus0 Root mount waiting for: usbus0 uhub0: 15 ports with 15 removable, self powered mountroot: waiting for device /dev/gpt/rootfs... +vtballoon0: <VirtIO Balloon Adapter> on virtio_mmio0 lo0: link state changed to UP vtnet0: link state changed to UP Invoking IPv6 network device address event may sleep with the following non-sleepable locks held: With "Windows 11" it's almost like "Other Linux", except you also get: +uhid2: <Pen Interface> on usbus1 Comment Actions Now, regarding "vm.efi.dtb=0": with "Other", things change like this (I've omitted changes to USB enumeration order, I guess that's random): --- other.txt 2021-09-01 16:24:24.720243000 +0000 +++ other-nodtb.txt 2021-09-01 16:29:38.989275000 +0000 @@ -2,7 +2,6 @@ GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb -WARNING: Cannot find freebsd,dts-version property, cannot check DTB compliance Copyright (c) 1992-2021 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. @@ -14,7 +13,7 @@ WARNING: WITNESS option enabled, expect reduced perfor VT(efifb): resolution 1024x768 module firmware already present! real memory = 8586985472 (8189 MB) -avail memory = 8345776128 (7959 MB) +avail memory = 8345804800 (7959 MB) Starting CPU 1 (1) Starting CPU 2 (2) Starting CPU 3 (3) @@ -31,39 +30,22 @@ MAP 23f800000 mode 2 pages 592 MAP 1e00000 mode 0 pages 78 MAP 2120000 mode 0 pages 1 kbd0 at kbdmux0 -ofwbus0: <Open Firmware Device Tree> -clk_fixed0: <Fixed clock> on ofwbus0 -psci0: <ARM Power State Co-ordination Interface Driver> on ofwbus0 -gic0: <ARM Generic Interrupt Controller> mem 0x2010000-0x201ffff,0x2000000-0x2002fff on ofwbus0 +acpi0: <PRLS PRLS_OEM> +acpi0: Power Button (fixed) +acpi0: Sleep Button (fixed) +acpi0: Could not update all GPEs: AE_NOT_CONFIGURED +psci0: <ARM Power State Co-ordination Interface Driver> on acpi0 +gic0: <ARM Generic Interrupt Controller> iomem 0x2010000-0x2010fff,0x2000000-0x2000fff on acpi0 gic0: pn 0x0, arch 0x2, rev 0x0, implementer 0x0 irqs 128 +gic0: frame: 0 2250000 0 0 0 gicv2m0: <ARM Generic Interrupt Controller MSI/MSIX> mem 0x2250000-0x2250fff on gic0 -generic_timer0: <ARMv8 Generic Timer> irq 0,1,2,3 on ofwbus0 +generic_timer0: <ARM Generic Timer> irq 8,9,10 on acpi0 Timecounter "ARM MPCore Timecounter" frequency 24000000 Hz quality 1000 Event timer "ARM MPCore Eventtimer" frequency 24000000 Hz quality 1000 efirtc0: <EFI Realtime Clock> efirtc0: registered as a time-of-day clock, resolution 1.000000s -cpulist0: <Open Firmware CPU Group> on ofwbus0 -cpu0: <Open Firmware CPU> on cpulist0 -cpu1: <Open Firmware CPU> on cpulist0 -cpu2: <Open Firmware CPU> on cpulist0 -cpu3: <Open Firmware CPU> on cpulist0 -cpu4: <Open Firmware CPU> on cpulist0 -cpu5: <Open Firmware CPU> on cpulist0 -cpu6: <Open Firmware CPU> on cpulist0 -cpu7: <Open Firmware CPU> on cpulist0 -ahci0: <AHCI SATA controller> mem 0x2140000-0x2141fff irq 4 on ofwbus0 -ahci0: AHCI v1.10 with 6 3Gbps ports, Port Multiplier not supported -ahcich0: <AHCI channel> at channel 0 on ahci0 -ahcich1: <AHCI channel> at channel 1 on ahci0 -ahcich2: <AHCI channel> at channel 2 on ahci0 -ahcich3: <AHCI channel> at channel 3 on ahci0 -ahcich4: <AHCI channel> at channel 4 on ahci0 -ahcich5: <AHCI channel> at channel 5 on ahci0 -virtio_mmio0: <VirtIO MMIO adapter> mem 0x21c0000-0x21c3fff irq 10 on ofwbus0 -vtnet0: <VirtIO Networking Adapter> on virtio_mmio0 -vtnet0: Ethernet address: 00:1c:42:aa:d3:02 -pcib0: <Generic PCI host controller> mem 0x2300000-0x23fffff on ofwbus0 -pci0: <OFW PCI bus> on pcib0 +pcib0: <Generic PCI host controller> on acpi0 +pci0: <PCI bus> on pcib0 pci0: <multimedia, HDA> at device 1.0 (no driver attached) ehci0: <Intel 82801FB (ICH6) USB 2.0 controller> mem 0x10005000-0x100053ff at device 2.0 on pci0 usbus0: EHCI version 1.0 @@ -71,6 +53,21 @@ usbus0 on ehci0 xhci0: <NEC uPD720200 USB 3.0 controller> mem 0x10004000-0x10004fff at device 3.0 on pci0 xhci0: 32 bytes context size, 32-bit DMA usbus1 on xhci0 +cpu0: <ACPI CPU> on acpi0 +acpi_button0: <Power Button> on acpi0 +acpi_acad0: <AC Adapter> on acpi0 +battery0: <ACPI Control Method Battery> on acpi0 +virtio_mmio0: <VirtIO MMIO adapter> iomem 0x21c0000-0x21c3fff irq 6 on acpi0 +vtnet0: <VirtIO Networking Adapter> on virtio_mmio0 +vtnet0: Ethernet address: 00:1c:42:aa:d3:02 +ahci0: <AHCI SATA controller> iomem 0x2140000-0x2141fff irq 7 on acpi0 +ahci0: AHCI v1.10 with 6 3Gbps ports, Port Multiplier not supported +ahcich0: <AHCI channel> at channel 0 on ahci0 +ahcich1: <AHCI channel> at channel 1 on ahci0 +ahcich2: <AHCI channel> at channel 2 on ahci0 +ahcich3: <AHCI channel> at channel 3 on ahci0 +ahcich4: <AHCI channel> at channel 4 on ahci0 +ahcich5: <AHCI channel> at channel 5 on ahci0 armv8crypto0: <AES-CBC,AES-XTS,AES-GCM> Timecounters tick every 1.000 msec CPU 0: ARM Unknown CPU r0p0 (midr: 410f0000) affinity: 0 When set to "Windows 11" the difference seems to be the same. Both configurations seem to work fine. Comment Actions Does Parallels have a way to provide fine-grained control over guest devices? I.e., instead of "Other Linux" or such, a "Custom" setting that offers individual on/off controls? (Even if Parallels offers a "FreeBSD" setting, the appropriate devices to enable may change over time and with FreeBSD versions.) Comment Actions
Looks like "gic0: pn 0x0, arch 0x2" means that GICv2 is in use. If all works correctly, then when "Windows 11" is set, there should be "arch 0x3" and line about gicv2m0. Also XHCI should be configured to use MSI interrupts. I'd suggest also to recreate VM in Parallels v17. It is because in Parallels 17 some of the devices (like network adapters) are now located on PCI bus. But on earlier created VMs they are still on mmio (we had to implement that this way to not break networking configurations of existing VMs). Alternatively it is possible to edit the config.pvs file of the VM and change NetworkBusType setting to 0: <Runtime> ... <NetworkBusType>0</NetworkBusType> </Runtime> Comment Actions There are "undocumented" system flags. They are undocumented mostly because nobody wants to take responsibility to support such flags in later versions. But most of them are not secret and if something specific is needed, I can reply with corresponding flag, if it exists. There are not so many flags. Tendency is always try to make an option in config file. Comment Actions Hm, interesting. With "<NetworkBusType>0</NetworkBusType>" and the VM type set to "Windows 11", it panics (with the patch above) after successfully probing vtnet0 on virtio_pci0: panic: giv_v3_release_msix: Trying to release an unused MSI-X interrupt Backtrace looks roughly like this: gic_v3_release_msix() intr_release_msix() generic_pcie_fdt_release_msix() pci_release_msi_method() vtpci_free_interrupts() vtpci_setup_interrupts() vtnet_attach() device_attach() vtpci_legacy_probe_and_attach_child() vtpci_legacy_attach() device_attach() device_probe_and_attach() bus_generic_attach() pci_attach() device_attach() device_probe_and_attach() bus_generic_attach() device_attach() device_device_probe_and_attach() bus_generic_new_pass() bus_generic_new_pass() bus_generic_new_pass() bus_set_pass() mi_startup() virtdone() If, however, I add "vm.efi.dtb=0", the panic is gone and vtnet0 probes correctly on virtio_pci0 instead of virtio_mmio1. For some reason it reports as "<VirtIO PCI (legacy) Network adapter>", not "<VirtIO Networking Adapter>". Comment Actions Followup PR can be found at https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=258534. |