pci: Clear active PME# and disable PME# generation

[jhb]

Summary

The PCI power management specification requires that the OS clear any
pending PME# interrupt and generation of PME# interrupts during
"initial operating system load". Note that clearing a pending PME#
interrupt requires writing a 1 to the Read/Write-Clear PME bit in the
power management status register. To handle the boot time case, clear
PME# state in pci_read_cap() when scanning new PCI devices. This
should also cover hotplug devices.

In addition, clear this state on every PCI device after resume from
sleep in pci_resume_child before invoking the driver's DEVICE_RESUME
method.

[imp]

sys/dev/pci/pci.c

916

why were you able to remove this check?

[cperciva]

This doesn't cause any problems in EC2, but it doesn't solve the "recognizing that we're ready for an nvme disk to go away" problem either. They're definitely using a write to the power status register as a cue to remove the device (per https://reviews.freebsd.org/D49146 ).

[jhb]

sys/dev/pci/pci.c

916

Because it is incorrect. The register is always there. Some PCI capabilities do have optional registers indicated by some flag in another register, but this one is always present.

That said, we don't use pp_bse or pp_data anywhere and we should probably just remove them. Even pp_status is rather dubious. For most other capability register sets like this, we only save the base offset in cfg and use offsets like PCIR_POWER_STATUS inline when accessing registers.

[jhb]

In D49222#1122599, @cperciva wrote:

This doesn't cause any problems in EC2, but it doesn't solve the "recognizing that we're ready for an nvme disk to go away" problem either. They're definitely using a write to the power status register as a cue to remove the device (per https://reviews.freebsd.org/D49146 ).

Humm, I can't imagine what the goal of this behavior is on the part of EC2. Can the engineers there explain why they think the PME is still pending and needs to be cleared? Do they assert a PME as part of the eject notification? (You can read the PME bit to see when it changes from 0 to 1 if that's helpful.)

[imp]

sys/dev/pci/pci.c

916

yea, if we don't use them...that's a good idea...

[cperciva]

In D49222#1122616, @jhb wrote:

In D49222#1122599, @cperciva wrote:

This doesn't cause any problems in EC2, but it doesn't solve the "recognizing that we're ready for an nvme disk to go away" problem either. They're definitely using a write to the power status register as a cue to remove the device (per https://reviews.freebsd.org/D49146 ).

Humm, I can't imagine what the goal of this behavior is on the part of EC2. Can the engineers there explain why they think the PME is still pending and needs to be cleared? Do they assert a PME as part of the eject notification? (You can read the PME bit to see when it changes from 0 to 1 if that's helpful.)

Yes, the PCIM_PSTAT_PMEENABLE is set to 0 until the eject request arrives at which point it is set to 1.

[cperciva]

In D49222#1122856, @cperciva wrote:

Yes, the PCIM_PSTAT_PMEENABLE is set to 0 until the eject request arrives at which point it is set to 1.

Oops, I was misled by some earlier testing. In fact PME# never gets asserted. But we need to turn it off anyway to acknowledge the detach.

[jhb]

Clear PME earlier in resume

[jhb]

Existing device drivers cleared PME# first in their resume methods before messing with device registers, so do the same in pci_resume_child.

[jhb]

Add manpage changes

[jhb]

Rebase

[jhb]

The updated series survived multiple sleep/resume cycles on my X1 Carbon including waking due to key press and waking due to opening the lid. I don't have a way to test any of the WOL driver changes however.