iflib(4): Replace admin taskqueue group with per-interface taskqueues

[krzysztof.galazka_intel.com]

Summary

Using one taskqueue group with single thread to execute all admin
tasks may lead to unexpected timeouts when long running task (e.g.
handling a reset after FW update) for one interface prevents
tasks from other interfaces being executed. Taskqueue group API
doesn't let to dynamically add threads, and pre-allocating thread
for each CPU as it's done for traffic queues would be a waste
of resources on systems with small number of interfaces. Replace
global taskqueue group for admin tasks with taskqueue allocated
for each interface to allow independent execution.

Signed-off-by: Krzysztof Galazka <[e-mail]>

[krzysztof.galazka_intel.com]

Rebased on top of master.
Free taskqueue after releasing interrupts per jhb comment on Github.

[jhb]

This looks ok to me, though I'm not intimately familiar with all of iflib.

[kbowling]

Seems reasonable to me. If there is any work that depends or interacts with these changes it would be nice to see them linked/referenced so we can look at that too but I can't foresee any problems.

sys/net/iflib.c

6339

Can this line be removed entirely?

6401

Can you explain how this works? Looks like we used to have vflr_task but this will not be needed anymore with the per interface task which I suppose could do the same work?

[krzysztof.galazka_intel.com]

In D46062#1076736, @kbowling wrote:

Seems reasonable to me. If there is any work that depends or interacts with these changes it would be nice to see them linked/referenced so we can look at that too but I can't foresee any problems.

There is no dependent work.

sys/net/iflib.c

6401

Uh, I'm sorry, I don't know what I was thinking. I changed iflib_iov_intr_deferred to enqueue a task to admin taskqueue but this task is never initialized. I need to fix that. Thanks for pointing this out.