Start cubic epoch after having been app-limited.

[rscheff]

Summary

Cubic calculates the new cwnd based on absolute time
elapsed since the start of an epoch. A cubic epoch is
started on congestion events, or once the congestion
avoidance phase is started, after slow-start has
completed.

In transactional scenarios, it is likely that a
sender will be application limited for extended
periods of time. During this, the server only
transmits a limited amount of data, not fully
using the send window or current congestion window.

When after such a phase a subsequent larger
transfer is performed, which utilizes the full
congestion window, the cubic epoch could have
been lingering to an excessive amount of time.
Thus the next re-calculation of the cubic cwnd
would induce a massive jump of cwnd, leading to
a similar burst of sent data at line rate by
the cubic sender.

This Diff depends upon D23655, and adds the
condition of coming out from being app-limited
to again reset the cubic epoch.

Test Plan

Mount an NFS export, and perform a large read
request (e.g. 16MB using parallel threads to excercise
the senders congestion window). This should grow cwnd sufficiently
to have TCP in the congestion avoidance phase, where
cwnd is larger than ssthresh.
After completion of this request, run low-level
transfers, e.g. requesting 4 kB every 100ms, for
an extended period of time like 300 seconds.
In the final phase, perform another large request
(e.g. 16MB in parallel 64kB requests).
Observe the evolution of cwnd. Without this patch,
cwnd will jump dramatically during the last request,
potentially causing self-inflicted packet losses and
retransmission timeouts.

[cc]

Good discovery. Thanks.

[rscheff]

• update flags to uppercase and expand constant to 32 bit.

[rscheff]

• remove flag field microoptimization