tcp rack: fix switch to RACK when a FIN has already been sent

[tuexen]

Summary

Fix the rack sendmap entry in case a FIN has been sent when the stack is switched over to RACK.

This issue was found by syzkaller.

Test Plan

Run the following packetdrill-script:

rack.pkt2 KBDownload

[markj]

sys/netinet/tcp_stacks/rack.c

12613

Should this assignment move into the else block below? rsm->r_end is written to again in the other branch.

[tuexen]

sys/netinet/tcp_stacks/rack.c

12613

No. The code below will in a follow-up commit be changed to:

if (tp->snd_una == tp->iss)
    rsm->r_flags |= RACK_HAS_SYN;
rsm->r_start = tp->snd_una;

This is equivalent. I guess one should also add a condition which ensures that there hasn't been a wrap-around and SND.UNA is equal to ISS. This test can be based on the state of the TCP connection.
I plan to do that in a separate review once this change is in.