Tied to Mission: Backend Depth — deepens Lesson 4's RTT picture with what happens under load: the second half of diagnosing latency spikes is knowing how TCP behaves when it's the bottleneck.
You push a large payload to a host. The transfer starts fast, then stalls at a plateau, then crawls. Or: a service's p99 latency triples the day traffic doubles, while CPU and disk stay flat. In both cases the code didn't change, the hardware didn't change — the network did. And the network's behavior is dominated by one mechanism you don't see from the application: TCP's congestion control, the sender-side speed limit that stops the network from collapsing.
TCP has two different throttles, and they answer two different questions:
cwnd — how many segments it allows itself to have in flight.Without the second, every sender would blast at full speed, overflow the bottleneck buffer, and the network would be all loss and retransmission — throughput down, not up. Congestion control is the mechanism that keeps the internet usable; the cost is that a sender can never immediately use a link's full capacity.
A new connection starts with a small cwnd (on Linux, ~10 segments) and doubles it every RTT — 10, 20, 40, 80 — exponential growth, until it hits ssthresh. This is called slow start, and it means the first few RTTs of any connection carry very little data. A connection that lives for only a few RTTs (a small API call) spends its entire life in slow start, never reaching the link's real capacity.
This is the deeper half of Lesson 4's lesson: connection reuse isn't just about skipping the handshake. A reused connection also arrives with a warm cwnd — the sender already knows the path can carry more, so the first bytes of the next request don't wait for slow start to ramp again. For a backend making many small calls to the same host, keeping connections alive buys throughput on top of latency.
Once cwnd reaches ssthresh, growth switches to congestion avoidance: roughly one extra segment per RTT — linear, cautious growth. The question is when to stop, and TCP's answer is: when you lose data. Three duplicate ACKs tell the sender a segment was lost, triggering fast retransmit and halving cwnd; a full timeout is worse, cutting cwnd back to slow-start level. This AIMD shape — slow up, fast down — is how TCP shares links fairly: everyone grows gently and gets cut hard when the network is actually full. (Modern flavors: CUBIC is Linux's default; BBR from Google estimates capacity directly instead of treating loss as the only signal.)
Now the scenario that matters for your p99. When offered traffic reaches the link's capacity, packets don't immediately drop — they queue in the bottleneck buffer (the router/switch/interface queue). Each queued packet adds a full RTT's worth of delay for every packet behind it. Throughput stays flat (the link is full — that's the maximum), but latency grows with queue depth: the RTT inflates from base latency to base + queueing delay. This is bufferbloat, and it's the classic "p99 explodes while CPU is fine" signature: the service isn't slower, the path between it and its clients just started holding packets for longer.
Queueing is also a leading indicator: keep loading the link and the buffer eventually overflows, packets drop, cwnd collapses, and throughput falls off a cliff — the "saturation curve" where pushing more traffic in makes the network move less data, not more. Active queue management (CoDel, RED) exists precisely to drop a few packets early and keep queues short, trading a little throughput for stable latency.
For a backend engineer the practical signals are: (1) retransmission rate — check ss -s or netstat -s; retransmits climbing is the network warning light, long before the app shows symptoms; (2) RTT inflation — if ping/RTT to a dependency triples while its CPU is low, you're queueing at a network bottleneck, not at the service; (3) fewer, longer-lived connections beat many short ones (Lesson 4) because warm cwnds skip slow start.
TCP delivers one strict, ordered byte stream per connection. A segment arrives out of order because of a loss — the receiver holds everything after it until the missing segment arrives, even if the missing segment isn't needed by the application yet. One lost packet stalls the whole stream. This is head-of-line (HoL) blocking, and it's why HTTP/2 — which multiplexes many requests over a single TCP connection — can have one slow request delaying all others on that connection, and why HTTP/3 exists: QUIC runs over UDP with independent streams, so a loss on one stream doesn't block the rest. Worth knowing because it changes how you interpret "the connection is slow" — sometimes the problem is one packet and the protocol's insistence on order.
With two hosts you control (or your laptop and a server): run iperf3 -c <host> for throughput and, in parallel, ping -i 0.2 <host> to watch RTT while the link fills. You'll typically see RTT climb as throughput plateaus — queueing delay on a saturated path. On your own machine, check ss -s for the retransmission counters before and after a large transfer, and ss -tin to see a connection's cwnd — then compare cwnd on a fresh connection vs. a reused one, and connect it back to Lesson 4's connection reuse.
Ilya Grigorik's High Performance Browser Networking (already in the reference list) has the cleanest practical treatment of slow start and congestion control for application engineers. For the protocol-level detail, RFC 5681 is the canonical specification of congestion control (slow start, AIMD, fast retransmit). Bufferbloat is best explained in Gettys & Nichols, "Bufferbloat: Dark Buffers in the Internet" (CACM 2011), freely available — it's short and directly explains the latency-vs-throughput curve this lesson is about.
The latency-spike story so far has covered Postgres (Lessons 8–9) and the network (this lesson). Next is the other classic source of "slow under load" that lives entirely inside one machine: memory. Lesson 11.