Tied to Mission: Backend Depth — deepens the OS picture from Lesson 5 into memory: the other resource that accumulates, leaks, and gets misread in production.
You log into a server. free shows 90% of RAM used. Someone says "memory leak, let's restart." But the app's RSS is flat, and the biggest chunk of "used" memory is labeled buff/cache. Restarting won't help, because the machine isn't sick — that 90% is the operating system doing its job. Understanding why is the difference between restarting healthy servers and actually diagnosing the ones that are sick.
Each process sees a private, contiguous address space. Nothing in that space physically exists until it's touched — the OS maps it to real physical pages on demand through page tables. First access to a region faults the page in; that's demand paging. This indirection is why a process can "have" far more address space than RAM, and why two numbers on top mean different things:
And pages are shared, not copied. Every process that loads libc maps the same physical pages; each one's VSZ counts them, but RAM holds one copy. After fork(), parent and child share all their pages copy-on-write: the fork is nearly free, and the copying happens lazily, one page at a time, only when someone writes. That's the mechanism behind Lesson 6's multiprocessing — forking is cheap; the cost shows up as page faults when the child actually mutates data.
Every file read and write goes through the page cache: the OS keeps file pages in RAM. A read of a cached page never touches disk. A write lands in the cache and is marked dirty, to be flushed to disk in the background by the kernel's flusher threads. That's why reading the same file twice is dramatically faster the second time, and why a DB's "working set" staying in cache is often the difference between fast and unusable.
This is what buff/cache is, and it's not a leak — it's reclaimable. The kernel gives memory to the page cache freely because unused RAM is wasted RAM, and takes it back the moment an application actually needs it. The catch is how it takes it back: evicting clean pages is instant, but reclaiming dirty pages means writing them to disk first. So under memory pressure at the same moment the machine is busy, the kernel is doing disk I/O to free memory — which is exactly when latency spikes are most likely. (This is also why "the disk is doing something weird" during a memory crunch isn't a coincidence.)
Two things both show up as "memory is high," and telling them apart is a core production skill:
buff/cache, individual processes are flat, and the number drops when something needs memory. Restarting does nothing but throw away useful cache.Compare with Lesson 8's bloat: that grows on disk, not in RAM, and restart does not clear it. Three "uses lots of memory/space" phenomena, three different signatures, three different fixes.
When reclaiming cache isn't enough, the kernel falls back to swap: pushing anonymous (non-file) pages out to disk. A server that starts swapping turns every touched page into disk I/O — latency goes through the roof, and throughput collapses in a way that looks exactly like a network problem to the app (Lesson 10's RTT blowup is one candidate, swap is another; CPU being low and page-fault counters being high is how you tell). For latency-sensitive services you size RAM so this never happens; swap is a last resort, not a resource plan.
And if even swap can't satisfy demand, the kernel's OOM killer picks a victim — scored by a "badness" heuristic biased toward large processes — and kills it. The "my process just disappeared" incident is usually this, logged in dmesg. The OOM killer is a symptom, not a bug: it's the kernel choosing a crash over a hang. The fixes live upstream — bounded caches, bounded queues (Lesson 13), and actual leak detection.
free shows 90% of RAM used. A teammate says it's a memory leak and wants to restart the service. You look closer and most of the used memory is buff/cache; the app's RSS is flat. What do you do?On a Linux box: run free -m before and after cat some_large_file > /dev/null twice — the second read is near-instant and buff/cache jumped, because the first read populated the page cache. Then check a live process: cat /proc/<pid>/status and compare VmSize (VSZ) with VmRSS — the gap is mapped-but-not-touched address space. Finally, look at dmesg | grep -i oom on a box that has ever run out of memory, and read what the kernel says it killed and why.
OSTEP (already in the reference list) has the cleanest explanation of virtual memory, demand paging, and address translation in the "Memory Virtualization" part of the book. For the kernel side — page cache, dirty-page writeback, and the OOM killer's decision process — the Linux kernel documentation under admin-guide/sysctl/vm.rst is the primary source, and Mel Gorman's Understanding the Linux Virtual Memory Manager is the book-length treatment if you want the full mechanism.
Memory and the network both explain "everything got slow." Next: the one-threaded machinery that runs most Python services, and the single mistake that freezes all of it at once — Lesson 12.