Tied to Mission: Backend Depth — the OS/processes/threads phase, sitting between Postgres internals and Python concurrency.
A backend with 8 CPU cores can comfortably hold open thousands of client connections at once. That sounds like it shouldn't work — only 8 things can literally be executing at the same instant. It works because "handling a connection" and "executing on a core" are not the same thing most of the time. Most of a connection's life is spent waiting, not computing, and waiting doesn't need a core.
A process is an isolated unit: its own memory space, its own file descriptors, its own address space that no other process can see into directly. A thread is a unit of execution within a process — threads in the same process share that process's memory, so switching between them is cheaper than switching between processes, and they can read each other's data without explicit IPC. What every thread has that's genuinely its own is a stack and a set of CPU register values — the minimum state needed to know "where am I in the code, and what are my local variables right now."
Say a thread calls read() on a socket, and no data has arrived yet. The kernel can't return an answer — there's nothing to return. Instead of spinning the CPU waiting for bytes that aren't there, the kernel marks that thread blocked (also called "waiting") and takes it off the core entirely. The OS scheduler then picks a different runnable thread — from this process or any other — and runs it on that same core. When the network data finally arrives, a hardware interrupt tells the kernel, which moves the original thread from blocked back to runnable, eligible to run again whenever the scheduler next gives it a core.
The mechanism that makes this possible is a context switch: the OS saves the currently running thread's register values and stack pointer somewhere safe, then loads a different thread's saved registers and stack pointer so it can resume exactly where it left off. A context switch has real but small overhead compared to the I/O wait it enables — the CPU spends a little time bookkeeping so it can spend the rest of that wait doing something productive instead of idling.
Every thread the scheduler manages is in one of roughly three states: running (actually executing on a core right now), runnable (ready to run, just waiting for a core to become free), or blocked (waiting on something external — I/O, a lock, a timer — and not eligible to run even if a core is free). A blocking syscall is what moves a thread from running to blocked; an interrupt is what moves it back to runnable.
If a thread handling a connection spends most of its time blocked on read() waiting for the next request, or blocked on a database query waiting for disk or network, that thread isn't consuming a core while it waits. The scheduler is free to run other threads — handling other connections — on that same core during the gap. That's why a server with 8 cores can have thousands of threads alive: at any given instant, only a handful of them are actually running, and the rest are parked in the blocked state, costing essentially nothing except the memory for their stack.
This only holds for I/O-bound work — waiting on a network call, a disk read, a database round trip. It does not hold for CPU-bound work — image resizing, JSON parsing at scale, cryptographic hashing — because that work keeps a thread in the running state the entire time it's active. There's nothing to switch away from productively; the thread genuinely needs the core. Add more CPU-bound threads than you have cores and the scheduler just time-slices them, context-switching back and forth between threads that are all runnable and all want the CPU right now — pure overhead, no extra waiting was hiding for it to fill.
Run something that does blocking network calls (e.g. a script hammering a slow endpoint with several threads) and watch it in htop/top: thread count climbs, but CPU% per core stays low because most threads are blocked, not running. Then run something CPU-heavy (e.g. a tight loop or image processing) with the same thread count and watch CPU% pin near 100% instead. On Linux, you can also inspect a specific blocked thread directly: cat /proc/<pid>/status shows its state, and cat /proc/<pid>/wchan (or wchan in ps -o wchan) names the kernel function it's parked in — for a thread blocked on a socket read, you'll typically see something in the networking stack, not "0" or an empty string, which is what a runnable/running thread shows.
"Operating Systems: Three Easy Pieces" (OSTEP), free online — see the "CPU Virtualization" part of the book for the process abstraction and how scheduling works, including the chapters that walk through exactly how a blocked process gets moved off the CPU and back.
Want to trace this further into how epoll/select let a single thread juggle many blocked sockets without needing a thread per connection at all? Ask directly — that's what this curriculum is for.