A machine with 4 CPU cores can have hundreds of processes that all feel like they're running at once — the trick is that almost none of them actually are, at any given instant, and the switching between them is fast enough to be invisible.
3 min read
At any moment, every process the OS knows about is in one of a handful of well-defined states:
A process waiting on a slow disk read isn't burning CPU time while it waits — it's moved out of the Running state entirely, freeing the CPU core for a different, actually-runnable process. This is the whole reason a single CPU core can serve many I/O-bound programs efficiently: the moment one blocks on I/O, the OS immediately gives the core to someone else instead of sitting idle waiting for a disk that might take milliseconds (an eternity in CPU time).
A context switch is the OS saving one process's complete execution state (CPU registers, program counter, stack pointer) and loading another process's previously-saved state, so the second process resumes exactly where it left off. This is what makes a 4-core machine feel like it's running far more than 4 processes simultaneously — it's actually rapidly alternating between many of them, each getting a short slice of real CPU time, fast enough that the switching itself is imperceptible to a human.
Saving and restoring state takes real CPU cycles that produce no useful work by themselves — pure overhead. Beyond that, switching to a different process typically invalidates the CPU's cache (the fast on-chip memory holding recently-used data), so the new process's first several memory accesses are slower until its own data gets cached again. This is exactly why a system with too many actively-competing processes (or threads) can slow down even though each individual one is doing real work — an increasing share of total CPU time goes to switching between them rather than executing any of them.
A process can voluntarily give up the CPU by blocking on I/O (moving itself to Waiting) — this is a cheap, natural switch point. The OS can also force a switch even from a process that's still perfectly ready to keep running, once its allotted time slice expires — this is what actually guarantees fairness: without it, one CPU-bound process that never blocks on anything could simply monopolize a core forever.
Check your understanding
A quick comprehension check — not tracked, not graded, just for you.
1. Why does a process waiting on disk I/O get moved out of the Running state instead of just pausing in place?
2. What does a context switch actually involve?
3. Why isn't context switching free, even though it enables efficient multitasking?
Operating Systems Fundamentals