Clustering — scaling a Node app across CPU cores

A single Node process only ever uses one CPU core, no matter how many the machine has — clustering runs multiple independent copies of the same app, one per core, with the OS itself distributing incoming connections across them.

Advanced

3 min read

The problem: one Node process, one CPU core, regardless of how many exist

// This server, however well-optimized, uses at MOST one CPU core —
// even on a 16-core machine, 15 cores sit completely idle
const http = require("node:http");
http.createServer((req, res) => res.end("hello")).listen(3000);

Node's JavaScript execution is fundamentally single-threaded (the event loop, running on one thread) — a plain Node server, no matter how well it's written, genuinely cannot use more than one CPU core for running JavaScript, even on a machine with many available. This is a real, structural limitation distinct from the CPU-bound-work problem worker threads solve: even a server that's purely I/O-bound and never blocks on computation still can't take advantage of extra cores on its own, since there's only ever one event loop, on one thread, in one process.

`cluster`: multiple independent copies of the same process, one per core

const cluster = require("node:cluster");
const os = require("node:os");
 
if (cluster.isPrimary) {
  const numCPUs = os.availableParallelism(); // e.g., 8 on an 8-core machine
  for (let i = 0; i < numCPUs; i++) {
    cluster.fork(); // spawns a full WORKER PROCESS, each running the entire app independently
  }
} else {
  // this branch runs in EACH worker — the actual server code
  require("./server.js");
}

The cluster module's primary process spawns multiple worker processes (each a genuine, separate OS process, using the child_process.fork() mechanism from two lessons ago under the hood), and each worker runs an entirely independent copy of the same application code — its own event loop, its own memory, its own single CPU core. With one worker per available CPU core, the app as a whole can now genuinely use every core on the machine, even though each individual worker is still, on its own, exactly as single-threaded as before.

How incoming connections actually get distributed across workers

Every worker LISTENS on the same port (3000 here) — the operating system
itself (on most platforms, round-robin by default on Linux) decides which
worker actually receives each new incoming connection. The application
code doesn't manually route requests between workers at all.

Each worker independently calls .listen(3000), and the OS handles the actual distribution of incoming connections across the multiple processes all listening on that same port — the application itself doesn't implement any connection-routing logic; that's genuinely handled at the operating-system level, transparently to the app code running in each worker.

// Each worker has its OWN in-memory cache — a value cached in worker 1's
// memory does NOT exist in worker 2's memory. This is a real, common
// gotcha when moving from a single process to a clustered one.
const cache = new Map(); // this Map is DIFFERENT in every single worker

Because each worker is a genuinely separate OS process (not a thread within one process), they don't share memory at all — an in-memory cache, an in-memory session store, or any other process-local state exists independently, and differently, in every worker. This is a real, common source of bugs specifically when an app is moved from running as a single process to running clustered: a request handled by worker 2 has no visibility into anything worker 1 cached in its own memory, which is exactly why shared, cross-worker state (sessions, caches) needs to live somewhere genuinely external and shared — a real database or a cache like Redis — rather than in any individual worker's own process memory.

Clustering — scaling a Node app across CPU cores

The problem: one Node process, one CPU core, regardless of how many exist

cluster: multiple independent copies of the same process, one per core

How incoming connections actually get distributed across workers

The real trade-off: workers don't share memory, unlike threads within one process

Further reading

`cluster`: multiple independent copies of the same process, one per core