Building a Worker Thread Pool

Spawning a fresh worker for every task is wasteful: each new Worker(...) boots a new V8 isolate, allocates memory, and reparses your code — easily tens of milliseconds before any real work begins. A worker pool flips this around. You create a fixed set of long-lived workers once, then feed them a stream of tasks, keeping startup cost a one-time expense. This is the standard pattern for serving CPU-bound work under load without exhausting memory or thrashing the OS scheduler.

How a pool works

A pool has three moving parts: a set of reusable workers, a list of which workers are currently idle, and a queue of pending tasks. When a task arrives, the pool either hands it to an idle worker immediately or, if all workers are busy, parks it in the queue. The moment a worker finishes and reports back, the pool marks it idle and pulls the next queued task. This backpressure is the whole point — it bounds concurrency to the pool size instead of letting unlimited threads pile up.

Concept	Role
Worker set	Pre-spawned, long-lived threads that never restart between tasks
Idle list	Workers ready to accept a task right now
Task queue	FIFO buffer of work waiting for a free worker
Dispatch	Pairs a queued task with a freed-up worker

A reusable worker script

The worker needs to handle many tasks over its lifetime, so it listens on parentPort for messages in a loop rather than computing once and exiting. Each message carries a job id so the main thread can match the reply to the right caller.

// pool-worker.js
import { parentPort } from 'node:worker_threads';

function fib(n) {
  return n < 2 ? n : fib(n - 1) + fib(n - 2);
}

parentPort.on('message', ({ id, value }) => {
  const result = fib(value);
  parentPort.postMessage({ id, result });
});

Implementing the pool

The pool below spawns N workers up front, tracks idle ones, and resolves a promise per task. Pending callbacks are stored in a Map keyed by job id, so an out-of-order reply still reaches the correct resolve.

// worker-pool.js
import { Worker } from 'node:worker_threads';
import { availableParallelism } from 'node:os';

export class WorkerPool {
  #workers = [];
  #idle = [];
  #queue = [];
  #pending = new Map();
  #nextId = 0;

  constructor(script, size = availableParallelism()) {
    for (let i = 0; i < size; i++) {
      const worker = new Worker(script);
      worker.on('message', ({ id, result }) => {
        const { resolve } = this.#pending.get(id);
        this.#pending.delete(id);
        resolve(result);
        this.#idle.push(worker); // worker is free again
        this.#dispatch();
      });
      worker.on('error', (err) => this.#failAll(err));
      this.#workers.push(worker);
      this.#idle.push(worker);
    }
  }

  run(value) {
    return new Promise((resolve, reject) => {
      const id = this.#nextId++;
      this.#pending.set(id, { resolve, reject });
      this.#queue.push({ id, value });
      this.#dispatch();
    });
  }

  #dispatch() {
    while (this.#idle.length && this.#queue.length) {
      const worker = this.#idle.pop();
      const task = this.#queue.shift();
      worker.postMessage(task);
    }
  }

  #failAll(err) {
    for (const { reject } of this.#pending.values()) reject(err);
    this.#pending.clear();
  }

  async destroy() {
    await Promise.all(this.#workers.map((w) => w.terminate()));
  }
}

Using it feels like calling an ordinary async function — the pool hides all the dispatching and queueing:

// main.js
import { WorkerPool } from './worker-pool.js';

const pool = new WorkerPool(new URL('./pool-worker.js', import.meta.url), 4);

console.time('batch');
const numbers = [40, 41, 42, 38, 39, 43, 37, 44];
const results = await Promise.all(numbers.map((n) => pool.run(n)));
console.timeEnd('batch');

console.log(results);
await pool.destroy();

Output:

batch: 2104.587ms
[ 102334155, 165580141, 267914296, 39088169, 63245986, 433494437, 24157817, 701408733 ]

Eight tasks ran on four workers: the first four dispatched immediately, the rest waited in the queue and were picked up as workers freed. Total wall time is far below running them sequentially on one thread.

Always terminate() your workers when shutting down. Idle workers keep their V8 isolate and event loop alive, which prevents the process from exiting cleanly.

Using piscina instead

Writing your own pool is instructive, but in production most teams reach for piscina — a battle-tested pool with task cancellation, timeouts, AbortSignal support, transferable objects, and automatic queue management. Your worker file just exports a default function:

// piscina-worker.js
export default function fib(n) {
  return n < 2 ? n : fib(n - 1) + fib(n - 2);
}

// main.js
import Piscina from 'piscina';

const pool = new Piscina({
  filename: new URL('./piscina-worker.js', import.meta.url).href,
  maxThreads: 4,
});

const results = await Promise.all(
  [40, 41, 42, 38].map((n) => pool.run(n)),
);

console.log(results);
await pool.destroy();

Install it with:

npm install piscina

Piscina also accepts minThreads/maxThreads to grow and shrink the pool on demand, and an idleTimeout to retire threads that have gone quiet — features that are tedious to get right by hand.

Sizing the pool

The right pool size depends on the workload. For pure CPU-bound tasks, more threads than physical cores only causes context-switching contention, so cap the pool at the core count. os.availableParallelism() is the modern, container-aware way to read it (it respects cgroup CPU limits, unlike os.cpus().length).

Workload shape	Suggested size
Pure CPU (hashing, math, image work)	`availableParallelism()`
Mixed CPU + occasional I/O inside the worker	Slightly above core count
One worker per request, short tasks	Small fixed pool (2–8) + a queue

Don’t size the pool to your expected request rate. The queue absorbs bursts; a pool larger than your core count just spreads the same CPU thinner and adds memory pressure.

Best Practices

Spawn workers once and reuse them — pooling exists to amortize the per-worker startup cost.
Default the pool size to os.availableParallelism() for CPU-bound work, not a hand-picked constant.
Queue tasks when all workers are busy instead of spawning more; this caps concurrency and provides natural backpressure.
Tag each task with a unique id so replies map back to the correct caller even when they finish out of order.
Handle error and exit events on every worker and reject the affected pending tasks — never let a crashed worker hang a promise.
Always call terminate() (or pool.destroy()) on shutdown so lingering isolates don’t keep the process alive.
Prefer a maintained library like piscina in production for cancellation, timeouts, and dynamic sizing.