Why Your p99 Latency Destroys Node.js Event Loop Throughput - Theory and Practice

Everyone is tired of hearing that Node.js runs on a single event loop. I am aware of it.

"One slow task blocking that loop means every other request waits."

Engineers know this, but knowing is different from understanding and seeing in action.

This post is an interactive playground: tweak arrival rate and latency distribution, and watch what happens to queue depth and response time. The simulation runs entirely in your browser via a Web Worker, which keeps its calculations off the page's main thread.

New to queueing theory?

Terms like Markovian, service time, and general distribution are defined in the Queueing Theory Glossary.

The model

A single event loop is a single-server queue. Requests arrive, get processed one at a time, and every arriving request that finds the server busy joins the queue.

This is the classic M/G/1 queue:

• M — arrivals follow a Poisson process, a model for random independent arrivals
• G — service times follow a general distribution, meaning they may have any statistical shape
• 1 — one server (the event loop thread)

For an M/G/1 queue, the Pollaczek–Khinchine formula gives the expected waiting time:

$$W_q = \frac{\lambda \cdot E[S^2]}{2 \cdot (1 - \rho)}$$

Where:

• $\lambda$ = arrival rate (requests per second)
• $E[S]$ = mean service time
• $E[S^2]$ = second moment of service time (variance + mean²)
• $\rho = \lambda \cdot E[S]$ = utilization, which must be below 1 for stability

The key insight: waiting time depends not just on average service time, but on $E[S^2]$—and therefore on variance, or how spread out the service times are. A long tail (high p99 relative to p50) blows up $E[S^2]$ and sends wait times skyrocketing even at moderate utilization.

Try it yourself

Set your arrival rate in requests per second (RPS) and your latency distribution in milliseconds. The simulator uses piecewise-linear interpolation between your chosen percentiles, then falls off exponentially in the tail beyond p99. Try switching these parameters and then compare them with the Node.js version here!

• 100 RPS at 30 seconds
• 130 RPS at 30 seconds
• 150 RPS to 150 at 30 seconds
• 200 RPS to 300 at 30 seconds

You will understand in the next section!

RPSDuration (seconds)p50 latency (ms)p75 latency (ms)p95 latency (ms)p99 latency (ms)

What to notice

1. The hidden tax of variance

Two latency distributions can have similar means (p75) but wildly different mean service time and queueing behavior.

Scenario	p50	p75	p95	p99	Mean service
A	2 ms	3 ms	5 ms	8 ms	~3.2 ms
B	2 ms	5 ms	20 ms	50 ms	~5.6 ms

Scenario B is what happens when a database hiccups, a downstream service flakes, or a CPU-bound task sneaks into the event loop. Even though the median (p50) is the same, the heavier tail means $E[S^2]$ is much larger—and the queueing penalty explodes.

2. The ρ → 1 wall

At 100 RPS, $\rho < 0.6$, queueing delays are modest. You might not notice the event loop constraint at all.

At roughly 130 RPS, $\rho = 0.8$, things get spicy. The tail of the response time distribution stretches out.

At 150 RPS, $\rho = 0.95$, the queue is almost never empty. One slow request kicks off a backlog that takes many fast requests to drain.

At 200 RPS above with $\rho \geq 1$, the queue grows without bound. The theoretical mean wait becomes infinite. In practice, memory runs out or requests time out first.

3. Why this matters for Node.js

In Node.js, that "queue" is the event loop itself. There is no separate thread pool handling your synchronous JavaScript — just one loop.

A single 50 ms blocking call at the wrong moment forces every other callback, timer, and input/output (I/O) completion to wait. The p99 latency of your service becomes the p99 of the worst thing currently running, plus whatever queue has built up behind it.

That is why setTimeout(fn, 0) and process.nextTick do not help when the loop itself is the bottleneck. They just reshuffle the same queue.

Takeaways

• Monitor event loop lag, not just endpoint latency. A healthy service can have terrible loop lag.
• Treat high-variance tasks as a capacity problem. Even if they are rare, they determine your queueing penalty through $E[S^2]$.
• Offload CPU-bound work to worker threads, child processes, or other services. Keep the event loop for I/O coordination.
• Set latency objectives at p99 or p999, not just average. The tail is what breaks you under load.
• M/G/1 does not punish only $E[S]$ (mean). It punishes at $E[S^2]$ (p99 variance to p50).

Feel free to play with the numbers above. Find the $\rho$ where your setup transitions from "fine" to "on fire." That is your real operating limit.