Throughput

Throughput refers to the number of requests or operations a system can handle within a specific period of time, typically measured in requests per second (RPS) or transactions per second (TPS).

Key Concepts of Throughput

Term	Description
Throughput	The number of successful requests handled by the system per second/minute/hour.
Latency	The time it takes to process a single request.
Concurrency	The number of requests being processed at the same time.

High throughput with low latency is ideal for performance.

$$Throughput = \frac{Number of requests completed}{Time taken}$$

Why Throughput Matters in System Design

• It shows how well your system scales under load.
• It helps you determine resource needs (CPU, memory).
• It affects your user capacity and SLA performance.

Throughput vs Latency

Metric	Focus	Measured In
Latency	Time per request	Milliseconds (ms)
Throughput	Requests handled per second	RPS (req/sec)

How to Improve Throughput in Node.js

Method	Description
Use Clustering	Utilize multiple CPU cores using Node.js cluster module
Offload Work	Delegate work to background workers or queues (e.g., RabbitMQ, Bull)
Use Caching	Cache DB/API responses to avoid repeated expensive processing
Use Load Balancer	Distribute load across multiple Node.js instances
Use Asynchronous Code	Avoid blocking the event loop with CPU-intensive tasks

Measure Throughput

Install tool:

npm install -g autocannon

Run test:

autocannon -d 10 -c 50 http://localhost:3000/heavy

Explanation:

• -d 10 = duration: 10 seconds
• -c 50 = concurrency: 50 clients sending requests Output (example):

Requests/sec: 18
Latency: 51 ms

This confirms our estimate. Each request takes ~50ms → throughput ~20 RPS.

Example of Throughput

const cluster = require("cluster");
const http = require("http");
const os = require("os");

if (cluster.isMaster) {
  const cpuCount = os.cpus().length;
  console.log(`Master PID: ${process.pid}, forking ${cpuCount} workers...`);
  for (let i = 0; i < cpuCount; i++) {
    cluster.fork();
  }
} else {
  const app = require("express")();
  app.get("/", (req, res) => {
    const start = Date.now();
    while (Date.now() - start < 50) {}
    res.send(`Handled by PID: ${process.pid}`);
  });

  app.listen(3000, () => {
    console.log(`Worker PID: ${process.pid} running...`);
  });
}