Load Balancing Strategy: Handling Millions of Requests with HAProxy and Nginx

System Load Balancing: Combining Multiple VPS to Handle High Traffic

In the growth journey of a web application, there comes a point where even the most powerful single VPS can no longer withstand a sudden surge in traffic. This is when the "bottleneck" effect occurs—CPU stays at 100% and users begin seeing 504 Gateway Timeout errors. The sustainable solution here is not just continuing to upgrade hardware (Vertical Scaling), but expanding horizontally (Horizontal Scaling) using Load Balancing. This article will guide you through building a professional distribution system using HAProxy and Nginx.

1. Load Balancer - The "Traffic Police" of Your Infrastructure

A Load Balancer acts as the single point of entry for all internet requests. Instead of sending traffic directly to the Web Server, users send it to the Load Balancer. From there, based on pre-defined algorithms, the Load Balancer distributes traffic to the "Backend" VPS that is currently the least busy.

High Availability (HA): If one Web Server fails, the Load Balancer automatically removes it from the rotation, ensuring the website stays online via the remaining healthy servers.
Infinite Scalability: You can add 3, 5, or even 100 Web Servers behind a single Load Balancer without changing the application's public IP address.
SSL Offloading: The Load Balancer can handle HTTPS decryption, significantly reducing CPU resource usage for the Backend Web Servers.


// Example simulating a basic Round Robin algorithm for Load Balancing
interface WebServer {
 id: string;
 ip: string;
 weight: number;
 isHealthy: boolean;
}

class RoundRobinBalancer {
 private servers: WebServer[];
 private currentIndex: number = 0;

 constructor(servers: WebServer[]) {
  this.servers = servers.filter(s => s.isHealthy);
 }

 getNextServer(): WebServer | null {
  if (this.servers.length === 0) return null;
  const server = this.servers[this.currentIndex];
  this.currentIndex = (this.currentIndex + 1) % this.servers.length;
  return server;
 }
}

const cluster: WebServer[] = [
 { id: 'vps-01', ip: '10.0.0.1', weight: 1, isHealthy: true },
 { id: 'vps-02', ip: '10.0.0.2', weight: 1, isHealthy: true }
];

const balancer = new RoundRobinBalancer(cluster);
console.log(`Traffic 1 sent to: ${balancer.getNextServer()?.id}`);
console.log(`Traffic 2 sent to: ${balancer.getNextServer()?.id}`);

2. HAProxy - The Top Choice for Layer 4 and Layer 7

HAProxy (High Availability Proxy) is an extremely powerful open-source software dedicated to load balancing. It is renowned for handling tens of thousands of concurrent connections with ultra-low latency. HAProxy can operate at both the TCP level (Layer 4) and the HTTP level (Layer 7).

When configuring HAProxy, you focus on two main components: the Frontend (where requests are received) and the Backend (the list of processing servers).


// HAProxy configuration structure (simulated via JSON for UI display)
interface HAProxyConfig {
 global: { maxconn: number; user: string };
 frontend: { bindPort: number; defaultBackend: string };
 backend: {
  name: string;
  algorithm: "roundrobin" | "leastconn" | "source";
  nodes: Array<{ name: string; address: string; check: boolean }>;
 };
}

const haproxySetup: HAProxyConfig = {
 global: { maxconn: 50000, user: "haproxy" },
 frontend: { bindPort: 80, defaultBackend: "web_servers" },
 backend: {
  name: "web_servers",
  algorithm: "leastconn", // Prioritize server with fewest connections
  nodes: [
   { name: "web-01", address: "192.168.1.10:80", check: true },
   { name: "web-02", address: "192.168.1.11:80", check: true }
  ]
 }
};

console.log(`HAProxy is distributing traffic using: ${haproxySetup.backend.algorithm}`);

3. Nginx Load Balancer - Flexibility and Ease of Deployment

If you are already familiar with Nginx as a Web Server, converting it into a Load Balancer is incredibly simple. Nginx uses the upstream block to define the backend server group.

Nginx is often preferred for systems that require both Load Balancing and Static Content Caching (storing cache for static files like images and CSS).

Feature	HAProxy	Nginx
Pure Performance	Ultra-high (dedicated)	Very high
Ease of Config	Moderate	Easy
Operational Layer	L4 / L7	L7 (L4 requires stream module)
Statistics UI	Built-in (Stats page)	Requires third-party or Nginx Plus

4. Health Check Techniques - Ensuring Survival

A smart Load Balancer must not send traffic to a "dead" server. Health Checks allow the Load Balancer to periodically send small requests (like a ping or checking for HTTP 200 status) to Backend servers.

If a server fails to respond after a set number of attempts, it is marked as "Down," and traffic is immediately rerouted to other servers without interrupting the user experience.


// Logic simulating an automated Health Check process
interface HealthStatus {
 serverId: string;
 lastCheckStatus: number;
 failCount: number;
}

function performHealthCheck(server: HealthStatus): "UP" | "DOWN" {
 const HTTP_OK = 200;
 if (server.lastCheckStatus !== HTTP_OK) {
  server.failCount++;
 } else {
  server.failCount = 0;
 }

 // If more than 3 failures, mark as Down
 return server.failCount > 3 ? "DOWN" : "UP";
}

const status: HealthStatus = { serverId: 'node-02', lastCheckStatus: 502, failCount: 2 };
console.log(`Current server status: ${performHealthCheck(status)}`);

5. Session Stickiness - Maintaining User State

In applications with logins, a problem arises: A user logs into Server 1, but their next request is routed to Server 2. If Server 2 doesn't have the user's session info, they get logged out.

IP Hash: Routes users based on their IP address. The same IP always hits the same server.
Sticky Cookies: The Load Balancer inserts a cookie into the user's browser to identify and keep them on the original server.
Centralized Session: The best solution—store sessions centrally in Redis or Memcached so all Web Servers can access the same data.

6. Cost Analysis and Infrastructure Efficiency

Implementing a Load Balancer requires at least 3 VPS units (1 Load Balancer + 2 Web Servers). However, this cost is far lower than the damage caused by a website crash during a major sale or event.

A budget-optimized strategy for 2026 is to use the cheapest VPS for Web Servers and a VPS with high bandwidth for the Load Balancer. Since Web Servers only process logic, the Load Balancer is the one handling the bulk of the data throughput.


// Calculating the capacity of a Cluster
function calculateClusterCapacity(avgRequestTimeMs: number, serverCount: number): number {
 const requestsPerSecondPerServer = 1000 / avgRequestTimeMs;
 const totalRPS = requestsPerSecondPerServer * serverCount;
 return Math.floor(totalRPS);
}

const rps = calculateClusterCapacity(50, 4); // 4 servers, 50ms per request
console.log(`Total system capacity: ${rps} requests/second`);

7. Conclusion: Load Balancing Checklist

Before configuring Load Balancing for your project, ensure you have completed these steps:

Are images/videos moved to S3 or centralized storage (NFS)?
Has the Database been separated from the Web Servers?
Are all Web Servers configured with the exact same codebase?
Does the Load Balancer have an SSL certificate to secure incoming traffic?

Load Balancing is more than just software; it is a mindset of building infrastructure that is "self-healing" and infinitely scalable. Good luck with your high-traffic projects!

Load balancing using multiple VPSs to handle high traffic