Load Balancers & Reverse Proxies

Volume 2 — Networking Foundations

This closes out Volume 2. Load balancers and reverse proxies are often the same underlying technology wearing different hats — this chapter covers both roles, the concepts behind them, and how they map onto Azure's equivalents.

1. Reverse Proxy vs. Load Balancer

They're frequently confused because the same product (NGINX, Azure Application Gateway) can do both — the distinction is about purpose, not the tool:

	Reverse Proxy	Load Balancer
Primary job	Sits in front of one or more backend services, forwarding requests on their behalf — often adding TLS termination, caching, routing rules	Distributes incoming traffic across multiple instances of the same backend for scale and redundancy
Backend count	Can front a single backend	Specifically implies multiple backend instances
Typical extras	URL rewriting, caching, request/response header manipulation	Health checks, algorithms for which instance gets the next request

In practice, most production setups do both simultaneously: a single edge component that terminates TLS, applies routing rules, and distributes traffic across several backend instances. Cloudflare (in front of the Traxs frontends) and any future Azure Application Gateway deployment both fill this dual role.

2. Load Balancing Algorithms

Algorithm	How it decides
Round robin	Cycles through backends in order, one request each, repeat
Least connections	Sends the next request to whichever backend currently has the fewest active connections
IP hash	Routes based on a hash of the client's IP, so the same client consistently lands on the same backend — a simple way to get session affinity without cookies
Weighted	Same as round robin or least-connections, but backends can be given different weights — useful when instances have different capacity, or during a gradual rollout

Round robin is the simplest and most common default. Least connections tends to perform better when requests have widely varying processing times (a mix of fast health checks and slow report-generation requests, for example) since round robin alone doesn't account for how busy each backend currently is.

3. Health Probes

A load balancer only sends traffic to backends it currently believes are healthy — determined by periodically hitting a defined health check endpoint.

GET /health
Expected: 200 OK within 5 seconds, checked every 30 seconds

If a backend fails N consecutive checks, it's pulled out of rotation automatically until it starts passing again — this is what makes a rolling deploy or a single crashed instance invisible to end users, as long as at least one other backend is still healthy and passing its probe.

This is exactly why a real /health endpoint matters — one that actually checks the things that make the service genuinely able to serve traffic (can it reach its database, are its critical dependencies up), not just "the process is running and can return a 200." A health endpoint that always returns 200 regardless of actual backend health defeats the entire purpose of the probe.

4. SSL/TLS Offload (Termination)

"SSL offload" means the load balancer/reverse proxy terminates TLS — decrypts incoming HTTPS traffic — and then talks to the backend instances over plain HTTP internally (typically over a private network where that's an acceptable trade-off).

Why this matters practically: it centralizes certificate management in one place instead of every backend instance needing its own certificate, and it takes the CPU cost of encryption/decryption off the application servers. The trade-off is that traffic between the load balancer and the backend is unencrypted — acceptable when that segment is on a private network the load balancer and backends both trust, not acceptable if that internal segment could itself be exposed.

An alternative, end-to-end TLS (sometimes called "SSL passthrough" or "re-encryption"), keeps traffic encrypted all the way to the backend — the load balancer either passes the encrypted traffic through untouched, or terminates and immediately re-encrypts before forwarding. This costs more (certificate management and CPU overhead at every layer) in exchange for encryption on every segment of the path, which some compliance requirements mandate regardless of whether the internal network is otherwise trusted.

5. Azure Application Gateway vs. Azure Front Door

Both are Azure's Layer-7 (application-aware) load balancing services, but at different scopes:

	Application Gateway	Front Door
Scope	Regional — load balances across backends within one Azure region	Global — routes across multiple regions, using Microsoft's global edge network
Typical use	Distributing traffic across multiple instances within one region's VNet	Multi-region deployments, global latency-based routing, edge caching
WAF	Available as an add-on (WAF-enabled SKU)	Available as an add-on
SSL offload	Yes	Yes

For a single-region deployment (the current Traxs setup), Application Gateway is the directly relevant option if a dedicated Azure load balancer/reverse proxy layer is ever introduced ahead of App Service's own built-in load balancing. Front Door becomes relevant specifically if Traxs infrastructure ever expands to multiple Azure regions or needs global edge-level routing — not a near-term concern for the current architecture.

6. WAF (Web Application Firewall)

A WAF sits at the same layer as a reverse proxy and inspects requests for known attack patterns (SQL injection attempts, cross-site scripting payloads, known malicious request signatures) before they ever reach the application — an additional layer on top of, not a replacement for, secure application code.

Client request → WAF (inspects, blocks known-bad patterns) → Load Balancer → Backend

Both Application Gateway and Front Door offer WAF as an add-on SKU with managed rule sets (commonly based on the OWASP Core Rule Set) that get updated centrally rather than requiring the application itself to be redeployed to respond to newly discovered attack patterns.

7. NGINX as a Reverse Proxy — a Minimal Example

Even without Azure's managed services, this is the shape of what they're doing under the hood:

upstream backend_pool {
    least_conn;
    server 10.0.1.4:5000;
    server 10.0.1.5:5000;
    server 10.0.1.6:5000;
}

server {
    listen 443 ssl;
    server_name api.roundtrips.app;

    ssl_certificate     /etc/ssl/certs/roundtrips.crt;
    ssl_certificate_key /etc/ssl/private/roundtrips.key;

    location /health {
        proxy_pass http://backend_pool;
        access_log off;
    }

    location / {
        proxy_pass http://backend_pool;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

The X-Forwarded-* headers matter specifically because, once TLS is offloaded and requests are proxied, the backend application sees the proxy's IP and protocol as the connection's origin unless these headers are explicitly forwarded and the backend framework is configured to trust and read them — a common source of "the app thinks every request is HTTP" or "every request appears to come from the same IP" bugs once a reverse proxy is introduced in front of a previously direct-facing service.

8. Troubleshooting Playbook

Symptom	Likely cause	What to check
One backend instance never receives traffic	Failing its health probe	Check the health endpoint directly on that instance; confirm it reflects real backend health, not just process liveness
App sees every request as HTTP even though the client used HTTPS	TLS was offloaded at the load balancer, and X-Forwarded-Proto isn't being read/trusted by the backend framework	Confirm the proxy sets X-Forwarded-Proto, and the backend framework is configured to honor forwarded headers
App sees every request coming from the same IP	Backend logging remote_addr/connection IP instead of X-Forwarded-For	Configure the backend to read the client IP from X-Forwarded-For (from a trusted proxy) instead of the raw connection
Uneven load across backend instances	Round robin with significantly uneven request costs, or a broken health check pulling instances in/out unpredictably	Consider least-connections; verify health check stability
Sudden spike in errors during a deploy	Load balancer routing to instances mid-restart before they're ready, or health probe grace period too short	Increase the health probe's initial grace period; confirm the rolling deploy waits for health checks to pass before shifting traffic
WAF blocking legitimate requests	An overly aggressive managed rule matching valid input as an attack pattern (a false positive)	Check WAF logs for which rule triggered; add a scoped exclusion for that specific rule/pattern rather than disabling the WAF broadly

9. Quick Reference

Category	Concept	Detail
Algorithm	Round robin	Simple, even distribution, ignores current backend load
Algorithm	Least connections	Accounts for current load, better for uneven request costs
Health	Health probe	Periodic check pulling unhealthy backends out of rotation automatically
TLS	SSL offload	Load balancer decrypts, talks plain HTTP to backends internally
TLS	End-to-end TLS	Encrypted all the way to the backend, higher overhead
Azure	Application Gateway	Regional Layer-7 load balancer, WAF add-on available
Azure	Front Door	Global Layer-7 load balancer, multi-region, edge routing
Security	WAF	Blocks known attack patterns before they reach the application
Headers	X-Forwarded-For	Original client IP, once a proxy sits in front
Headers	X-Forwarded-Proto	Original protocol (http/https) the client actually used

---

Part of the Traxs Engineering Handbook — Volume 2: Networking Foundations. Companion chapters in this volume: TCP/IP Fundamentals, DNS Deep Dive, SSL/TLS Explained, VPN Fundamentals, Network Troubleshooting.

Volume 2 — Networking Foundations is now complete: TCP/IP Fundamentals, DNS Deep Dive, SSL/TLS Explained, VPN Fundamentals, Network Troubleshooting, and Load Balancers & Reverse Proxies.