FjellSystem Design Lab
Lesson 3.4• 5 min read

Reverse proxies and edge security

TLS termination, compression and caching, web application firewalls, and rate limiting.

What you will learn

  • List what a reverse proxy does in front of your servers.
  • Say what a web application firewall blocks.
  • Compare the token bucket and leaky bucket rate limiters.

Reverse Proxy Core Functions

A reverse proxy is a web server that sits in front of internal services and gives the public one unified interface: it forwards each request to a server that can fulfill it and returns that server's response. Useful even in front of a single server, it provides:

  • Security: Hides information about backend servers, can block IP addresses, and can limit connections per client.
  • Scalability and flexibility: Clients see only the proxy's address, so you can add, remove or reconfigure servers behind it.
  • TLS termination: Decrypts incoming requests and encrypts responses, so backend servers do not have to do this potentially expensive work. Whether traffic is re-encrypted on the way to the backends is a separate security decision.
  • Compression of responses, caching of responses it has already seen, and serving static content directly.

NGINX and HAProxy can act as both a layer 7 reverse proxy and a load balancer.

Web Application Firewall (WAF)

A web application firewall applies a set of rules to HTTP traffic. Generally these rules cover common attacks such as:

  • SQL injection (SQLi): e.g., ' OR 1=1--
  • Cross-site scripting (XSS): e.g., <script>alert(1)</script>

Rate Limiting Algorithms

To protect backends from brute force and traffic surges, reverse proxies implement rate limiters:

Token bucket algorithm

A bucket holds up to BB tokens (the burst), and tokens are added at a steady rate of rr per second (the rate). Each request consumes 11 token.

  • Allowed: If tokens ≥1\ge 1, consume 11 token and forward the request.
  • Rejected: If tokens <1< 1, respond with HTTP 429 Too Many Requests, optionally with a Retry-After header telling the client how long to wait.
  • Advantage: Allows bursts up to BB requests while holding the long-run average to rr. AWS API Gateway, for example, throttles with a token bucket configured exactly this way.

Leaky bucket algorithm

Requests enter a queue and leave it at a constant rate rr; when the queue is full, new requests are rejected. This smooths bursty traffic into a steady stream. NGINX's request-rate limiting uses the leaky bucket method (and by default rejects excess requests with 503 rather than 429).

Token bucket rate limiting and a firewall

Tokens in bucket
10 / 10
Refill rate
+2 tokens/sec
Burst tolerance
Max 10 requests
Edge rate limiter & WAF decision log:
203.0.113.19GET /api/v1/feedHTTP 200Allowed (1 token consumed)
203.0.113.19GET /api/v1/feedHTTP 200Allowed (1 token consumed)

Token bucket algorithm: Tokens refill at a fixed rate (r per second) up to the bucket's capacity (B). Each incoming request consumes 1 token. When the bucket is empty, requests are rejected with 429 Too Many Requests, optionally with a Retry-After header, protecting the services behind the proxy from sudden traffic spikes.

Figure 4: Reverse proxy security perimeter terminating TLS, buffering slow clients, and rate limiting.
  • Untrusted Public Internet: 1 × Clients & Users
  • WAF & Reverse Proxy: 2 × Application Load Balancer
  • Private Backend Microservices: 6 × App Compute Server
  • Untrusted Public Internet sends requests to WAF & Reverse Proxy
  • WAF & Reverse Proxy sends requests to Private Backend Microservices

Key takeaways

  • A reverse proxy gives the public one address, hides and shields the backends, terminates TLS, compresses, caches and serves static files; it helps even in front of one server.
  • A WAF applies rules to HTTP traffic against attacks such as SQL injection and cross-site scripting.
  • A token bucket allows bursts up to BB and holds the average to rr; refused requests get 429 Too Many Requests, optionally with Retry-After.
  • A leaky bucket queues requests and lets them out at a constant rate, smoothing bursts into a steady stream.

Sources

Adapted from The System Design Primer by Donne Martin, licensed under CC BY 4.0. Changes were made.