How DDoS protection works: scrubbing, filtering and rate limits

A flood of 100 Gbit/s aimed at a server with a 1 Gbit/s port is decided before a single packet reaches the machine. The link in front of it is full a hundred times over, and whatever runs on the server, however clever, sees one gigabit of the hundred and can do nothing about the other ninety-nine. A flood of a few thousand ordinary-looking requests a second, carried inside encrypted connections, is the opposite problem. It crosses every router and filter on the way without looking unusual, and only the program that decrypts it can tell that anything is wrong.

So the question worth asking about DDoS protection is where it sits. Every place that can stop traffic trades what it can see against how much it can absorb, and following the path from the internet inward to your application shows what each one is good for and where it goes blind.

Every layer trades sight for capacity

An attack needs something to run out (the guide to DDoS attacks sorts them by what that is), and the simplest is the link. Traffic that ignores congestion, sent faster than a link can carry it, fills the queues in front of that link until real packets are lost along with the attack's.¹ A filter on the server cannot undo that, because by the time it sees a packet the packet has already used the scarce link, so this kind of flood has to be stopped somewhere nearer the attacker.¹

Closer to the attacker means further from you, and that is the trade. A network many hops upstream has room to take a flood in, but it carries everyone's traffic, has never seen your protocol, and must judge millions of packets a second with rules general enough for all its customers. Your application knows exactly what a valid request looks like, but by the time it can check one, the request has crossed the link and the kernel and, on HTTPS, been decrypted.

Short of having vastly more capacity than any attacker, there is one real defence against an attack that uses up resources: treat attackers differently from everyone else, which means finding "something that legitimate users can do but attackers can't".¹

Upstream: capacity and scrubbing, far from your server

Upstream capacity exists to take in a flood that no single customer's link could carry, and to sort it before the traffic narrows down to that link. That work falls to your providers, which is where the UK National Cyber Security Centre's guidance on volume starts.²

Sorting a flood is called scrubbing, and it only works if the traffic reaches the scrubber before it reaches you. A protection service has to pull your traffic away from your network, drop the attack and hand the rest back.³ What gets dropped is decided by rules that range from blocking any address that has sent too much to far more involved logic.²

How traffic gets to the scrubber and back

The pulling is done with routing. Either the scrubbing network announces your prefix over BGP, so that the rest of the internet prefers the path through it, or your provider points its own routes for those addresses at its scrubbers. Routes more specific than a /24 are usually not accepted across the internet,⁵ so a service that diverts by announcement moves at least a whole /24, 256 addresses, at a time. What survives the filtering is handed back to your network over a separate path, such as a direct interconnect, rather than across the open internet, where it would meet the flood again.

The timing depends on how the service runs. Always-on scrubbing keeps your traffic flowing through the filters all the time. On-demand scrubbing diverts only after it has detected an attack, so every flood gets a head start: from its first packet until it has been noticed and the new routes have spread, it lands on your link as if there were no protection at all.

What an upstream filter can see

A scrubbing network sees packets in bulk: addresses, ports, protocols, sizes and rates, and how they change. That is enough for the crudest floods. A server that asked no DNS questions has no reason to receive hundreds of thousands of DNS answers a second, which is what a reflected DNS flood looks like, so those packets can be dropped without knowing what the server runs. The guide to amplification attacks explains how a small spoofed request becomes that large reply.

What it cannot see is what your traffic means. A provider filtering for many customers has little feel for the fine detail of any one application, and once the traffic is encrypted, as HTTPS is, it has even less to go on when looking for anything unusual.² An upstream filter can tell that 300 Mbit/s of UDP is arriving at your game port, but not whether it is players or a flood built to look like them. That judgement has to happen closer to you.

Blackholing: the emergency brake that finishes the attack

When a flood is bigger than the links it has to cross, it hurts more than its target. If more attack traffic heads for one customer than the provider's links to that customer can carry, real traffic to that customer is crowded out, and other customers and the provider's own network often suffer with it.⁴

The provider's tool for that situation is the blackhole. Remotely triggered blackhole filtering, or RTBH, is a widely used way of mitigating denial of service,⁴ and it works through the routers' forwarding tables rather than their filters. Every edge router is given a discard route in advance, a route to a null interface. When an address comes under attack, one trigger router announces a route for that address over BGP, tagged with an agreed community and with NO_EXPORT so it stays inside the network, and each router that receives it points the address at the discard route.⁴ Routes are used instead of filters for scale: a router's forwarding path handles more traffic, and far more entries, than its filters can.⁴

To ask a neighbouring network to do the same, there is a standard tag: BLACKHOLE, a well-known BGP community that asks the neighbour to discard everything sent to the tagged prefix.⁵ Its value, 0xFFFF029A, splits into two 16-bit halves: 0xFFFF is 65,535 and 0x029A is 2 × 256 + 154 = 666, so it is written 65535:666, 666 being the number operators already associated with blackholing.⁵ It comes with guard rails. The receiving network should stop the route spreading further (NO_EXPORT or NO_ADVERTISE), and the prefix should be as specific as possible, typically one address: a /32 in IPv4, a /128 in IPv6.⁵ That cuts across normal practice, in which routes more specific than a /24 are usually refused, so networks that honour BLACKHOLE accept these longer routes as an exception: only when tagged, only from a neighbour authorised to announce a prefix covering them, and only on sessions where both sides agreed to it.⁵ Blackholing a whole /24, 256 addresses, to deal with one server would take 255 innocent neighbours down with it.

Why a blackhole completes the attack

RFC 5635 is candid that a destination blackhole finishes the job for the attacker. Every packet to that address is discarded, the attack's and your users' alike, and the target is offline. The rest of the network is spared, and the target, if anything, is worse off than before.⁴ An attacker who wanted your server unreachable has got exactly that, and can stop spending.

That makes the blackhole the provider's emergency brake, not a defence of the target. It can be pulled at the customer's request, or by the provider on its own judgement to keep the network running.⁴ Source-based RTBH turns the idea around and discards by source address, which cuts off all traffic to and from each listed address, not only what it sends to the victim.⁴ Against a directly spoofed flood that is useless, because the source addresses are invented; the real fix for forged sources is ingress filtering at the edge of the networks they come from,⁶ which the victim does not control. Against a reflected flood it can work, because reflectors are real servers at fixed addresses. The cost is losing contact with each of those servers in both directions, and a large flood can come from a great many of them.

FlowSpec: dropping one kind of traffic, not everything

Between the blackhole and full scrubbing sits BGP Flow Specification, or FlowSpec. It uses BGP to carry filter rules instead of routes: each rule matches fields such as the destination prefix, the IP protocol, source and destination ports and the packet's length, and carries an action, such as discarding what matches or capping its rate.⁷ It exists mainly so that a network can have its upstream drop unwanted traffic at the upstream's own ingress routers.⁷ A rule to discard UDP from source port 53 to 203.0.113.10 removes a reflected DNS flood at the provider's edge while every other packet to that address keeps flowing, at the price of any DNS answers the server really asked for.

The rules run on the routers' access-list and firewall features,⁷ the slower and scarcer resource that RTBH was designed to avoid,⁴ so a network can hold only so many of them. And they match headers, not meaning: a forged packet with the right protocol, port and length passes as easily as a real one.

On the host: every packet that arrives, and nothing that did not

Whatever survives upstream meets your server's link, which has a fixed speed and no judgement. Anything that fits through it can be judged with far more context than any upstream network has.

On Linux the earliest place to judge a packet is XDP, which runs an eBPF program at the earliest possible point in software, the moment the network driver picks the packet up,⁸ and in native mode before the kernel allocates a socket buffer for it.⁹ A program that returns XDP_DROP discards the packet there without spending anything more on it, which is why DDoS mitigation is a standard use.⁸⁹ The guide to XDP and eBPF covers how it works. What matters here is that a drop is paid for once, at the cheapest place on the machine, not again in a firewall, the TCP stack and the application. That counts most in a packet flood: a 1 Gbit/s port can deliver about 1.49 million minimum-size packets a second (the arithmetic is in the guide to DDoS attacks), each a separate decision.

What a host filter gains is context. It sees every packet that arrives, in full, including the payload of unencrypted protocols, and it can keep state per source: which addresses completed a handshake, which answered a challenge. Its rules can be specific to one server's services instead of general enough for a whole network. Its blind spots are the two ends of the path: a flood the link has already dropped, and anything inside TLS.

State has a price. A stateful firewall can be attacked in a way a stateless one cannot: by making it keep too many entries, or entries arranged to make every lookup slow. An attacker who can predict its hash function can push each lookup from O(1) to O(n).¹ So a host filter should hold state only for sources that have earned it, the principle behind SYN cookies, which SYN floods explained covers.

Rate limits count, they do not understand

A rate limit counts something per key over time and drops what goes over a threshold. The key can be a source address, a destination port, a connection, or all traffic at once. Rate limits are cheap and predictable, and good at one job: stopping a handful of sources that each send far too much. They have two blind spots, and attacks aim at both.

Floods inside established connections

A limit on new connections counts handshakes, and a modern client needs few. HTTP/2 multiplexes requests: each request and its response travel as a separate stream inside one connection.¹⁰ One bot can open a single TLS connection and send requests down it for as long as the server keeps it open, and a connection-rate limit counts it once. A filter on the path sees encrypted records of ordinary sizes, because TLS is designed to prevent eavesdropping and tampering¹¹ and the requests are inside it.

Rapid Reset, exploited in the wild from August to October 2023,¹² went further: HTTP/2 lets a client cancel requests, and resetting many streams very quickly tied up server resources,¹³ all inside connections that were already open. HTTP/2 lets each side cap how many streams the other may have open at once, but only open and half-closed streams count against that cap, and cancelling a stream closes it at once.¹⁰ A stream cancelled a moment after it starts frees its place for the next one, while the server may still be working on the request it carried. That is exactly the room Rapid Reset used. The protocol's own advice covers the case: an endpoint should keep track of features that can be abused, set limits on how much each is used, and may close a connection that behaves suspiciously.¹⁰ Only the endpoint that decrypts the traffic can apply limits like that, or limit requests per client, per URL or per connection.

Shared addresses and floods spread thin

The second blind spot is the key. A per-source limit assumes that one address means one sender, and on today's internet it often does not: carrier-grade NAT puts many subscribers behind one public IPv4 address. A server that counts requests per address and sends a noisy one to a penalty box then locks out everyone behind it, and once sharing is widespread, penalty boxes stop working as a defence against abuse.¹⁴

Forged source addresses exploit the same assumption from the other side, spreading a flood so thin that each address looks quiet: 375,000 packets a second from 150,000 forged addresses is 375,000 ÷ 150,000 = 2.5 packets a second each, and a limit low enough to catch that would cut off any real client sending more than twice a second. A limit on the port's total catches the volume but cannot tell who is who, so it drops real and fake packets in proportion.

False positives, and why protocol awareness cuts them

A false positive is a real user dropped as an attacker. Every generic rule produces them, because a rule that only counts or matches shapes cannot tell a surge of real users from a flood. In principle, a subtle enough attack is indistinguishable from a sudden crowd of real visitors.¹ The worked example below has a small case of it: 40 players behind one carrier address, each sending 20 packets a second, add up to 40 × 20 = 800 a second from that address, which a counter set for one player reads as an attack.

The way out is the test from the start of this guide, something real users can do and attackers cannot. On the network, the most dependable such thing is speaking the protocol. A real client completes the service's handshake, answers the server's challenge and sends packets structured the way the service expects. A forged source cannot complete a handshake at all, because the server's reply goes to the address it forged, and a flood built from a recorded packet repeats one shape endlessly. A protocol check asks "is this the service speaking?" instead of "is this too much?", and the answer does not depend on how many people share an address.

Protocol awareness has costs. A validator has to be written for each protocol, and a check running on a port that serves something else drops real traffic as surely as a crude limit, so protocol checks are tied to known services on known ports. It also stops at encryption: a check can read a game's handshake or a DNS query, but not an HTTPS request.

Application-layer filtering: full sight, but only after TLS

At the end of the path is the program that answers: nginx, your application server, the game. It sees what no other layer can: the URL, the user, the session and the cost of each request. Attacks that look legitimate everywhere else can finally be recognised here, such as a flood of searches that each cost a database query.

For HTTPS, "application layer" has a precise meaning: after TLS is terminated. Anything before that point sees encrypted records, so a third party can filter your requests only if it terminates TLS for you. For a CDN in front of an HTTPS site, that means the CDN holds your private key, or a certificate that clients accept as yours.² In return it can work on requests, for example by rate-limiting particular URLs,² which is how the search flood in part three of the worked example below could be stopped before it reaches your server. It only helps if the attacker cannot go around it: an origin whose real address leaks can be attacked directly, so the CDN has to keep that address hidden.²

The price of full sight is capacity. Every request that reaches the application has already cost a TCP handshake, a TLS handshake, a connection-table entry and decryption before any of your code runs. Filtering here is the most precise and the most expensive per decision, so it has to be the last layer, never the only one.

One attack, five layers: a worked example

Take one server with a 1 Gbit/s port. It runs a game on UDP and a website on HTTPS, and on a normal evening it receives 20 Mbit/s of real traffic, including 5,000 packets a second from 250 players sending 20 each. Forty of those players are on one mobile carrier that puts them all behind one shared address. An attack arrives in three parts at once. The numbers are illustrative, not from a real incident.

Part one: 80 Gbit/s of reflected DNS answers. With nothing upstream, 80 Gbit/s of attack and 20 Mbit/s of real traffic compete for a 1 Gbit/s link, which carries about one bit in eighty of what arrives: 1 ÷ 80.02 ≈ 1.25%. Real traffic gets through in the same proportion, 20 × 0.0125 = 0.25 Mbit/s, which for players is no service at all. The server's own filter can drop every DNS answer it sees and achieve nothing, because the loss happened in front of it. A blackhole takes real traffic from 1.25% to none and completes the attack. Only something upstream helps, scrubbing or a FlowSpec rule for UDP from port 53: 80 Gbit/s of DNS answers to a server that asked no questions is easy to recognise by its shape, and it never reaches the link.

Part two: 300 Mbit/s of forged game traffic. Packets of 100 bytes built to resemble the game's own: 300,000,000 ÷ 8 ÷ 100 = 375,000 packets a second, from 150,000 forged addresses. Upstream it is unremarkable, UDP to the right port at a rate the port can carry. Per-source limits get it backwards: each forged address sends 2.5 packets a second, under any limit a real player could live with, while a limit of 100 a second, five times one player's rate, cuts off the 40 players on the shared address, who send 40 × 20 = 800 between them. A cap of 50,000 packets a second on the whole port lets through 50,000 of the 380,000 arriving, about 13%, so the players lose 87% of their packets along with the attack. A protocol check on the host drops all 375,000, because none of the forged sources can finish the game's handshake, and passes the 5,000 that can. Dropped at the driver, the forged packets never cost the kernel a socket buffer.

Part three: an HTTPS request flood. 2,000 bots each open one connection to the website and send 10 requests a second over HTTP/2, which makes 20,000 requests a second. At about 500 bytes a request that is 20,000 × 500 × 8 = 80 Mbit/s, 8% of the port. Upstream it is ordinary HTTPS. A connection-rate limit sees 2,000 connections opened once, then nothing new. The host filter can judge the connections, how many each address holds and whether their handshakes were genuine, but not what travels inside them. The TLS endpoint, your web server or a proxy holding your key, sees 20,000 requests a second for the search page, and is the only place a per-client request limit can stop them.

Real traffic that crosses the flooded port with nothing upstream

1.25%

Real traffic after a blackhole: the attack completed

0%

From each forged address, under any limit a real client could pass

2.5packets/s

What a connection limit sees of each bot in part three

1connection

Putting the layers together

Mistakes that break the stack

Counting a blackhole as protection. It protects the provider's other customers. For the target it delivers the attacker's goal.

Tightening rate limits until the attack stops. It stops when the limit is low enough to block your busiest real users too, often the ones sharing an address. If a limit has to sit below a real client's normal rate to catch a flood, it is the wrong tool for that flood.

Reading one big capacity figure as complete protection. Capacity answers volume. Parts two and three of the worked example came to 380 Mbit/s between them, 38% of a single port, and capacity did nothing about either.

What to set up before you need it

• Ask your provider the awkward questions. Find out what mitigation is on your account, whether it is applied automatically and whether you are told, and whether the provider reserves the right to sink-hole your traffic to protect its other customers.²
• Tell the filter what each port serves. Protocol checks run only on ports they know about. List the ports your services really use, describe each one, and close the rest.
• Put request limits where TLS ends. Rate-limit the expensive URLs per client in your web server or proxy, and patch its HTTP/2 support (see Rapid Reset above).
• Know what normal looks like. Monitoring that records a baseline of normal traffic has to be running before an attack to be of any use during one.¹ A limit set without one is a guess.

One piece of that baseline is who holds your connections. This lists the addresses with the most established connections to port 443 on your own server:

$ ss -Htn state established '( sport = :443 )' | awk '{print $4}' | sed 's/:[0-9]*$//' | sort | uniq -c | sort -rn | head -5    212 203.0.113.40     31 198.51.100.7      4 192.0.2.19      3 192.0.2.88      2 198.51.100.23

Here 203.0.113.40 holds 212. That could be a crawler, or an office, a university or a mobile carrier with hundreds of people behind one address. Find out which before a limit decides for you.