Where should your server be? Latency, distance and the speed of light

A shop's server sits in Limburg, and a customer opens it from Mumbai, 6,612 km away. Before the customer's browser holds the first byte of the page, packets have crossed that distance and come back three times, and even along a perfectly straight fibre, light in glass needs about 66 ms for each of those round trips. A faster processor does not change that, and neither does a bigger network port. The limit is the speed of light.

That makes latency unusually easy to reason about. You can work out the floor for any two places with one line of arithmetic, see how far a real connection sits above it, and tell whether a slow connection is a distance problem, which only moving the server fixes, or a problem with the route, a queue or the server itself, which you can find, and then fix or report.

Latency is a round trip, and four things fill it

Latency is the time a message takes to get where it is going. It can be measured one way, as the time a request takes to reach the server, or as the whole round trip there and back.¹ In practice the round trip is the number that matters. You can measure it from one end without synchronised clocks, and it is what a browser actually waits for: almost every step in setting up a connection is a message that has to be answered before the next step can start. The usual abbreviation is RTT, round-trip time.

When a round trip is slow, the time went into one or more of four places.

• Distance. The signal goes there and back, and it cannot travel faster than light in the medium it is in. This part is fixed the moment you choose where the server is.
• Detours. The path the packets take is always longer than the straight line between the two ends, sometimes by a little and sometimes by a lot.
• Queues. Every router on the way holds packets in a buffer while its outgoing link is busy. Buffers exist to absorb bursts, and when they stay nearly full for long periods, every packet waits its turn.²
• The far end. A server that has to run code, query a database or build a page adds its own time before it replies.

The classic tool, ping, sends an ICMP echo request and times the echo reply.³No web server, database or game is involved in answering it, so ping sees the first three and not the fourth. The first two depend on where the server is, and they are what most of this guide is about.

The floor: a millisecond for every 100 km

The speed of light in a vacuum is exactly 299,792,458 metres a second, a defined value with no uncertainty at all.⁴ Long-distance internet traffic does not travel through a vacuum, though. It travels as pulses of light in optical fibre, and glass slows light down. The factor is the material's refractive index, n: light moves through the material at c divided by n.⁵ Ordinary telecom fibre has a core of doped silica with an index of about 1.4475 inside a cladding of pure silica at about 1.444,⁶so light in the core travels at about 299,792 ÷ 1.4475 = 207,000 km a second.

Data rides on pulses, and a pulse travels at the group velocity. In glass that is a little slower than the speed of a single wave crest, because the group index of glass is somewhat larger than its ordinary index.⁷The usual rule of thumb rounds all of this to one figure: a signal in fibre covers about 200,000 km a second,⁶ which is roughly two thirds of the speed of light in a vacuum (two thirds of 299,792 is 199,861).⁸

Now the arithmetic. At 200,000 km a second, one kilometre of fibre takes 5 microseconds. A round trip covers the distance twice, so each kilometre between the two ends costs 10 microseconds of round trip, and every 100 km costs a millisecond. That is the rule to keep: about 1 ms of round trip for every 100 km of fibre.

Light in a vacuum

299,792km/s

A signal in optical fibre, the working figure

200,000km/s

Of round trip for every 100 km of fibre, the working floor

1ms

The rule is a working floor, rounded a few per cent to the slow side. The core's index alone would allow 207,000 km a second, about 3.5% faster than the rounded figure, and the slower group velocity takes back only part of that gap, so a perfectly straight fibre could come in a few per cent under 1 ms per 100 km. In practice it hardly matters: no real cable runs along the shortest line, and the detours in the next section add far more than a few per cent.

Fibre is not the only medium, either. On a few routes where a millisecond is worth money, microwave links through the air have beaten it. Between the New York and Chicago stock exchanges, shortening the fibre route had brought the round trip down to 13.1 ms by 2010; in 2012 microwave links cut it to 9 ms, and later to 8.5 ms, only 0.6 ms more than light in a vacuum would need along the shortest path.⁸ That vacuum figure is the one floor nothing beats: 1 ms of round trip for every 150 km, since 300 km there and back at 299,792 km a second takes 1.0 ms.

Distance means the great circle

The distance in the rule is the great-circle distance, the shortest path between two points over the surface of a sphere.⁹ With the Earth's mean radius of about 6,371 km, the spherical formula is accurate to within about 0.5%,⁹ far closer than anything else in this calculation. Every distance in this guide comes from that formula and city-centre coordinates.

Check the rule against a published example. Sydney to New York is about 16,000 km, which puts the minimum one-way delay over fibre at 80 ms.⁶ The great-circle formula gives 15,988 km; at 5 microseconds a kilometre that is 79.9 ms one way, and 159.9 ms for the round trip. The chart at the top of this page draws the same line, with the floors for a few more city pairs on it.

Why real routes run longer than the line

The great circle is the path a packet would take if someone had laid one fibre straight from your user to your server. Nobody did. A real path is a chain of links built at different times by different companies for their own reasons, and it bends away from the line in three ways.

Cables go where cables can be laid. Across land a cable needs a route someone could build along, and under the sea it has to come ashore somewhere, so the fibre between two cities is longer than the distance between them. A route between continents also has to pass through the few places where the long cables land.

Traffic goes where networks have agreed to meet. Your user's provider hands packets to another network only where the two have agreed to connect, and that meeting point can be well off the straight line. A large measurement study counts these routing detours, together with fibre that runs longer than the geographic distance, as infrastructural inflation, and calls these inefficiencies a significant and under-explored source of latency.⁸

Sometimes the detour is absurd. In the same study's traces, packets between Eastern China and Taiwan were seen travelling first to California.⁸ This is called hairpinning, and it is why two nearby cities can have a worse connection than two distant ones. Milder versions are common: the study saw extra delay to countries whose connections may be more roundabout than average, such as Portugal, because transatlantic links from the eastern United States come ashore in Northern Europe.⁸

How much longer, measured

The most useful large measurement of this is a 2015 study by Ankit Singla, Balakrishnan Chandrasekaran, Brighten Godfrey and Bruce Maggs, who fetched the home pages of 28,000 popular websites from 186 locations and timed every stage.⁸ Their yardstick is the round trip at the speed of light in a vacuum along the great circle, which they call the c-latency. Light in fibre along that same line would already take 1.5 times as long. To see the route itself, they geolocated every router a traceroute passed through, joined the routers with the shortest lines over the Earth's surface and priced that path at the speed of light in fibre. In the median it came to 2 times the c-latency, an estimate the authors call optimistic.⁸ That is 2 ÷ 1.5 ≈ 1.33 times the straight-fibre figure: even on this optimistic estimate, the route runs about a third longer than the great circle. The timed stages sit higher again. In the median, the minimum ping took 3.2 times the c-latency and the TCP handshake 3.4 times.⁸

To compare with the floor in this guide, divide by 1.5, since the floor already assumes fibre: a median minimum ping of 3.2 times the vacuum figure is 3.2 ÷ 1.5 ≈ 2.1 times the fibre floor. On a typical path in that study the fastest ping took about twice as long as a straight fibre would allow. The data is from 2015 and paths differ a lot, so treat this as a rough expectation, not a promise: about 1 ms per 100 km is the fibre floor, and about 2 ms per 100 km is an ordinary real path. A minimum ping far above that means the path has a detour worth asking about.

A website pays the distance several times

The last bar in that chart is the one that matters most if you run a website, and it is mostly round trips. The study splits it into parts, each a median multiple of the c-latency: the DNS lookup came to 7.4 times, the TCP handshake to 3.4, the request and response (which includes the server's own time) to 6.6, and the transfer of the HTML to 10.2, because TCP's slow start needs several round trips even for a page of tens of kilobytes.⁸ Medians of the parts need not add up to the median of the whole, so these do not sum to 35.4. What they share is the unit: every part is priced in round trips, so, as the authors point out, inflation in the round trip multiplies into each of them.⁸ How slow start grows a connection is explained under big responses cost extra round trips.

Distance sets the price of one round trip; the protocols decide how many you pay before anything useful happens. A first HTTPS visit over TCP and TLS 1.3 pays one round trip for the TCP handshake,¹⁰ one for TLS¹¹ and one for the request before the first byte of HTML arrives.¹ Before all that comes a DNS lookup, the second-largest part in the study above: often one round trip to a nearby resolver, none if the answer is cached, more if the resolver has to ask other servers first (see how DNS works). The full sequence, and what the browser does once the HTML arrives, is in what happens when you open a website.

The same shop in two places

Back to the shop in Limburg and the customer in Mumbai. The great-circle distance is 6,612 km, so one round trip has a floor of about 2 × 6,612 km ÷ 200,000 km/s = 66.1 ms, and the three before the first byte about 198.3 ms. That is before any DNS lookup, before the server does any work, and on a fibre that runs exactly along the great circle, which no real path does.

Now move the shop to Mumbai and serve a customer in Delhi, 1,149 km away. One round trip has a floor of about 11.5 ms, and three come to 34.5 ms. Nothing about the server got faster. The distance shrank by a factor of about 5.8, and the waiting shrank with it. The drawing below does the same arithmetic for any distance.

Fewer round trips, same distance

Protocols can cut the count: HTTP/3 runs over QUIC, which secures the connection inside its own handshake, so when no packets are lost a new connection is usually ready after one round trip,¹² and a connection that is reused pays no setup at all (HTTP/1.1, HTTP/2 and HTTP/3 has the details). Every round trip that remains is still paid at the full distance, and a page that loads fonts, scripts and API calls from other hosts pays the setup again for each of them.

Measuring your own path: ping, traceroute and mtr

Floors tell you what is possible. To see what your users actually get, measure, and measure from where they are: a test from an office five kilometres from the server says nothing about users five thousand kilometres away.

ping: the round trip, and how much it wobbles

Given -c 5, ping sends five echo requests and stops. It prints the round-trip time of each reply, then the minimum, average and maximum, plus mdev, the population standard deviation: roughly, how far a typical reply sits from the average.³

$ ping -c 5 203.0.113.10PING 203.0.113.10 (203.0.113.10) 56(84) bytes of data.64 bytes from 203.0.113.10: icmp_seq=1 ttl=54 time=41.8 ms64 bytes from 203.0.113.10: icmp_seq=2 ttl=54 time=41.2 ms64 bytes from 203.0.113.10: icmp_seq=3 ttl=54 time=43.1 ms64 bytes from 203.0.113.10: icmp_seq=4 ttl=54 time=41.5 ms64 bytes from 203.0.113.10: icmp_seq=5 ttl=54 time=41.6 ms--- 203.0.113.10 ping statistics ---5 packets transmitted, 5 received, 0% packet loss, time 4006msrtt min/avg/max/mdev = 41.200/41.840/43.100/0.659 ms

Read the minimum first. 41.2 ms is the closest this path came to its floor: the reply that met the least queueing on the way. The average sits a little above it, and that gap, like mdev, is time spent waiting behind other traffic. On a quiet path the spread is tiny. In the 2015 study, the second-longest of 30 pings was only 1.1% slower than the fastest, in the median case.⁸ An average far above the minimum means something on the path is busy.

Then compare the minimum with the floor. Say this server is 2,000 km from the user: the floor is 20 ms, so 41.2 ms is about twice the floor, an ordinary path by the rule of thumb above. If the minimum were 120 ms, six times the floor, the packets would be taking a long way round, and the next tool shows you where.

traceroute: where the path goes

traceroute uses a safety feature of IP. Every router that handles a packet must lower its time-to-live field, the TTL, by at least one, and a packet whose TTL reaches zero is destroyed,¹³ so a packet caught in a loop cannot circle forever. A router that discards a packet this way may tell the sender with an ICMP Time Exceeded message.¹⁴ traceroute sends probes with a TTL of 1, then 2, then 3, so each one expires a router further along and that router answers. By default it sends three probes for each TTL, goes up to 30 hops, and prints each router's address with the round-trip time of every probe.¹⁵

Notice the word "may". A router is allowed not to answer, and some routers limit how quickly they send ICMP replies,¹⁵ which is why some lines show asterisks instead of times.

mtr: both at once, over time

mtr combines traceroute and ping. It keeps probing every hop and reports, for each one, how many probes were answered and how long the answers took. Its manual points out that where loss or delay climbs suddenly at one hop, the link there is often faulty or overloaded.¹⁶ In report mode, -r, it runs for the number of cycles given by -c, prints its statistics and exits,¹⁶ which makes it easy to paste into a support ticket.

$ mtr -r -c 50 203.0.113.10HOST: laptop                  Loss%   Snt   Last   Avg  Best  Wrst StDev  1.|-- 192.168.1.1            0.0%    50    1.1   1.3   0.9   3.2   0.4  2.|-- 198.51.100.1           0.0%    50    6.8   7.4   6.1  12.0   1.1  3.|-- 198.51.100.45         40.0%    50   12.3  13.0  11.8  19.5   1.6  4.|-- 192.0.2.17             0.0%    50   40.9  41.5  40.2  45.0   0.9  5.|-- 203.0.113.10           0.0%    50   41.7  42.1  41.0  46.3   0.8

Two things to read in it.

• The jump is where the distance is. Times climb a little from hop to hop, then leap from about 13 ms at hop 3 to about 41 ms at hop 4. That leap is the long-distance link, and every hop after it inherits it. The jump tells you which link carries the distance, and whether it is a distance you expected.
• Loss and delay count only if they last to the end. Hop 3 shows 40% loss, yet hops 4 and 5 lose nothing. Packets really dropped at hop 3 could never have reached hop 5, so this is hop 3 not replying to every probe that expires there (routers may rate-limit those ICMP replies), not loss of your traffic. Loss matters when it starts at a hop and carries on all the way to the last line. The same goes for time: a hop that looks slow while every hop after it is fast is only slow to reply, not slowing your packets.

A trace also shows only the way out. Replies can come back by a different route, which a trace from your side cannot see. That is why, when a path to your server is slower than its floor suggests, the Clouduxe help centre's guide to diagnosing high latency asks for an mtr in both directions, along with a long ping from an affected machine.

Mistakes that add latency, or hide it

• Putting the application and its database in different regions. Every query is a round trip. With the application in Mumbai and the database in Limburg, each one costs about 66.1 ms at the floor before the database does any work, so a page that runs 20 queries one after another waits 20 × 66.1 ms, about 1.3 seconds, and more on a real route. Keep the two in the same place, and put that place near the users.
• Moving the server to fix a slow application. If the minimum ping sits near the floor and pages are still slow, the time is going into the fourth part, the far end. A move saves distance, not the time your code spends building each page; the web server stack is where to look instead.
• Reading a speed test's ping as your server's latency. A speed test measures the path to whichever test server it picked, which is not your server. It tells you about the user's own connection, not about the distance to you.
• Choosing from the map alone. The table below gives the best case. The route decides the rest, and a hairpin can make a near region slower than a far one. Where you can, measure from your users' networks before you commit.

Choosing a region for users spread across a continent

With a floor you can calculate, choosing a region becomes arithmetic. List where your users are, work out the floor from each candidate region, and weight it by how much each group talks to the server. The table does the first two steps for a dozen cities and the three regions Clouduxe runs in.

Weigh users by the round trips they make

Head count is the obvious weight, and the wrong one. A visitor reading an article pays a handful of round trips per page. A logged-in user of a web app pays one for every click, and a player on a game server exchanges packets for the whole session; how often the server updates the game world is its own subject, covered in tick rate and the main thread.

Say 60% of an audience are readers in Southeast Asia and 40% are logged-in users of an app in India, with Jakarta and Delhi standing in for each group, and say a reader's visit makes about 20 round trips and an app session about 300. Count heads and Singapore looks better: its floors are 9.1 ms to Jakarta and 41.4 ms to Delhi, an average of 0.6 × 9.1 + 0.4 × 41.4 = 22.0 ms, against 0.6 × 46.6 + 0.4 × 11.5 = 32.6 ms from Mumbai.

Now count round trips. For every 100 people there are 60 × 20 = 1,200 reader round trips and 40 × 300 = 12,000 app round trips. From Mumbai, 1,200 × 46.6 ms comes to about 56 seconds and 12,000 × 11.5 ms to about 138: 194 seconds of waiting in all. From Singapore, 1,200 × 9.1 ms is about 11 seconds and 12,000 × 41.4 ms about 497: 508 seconds. The region that lost on head count comes out about 2.6 times better, because the group it serves well is the one making the round trips. Put the server nearest the group whose work is interactive, and make everything else cheap for the rest.

When the groups are a continent apart, no single region is kind to both. Delhi is 11.5 ms from Mumbai and 61.6 ms from Limburg; London is 5.9 ms from Limburg and 71.9 ms from Mumbai. A server halfway between Delhi and London would give each side half of the 67.1 ms floor between them, about 33.6 ms: roughly 3 times what Mumbai gives Delhi and 6 times what Limburg gives London, with no promise about the routes to it. The honest options are to pick the side where the interactive work is, or to run a server on each side and take on the work of keeping their data in step.

Let a CDN carry the static files

Much of what a page downloads is the same for every visitor: images, stylesheets, scripts, fonts. A content delivery network keeps copies of that kind of data on servers in many places and answers each request from one close to the user.¹⁷ The round trips for those files then happen over a short distance, and only the dynamic requests travel all the way to your server: the page built for one user, the API call, the checkout. That split is what makes one region workable for an audience spread across a continent. Put the server where the interactive users are, and let the CDN shorten the distance for everything that is the same for everyone.