Global Anycast load balancing isn’t just about sending traffic to the "closest" server; it’s a sophisticated illusion that leverages the internet’s routing protocol to make the entire network appear as if it’s hosted in one massive, distributed location.
Imagine you’re trying to reach a popular website. Your request doesn’t just magically find the closest server. Instead, it enters the global Border Gateway Protocol (BGP) routing system. Anycast works by advertising the same IP address from multiple physical locations around the world. When your local router needs to send traffic to that IP, BGP, through a complex set of algorithms, determines the "best" path. This "best" path is typically the one with the fewest routing hops, which usually correlates to geographical proximity, but it’s actually about BGP’s view of network topology, not raw distance.
Let’s see this in action. We’ll simulate a simple Anycast setup using BGP.
First, we need two routers (simulated here with docker containers running frr - Free Range Routing) in different "regions" that will both announce the same public IP address. Let’s say our Anycast IP is 192.0.2.100.
Router A (e.g., US-West)
# Dockerfile for Router A
FROM frrouting/frr:latest
RUN apt-get update && apt-get install -y iproute2 && rm -rf /var/lib/apt/lists/*
COPY frr.conf /etc/frr/frr.conf
EXPOSE 179 # BGP port
frr.conf for Router A:
! frr.conf for Router A
hostname RouterA
log syslog
!
password frr
enable password frr
!
! Basic interface configuration (e.g., loopback and an "external" interface)
interface lo
ip address 10.0.0.1/32
!
interface eth0
ip address 10.0.1.1/24
!
! BGP configuration
router bgp 65001
bgp router-id 10.0.0.1
bgp cluster-id 10.0.0.1
! Neighbor config for an iBGP peer or a real upstream provider
! neighbor 10.0.1.2 remote-as 65001 # Example iBGP peer
!
! Announce the Anycast IP prefix
network 192.0.2.100/32
!
! Static routes if needed for connectivity
! ip route 0.0.0.0/0 via 10.0.1.254
Router B (e.g., EU-Central)
# Dockerfile for Router B
FROM frrouting/frr:latest
RUN apt-get update && apt-get install -y iproute2 && rm -rf /var/lib/apt/lists/*
COPY frr.conf /etc/frr/frr.conf
EXPOSE 179 # BGP port
frr.conf for Router B:
! frr.conf for Router B
hostname RouterB
log syslog
!
password frr
enable password frr
!
! Basic interface configuration
interface lo
ip address 10.0.0.2/32
!
interface eth0
ip address 10.0.2.1/24
!
! BGP configuration
router bgp 65001
bgp router-id 10.0.0.2
bgp cluster-id 10.0.0.2
! Neighbor config
! neighbor 10.0.2.2 remote-as 65001
!
! Announce the SAME Anycast IP prefix
network 192.0.2.100/32
!
! Static routes if needed
! ip route 0.0.0.0/0 via 10.0.2.254
Now, imagine these routers are connected to different upstream Internet Service Providers (ISPs) that are part of the global BGP routing fabric. Both Router A and Router B will advertise to their respective peers that they can reach 192.0.2.100.
A client in New York trying to reach 192.0.2.100 will query its local DNS resolver. That resolver, or the client’s local router, will then perform a BGP lookup. The BGP decision process will analyze the advertised paths from various Autonomous Systems (AS) and select the AS that offers the "best" path. If Router A is connected to an ISP that has a more direct route (fewer AS hops) to the client’s network than Router B, the traffic will be routed towards Router A. The client thinks it’s talking to a server in the US, even though there’s an identical server in Europe.
The critical lever here is the BGP attribute tuning. While the network command tells BGP to advertise a prefix, the actual path selection is influenced by attributes like AS-Path length, MED (Multi-Exit Discriminator), Local Preference, and community tags. For instance, if you have two BGP sessions to the same AS, you might use MED to influence which exit point the upstream AS prefers. By default, BGP aims for the shortest AS-Path, which is why geographical proximity often wins, but this can be manipulated.
What most people don’t realize is that Anycast isn’t inherently about low latency; it’s about routing efficiency as perceived by BGP. A server in Australia could, technically, be the "closest" Anycast endpoint to a user in Europe if the BGP routing table dictates a shorter path through the Australian AS. The goal is typically to reduce the number of AS hops, which indirectly leads to lower latency and higher reliability by avoiding congested peering points.
The next concept you’ll grapple with is how to ensure your Anycast IP is reachable even if one of your edge locations goes down, which leads to discussions on health checks and dynamic BGP route withdrawal.