NATS Stream Mirrors allow you to replicate streams across different NATS clusters, acting as a robust disaster recovery and high availability solution.
Let’s see this in action. Imagine we have two NATS clusters, cluster-a and cluster-b. We want to mirror a stream named my-stream from cluster-a to cluster-b.
On cluster-a, we have a stream configured like this:
{
"name": "my-stream",
"subjects": ["events.>" ],
"storage": "file",
"retention": "interest",
"max_age": "24h",
"max_msgs": 1000000,
"replicas": 3
}
Now, on cluster-b, we’ll define a mirror for my-stream. This involves creating a stream on cluster-b that points to the stream on cluster-a.
Here’s the configuration for the mirrored stream on cluster-b:
{
"name": "my-stream",
"subjects": ["events.>"],
"mirror": {
"name": "my-stream",
"cluster": "cluster-a"
},
"storage": "file",
"retention": "interest",
"max_age": "24h",
"max_msgs": 1000000,
"replicas": 3
}
Notice the "mirror" section. This tells NATS on cluster-b to look for a stream named my-stream on a cluster identified as cluster-a. When cluster-b starts up, it will discover cluster-a (assuming they are configured for inter-cluster communication, typically via routes or connect) and begin replicating messages.
The primary use case is disaster recovery. If cluster-a goes down, cluster-b will have a near real-time copy of the my-stream data. Applications can then failover to cluster-b and continue processing messages from the mirrored stream. This is achieved because the mirrored stream on cluster-b will become active and serve messages once cluster-a is unavailable.
Internally, NATS uses a specialized consumer type for mirroring. When a stream is configured with a mirror, the NATS server on the "destination" cluster (e.g., cluster-b) creates a persistent consumer that subscribes to a special internal subject on the "source" cluster (e.g., cluster-a). This internal subject effectively mirrors all new messages published to the original stream. The destination cluster then writes these messages to its local copy of the stream. The configuration for subjects, storage, retention, max_age, and max_msgs on the mirrored stream defines how the local copy behaves, while the mirror section dictates where it gets its data from.
The cluster field in the mirror configuration is crucial. It must match the name field of the source NATS cluster as defined in its nats-server configuration. If the names don’t match, or if the clusters cannot discover each other, mirroring will not establish.
A common pitfall is network configuration between clusters. For mirroring to work, NATS servers must be able to route traffic between the clusters. This is typically set up using routes in the nats-server configuration, defining direct connections between servers in different clusters, or via NATS Connectors. Without this connectivity, the mirror will simply never establish a connection to the source stream.
The replicas setting on the mirrored stream determines how many NATS servers within cluster-b will hold a copy of the data. This is independent of the replication factor on the source stream in cluster-a. For high availability within cluster-b, you’d want replicas > 1.
When a failover scenario occurs, applications reading from my-stream would typically be configured to try cluster-a first. If it’s unreachable, they would then attempt to connect to cluster-b and read from the my-stream there. The mirrored stream on cluster-b will seamlessly take over serving traffic.
One subtle aspect of mirroring is how it handles stream state. The mirrored stream on the destination cluster is not just a passive copy; it actively tracks its own sequence number. When the source cluster becomes unavailable, the destination cluster’s mirrored stream takes over and continues assigning sequence numbers from where it last received them. This ensures continuity.
The next challenge you’ll likely encounter is managing the failback process once the primary cluster is restored.