NATS JetStream’s acknowledgment policies are the unsung heroes of reliable message delivery, and the "All" policy is the most surprising because it doesn’t actually require you to acknowledge all messages.
Let’s see this in action. Imagine a simple JetStream stream and consumer setup. We’re publishing messages, and our consumer is set to use the All ack policy.
package main
import (
"context"
"fmt"
"log"
"time"
"github.com/nats-io/nats.go"
)
func main() {
// Connect to NATS
nc, err := nats.Connect(nats.DefaultURL)
if err != nil {
log.Fatalf("Failed to connect to NATS: %v", err)
}
defer nc.Close()
js, err := nc.JetStream()
if err != nil {
log.Fatalf("Failed to get JetStream context: %v", err)
}
streamName := "ORDERS"
subjectName := "order.created"
consumerName := "order_processor_all"
// Ensure stream exists
_, err = js.Stream(streamName)
if err != nil {
// Stream doesn't exist, create it
_, err = js.AddStream(&nats.StreamConfig{
Name: streamName,
Subjects: []string{subjectName},
})
if err != nil {
log.Fatalf("Failed to add stream: %v", err)
}
log.Printf("Stream '%s' created.", streamName)
} else {
log.Printf("Stream '%s' already exists.", streamName)
}
// Ensure consumer exists with ALL ack policy
_, err = js.Consumer(streamName, consumerName)
if err != nil {
// Consumer doesn't exist, create it
_, err = js.AddConsumer(streamName, &nats.ConsumerConfig{
Name: consumerName,
Durable: consumerName, // Durable consumers persist their state
AckPolicy: nats.AckAll,
DeliverPolicy: nats.DeliverNew,
FilterSubject: subjectName,
})
if err != nil {
log.Fatalf("Failed to add consumer: %v", err)
}
log.Printf("Consumer '%s' created with AckPolicy: All.", consumerName)
} else {
log.Printf("Consumer '%s' already exists.", consumerName)
}
// Publish some messages
for i := 0; i < 5; i++ {
msgPayload := fmt.Sprintf(`{"order_id": %d, "item": "widget"}`, i)
_, err := js.Publish(subjectName, []byte(msgPayload))
if err != nil {
log.Printf("Failed to publish message %d: %v", i, err)
} else {
log.Printf("Published: %s", msgPayload)
}
time.Sleep(100 * time.Millisecond)
}
// Subscribe to the consumer
sub, err := js.ChanQueueSubscribe(subjectName, consumerName, func(msg *nats.Msg) {
log.Printf("Received message: %s", string(msg.Data))
// With AckAll policy, we don't explicitly ack here.
// The client library and NATS server handle it.
// If we wanted to ack explicitly, we'd use msg.Ack() here.
// For AckAll, NATS tracks the highest acknowledged sequence number for this consumer.
}, nats.BindStream(streamName), nats.BindConsumer(consumerName))
if err != nil {
log.Fatalf("Failed to subscribe: %v", err)
}
defer sub.Unsubscribe()
log.Println("Subscribed to consumer. Waiting for messages...")
// Keep the application running to receive messages
time.Sleep(5 * time.Second)
log.Println("Exiting.")
}
When you run this, you’ll see messages being published and then received. The key here is that within the ChanQueueSubscribe callback, you don’t need to call msg.Ack(). The AckAll policy tells JetStream that once any message is acknowledged for a given consumer, all messages before that acknowledged message in the stream sequence are considered delivered and processed. JetStream then automatically advances the consumer’s internal sequence pointer to that acknowledged message’s sequence number. This is why you don’t explicitly ack each message; NATS handles the tracking based on the highest acknowledged sequence.
The problem JetStream’s acknowledgment policies solve is ensuring that messages are reliably delivered and processed, even in the face of network partitions or application crashes. Without them, a message might be sent to a consumer, the consumer might crash before processing it, and the message would be lost.
Let’s break down the policies:
-
Explicit: This is the most common and granular. For every message received, the consumer must send an acknowledgment back to JetStream. This is done by callingmsg.Ack()on the received message. JetStream only advances the consumer’s position when it receives an explicitAckfor a message. If the consumer crashes before acknowledging, the message remains "unacknowledged" and will be redelivered on restart. This offers the highest guarantee of processing but requires careful management of acknowledgments. -
None: With this policy, the consumer never sends acknowledgments. JetStream treats a message as delivered as soon as it’s sent to the consumer. This is akin to "fire and forget." It’s the fastest and simplest, but offers no guarantee of delivery or processing. If the consumer crashes immediately after receiving a message, that message is lost. This is suitable for idempotent consumers where reprocessing is either impossible or harmless. -
All: This is where the surprise lies. When you set theAckPolicytoAll, you still need to acknowledge messages, but with a crucial difference. You can acknowledge any message usingmsg.Ack(). Once one message is acknowledged, JetStream considers all messages with a sequence number less than or equal to the acknowledged message’s sequence number as acknowledged for that consumer. JetStream then automatically updates the consumer’s state to reflect that all preceding messages are delivered. This is incredibly useful for consumers that process messages sequentially and can determine when a batch of work is done. For instance, if your consumer receives messages 1, 2, and 3, and it successfully processes message 3, it can acknowledge message 3. JetStream then knows that messages 1 and 2 (which came before 3) are also safe to consider delivered.
When using AckAll, the client library often handles the acknowledgment implicitly by tracking the highest sequence number it has processed. However, if you were to manually acknowledge, you’d pick a message and Ack() it. JetStream then advances the consumer’s high-water mark. This means that if your consumer receives messages 1 through 10, and it successfully processes up to message 5, acknowledging message 5 effectively acknowledges messages 1 through 5. If your consumer then crashes, messages 6 through 10 will be redelivered. This policy is beneficial when your consumer’s internal state can reliably determine the completion of a sequence of messages.
The most overlooked aspect of AckAll is that it still requires an explicit acknowledgment call from the client, but the effect of that acknowledgment is to mark all preceding messages as delivered, not just the one explicitly acknowledged. This allows for efficient batch-like processing where the consumer can signal completion up to a certain point.
The next logical step after understanding acknowledgment policies is exploring how to configure delivery guarantees, such as AtLeastOnce and AtMostOnce, which build upon these acknowledgment mechanisms.