Fix Kafka OffsetOutOfRangeException Errors (2026)

The Kafka broker rejected your consumer’s fetch request because the offset it provided was beyond the current end of the log for the partition.

Common Causes and Fixes

1. Consumer Lagging Significantly

Diagnosis: Check consumer group lag using kafka-consumer-groups.sh.
```
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my-consumer-group
```
Look for partitions where the LAG is very high or LAST OFFSET is -1 (meaning the consumer has never committed an offset or lost its last committed offset).
Fix: If the consumer is genuinely lagging and you want to catch up, there’s no direct "fix" for the OffsetOutOfRangeException itself other than letting the consumer catch up or resetting its offset. If you want to reset the offset to the latest available:
```
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group my-consumer-group --topic my-topic --reset-offsets --to-latest --execute
```
This command tells Kafka to update the committed offset for my-consumer-group on my-topic to the current end of the log for all partitions. The consumer will then resume fetching from the latest messages.
Why it works: The OffsetOutOfRangeException happens because the consumer thinks it processed up to offset X, but the broker’s log for that partition has already advanced past X. Resetting to latest aligns the consumer’s expected offset with the broker’s current log end, allowing it to fetch new messages.

2. Consumer Restarts After Broker Log Compaction/Deletion

Diagnosis: Examine broker logs for INFO messages related to log retention and compaction. Check log.retention.hours, log.retention.bytes, or log.cleanup.policy in server.properties. If log.cleanup.policy=compact, older messages are deleted. If retention periods are short and the consumer was down for longer than that, its committed offset might be gone.

Fix: If log compaction or short retention is the culprit and you need to reprocess old messages, you’ll need to manually reset the consumer’s offset.

To reset to the earliest available offset:

kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group my-consumer-group --topic my-topic --reset-offsets --to-earliest --execute

To reset to a specific offset (e.g., offset 1000):

kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group my-consumer-group --topic my-topic --partition 0 --reset-offsets --to-offset 1000 --execute

Replace 0 with the partition number and 1000 with the desired offset.

Why it works: Log compaction and retention policies actively prune old data. If a consumer’s committed offset points to data that has been purged, it will trigger this exception. Resetting to earliest or a specific older offset makes the consumer’s expected position valid again.

3. Incorrect Consumer Offset Commit Strategy

Diagnosis: Review the consumer code. Are you using enable.auto.commit=true with a short auto.commit.interval.ms? Or are you manually committing offsets? If manually committing, ensure it’s happening after processing messages and that the commit is successful. A failure to commit means the consumer might restart with an old, now invalid, offset.

Fix: If using manual commits, ensure you commit after successfully processing messages.

// In your consumer loop
try {
    ConsumerRecords<Key, Value> records = consumer.poll(Duration.ofMillis(1000));
    for (ConsumerRecord<Key, Value> record : records) {
        // Process record
        System.printf("Processing offset = %d, key = %s, value = %s%n", record.offset(), record.key(), record.value());
        // Assume processing is successful
    }
    // Commit offsets AFTER successful processing
    consumer.commitSync(); // Or commitAsync()
} catch (CommitFailedException e) {
    // Handle commit failure, potentially restart or log
    System.err.println("Offset commit failed: " + e.getMessage());
    // Re-throw or handle appropriately
} catch (Exception e) {
    // Handle processing exceptions
    e.printStackTrace();
    // If processing failed, DO NOT commitSync()
}

If enable.auto.commit=true is used and you suspect issues, consider switching to manual commits for more control, or increase auto.commit.interval.ms if processing is slow.

Why it works: OffsetOutOfRangeException often arises when a consumer restarts and its last committed offset is no longer valid. By ensuring commits happen only after successful processing and by handling commit failures, you prevent the consumer from losing its progress and trying to fetch from a non-existent offset.

4. Network Interruption Leading to Stale Offset

Diagnosis: Check consumer client logs for network-related errors (e.g., Connection reset by peer, SocketTimeoutException). If the consumer was disconnected for an extended period and then reconnected, it might try to resume from an offset that’s no longer available on the broker due to retention policies.
Fix: Similar to cause #2, if the data has been purged, you need to reset the offset.
```
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group my-consumer-group --topic my-topic --reset-offsets --to-latest --execute
```
This will reposition the consumer to the current end of the log. If you need to reprocess data, use --to-earliest or --to-offset.
Why it works: Network issues can interrupt the consumer’s communication with the broker. If the consumer restarts without a successful offset commit due to the interruption, it will attempt to fetch from its last known (but potentially stale) offset, leading to the exception if that offset is gone.

5. Consumer Group ID Mismatch or New Consumer Group

Diagnosis: Verify the group.id configuration in your consumer application. Ensure it matches the group ID you expect or are inspecting. If a new group.id is used, Kafka treats it as a brand new consumer. By default, new consumer groups start from the latest offset. If your application code or configuration expects it to start from the beginning (auto.offset.reset=earliest), and the topic already has messages beyond what this new consumer has "processed" (which is nothing), you might see this if the topic was created after the last message was produced. This is less common for OffsetOutOfRangeException and more for UnknownTopicOrPartitionException or starting from latest when you wanted earliest, but can manifest if the consumer logic is complex.
Fix: Ensure the group.id is consistent. If a new group ID is intentional, configure auto.offset.reset appropriately.
```
# In consumer properties
group.id=my-new-consumer-group
auto.offset.reset=earliest # Or latest, depending on desired behavior
```
If the exception is occurring because the consumer was running, stopped, and a new group ID was assigned, you might need to reset the new group’s offset manually as described in cause #2 or #1.
Why it works: Kafka tracks offsets per group.id. A new group.id has no committed offsets, so Kafka’s default behavior (usually latest) applies. If your application logic relies on starting from the beginning and the topic already has data, you need to explicitly set auto.offset.reset=earliest or manually reset the offset for the new group.

6. Clock Skew Between Broker and Consumer

Diagnosis: This is a very rare cause for OffsetOutOfRangeException but can cause subtle timing issues. Check system clocks on both the Kafka brokers and the consumer machines. Significant clock skew can disrupt the timing of offset commits and fetches.
Fix: Synchronize system clocks using NTP (Network Time Protocol).
```
# On Linux/macOS
sudo ntpdate pool.ntp.org
```
Or configure systemd-timesyncd or chronyd.
Why it works: While Kafka’s internal mechanisms are robust, extreme clock skew can theoretically lead to race conditions where a commit appears to happen "after" a fetch request in a way that confuses the broker’s offset tracking, especially if combined with other transient issues. Synchronizing clocks ensures consistent time-based operations.

The next error you’ll likely encounter if you’ve fixed the OffsetOutOfRangeException and your consumer is still not processing correctly is a RecordTooLargeException if you try to fetch more data than the broker’s message.max.bytes or the consumer’s fetch.max.bytes allows, or a NotEnoughReplicasException if your topic replication is misconfigured.