Java Native Memory Tracking (NMT) is a powerful debugging tool that lets you see exactly where your JVM is allocating memory outside the Java heap.
Here’s what you’d see if you enabled NMT on a simple Java application:
public class NmtExample {
public static void main(String[] args) {
// This is a simple example, NMT shines in larger, more complex apps
System.out.println("Starting NMT example...");
try {
// A small sleep to allow attaching tools if needed
Thread.sleep(60000);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
System.out.println("Exiting NMT example.");
}
}
To run this with NMT enabled, you’d use a JVM flag like this:
java -XX:NativeMemoryTracking=detail -jar NmtExample.jar
After the application runs (or while it’s running, if you attach a tool like jcmd), you can inspect the native memory usage. The key command is jcmd with the VM.native_memory subcommand.
jcmd <pid> VM.native_memory summary
This will output a report detailing memory allocations categorized by "reserved" and "committed" memory. "Reserved" is the total address space the JVM has asked the OS for, while "committed" is the portion of that space actually in use by the OS. You’ll see categories like arena, compiler, gc, internal, jni, malloc, mmap, threads, and vm_total.
The real power comes from jcmd <pid> VM.native_memory detail. This gives you a granular breakdown, showing the exact call sites within the JVM (and sometimes C/C++ libraries) where memory was allocated. You’ll see memory usage attributed to specific threads, JNI calls, internal JVM structures, and even JIT compiler allocations.
Total: reserved=150000KB, committed=120000KB, used=110000KB
- Internal: reserved=20000KB, committed=20000KB, used=19500KB
- mmap: reserved=10000KB, committed=10000KB, used=9800KB
- arena: reserved=10000KB, committed=10000KB, used=9700KB
- Compiler: reserved=30000KB, committed=30000KB, used=28000KB
- Threads: reserved=40000KB, committed=30000KB, used=29500KB
- thread stack: reserved=30000KB, committed=20000KB, used=19800KB
- vm reserves: reserved=10000KB, committed=10000KB, used=9700KB
- JNI: reserved=5000KB, committed=5000KB, used=4500KB
- malloc: reserved=25000KB, committed=25000KB, used=24000KB
- gc: reserved=10000KB, committed=10000KB, used=9000KB
- vm_total: reserved=150000KB, committed=120000KB, used=110000KB
NMT is crucial for diagnosing memory leaks or excessive memory consumption that isn’t happening on the Java heap. This often involves JNI code, native libraries, or internal JVM components like thread stacks or the JIT compiler. It helps you understand the JVM’s footprint beyond what jmap or heap dumps show.
The most surprising thing about NMT is that it doesn’t just show memory allocated by your Java code; it tracks all memory managed by the JVM process, including memory used by the JVM’s internal structures, the garbage collector, the JIT compiler, and any native code loaded via JNI. This holistic view is what makes it indispensable for diagnosing native memory issues.
When you look at the jcmd VM.native_memory detail output, you’ll see a hierarchical breakdown. For example, under Threads, you might see thread stack allocations. If a specific thread is consuming an enormous amount of native memory for its stack, this report will pinpoint it. Similarly, if a JNI library is leaking memory, you’ll see large allocations attributed to the JNI category, and the detailed output will often show which specific JNI call or library is responsible.
The arena category, for instance, often refers to memory used for internal JVM data structures, like the memory manager’s internal caches or reusable memory pools. High usage here might indicate internal fragmentation or a specific JVM feature aggressively caching data. mmap shows memory directly mapped from files or allocated via mmap system calls, which can include memory for loaded libraries or large data buffers.
To effectively use NMT, you need to enable it at JVM startup with -XX:NativeMemoryTracking=detail. Without detail, you only get a summary, which is less useful for pinpointing specific issues. The jcmd <pid> VM.native_memory command is your primary tool for querying the collected data. You can also capture the output to a file for later analysis.
A common scenario where NMT is a lifesaver is when your application experiences OutOfMemoryError: unable to create new native thread. This error, despite its name, doesn’t always mean you’re out of Java heap. It often means the OS has run out of resources to create new threads, and NMT can help you see if excessive thread stack sizes or a large number of threads are contributing to this by showing the Threads and thread stack components of native memory usage.
If you’re seeing significant malloc usage that seems out of proportion, it often points to native libraries (or potentially the JVM itself using malloc for certain allocations) that are not being properly deallocated. Tracking down the source of these allocations in the detail output is key.
Finally, after you’ve fixed all your native memory issues, the next thing you might run into is tracking down subtle performance regressions in garbage collection.