Access Off-Heap or On-Heap Memory with Memory Segments

Overview of On-Heap and Off-Heap Memory

In Java, the JVM's garbage collector manages on-heap memory for storing objects. The heap can grow or shrink while the application runs, and when it becomes full, the JVM performs garbage collection, making space for new allocations.

The off-heap memory is memory outside the Java heap for tasks that require faster access or interoperability with native code. For example, to invoke a C function from a Java application, its arguments must be in off-heap memory. As garbage collection does not reach off-heap memory, you can control how and when those arguments are deallocated.

You can use a MemorySegment object to interact with off-heap memory. A MemorySegment is associated with a contiguous region of memory. There are two kinds of memory segments:

• Heap segment - a memory segment backed by an on-heap region of memory.
• Native segment - a memory segment backed by an off-heap region of memory.

This tutorial covers how to allocate and access native segments.

Allocate a MemorySegment with an Arena

You should allocate a MemorySegment object with an arena, which enables you to control the lifecycle of native memory segments. Each arena has a scope, which specifies when the off-heap memory (associated with the MemorySegment object) should be deallocated and no longer valid. A memory segment is accessible if the scope associated with it is still valid or alive.

There are four kinds of arenas:

• A confined arena (Arena.ofConfined()) provides a bounded and deterministic lifetime to its memory segments, being alive from when it is created to when it is closed. A confined arena has an owner thread, typically the thread that created it. Only the owner thread can access the memory segments allocated in a confined arena. You'll get an exception if you try to close a confined arena with a thread other than the owner thread.
• A shared arena (Arena.ofShared()) has no owner thread. Multiple threads can access the memory segments allocated in a shared arena. Also, any thread may close a shared arena, and this operation is guaranteed to be safe and atomic. Checkout Slice a MemorySegment for an example using a shared arena.
• An automatic arena (Arena.ofAuto()) is an arena managed automatically by the garbage collector. Any thread can access memory segments allocated by an automatic arena. Calling Arena.close() on an automatic arena will throw a UnsupportedOperationException.
• A global arena (Arena.global()) has as main trait: any thread can access memory segments allocated with this arena. In addition, the region of memory of these memory segments never gets deallocated. Calling Arena::close on an global arena will throw a UnsupportedOperationException.

The following example allocates a memory segment with a confined arena and stores a String in the off-heap memory associated with the memory segment:

String pattern = "********";

try (Arena arena = Arena.ofConfined()) {

    // Allocate off-heap memory and copy the argument, a Java string, into off-heap memory
    MemorySegment nativeString = arena.allocateFrom(pattern);
    
} // Off-heap memory is deallocated 

The Arena interface extends the SegmentAllocator interface, which contains methods that both allocate off-heap memory and copy Java data into it. The previous example calls the method SegmentAllocator.allocateFrom(String), which performs the following:

• Allocates a memory segment with an arena,
• Converts a string into a UTF-8 encoded, null-terminated C string,
• Stores the string into the memory segment.

Next, let's check how you can get access to a value stored in a memory segment.

Print a Value Stored in the Off-Heap Memory

With the methods within MemorySegment interface you can read from or write to memory segments. Each of these methods takes as an argument a value layout, which models the memory layout associated with values of basic data types such as primitives.

A value layout encodes:

• the size,
• the byte order,
• the bit alignment of the piece of memory to be accessed,
• and the Java type to be used for the access operation.

For example, MemoryLayout.get(ValueLayout.OfByte,long) takes as an argument ValueLayout.JAVA_BYTE.

// Access off-heap memory
for (int i = 0; i < s.length(); i++ ) {
    System.out.print((char)nativeString.get(ValueLayout.JAVA_BYTE, i)); 
}

This ValueLayout.JAVA_BYTE has the following characteristics:

• Has the same size as a Java byte.
• Byte alignment is set to 1, meaning that the memory layout is stored at a memory address that is a multiple of 8 bits.
• Byte order is set to ByteOrder.nativeOrder() that returns the native byte order of the hardware upon which the Java virtual machine is running.

Close an Arena

When working with an arena is a good practice to access it through a try-with-resources statement.

try (Arena arena = Arena.ofConfined()) {
    
    // Allocate off-heap memory etc
        
} // Off-heap memory is deallocated 

In the previous example, the arena's scope is no longer alive once outside the try-with-resources statement. All memory segments associated with its scope are invalidated, and the memory regions backing them are deallocated.

If you try to access a memory segment associated with an arena scope that is closed, you'll get an IllegalStateException. You can check if that happens by running the following statement in a jshell session:

String pattern = "********";

try (Arena arena = Arena.ofConfined()) {
    MemorySegment nativeString = arena.allocateFrom(pattern);
}

for (int i = 0; i < s.length(); i++ ) {
    // Exception in thread "main" java.lang.IllegalStateException: Already closed
    System.out.print((char)nativeString.get(ValueLayout.JAVA_BYTE, i)); 
}