-Jep, JNI, and GIL-

1. Introduction: Challenges in Integrating Heterogeneous Runtimes

In modern backend architectures, service logic is increasingly less likely to be composed in a single language. While the Python ecosystem, led by Numpy and Scipy, has become the de facto standard for numerical analysis and machine learning inference, JVM-based stacks still maintain an advantage in transaction management and operational stability. As a result, backend engineers face the challenge of integrating the strengths of two different runtimes into a single service flow.

The vessel data platform faced the same challenge. Data on engine and generator status, tank levels, and hull attitude arrives from vessels at sea every minute, but the platform's essential role is not to store this data. It is to perform computations that interpret hull motion and calculate the degree of degradation compared with normal performance. The problem was that these computations consisted of naval architecture formulas that had already been written and validated in Python by domain experts. They were built on pretrained machine learning models and C extension libraries, for which no corresponding Java implementations even existed. In addition, formulas subject to Class approval had to produce results that matched the existing results down to the decimal places.

This technical document describes a case in which Java Embedded Python (hereafter Jep) was used to embed a Python interpreter inside a JVM process. It first describes two issues encountered in the initial implementation and how they were addressed, then analyzes Jep's internal operation from the perspectives of JNI and the GIL to explain in detail why those issues were structurally inevitable.

2. Background to the Technology Choice: Trade-offs Between Process Separation and In-Process Embedding

There are two primary approaches to integrating validated Python assets into a service. Separating them into a standalone Python service provides benefits in terms of fault isolation, but incurs network round-trip and serialization costs for every call. In-process embedding, by contrast, exchanges data directly within the same process memory space.

The deciding criterion was whether the computation was located inside the request-processing flow. Most of the platform's computations had to be performed immediately when sensor data arriving every minute was received, and then persisted as domain objects. We therefore determined that inserting HTTP round trips and serialization into this path would provide no practical benefit and adopted the in-process approach. Jep is the library that makes this possible.

Unlike Jython, Jep does not reimplement Python on top of the JVM. Instead, it invokes the original Python interpreter written in C through JNI (Java Native Interface). Numpy and Scipy are mostly C extension modules and do not run in a Jython environment. Because compatibility with C extensions was an essential requirement, Jep was effectively the only option.

try (Interpreter interp = new SharedInterpreter()) {
    interp.exec("import numpy as np");
    interp.set("xs", new double[]{1.0, 2.0, 3.0});
    interp.exec("result = float(np.mean(np.array(xs)))");
    double mean = (Double) interp.getValue("result");
}

The interface consists of three verbs. set() binds a Java object to a Python global variable, exec() executes code, and getValue() extracts the result. Python dict objects are automatically converted to Java Map objects and lists to List objects, including nested structures. If the process had been separated, JSON schema definitions and serialization code would have been added at exactly this point.

3. Limitations of the Initial Implementation and Technical Pain Points

The initial implementation had a simple structure: it created one Jep instance for reuse and delegated the native library path to the container environment's default lookup rules. Although the defect did not appear in the validation environment, two problems of different natures emerged when concurrency and infrastructure changes were introduced in the production environment.

3.1. Instance-Reuse Design: Overlooking the Thread-Affinity Constraint

In the initial implementation, the Jep instance was stored in a field of a specific class and reused across multiple calls to reduce the cost of creating interpreters. This was a natural design from the perspective of handling a connection pool, and it worked correctly during single-threaded validation.

However, a Jep instance was not an object that could be shared in this way. The detailed rationale is described in Section 4.4, but the reason is that the tstate field held internally by the Jep object is a native PyThreadState pointer that must exist once per thread. The problem surfaced when a worker thread other than the thread that created the instance accessed it.

There were two responses. First, the interpreter implementation was switched to SharedInterpreter. The previous implementation also had a separate problem: it was incompatible with C extension modules such as Numpy (Section 4.3). Second, to address the thread-affinity issue itself, we abandoned the approach of storing the instance in a field and changed the structure to create it at the point of use within a try-with-resources statement and release it immediately. The latter change resolved the threading issue.

try (SharedInterpreter jep = new SharedInterpreter()) {
    // 사용하는 스레드에서 직접 생성되고, 블록을 벗어나면 해제됨
}

This change resolved the thread-affinity issue, but introduced a new trade-off: the cost of importing modules had to be paid again for every call. The adjustment for this is discussed in Section 5.2.

3.2. Implicit Library Lookup: Vulnerability to Infrastructure Changes

The second problem occurred not in the application code but in the deployment configuration layer. Initially, we relied on the container environment's default lookup rules for the locations of the Jep native library and the Python runtime. Because the system worked normally without any additional configuration, we had not recognized the need for explicit declarations.

The problem surfaced after the Linux kernel version of the execution node was upgraded. The runtime could not resolve the library path, and the application failed during startup. The service failed to start solely because of a change in the infrastructure layer, while not a single line of application code had changed.

The response was to explicitly declare the native library path using the -Djava.library.path JVM option. We directly added this option, together with the path to the library directory inside the container image (for example, native-libs), to Deployment manifest's spec.template.spec.containers[].args, thereby replacing reliance on the runtime's lookup rules with an explicit declaration.

# deployment.yml
spec:
  template:
    spec:
      containers:
        - args:
            - "-Djava.library.path=/app/native-libs"

The implication of this case is clear. For a library with native dependencies, the state of “working without separate configuration” does not mean that stability has been secured; it merely means that the environment has not changed yet.

4. Decomposing Jep's Internal Operation

Both failures in Section 3 were inevitable consequences of Jep's internal structure. The following five mechanisms determine the operation and constraints of Jep-based systems.

4.1. Process Layout: One Process, Two Independent Runtimes

Internal Process Structure

Figure 1. A structure in which the JVM area and Python area coexist within a single OS process across a JNI boundary

Three areas coexist within a single OS process. The JVM area (thread pool, Jep instances, and JVM Heap) is entirely managed by the GC, whereas the Python area (PyThreadState, sys.modules/sys.path, C extension .so files, and ndarray buffers) is not managed by the GC. There is also a thread dedicated to JepMainInterpreter that persists until the process terminates.

The most important fact is that Python memory is allocated outside the JVM heap. Numpy ndarray objects, pandas DataFrame objects, and joblib models all reside in native memory areas unknown to the JVM, and -Xmx settings and the GC do not affect these areas.

4.2. Initialization Sequence: Lazy Initialization and Library Lookup

// jep/MainInterpreter.java
protected static synchronized MainInterpreter getMainInterpreter() throws Error {
    if (null == instance) {
        instance = new MainInterpreter();
        instance.initialize();
    }
    ...
}

It is a combination of the synchronized keyword and a lazy initialization pattern. Python is initialized not when the application starts, but when new SharedInterpreter() is called for the first time, and this occurs only once during the process lifecycle. Subsequent calls to new SharedInterpreter() are responsible only for creating PyThreadState objects.

The first step of initialize() is loading the native library through System.loadLibrary("jep"), and this is the source of the failure described in Section 3.2. It consults -Djava.library.path and LD_LIBRARY_PATH in sequence, and if both fail, LibraryLocator traverses the contents of site-packages. The system may work even when the path is not explicitly specified, but that behavior depends entirely on the defaults of the execution environment.

Py_Initialize() is then executed on a separate thread called JepMainInterpreter. According to the source comments, this is to avoid GIL issues that occur when a subinterpreter is located on the same thread as the main interpreter. This thread persists in an infinite loop and does not terminate. If this thread terminates while another thread is executing in the Python area, the state may be corrupted. Consequently, this thread remains in any JVM using Jep until the process terminates, so it should be recognized in advance to avoid misidentifying it as a leak during thread-dump analysis.

4.3. Choosing an Interpreter Model: SharedInterpreter and Global State Sharing

	SubInterpreter	SharedInterpreter
Based on	Py_NewInterpreter()	Shares the main interpreter
sys.modules	Isolated	Shared
C extension module compatibility	Unstable (many Numpy/Scipy components unsupported)	Stable
Global state contamination	None	Present

A subinterpreter is a feature that creates multiple interpreter states. However, when a C extension module uses global static variables, that state is not separated by interpreter. Large C extensions such as Numpy have exactly this structure, so they may malfunction or terminate the process in a subinterpreter environment. Although improvements are underway through PEP 554 and PEP 684, we determined that they were difficult to apply to the current stack and switched to SharedInterpreter in Section 3.1.

However, this choice comes at a cost, and the Javadoc also warns about it. Any action that changes module behavior affects the entire SharedInterpreter; typical examples include modifying sys.path or calling Numpy.seterr(). Because sys.modules is shared, all interpreters reference the same sys.path list, and repeatedly calling sys.path.append(...) without a condition check causes paths to accumulate. It is safer to register module paths through environment variables rather than in code.

4.4. Thread Affinity: Constraints Enforced by tstate

// jep/Jep.java
public void isValidThread() throws JepException {
    if (this.thread != Thread.currentThread())
        throw new JepException("Invalid thread access.");
    ...
}

Jep instances can only be used on the thread on which they were created, and all public methods verify this upon entry. This verification logic is the direct cause of the failure of the instance-reuse design described in Section 3.1.

Inside the Python interpreter, there is a data structure that stores “how far this thread has currently progressed in execution.” It contains information such as the location of the executing code, exception-handling state, and the call stack. This structure is PyThreadState. The key point is that exactly one such structure must exist for each thread.

On the Java side, Jep stores the memory address of this structure in a field.

private long tstate;  // 실제로는 PyThreadState의 메모리 주소

tstate is the name of this field and is an abbreviation for thread state. Because Java has no concept equivalent to a C pointer, it stores only the address value as a long type.

The fact that this value is thread-specific is the fundamental cause of the failure described in Section 3.1. If another thread attempts to execute Python code using a tstate that does not belong to it, the interpreter's internal state is corrupted. To prevent this, Jep checks upon every method entry whether the calling thread matches the thread on which the instance was created. The Javadoc also explicitly states that activating multiple interpreters simultaneously on a single thread is impossible.

The simplest way to satisfy the two constraints—one per thread, and only on that thread—is a try-with-resources structure that creates the instance at the point of use and immediately releases it.

4.5. Serialization by the Global Interpreter Lock (GIL)

Jep does not remove the GIL. When a Java thread calls jep.eval(...), it acquires the GIL and releases it when returning. Therefore, even if dozens of Java threads call it simultaneously, there is always only one thread actually executing Python bytecode at a time.

GIL Serialization Timeline

Figure 2. While Worker-1 performs an operation, Worker-2 waits for the GIL, resulting in serialized execution

However, many of Numpy's heavyweight operations release the GIL internally. Pure Python loops are completely serialized, but actual parallel processing occurs during ndarray operations, so it is advantageous to delegate computation logic to the Numpy layer as much as possible.

Therefore, expanding the thread pool for the Python execution path does not increase throughput. The GIL acts as an upper limit, while creating a PyThreadState and globals dict for each thread only increases native memory usage. Process separation is required for meaningful parallelization of CPU-bound work.

5. Three Operational Principles Derived from the Structural Constraints

5.1. Recalculating the Container Memory Budget

As seen in Section 4.1, memory used by the Python layer is located outside the JVM heap. -XX:MaxRAMPercentage=80 is reasonable for a typical Spring Boot container, but in a Jep service it means that the Python interpreter, modules, ndarray objects, and ML models must all fit within the remaining 20%. If the budget is exceeded, the kernel's OOM Killer terminates the container rather than an OutOfMemoryError being thrown, and only exit code 137 is observed without a heap dump. The appropriate value depends on how much memory the Python side actually uses, so the only viable approach is to directly measure the container's actual memory usage with monitoring tools and calculate backward from there.

5.2. Aligning Interpreter Lifetime with the Unit of Work

The response described in Section 3.1 resolved the thread-affinity issue, but a problem remained: creating an interpreter for each call causes the module import cost to recur on every call. It is reasonable to align the interpreter's lifetime with a logical Unit of Work rather than with an individual call.

// getJep(): SharedInterpreter 생성 + 모듈 import
try (SharedInterpreter jep = this.getJep()) {
   flowA.createBaseline(jep, targetId, param);
   flowB.createBaseline(jep, targetId, param);
   flowC.createBaseline(jep, targetId, param);
}

Because the instance is not stored in a field, the thread-affinity constraint is preserved while the initialization cost is amortized across the batch.

5.3. Designing the Blast Radius in Advance

If C extension code accesses an invalid memory area—a situation commonly called a segfault—the process itself dies rather than a Java exception being thrown. This means it cannot be caught with try-catch, and there is no way to recover at the JVM layer. By separating the service that performs Python calls from pure domain services, the scope of an incident like this can be confined to that service.

6. Conclusion: Lessons from Coexistence Rather Than Embedding

The most valuable lesson learned from this implementation process is “Although the term embedding sounds like a structure in which one runtime controls the other, in reality it means that two independent runtimes coexist within the same process”.

The two decisions made without recognizing this fact led to the failures described in Section 3. Reusing Jep instances as if they were connection-pool objects resulted from applying the JVM-object lifecycle mindset directly to objects holding native pointers. Delegating the library path to the environment default resulted from overlooking the fact that native dependencies lie outside the usual deployment boundary of a JVM application. In both cases, nothing prevented the flawed design at design time; the warning signs appeared only at runtime, after the incorrect implementation had already been created.

I conclude that adopting Jep itself was a valid choice. It allowed us to integrate proven Python assets without rewriting them and maintain a structure in which modifications to formulas by domain experts could be reflected immediately. However, the trade-off was that the memory, threading, and deployment models had to be redesigned from the ground up. The conclusion of this experience is that understanding the operational mechanisms behind the convenience first is essential for identifying the causes of failures in production.

Pancake Maker