blog

Spring Boot Kubernetes Client

NEXTREE.

• 8 min read
Spring Boot Kubernetes Client

Introduction

When operating Kubernetes, it is natural to have thoughts like this: "It would be great to be able to see at a glance whether the Pods in the cluster are operating normally and to collect the status of each Pod." What starts as a simple curiosity turns into a daily concern for operators as the number of servers they manage begins to increase.

The Kubernetes client library was the solution to this demand. It is a library that allows a Spring Boot application to directly check the status of the Kubernetes cluster and react in real-time to changes. Rather than merely being a tool to 'look into the cluster from the outside', it plays a bridging role that enables the application itself to become part of the cluster and breathe together with it.

In this article, I will sequentially discuss the motivations for introducing the Kubernetes client to Spring Boot, the principles of operation, and the security precautions that had to be addressed in the operational environment, based on my actual project experience.

Motivation for Introduction

In short, the reason I began exploring this technology was the need for operational automation and server status collection. The current project involves over 100 servers running, with plans for further expansion. This goes beyond just increasing the numbers; it signifies that it has reached a stage where it is impossible for a single operator to visualize the status of all servers in their mind.

In the existing method, checking the cluster status required the hassle of directly accessing the Shell to execute kubectl commands or having to connect separately to ArgoCD. The process starting from kubectl get pods — I increasingly felt that this process itself was a factor that consumed response time for issues.

Limitations of the existing GitOps environment

I was managing Pod configurations such as ConfigMap and Deployment on the server by modifying the GitOps environment through JGit in Spring Boot and triggering ArgoCD Sync.

However, since not all servers have the same environment, there were situations where human errors occurred intermittently due to network or firewall issues during the deployment process. For example, this could be due to a single key in the ConfigMap being different or an image pull failing only in specific environments. Even if changes are properly reflected in Git, I constantly face the fact that the actual state of the cluster may differ.

This sometimes made it difficult to quickly identify when a pod didn't start correctly or when there were issues with the Pod image or ConfigMap. Eventually, the situation accumulated where changes were applied in Git but not reflected in production, leading to a vicious cycle where operators had to manually verify things again to resolve the issue.

Expected behavior

By applying the Kubernetes client library to Spring Boot, the application itself can actively observe the cluster and respond to anomalies. The behavior I expect is clear.

  • If a ConfigMap change is applied differently than intended, it immediately notifies the server managing the data.

  • When the Pod does not operate normally due to various specific situations, it triggers an alert.

Ultimately, the key is to automate the process that required manual verification, preventing outages caused by human errors in advance. It's not just about becoming 'more convenient'; it significantly reduces the operational infrastructure aspect of the time taken to recognize an outage (MTTD).

How it works — Principles

In fact, what the Kubernetes client library does is essentially simple. It is a wrapper that conveniently wraps the REST API of kube-apiserver for developers to use. When you write a single line of code to retrieve the list of Pods, the library internally sends an HTTP GET request to the API Server and returns the response converted into a Java object.

In other words, the same flow that we usually input in the terminal with kubectl get pods changes to a single line of library call. The difference is that we receive the result as a Java object like PodList instead of human-readable text, and we can directly pass that object to the business logic.

Where does the API Server connection information come from?

This raises the question, 'Do developers need to manually set the API Server address or authentication information?' The answer is 'No'. Once the application is deployed as a Pod on the Kubernetes cluster, meaning it runs within the same cluster, Kubernetes automatically injects all the necessary connection information.

Specifically, the following three items are automatically prepared. The IP of the API Server is injected as environment variables called KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT. The client library reads this information to configure the connection, so from a developer's perspective, a single line is essentially all that is needed for setup.

ApiClient client = new KubernetesClientBuilder().build();

The reason this single line is possible is that the library automatically determines whether it is currently inside a cluster or not, and if it is inside the cluster, it uses the in-cluster settings, while if outside, it configures the connection using the context from ~/.kube/config. This is the background that allows the same code to run in both local development and production environments.

Real-time Change Detection — Watch and Informer

Real-time change detection is also supported. In addition to simple queries, you can use a feature called Watch to receive events from the cluster in real-time via HTTP streaming. This allows you to implement logic that reacts immediately when Pods are added or deleted or when a ConfigMap is updated.

Watch is essentially a long-running HTTP connection. Once a request is sent, the connection remains open, and data flows in as JSON line by line whenever an event occurs. Therefore, when using it in Spring Boot, it is common to handle it in a separate thread or asynchronous context.

In production environments, we often use the Informer pattern, which takes it a step further. The Informer initially retrieves the entire state using the List API and then receives only the changes via Watch to maintain a local cache. This means that we always hold a snapshot in memory that is close to the 'latest state of the cluster', and query requests receive responses directly from the cache without going to the API Server.

The advantages of this structure are twofold. First, it can significantly reduce the load on the API Server. Second, it allows for quick retrieval of the most up-to-date information. From a business logic perspective, all it takes is asking, 'Give me the list of Pods,' and the Informer will respond immediately from the cache.

To summarize, Single synchronization → REST call, Simple Change Detection → Watch, Operation grade status tracking → InformerYou can select among three usage patterns to suit the situation.

Client Lifecycle Management

The KubernetesClient object internally holds an HTTP connection pool, so creating a new one for each request is inefficient. I registered it as a singleton bean in the @Configuration class and implemented @PreDestroy or DisposableBean to ensure close() is called at the application termination. This small difference, when accumulated, can lead to connection leaks and zombie threads, so it's better to pay attention to this when setting up the structure initially.

Points to be cautious about from a security perspective

While it offers high convenience, you should be aware from the outset that the repercussions of misuse can be significant. I have compiled a list of the most common mistakes that occur in operational environments, along with minimal guidelines to prevent them.

RBAC

The most common mistake is granting excessive permissions to applications. When configuring RBAC (Role-Based Access Control) settings, it's easy to quickly grant all permissions or simply bind the cluster-admin role while developing. During the development phase, it's tempting to think, 'Let's just get it working first,' but this setting often carries over into production more frequently than expected.

This can lead to a serious situation where if the Pod is compromised, the attacker gains access to the entire cluster. Vulnerabilities in container images, RCE in dependency libraries, or even a simple SSRF case can directly lead to the compromise of the entire cluster.

It is advisable to explicitly allow only the resources and actions (get, list, watch) that are actually needed, and to prioritize using Roles that apply to specific namespaces over ClusterRoles whenever possible. Developing the habit of asking, 'Is this function really necessary for additional permissions?' each time a new feature is added becomes the strongest line of defense.

ServiceAccount token management

You need to be careful with ServiceAccount token management as well. By default, Kubernetes automatically mounts ServiceAccount tokens to all Pods. This also means that tokens are mounted to Pods that do not need to access the Kubernetes API. In other words, this implies that all Pods within the cluster can be potential entry points.

To prevent this, in the case of Pods that do not require API access, you should explicitly add the setting automountServiceAccountToken: false to prevent unnecessary token exposure. Since it can be configured both at the ServiceAccount level and the Pod spec level, it is recommended to use both if possible to establish a dual defense.

Audit logs and notifications

Lastly, it is just as important to log 'who did what' as it is to control the permissions themselves. Kubernetes provides an audit log feature by default, so it is advisable to include a procedure in the operational routine to periodically check whether the ServiceAccount used by our application attempts to make calls outside of the intended scope. Permissions should not be set and forgotten; they must be accompanied by checks and reflections to have proper meaning.

Conclusion

Using the Kubernetes client in Spring Boot means that the application moves beyond merely running on the cluster to actively recognizing and utilizing the cluster. Until now, it has involved pulling a part of the operations that were previously left to external platforms into the application code, thereby taking shared responsibility.

While it can be a powerful choice when creating automated operational tools or platforms, one should carefully approach permission settings due to the significant security risks associated with improper usage. If convenience is guaranteed, I believe it is also important to meticulously check the default settings and configurations.

Personally, the biggest gain from this introduction has been the sense that 'the boundary between applications and infrastructure is becoming increasingly blurred.' Previously, cluster state observation was considered the domain of the infrastructure team, but now it has become a natural part of our service code that we need to handle daily. This change leads to faster responses for operators and shorter downtime for users.

Reference material

Bang

Site footer