Dynamic Routing Infrastructure and the Challenges Ahead
In the previous article, I described the process of moving away from the rigid, static YAML configuration structure by using Spring Cloud Gateway (hereafter, SCG) and building a database-driven, zero-downtime dynamic routing architecture. By moving the core infrastructure configuration elements into the application's business data domain, we secured the flexibility to change routing rules in real time during operation without rebuilding or redeploying the system. In a complex multi-vendor environment, this became a powerful buffer that enabled us to respond flexibly to differences in deployment schedules among development companies and to endpoint changes.
However, from a practical perspective, completing a zero-downtime dynamic routing environment was merely the first step toward a much larger goal. The moment we flexibly opened traffic channels at the infrastructure level, more fundamental and critical practical challenges faced by a distributed microservices architecture (MSA) began to emerge. At the center were two key challenges: how to maintain common security standards across dozens of fragmented services, and how to safely protect internal systems from large-scale traffic that arrives unpredictably at specific times. These were the centralization of security functions and traffic control and queue management to ensure system availability.
In this article, we will take an in-depth look at the implementation of integrated authentication and authorization filters, as well as the practical implementation strategy for a queue management service that reliably accepts and controls large-scale traffic, based on the GatewayRoute domain model and dynamic infrastructure designed in the previous article.
Integrated Authentication and Authorization Filter
The first technical Achilles' heel encountered when implementing MSA in a multi-vendor environment is the consistency of security standards. In a structure where multiple development companies specializing in different domains independently develop and deploy their own microservices, authentication and authorization mechanisms are easily fragmented within each service. In particular, the infrastructure built for this project operated in an internal closed network that blocked indiscriminate contact with external systems. Therefore, it was essential to have a powerful Secure Gate that could consistently verify the identity of every incoming request and fully control it according to business permissions, without directly exposing the network topology of internal microservices to external integration points. We implemented a global asynchronous authentication/authorization filter at the Spring Cloud Gateway layer to completely remove security overhead, allowing each service to focus solely on its business logic while synchronizing the security level of the entire system upward.
First, Spring Cloud Gateway has a reactive architecture internally based on Spring WebFlux and the Netty engine. Therefore, rather than a traditional synchronous filter based on the Servlet model, which processes one request per thread, it requires a reactive WebFilter design capable of processing a large number of concurrent connections in a non-blocking manner using a small number of Event Loop threads. The integrated authentication and authorization filter we implemented goes beyond simple JWT (JSON Web Token) decryption. It organically mediates complex integration specifications between multiple vendors within a single filter and performs authorization control by mapping requests in real time to the services permitted for each API path based on the database. The reason this complex filter architecture was able to operate stably and responsively in a non-blocking API gateway environment without even a single moment of delay lies in three key technical mechanisms that perfectly understand and appropriately apply the characteristics of reactive streams.
Second, it uses a dynamic token swap and protocol adapter mechanism for transparently accommodating multiple integration specifications. This filter goes beyond serving as a simple gateway entry point and effectively performs the role of a centralized adapter that flexibly mediates communication specifications within an internal closed network involving multiple development companies and legacy platforms. In a multi-vendor environment, each system often uses a different credential scheme. Instead of requiring the client to handle each case individually, we designed the gateway to identify the destination of a request and dynamically modify the request in real time into the format required by the downstream service. In particular, when integrating with a legacy platform, we implemented a dynamic token swap architecture that validates the original external token, dynamically obtains an infrastructure token required for communication with the internal business domain in the background, replaces the header with the new credentials, and forwards the request. As a result, we were able to completely conceal the diverse and complex authentication/authorization specifications imposed by multiple development companies and legacy interfaces beneath the gateway layer, achieving an unprecedented reduction in coupling among internal systems.
Third, it uses a non-blocking log processing process. The process of storing logs (GateLogCdo) for maintaining security confidentiality was isolated with extreme precision so that it would not negatively affect the response speed of the main routing stream. The entire I/O overhead of writing audit logs to the database and transmitting them over the network is delegated to an independent background thread pool that is completely decoupled from the main runtime flow. This fundamentally prevented even a single request from being delayed by log storage, enabling us to build an architecture that preserves the gateway's essential responsibilities of security validation and routing mediation with complete integrity.
Queue Management System
In an MSA environment, a failure or performance degradation in a particular domain service does not end with the interruption of a single component. In a distributed system, when a marketing event or sudden traffic spike is concentrated on a specific microservice, delayed responses from that service frequently lead to a cascading failure in which the connection pool of the API gateway, the upper calling layer, becomes exhausted. Particularly in a multi-vendor environment where multiple development companies independently manage their respective domains, it is difficult to respond flexibly to business requirements using only uniform IP blocking at the hardware equipment level or infrastructure bandwidth limits. This is why an architecture for queueing and availability control is required—one that is not dependent on infrastructure settings, dynamically changes traffic thresholds according to the importance and available capacity of each API route, and grants users who exceed the threshold an opportunity to enter sequentially instead of unconditionally rejecting them. To this end, we built a custom dynamic queue system based on a database that configures the available capacity of each service in real time during gateway runtime and safely queues excess traffic within a reactive, non-blocking stream.
QueueFilter, the entry point for queue control, was implemented by inheriting from AbstractGatewayFilterFactory, SCG's standard extension specification. On every request, this filter extracts the route ID to establish an independent traffic isolation barrier and mediates waiting signals in a non-blocking manner that does not occupy Netty's event loop threads until the user's turn arrives.
The key to enabling this queue system to protect the backend without exhausting the gateway's own resources lies in its architectural integration, which seamlessly connects reactive streams with the Java standard concurrency API.
First, it provides a non-blocking bridge between CompletableFuture and reactive streams. In a traditional blocking structure, implementing a queue requires putting threads to sleep, so the gateway collapses first as threads are exhausted in proportion to the number of waiting users. To overcome this, we designed QueueContext to create an empty CompletableFuture for each waiting session and manage them in a ConcurrentHashMap. Users return their available thread without holding onto Netty's valuable event loop threads for even 1ms until their turn arrives. Only when the internal operation is complete and QueueContext emits a turn-complete signal does the reactive pipeline wake up and proceed through the subsequent routing chain, achieving extremely high resource efficiency.
Second, it uses a two-track blocking strategy for handling system limits, along with asynchronous backend integration. Traffic within the range permitted by the in-memory queue is stored in QueueContext and safely waits in a reactive pending state. However, when even the in-memory queue reaches capacity, the waiting session information is passed to the backend to trigger a distributed event, after which a QueueFullException is thrown and handled. As a result, the gateway is fully protected from the risk of in-memory overflow, while the client receives a queue ID through a standard HTTP response and gains a robust architectural buffer for communicating with the backend out of band.
Third, it provides a mechanism for fundamentally preventing resource leaks based on the reactive lifecycle. One of the most common practical failures in a distributed queue environment occurs when a user refreshes the page or closes the window while waiting, causing the connection to be lost while in-memory resources or concurrency slots remain unreleased, resulting in a deadlock. This system fundamentally prevents the problem by precisely intercepting the lifecycle signals of reactive streams. Specifically, doOnCancel, which is triggered when a stream is canceled, immediately removes ghost sessions, while the doFinally block sequentially wakes the next waiting user whenever the HTTP request-response cycle finally ends for any reason, including successful completion, network errors, and timeouts. If it encounters a phantom waiter that was canceled before acquiring an available slot, it also performs cleanup logic to quietly remove that waiter from memory. Through this interlocking lifecycle control, we achieved the notable result of maintaining stability without a single resource leak or deadlock.
Conclusion
So far, we have examined how to implement an integrated authentication and authorization filter and a queue management service. By using an API gateway that had previously served merely to forward external requests to the backend, we centralized security and traffic control, maximized development productivity in a multi-vendor environment, and secured infrastructure stability through a queue management service.
Ultimately, architectural evolution is not about fitting the business to the constraints of rigid hardware or infrastructure. It is the process of flexibly evolving software itself so that it can support rapidly changing business requirements and complex collaboration structures. I hope that the dynamic control architecture centered on SCG that we created in this project will serve as a useful reference for other developers considering MSA in multi-vendor environments and working to address fragmented distributed security and large-scale traffic control.
Hustle Paul