Implementation of Non-Stop Dynamic Routing

Project Background and Reasons for Choosing Spring Cloud Gateway

This project fully adopted a microservices architecture (MSA) to flexibly respond to rapidly changing business requirements and secure individual scalability without interference between services. It was carried out in a 'multi-vendor collaboration structure' involving numerous developers with domain expertise, enabling parallel development. However, from an architectural perspective, this led to the challenge of increased administrative complexity due to fragmented service boundaries and a surge in management points.

The crucial issues that needed to be resolved for successful project execution are as follows.

• Resolving management complexity in a multi-developer environment: It is impossible to synchronize each service's endpoint individually during the process of tens of services being independently deployed and rolled back. A centralized routing system that abstracts network locations and links requests was essential.

• Maximizing productivity through centralization of security functions: Implementing authentication and authorization individually for each service leads to redundant development and creates overall security vulnerabilities in the system. It was necessary to integrate this in the Gateway to upwardly synchronize the security level of all services.

• Stable external integration within a closed network infrastructure: This system was built on an internal closed network to maintain confidentiality, but there was a necessity for external integration points. The 'Secure Gate' that mediates communication without directly exposing internal services and hides the internal topology was key.

• Securing flexibility for functional changes and additions: To respond to rapidly changing requirements and integration standards, a flexible gateway was needed that allowed control of business logic through code rather than relying on hardware equipment dependent on infrastructure settings.

To comprehensively meet these requirements, we chose Spring Cloud Gateway (SCG). SCG is not only perfectly integrated with the JVM ecosystem, but also its Netty-based asynchronous non-blocking architecturecan reliably handle a large number of concurrent connections with minimal resources.

Above all, the ability to freely customize filter logic using Java code was decisive. This enabled us to implement complex permission control, closed network-specific security filtering, and dynamic routing between multiple developers promptly at the application level, beyond merely the role of traffic forwarding.

Implementing Seamless Dynamic Routing

In the initial setup phase, we adopted the most standard method provided by Spring Cloud Gateway (hereinafter referred to as SCG), which is static routing. This is done by explicitly specifying the routing rules in the application.yml configuration file as shown below.

spring:
   cloud:
       gateway:
           server:
               webflux:
                   routes:
                      - id: shared
                         predicates:
                             - Path=${server.servlet.context-path}/shared/**
                         uri: <http://foo-bar.com/api/shared>
                         filters:
                             - RewritePath=${server.servlet.context-path}/, /api/
                             - AddRequestHeader=xgate-id, gate1234
                         metadata:
                             response-timeout: 1800000 #30분
                             connect-timeout: 3000 #30초

This method has the advantage of being intuitively structured and quickly implemented, but it revealed critical drawbacks in environments where multiple developers are involved and operational requirements change frequently. This is because even adding a single route or modifying a timeout value requires changing the configuration file and going through the entire process of rebuilding and deploying the gateway service.

In particular, the fact that service operators cannot immediately control the routes, and must pass modification requests to developers and wait for them to be reflected according to the deployment schedule, has become a significant overhead that hinders the agility of the business.

To overcome these operational constraints, manage routing metadata in the database and synchronize it with the gateway engine in real-time.Dynamic RoutingWe have decided to switch to a method. First, we designed the GatewayRoute domain as follows to normalize the scattered routing information from the existing YAML file into a relational data model.

public class GatewayRoute {
    private String id;
    private String routeName;
    private String pathPattern;
    private String rewriteFrom;
    private String rewriteTo;
    private String uri;
    private long responseTimeout;
    private long connectTimeout;
    private boolean active;
    private List<ApiPermission> apiPermissions;
    private boolean useQueue;
    private int maxQueueSize;
    private int maxConcurrent;
    private List<KeyValue> requestHeaders;
}

public class ApiPermission {
    private String path;
    private List<String> permittedServices;
}

This domain model has been extended to manage not only all the routing information previously handled in a static manner but also the metadata for [integrated authentication and authorization filters] and [queue management] that will be described later.

Additionally, the technical core for applying dynamically stored data to actual routing lies in utilizing the internal mechanism of SCG. SCG automatically detects implementations of the RouteDefinitionLocator interface at initialization time to load the list of routes.

To achieve this, we created a custom GatewayRouteConfig class and implemented logic that retrieves route information from the database and converts it into a RouteDefinition format that SCG can understand. This allows operators to apply routing rules in real-time simply by changing the configuration in the admin UI, completing an environment where the gateway can be updated without a restart.

@Configuration
public class GatewayRouteConfig {

        @Bean
        public RouteDefinitionLocator dbRouteDefinitionLocator() {
        return () -> policySeek.findAllGatewayRoutes()
        .filter(GatewayRoute::isActive)
        .doOnNext(route -> {
                queueManager.updateRouteConfig(
                   route.getId();
                   route.getMaxQueueSize(),
                   route.getMaxConcurrent()
                );
           }
        .map(this::convertToRouteDefinition);
        }

        private RouteDefinition convertToRouteDefinition(GatewayRoute route){
        RouteDefinition definition = new RouteDefinition();
        definition.setId(route.getId());
        definition.setUri(route.getUri());
        String fullPath = normalizedContexPath() + route.getPathPattern();
        definition.setPredicates(List.of(new PredicateDefinition("Path=": + fullPath)));

        List<FilterDefinition> filters = new ArrayList<>();
        if (route.isUseQueue()){
                filters.add(new FilterDefinition("Queue"));
        }
        filters.add(new FilterDefinition("RewritePath=" + normalizeContextPath() + route.getRewriteFrom() + "," + route.getRewriteTo()));
        route.getRequestHeaders().forEach(keyValue -> filters.add((new FilterDefinition("AddRequestHeader=" + keyValue.getKey() + "," + keyValue.getValue())));
        definition.setFilters(filters);
     
        definition.getMetadata().put("response-timeout", route.getResponseTimeout());
        definition.getMetadata().put("connect-timeout", route.getConnectTimeout());
        }
}

In conclusion

So far, by utilizing Spring Cloud Gateway, we have moved away from the existing static YAML configuration method to a database-based one.Zero Downtime Dynamic Routing ArchitectureWe examined the process of building it. This enhancement, which elevated the infrastructure setup to the realm of business data, provided clear practical advantages as follows.

• Complete separation of operations and deployment:The inefficiency of having to rebuild and deploy the entire system to add a single API path or adjust timeout values has disappeared. Now, routing rules can be applied in real-time without service interruptions even during operation.

• Reducing communication costs in a multi-vendor environment:A buffer area has been established that allows agile responses by simply modifying data at the gateway level, even if microservices from different developers are deployed at different times or change endpoints.

• Flexible architectural scalability:By managing routing metadata with a normalized domain model, we have completed a solid framework that organically combines security and traffic control beyond simple traffic forwarding.

However, building a dynamic routing environment stably is just the beginning. The real practical concern is how to maintain security standards on this distributed infrastructure and safely control the massive traffic that is pouring in. In the next part, we will discuss the core business security and control architecture operating based on the dynamic routing infrastructure completed in this part.

Hustle Paul