1. Why Flux?
The existing business logic was operating in a sequential loop-based structure. It would iterate through a list of data, calling external APIs or databases one by one, collecting each result before proceeding to the next logic. This was a very typical structure.
Initially, there weren't many data, so there weren't any significant issues. However, as the service scale grew and the options and queries to be processed increased, performance degradation began to occur in earnest.
In particular, in structures with a high dependency on external services or databases, the accumulated network I/O wait time caused the overall response time to increase dramatically.
For example, if an API with an average response time of 200ms is called sequentially 100 times, it can take over 20 seconds just through simple calculations. The problem was that, even though there were sufficient CPU resources available, most of the time was spent in I/O wait states.
In other words, it was an inefficient structure that did not make proper use of system resources.
The existing method had the following characteristics.
First, the next request could only start once the previous request had been completed.
Secondly, the network response waiting time has accumulated in the total response time.
Thirdly, as the number of data entities increased, the response time increased linearly.
Fourth, if there is a delay in the external API, the overall service response time also slows down.
This issue has become even more serious in environments where large volumes of option data are queried. In actual operating environments, there were cases where hundreds of query requests occurred simultaneously during specific time periods, and in this process, response delays and timeout issues occurred repeatedly.
The first thing I considered to solve this was parallel processing.
This was because there was no need to process sequentially if it was an independent query operation.
In particular, most tasks were centered on network I/O waiting rather than CPU computation, so significant performance improvements could be expected just by appropriately applying parallel processing. The technology chosen for this process was Reactor-based Flux. Flux is not just a simple collection iteration tool, but a Reactive Stream-based library that can naturally combine parallel processing and asynchronous execution while processing data in a stream form.
In particular, Flux was chosen for the following reasons.
First, we were able to reliably process large amounts of data in a streaming fashion.
Second, we could easily set up parallel processing using parallel rails.
Third, it was possible to finely control thread resources based on Scheduler.
Fourth, we were able to prevent indiscriminate thread increases through a Backpressure-based structure.
Fifth, it was possible to seamlessly integrate with existing Java and Spring-based systems. Most importantly, the possibility of 'reducing the total processing time to the longest single request level' was a significant factor.
In the existing sequential processing method, if there are n requests, the total execution time had the structure as follows.
Existing Method:
n × Average Response Time
On the other hand, the parallel processing structure can be improved in the following form.
Improvement method:
MAX(individual response time) + α(scheduling overhead)
That is, if the number of parallel processes is sufficiently secured, the overall response time will converge to the level of the longest single task time.
2. Application Process and Implementation Strategy
In the actual application process, the problem was not solved simply by applying asynchronous calls. In the operating environment, performance, stability, data consistency, and thread resource management all had to be considered together.
The most important part, in particular, was the constraint that 'the results must be retrieved synchronously after parallel processing.' In actual business logic, all retrieval results must be collected before the next step of calculations and subsequent processing logic can be executed.
In other words, while the retrieval itself is processed in parallel, the final results must be combined synchronously.
(1) Flux-based stream configuration
First, the target data was converted into a stream format using Flux.fromIterable(). Then, the .parallel(size) method was used to configure parallel rails.
The size value here was not determined solely based on the number of CPU cores. We adjusted the level of parallel processing by considering external API response times, database load, network latency, and server memory conditions.
If the level of parallel processing is set too high, it can actually cause excessive load on external services. Therefore, we took the following criteria into consideration in the operational environment.
• External API Rate Limit
• DB Connection Pool Size
• Average response time
• Server memory usage
• CPU usage
• Network latency situation
(2) Scheduler Optimization
Another important factor in parallel processing was thread management.
Initial usage was simply with parallel(), but it was confirmed that thread management strategies are very important in actual operational environments.
Especially in network I/O-centric tasks, there is a possibility of blocking, so it was not appropriate to use only a standard CPU-bound thread pool. To address this, Schedulers.boundedElastic() was used. boundedElastic is a flexible thread pool provided by Reactor. It allows for thread expansion when necessary, but has limits to prevent unlimited growth.
In other words, it provided a structure suitable for I/O wait-centric tasks while maintaining system stability.
In particular, it had the following advantages.
First, it was possible to flexibly scale the threads as needed.
Second, idle threads were automatically cleaned up.
Third, we were able to reduce the risk of OOM caused by unlimited thread creation.
Fourth, it operated reliably even in a Blocking I/O environment.
In actual operating environments, “stable resource usage” was much more important than simple performance. BoundedElastic was a very suitable choice in terms of operational stability.
(3) CountDownLatch based synchronization
The most critical concern was how to combine asynchronous and synchronous processing. While asynchronous processing significantly improved the retrieval speed, ultimately, all retrieval results had to be gathered before the next business logic could be executed.
In other words, the retrieval process was asynchronous, but the final flow needed to be controlled synchronously. To resolve this, CountDownLatch was used. Each thread called the API in parallel and stored the results in responseBodyList.
And the countDown() was called each time a task was completed. In the main flow, it was structured to wait until all tasks are finished using await().
To summarize the structure, it is as follows.
• Asynchronous: Each thread makes API calls in parallel and stores the results in responseBodyList
• Synchronous: The main flow waits for all threads to finish using CountDownLatch.await()
The biggest advantage of this approach is that it safely introduced parallel processing without significantly changing the existing business logic structure.
In other words, it allowed us to parallelize only the internal inquiry logic while maintaining the existing synchronous-based business structure.
3. Problem Solving and Performance Improvement Results
Performance has significantly improved after applying a Flux-based parallel processing architecture. In the previous sequential processing method, the response time increased linearly as the amount of data increased.
For example, if 100 tasks are performed with an average response time of 300ms, the total response time can increase to about 30 seconds.
However, in a parallel processing structure, most requests were executed simultaneously, so the total response time converged to the duration of the longest single request. In actual operating environments, improvements such as the following could be observed.
• Existing method:
n × Average response time
• Improvement method:
MAX(individual response time) + α(scheduling and merging cost)
In particular, the effect of parallel processing was very significant in environments where network latency occurred. In the existing method, if a specific API response was slow, the entire flow was delayed.
On the other hand, in the parallel processing structure, even if some requests were slow, other requests could be processed simultaneously, greatly improving the overall perceived performance.
We were also able to achieve much more efficient results in terms of CPU usage. Previously, most of the time was spent in I/O waiting, but after parallel processing, idle time has significantly decreased.
There were also positive changes from an operational perspective.
First, the frequency of timeouts during bulk option queries has decreased.
Second, the response stability during peak hours has improved.
Thirdly, the proportion of overall service failures due to external API delays has decreased.
Fourthly, the ability to adjust the number of parallel processes by environment has increased operational flexibility.
It was very useful that the balance between performance and stability can easily be adjusted, especially through parallel control of the number of cores.
4. Conclusion
This optimization experience made me realize once again that what is more important than simply using the latest technology is finding the optimal combination of tools that fit the business constraints. At first, I thought that simply applying asynchronous processing would solve all the problems.
However, in an actual operating environment, we had to consider not only performance but also stability, data integrity, and operational maintainability all together.
I learned that especially changing all operations to a completely asynchronous structure is not always the right answer.
In actual projects, it was necessary to collect synchronous results after asynchronous processing because all results were essential to perform the next stage logic.
We found that a hybrid model using synchronization tools like CountDownLatch can be a very effective solution.
Additionally, it was a very meaningful experience to approach Flux not just from the perspective of a 'latest technology', but as a suitable execution model for high-volume I/O processing environments.
Ultimately, what matters is not the technology itself, but choosing the method that best fits the current system architecture and operating environment.
This experience was meaningful beyond simple performance improvement. I believe it was a practical optimization experience that considered stability and maintainability in the operational environment.
Jack