Response Delay Issues Solved with Cache Strategy

1. Overview

The first challenge I faced while operating the MyData service was the response delay of the main screen that occurred during the rapid increase in user numbers. Initially, the structure did not pose a significant problem, but as the user base grew, it manifested as a noticeable delay. This article documents how the issue was diagnosed and resolved at that time, and how the solution is connected to current technological trends.

At that time, the service was a Vue 2 based WebApp, and the cache was implemented on the server side rather than the client. In this regard, the caching layer is fundamentally different from the client memory cache provided by today's data fetching libraries. However, the strategy of 'showing outdated data first and updating in the background' is essentially in line with the Stale-While-Revalidate (SWR) cache strategy, which is now standardized in the TanStack Query (formerly Vue Query) in the Vue ecosystem and SWR in the React ecosystem. This article will cover everything from problem diagnosis to solution design, application results, and reinterpretation from the SWR perspective, while discussing why we were able to arrive at the same conclusion without being aware of the trends.

2. Problem Situation: Main Screen Response Delay

2-1. Existing Structure

The core value of the MyData service was to aggregate and display asset and transaction information registered with various financial institutions on the main screen as soon as the user entered. To achieve this, we adopted a method of querying real-time information of active users at each screen entry point to show the latest data. Initially, the user count was low, so this structure did not face major issues, and rather, the simple and clear design principle of 'always showing the latest data' worked to our advantage.

2-2. Point of Problem Revelation

As the number of users increased, the problem gradually became apparent. Since the structure sent simultaneous query requests to multiple external institutions at each screen entry, the more simultaneous users there were, the cumulative waiting time for external responses translated into user-perceived delays. Through operation monitoring metrics, we confirmed a pattern where the time taken for the screen to display after entering the main screen increased proportionally with user count growth.

In particular, when some of the external institutions responded slowly, it revealed a structural vulnerability that extended the overall screen loading time. The structure designed to uphold the principle of 'always showing the latest data' became a bottleneck affecting the perceived performance of the entire service as the user base grew. What became clear at this point was that it was difficult to satisfy the structure that guarantees 100% latest data at every screen entry while also allowing users to immediately receive the screen. We needed to prioritize one of the two values and redesign the structure accordingly.

3. Solution Strategy: Cache-Priority Rendering + Background Update

3-1. Design Principles

The principle adopted while redefining the problem was to separate 'immediate response' and 'ensuring currency' into two stages instead of processing them simultaneously in a single request. At the point of entry to the screen, we would immediately show the most recently queried and saved data, and right after that, query for the latest data in the background to refresh the screen. Users receive meaningful information immediately instead of staring at a blank screen or loading status, and they experience a natural transition to the latest information after a slight time delay.

To apply this principle, we needed a cache storage for saving the most recent query results and an asynchronous processing structure capable of separating data retrieval for screen exposure and retrieval for updates.

3-2. Processing Flow

The processing flow can be summarized in pseudocode as follows.

function 메인화면진입(사용자ID):
  캐시데이터 = 캐시저장소.조회(사용자ID)
 
  if 캐시데이터 존재:
    화면.즉시렌더링(캐시데이터)
  else:
    화면.로딩상태표시()
 
  비동기작업큐.등록(갱신작업, 사용자ID)
  return
 
function 갱신작업(사용자ID):
  최신데이터 = 외부기관조회.전체조회(사용자ID)
  캐시저장소.갱신(사용자ID, 최신데이터)
  if 화면.현재활성상태(사용자ID):
    화면.부분갱신(최신데이터)

The key point of this structure is that it separates the screen rendering path from the data update path. The screen responds immediately by looking only at the cache store, while heavy tasks like external agency queries are processed asynchronously on a separate path, not affecting screen response time.

3-3. Distribution of update tasks through message queues

Merely running update tasks asynchronously was not sufficient. At the point of increasing simultaneous users, the update tasks themselves could surge, causing a flood of requests to external agencies. To mitigate this, update tasks were loaded into a message queue, and workers were configured to consume the queue at a constant throughput.

From the user's perspective, upon entering the screen, they receive cache data immediately, so the perceived response speed is maintained regardless of queue processing speed. From the system's perspective, the volume of requests to external agencies is smoothed through the queue, reducing the phenomenon of load concentrating at specific points in time.

3-4. Structure comparison

[Table 1] Comparison before and after structural improvements

4. Results of Implementation

The most notable change after structural improvements is that the perceived waiting time when entering the screen has been significantly reduced. Users can always receive data at a consistent speed, regardless of the response speed of external institutions, and even if the response from a specific institution is slow, its impact does not propagate to the overall screen loading.

From an operational perspective, external query requests are smoothed through message queues, allowing stable processing of refresh operations even during times of increased simultaneous access. Most importantly, rather than giving up on 'freshness', we only deferred the 'point that guarantees freshness' from the screen entry point to shortly thereafter, thus maintaining both data reliability and user experience.

5. Looking at it now: Connection with SWR Pattern

5-1. What is the SWR Pattern?

Stale-While-Revalidate is a caching strategy that first returns cached (stale) data while simultaneously validating (revalidating) in the background. Once validation is complete, it replaces it with the latest data. This strategy includes three branches with different mechanisms. The first is at the HTTP protocol level, where the Cache-Control: stale-while-revalidate header (RFC 5861) is seen by the browser, CDN, or reverse proxy, which automatically serves the cached response first and then validates in the background. The second is at the application level, where cache retrieval and background update logic are directly implemented in the backend code without relying on protocols or libraries. The third is at the client library level, which declaratively offers the same strategy on browser memory, akin to TanStack Query or SWR.js.

This case falls under the application level. It is significant in that it was not borrowed from a protocol or library, but rather independently moved the same strategy into code.

5-2. Comparison of Self-implemented Method and Client Library Implementation

While the layers of mechanisms are different, when comparing the strategy of 'showing old data first and updating in the background' from an implementation perspective, it is as follows.

[Table 2] Comparison of Implementation Methods

5-3. Reasons for reaching the same conclusion

At the time, I was unaware of the name Stale-While-Revalidate, nor did I know of the existence of protocols or libraries that standardize such strategies. However, I felt through operational data that the data 'to be shown to the user immediately' and 'the data that needs to be guaranteed for accuracy' do not need to be processed in the same way at the same time, and the result of transferring that realization to structure naturally concluded in the same pattern. I believe this is an example that shows good architectural patterns are often rediscovered independently in the process of closely examining actual problems rather than starting from specific technology trends.

6. Limitations and Trade-offs

This structure also had a clear trade-off. The data displayed when entering the screen is, by definition, slightly outdated (Stale), making it difficult to apply to screens or functions where data freshness is important. Additionally, if the cache invalidation timing is poorly designed, there is a risk that users may see information that is older than it actually is for a longer period.

In practice, cache retention times and update priorities had to be applied differently depending on the data characteristics for each screen. For example, data like total assets, which has a high exposure frequency and relatively low volatility, was given a longer cache retention time, while data like transaction histories, which require timeliness, was given a shorter cache retention time with a higher update priority. The process of establishing these criteria itself was a separate tuning task, requiring individual judgment tailored to the data characteristics rather than a uniform answer.

7. Settling into an operational guide

Instead of stopping at one structural improvement, I organized the cache retention times and update priority criteria by data type into an internal team guide. This allows us to quickly decide by matching the characteristics of the data against the criteria below instead of rethinking the cache policy from scratch every time a new screen is added.

[Table 3] Cache operation standards by data type (example)

The significance of this criterion table is not merely a collection of configuration values, but rather a shared thought process within the team that first answers the question, 'How often does this data change, and how quickly does the user expect recency?' and then translates that answer into a caching policy. Subsequently, even in the stage of planning a new screen, this procedure was followed, establishing the caching strategy as a replicable operational criterion rather than a one-time problem-solving task.

8. Conclusion

Looking back on this experience, I reached solutions not by first learning and applying trends or established patterns, but by accurately decomposing the issues encountered during operations. Nowadays, libraries like TanStack Query or SWR.js allow us to apply the same strategies with just a few lines of declarative code, but the prior experience of contemplating 'Why is this strategy necessary?' has provided a deeper understanding than merely using libraries.

When introducing new tools, I reaffirmed the experience that understanding the essence of the problem that the tool solves alters the scope and depth of its application.

cryss