1. Background of technology selection: Maximizing the given environment (Ag-Grid) and the necessity of TypeScript
One of the biggest challenges of the project was to provide a user-friendly interface that allows users to quickly and intuitively query tens of thousands of data entries. Particularly, since it needed to offer a user experience similar to Excel rather than just simple query functionality, the difficulty of the requirements was quite high.
Users were naturally expecting the following features.
● Real-time search and filtering
● Column alignment and fixation
● Multi-selection
● Scroll optimization and bulk data rendering
● Excel download
When I was assigned to the project, Ag-Grid-based React, specialized in handling large-scale data processing, was already implemented as a core component. Since the amount of data was beyond what a typical HTML Table could handle due to browser DOM overload, the adoption of Ag-Grid, which supports virtual scrolling by default, was an essential choice.
My main task was to push the performance of the already implemented Ag-Grid to its limits and customize it according to the requirements. I optimized the handling of tens of thousands of records by utilizing Ag-Grid's server-side Row Model, multi-sorting, and custom Cell Renderer.
Additionally, the project has enforced the use of TypeScript. As the scale of the service increased, the API response structure and state management structure became increasingly complex. In cases where only JavaScript was used, errors were often encountered only at runtime; however, this was resolved by clearly defining the API response structure based on interfaces.
● API Response Interface
● Grid Row Model / Domain Type
● State Type / Event Payload Type
As a result, just by looking at the code, I could immediately grasp the data structure, and it greatly reduced communication costs during the collaboration process.
2. Architecture design and separation of layers: two project experiences
As the project grew larger, maintainability and structural stability began to become much more important than simply implementing functionalities. Through two projects at different times, I was able to directly experience the impact of frontend architecture on development productivity and maintenance.
The first project I experienced adopted a View-Stub-State structure. In this structure, folders were separated by functionality, and the roles of each layer were clearly defined.
● View: Responsible for pure UI rendering
● Stub: Responsible for API communication and domain business logic
● State: Global State and Business State Management
The biggest advantage of this structure was that it allowed for a quick understanding of the project flow. In fact, when I was first brought onto the project as a new developer, this clear separation of layers enabled me to easily grasp the functional structure and significantly reduce onboarding time. The high independence of functional units also contributed to excellent maintainability and scalability.
In the second project that was subsequently implemented, we used the Container-Presenter pattern. This pattern is a representative structure that separates UI rendering (Presenter) from business logic (Container).
This structure has the advantage of clear Separation of Concerns. By keeping the Presenter as a pure UI component, it allows for good testing and reusability. However, as the scale and complexity of the service have increased, limitations compared to the existing View-Stub-State structure began to emerge.
● Increase in folder structure complexity
● Distributed state management
● Challenges in managing duplicate domain logic and API structure
As a result, through experience with both architectures, I gained insights that for large-scale projects operated over the long term, modularization of functional units in a View-Stub-State format, which further isolates domain logic and state, may be more advantageous.
3. Experience in Optimizing Large-Scale Data Processing
Large-scale data processing was more than just displaying data on the screen. In particular, optimizing rendering performance and state management was crucial in the existing Ag-Grid environment.
Initially, the entire Grid was re-rendered every time the data changed, resulting in issues such as scroll delays and increased browser memory. To solve this, several optimization strategies were implemented.
The first optimization applied was rendering optimization based on React.memo. The Cell Renderer and Row Component were memoized to reduce unnecessary re-renders. Additionally, useMemo and useCallback were actively utilized. Particularly in Ag-Grid, frequent re-creation of the column definition objects led to the entire Grid being re-initialized, so this was firmly fixed using useMemo.
Additionally, to reduce network load and initial loading times, we have advanced the server-side Pagination structure. By transferring the search and sorting logic to the server instead of the client, we significantly reduced the burden on the browser and established a structure that reliably handles thousands of data entries even in a real operating environment.
4. Problem-solving experience: Data integrity and asynchronous event handling
One of the major concerns in the project was the data consistency issue of the notification service. Within a single method, the DB notification record saving and the external API message sending were performed simultaneously.
The problem occurred when an exception arose after the external API call, causing the DB transaction to roll back. The user received the notification, but there was a serious consistency error where no record remained in the system. The external API call could not guarantee the same atomicity as the DB transaction inherently.
Ultimately, the project adopted an event-driven asynchronous processing structure. The core idea was to 'perform external tasks only after the DB commit.'
1. Main business logic and DB saving execution
2. Event issuance at the point of DB transaction commit completion
3. A separate event handler independently processes message sending (separate from the main transaction)
Through this structure, if the DB storage fails, the message will not be sent, and even if the external API fails, it can be completely isolated so that it does not affect the main business logic.
5. Improvement effects from the perspective of collaboration and maintenance.
The frontend project becomes increasingly complex over time with components and state management structures. The biggest takeaway from my experience with two large projects was realizing how much the initial architecture design and type definitions impact collaboration with colleagues.
Managing API responses with a common type through TypeScript has made my understanding of the data structure with the backend much more intuitive. Additionally, by experiencing a clearly separated architecture like View-Stub-State, I have learned firsthand how to minimize side effects when modifying specific features.
The quality of code reviews naturally improved. As the structure became unified, the time spent interpreting the implementation method decreased, allowing for reviews focused on the business logic itself.
6. Conclusion
These projects were a time to deeply feel how important architectural design and operational stability are in a large-scale data environment, going beyond merely implementing a React interface.
Although a great tool called Ag-Grid was already provided, the process of tuning and optimizing it to meet the project's requirements was by no means simple. Additionally, experiencing two different front-end architectures consecutively taught me that designing a structure suited to the team's scale and service characteristics is essential.
Ultimately, these experiences have meant more than just simple feature implementations and will serve as a solid and unwavering reference point when designing and troubleshooting large-scale React-based systems in the future.
LEE DAVID