Processing Large-scale Clinical Data and Legacy Integration

Experience in MSA backend optimization for large-scale clinical data processing and legacy integration (AMC PROMs construction)

1. Introduction and technical challenges

The core data supporting 'Patient-Centered Care', which is a central paradigm of modern healthcare, is PROMs (Patient-Reported Outcome Measures). I was responsible for the backend development of the AMC PROMs platform at Asan Medical Center, undertaking the mission of reliably collecting and analyzing vast clinical data while integrating it with the massive hospital core system (AMIS). I would like to share the process of addressing four technical challenges faced during this process: certification limitations, real-time data visualization bottlenecks, large-scale retroactive calculation OOM, and the risk of failure propagation, and how I resolved them from an architectural perspective.

2. A non-standard payload-based authentication adapter that overcomes HTTP header constraints

[Problem Situation: Limitations of HTTP Header Manipulation in Legacy Environment] The latest MSA environment and Spring Security adhere to the standard protocol of transmitting JWT via the HTTP Authorization Header. However, the hospital core system (AMIS) could not send tokens in standard HTTP headers due to structural constraints and had to transmit encrypted authentication information in a specific field (encToken) within the Request Body (AmcData).

[Solution: Implementation of ContentCachingRequestWrapper and Custom Filter] To accommodate non-standard communication while maintaining internal security standards, a custom Security Filter was implemented at the front end. Due to the nature of servlets, the InputStream is lost after reading once, so ContentCachingRequestWrapper was applied to cache the Request Body. The encToken from the cached Body was extracted in the filter, validated against the Keycloak server, and valid information was injected into a ThreadLocal-based ContextHolder. This allowed for the acceptance of non-standard external integrations while ensuring that the server's internal authentication adapter followed a perfectly standard authorization control flow.

3. Introduction of Kafka & CQRS patterns for real-time clinical data visualization

[Problem Situation: Real-time DB bottleneck due to complex medical formula calculations] Before the patient submits the questionnaire and enters the consultation room, pain index and quality of life (QoL) changes should already be visualized on the doctor's monitor. If healthcare professionals have to join a vast historical survey response table and calculate medical rules every time they open the dashboard, it will cause serious query latency, resulting in inconvenience for users.