blog

Improved DOM Capture

NEXTREE.

• 16 min read
Improved DOM Capture

Introduction

This time, I was tasked with developing the Snap feature for the echo service. Snap is a feature that captures the screen exactly as it appears when the user clicks the camera icon in the header. Instead of explaining something only in words, it sends a screenshot of the screen as well, so the captured image serves both as an explanation and as evidence of what went wrong.

That meant we had to bring both the speed and accuracy of this feature above a certain threshold. First, speed: the target was around one second, because capturing the screen must not interrupt the flow of submitting a report. The other was accuracy: the screen seen by the reporter and the screenshot had to be identical. If they differed, the screenshot would have no value as evidence.

When I first started working on improvements, I thought the problem would be solved by improving speed alone. While optimizing the speed and taking various screenshots, I discovered that there was also an accuracy problem. This article summarizes how I addressed those two stages and what the resulting pipeline looks like today. To state the results first:

Screen

Node count

Current

Clone coordinate comparison (accuracy)

Simple list screen

1,476

436–457ms

0 out of 996

Diagram screen

4,745

1,186ms

0 out of 4,281

A capture that took 3.3 seconds before the improvements now takes 0.45 seconds on a lightweight screen and 1.2 seconds even on a heavy screen with three times as many nodes. Accuracy has also improved: on the heavy screen, the coordinates of all 4,281 elements now match the actual screen exactly.

Limitations of DOM capture

The root of the problem was that Snap was not taking an actual screenshot. Libraries such as html-to-image cannot access the pixels in which a web page has actually been rendered, so in practice they work as follows:

① Clone the entire DOM tree of the screen

② Apply the CSS values currently applied to each element, one by one, as inline styles

③ Inject the result into <foreignObject> inside an SVG

④ Load that SVG as an <img> and draw it onto a canvas

In other words, it is similar to making the browser render the same HTML one more time. The problem is that the SVG inside the <img> in step ④ is completely isolated from the original page. It is designed this way for security reasons, and this isolation has three characteristics, each of which later came back as an actual bug.

  • It cannot access the web fonts of the parent document. If they are not embedded, they are replaced with system fonts, changing the character widths and shifting the layout.

  • Network access is blocked. Since image URLs cannot be fetched as-is, they must all be converted into data URI (a form in which the file contents are converted into a string and inserted) before capturing.

  • Scripting is disabled. Elements whose behavior changes depending on whether JavaScript is executed are rendered differently only in the capture.

Development process 1 — Finding the cause of the slowness and trying four libraries

The first request I received was, "Screenshot generation seems to be taking somewhat long." When I measured it, the dialog opened immediately, but a spinner continued running for more than three seconds until the screenshot was ready. During that time, the main thread stopped completely, and even clicks could not be processed. JavaScript in the browser uses a single-threaded structure that handles only one task at a time, so when the capture holds on to that thread, the screen appears to freeze.

Investigating where bottlenecks occur in each stage

At first, I thought the large number of elements on the screen was causing the delay, so I began by dividing the process into stages and measuring how long each one took. The measurements showed that one factor was the size of the serialized SVG. The analysis led to the conclusion that about 40KB of inline styles were attached per node. The other factor was cssText: I confirmed from the source that when this value is empty, the library takes a path that moves properties one by one instead of using the fast path that copies styles all at once.

When I extracted the actual list, I found that the app had about 1,000 computed value properties, half of which were CSS custom properties. These were variables beginning with -- that contained theme colors or icon paths. However, because computed value returns the final value after the variables have already been substituted, there was no reason to inject the variables themselves into the clone again. After also removing properties unrelated to visual appearance, I reduced the list to about 100 items.

// utils/inlineStyleProperties.ts — 인라인할 CSS 프로퍼티 127개
export const INLINE_STYLE_PROPERTIES = [
  'display', 'position', 'top', 'right', 'bottom', 'left', /* ... */

  // 'box-sizing' 은 반드시 있어야 합니다. Chrome의 computed width 는 그 요소의
  // box-sizing 기준값이라, 빼면 클론이 padding+border 만큼 넓어집니다.
  'width', 'height', 'box-sizing', /* ... */

  // ::before / ::after 규칙도 이 목록으로 만들어집니다.
  // 빠지면 의사요소(구분선·체크박스·화살표)가 통째로 사라집니다.
  'content',

  // 아이콘 대부분이 mask 로 렌더됩니다. 빠지면 아이콘이 전부 사라집니다.
  'mask-image', 'mask-size', '-webkit-mask-image', /* ... */
];

Code 1. Omitting an item from the list does not cause an error; it only causes the screenshot to quietly diverge. That is why I left the reason in a comment.

The most alarming item on this list was content. If it had been omitted, every element drawn with ::before / ::after would have been rendered empty. I did not catch this through testing; I happened to discover it while reading the library source. The biggest weakness at this point was that, while creating the list, I had not also established a way to verify what was actually needed. This was also the direct reason I later began evaluating other libraries.

I implemented and compared each of the four approaches

A list extracted by AI is inherently risky. Snap is part of a shared library intended to be used across multiple episodes, but I created the list based on only a few screens. If another team's app uses a property not included in the list, the screenshot will quietly diverge without producing an error. Since I judged silently incorrect results to be worse than slow results for a bug-reporting tool, I tested alternatives that would not require curation.

Approach

Method

Lightweight screen

Heavy screen

Increase rate

Option A

html-to-image + CSS properties extracted by AI

378–388ms

828–865ms

2.2×

Option B

modern-screenshot + CSS properties generated at runtime

575–863ms

1,870–2,045ms

2.8×

Option C

snapdom + class deduplication

1,235–1,985ms

5,186ms (26 seconds on the first run)

4.2×

Option D

html2canvas — interprets CSS in JS and draws it directly

516–610ms

(Different measurement conditions)

—

The final column was what determined the decision in this table. When the number of nodes increased 4.3-fold, the growth rates themselves differed. Since the number of properties read and written per node was about 100 for Option A and about 450 for Option B, it was inevitable that the gap would widen as the screen became heavier. If we had compared only lightweight screens, we would have chosen Option B.

  • Option B (modern-screenshot) — Replacing only the library produced no improvement. It makes the output smaller by inlining only the values that differ from the defaults for each tag, but you ultimately have to read everything to determine what differs. Passing in a list generated at runtime brought the time down to 806ms, but it exceeded two seconds on heavy screens.

  • Option C (snapdom) — Because it groups identical styles into classes, it produced the smallest markup, and its rendering fidelity was the only perfect one among the three. However, there was no public option for including only a font subset, so the entire app font had to be embedded as well. As a result, the first capture on a heavy screen took 26 seconds.

  • Option D (html2canvas) — We tested it because it is a widely used library. Since it belongs to a different category (it interprets CSS in JS and draws directly onto a canvas), the fact that font embedding was entirely unnecessary was appealing. However, support for mask-image, which most of the app's icons use, was completely absent from the bundle, so all seven header and toolbar icons were rendered as black squares. Its last release was in 2022, and it would sometimes encounter a modern CSS function, throw an exception, and crash.

Why we kept the library we had originally been using

The conclusion was to keep Option A. It was the only option that met the one-second requirement on heavy screens, and since we could not know which screens snap-view would be used on, we decided that barely passing on the screens we measured was not enough.

The argument supporting Option B was that "without curation, it is safer with unfamiliar CSS," but when we compared two captures pixel by pixel on screens we had not inspected while creating the whitelist, we found a difference of about 10%. At first, I thought that Option A must indeed be missing something, but when I checked each one against the actual screen coordinates, both turned out to be wrong in the same way. The difference was not that one was more accurate; it was that the two libraries rendered the same CSS slightly differently. We decided that we could not choose something 2.2 times slower for an unverified benefit. That said, this does not prove that "Option A is safer." It only means that the issue was not observed on the two screens, so this weakness still remains.

The Option D experiment also taught us one costly lesson. When html2canvas fails, it leaves an iframe on the page, but querySelectorAll('*') cannot see inside it. When we reran Option A with five of them accumulated, the markup grew to 66MB and the capture time ballooned to 4,960ms. The only clue was that it deviated 13-fold from the baseline; if we had trusted that number as it was, we would have produced a completely incorrect document saying, "Option A takes five seconds on this screen."

Development process ② — After making it faster, we discovered that the captures differed from the screen

Once we had addressed the speed, we placed the captures next to the actual screen and found four discrepancies. And the cause of all four differed from our initial assumptions. In particular, we suspected that issues 2, 3, and 4 were all font problems, but in reality they had nothing to do with fonts.

#

Symptom

Actual cause

Category

1

The ag-grid checkboxes and sort arrows were missing entirely

Pseudo-element backgrounds were not covered by the library's resource inlining path

Resource

2

The text appeared larger and pushed out of place

pixelRatio: 1 is based on CSS pixels (with the user's zoom at 90%)

Scale

3

The header breadcrumb wrapped onto two lines

A decimal rounding value of 0.007px flipped flex-wrap

Layout

4

An empty strip appeared below the header

<noscript> came back to life only inside the capture

Layout

Missing icons

The row-selection checkboxes and column-sort arrows disappeared entirely, but only in the capture. Meanwhile, the camera and notification icons in the header were fine. We did not know why the same icons behaved differently, so we dug into the library source and found that there were only two places where resources were converted to data URIs: the path that reads an element's own background and the path that reads <img> tags. However, pseudo-element rules are created as <style> text and attached, so they were covered by neither path. The problematic CSS was in the host app's SCSS, so we could not fix it ourselves either.

The clue was in an unexpected place. Looking at the source of the fontEmbedCSS option, which we had been using only to embed fonts, we found that it took the received string as-is, created a <style> from it, and inserted it at the very front of the clone. Despite its name, it was actually an arbitrary CSS injection point.

const cssText = options.fontEmbedCSS != null ? options.fontEmbedCSS : ...
if (cssText) {
  const styleNode = document.createElement('style')
  styleNode.appendChild(document.createTextNode(cssText))
  clonedNode.insertBefore(styleNode, clonedNode.firstChild)
}

Code 2. Checking this one line resolved the issue we had documented as having "no path to fix."

So immediately before capture, we scanned the pseudo-elements of every element, selected only those using url(), converted the relevant images to data URIs, added a marker attribute to the target elements, and passed in CSS rules targeting those markers. Two things were essential. !important was necessary because the rules created by the library are inserted later and therefore win through document order. Also, declarations on identical elements were made to share marker numbers; otherwise, each ag-grid row would copy the same checkbox image data once per row, causing the SVG to bloat.

One assumption was overturned here. Thinking that parsing the stylesheets would be cheaper than scanning every element, we built that approach first, but actual measurements showed that the stylesheet-scanning approach was much slower than traversing every element. getComputedStyle was cheaper than expected, while scanning the stylesheets was expensive. Switching to direct traversal shortened the code and actually reduced the capture time from 348ms to 332ms.

Captures that were larger than the screen

As we captured several screens, we noticed that text in the header appeared larger and pushed sideways. It looked as though the layout was broken, but it was actually a scaling issue. The screen is rendered with devicePixelRatio physical pixels for each CSS pixel, but the option was fixed at pixelRatio: 1. I was using 90% browser zoom, which made the DPR 0.9, so the capture became 1.112 times larger (1÷0.9).

const capturePixelRatio = (): number =>
  Math.min(window.devicePixelRatio || 1, 2);

Code 3. In a DPR 2 environment, the capture time was nearly unchanged; only the size increased (120KB → 316KB).

Empty strip below the header

Only in the capture, a 24px-high empty strip appeared directly below the header, and all the content below it was shifted down. However, when I remeasured the layout coordinates of the clone, they matched perfectly. If there was no margin at the DOM stage, the only possibility left was the rasterization stage, so I directly scanned the pixels of the captured image and extracted the "y ranges containing ink." Only the header was in its original position; everything else was shifted down by exactly 24px. Since it was shifted once rather than cumulatively, this meant that something at the very start of the flow was taking up the space of a single line.

The culprit was <noscript>. According to the HTML specification, the browser's default rule to "hide this" applies only when scripting is enabled. However, as mentioned earlier, SVG inside <img> is rendered in a context where scripting is disabled, so only in the capture this tag came back to life and occupied one line. The reason the header was unaffected is that position: fixed places it outside the flow, which is why it looked like an "empty strip below the header." This was behavior according to the specification, not a Chrome bug.

const EXCLUDED_TAG_NAMES = new Set(['NOSCRIPT', 'SCRIPT']);

filter: (node) => {
  if (!(node instanceof Element)) return true;
  if (EXCLUDED_TAG_NAMES.has(node.tagName)) return false;
  return !EXCLUDED_CLASS_NAMES.some((name) => node.classList?.contains(name));
},

Code 6. After the fix, even 1px lines such as tab underlines and grid header borders returned to their proper positions.

In the end, the key was the diagnostic tooling

Looking back, most of what we actually did in this task was build tools. While comparing screenshots by eye, we failed to catch even one of the four issues.

  • Step 1 · Compare coordinates against a hand-built clone — Only 1 of 996 nodes was misaligned (while animating). We caught nothing here.

  • Step 2 · Extract and compare the SVG actually generated by the library — At this point, pseudo-element styles and filters were all reflected. This is where we caught the flex line wrapping.

  • Step 3 · Directly scan the pixels of the captured image — We counted whether each row contained ink and extracted the boundaries of the content regions. This is where we caught <noscript>.

I think why Step 2 was insufficient is the main lesson of this entire task. The iframe used for comparison has scripting enabled, so <noscript> is hidden. As a result, the coordinates matched perfectly, while only the actual image was misaligned. We did not realize that coordinate comparisons at the DOM level cannot catch problems in the rasterization stage until we took the tooling one step further.

The current capture pipeline

This is the current code with all the decisions and fixes above applied. It contains three pieces of logic that temporarily modify the screen and then restore it, and I left comments explaining the reason for each step in this order.

export const captureViewport = async (): Promise<string | null> => {
  try {
    // 퀵메뉴 닫힘 등 직전 DOM 변경이 화면에 반영된 뒤 캡처합니다.
    await new Promise((r) =>
      requestAnimationFrame(() => requestAnimationFrame(r)));

    // 의사요소 배경을 data URI 로. pinScrollOffsets 보다 먼저
    const { css: pseudoCss, restore: restorePseudo } =
      await inlinePseudoBackgrounds();
    // 현재 라인을 파악해야 하므로 스크롤 처리보다 먼저 진행
    const unpinFlexLines = pinFlexLines();
    // scrollTop 은 CSS 가 아니라 런타임 상태여서 복제본이 찾지 못함
    //  transform 으로 번역해 두면 복제본이 그대로 복사함
    const unpinScrollOffsets = pinScrollOffsets();

    let result: string | null;
    try {
      const shot = toJpeg(document.body, {
        ...CAPTURE_OPTIONS,               // 화이트리스트
        pixelRatio: capturePixelRatio(),  // 실 기기 픽셀 기준
        // foreignObject 는 웹폰트에도 네트워크에도 접근할 수 없으므로 필요한 것은 전부 여기에 넣음
        fontEmbedCSS: CAPTURE_FONT_EMBED_CSS + GLOBAL_CAPTURE_CSS + pseudoCss,
      });
      const timeout = new Promise<null>((r) => setTimeout(r, TIMEOUT_MS, null));
      result = await Promise.race([shot, timeout]);
    } finally {
      // 복구는 반드시 역순. 순서를 바꾸면 스크롤 값이 엉뚱하게 잡힘
      unpinScrollOffsets();
      unpinFlexLines();
      restorePseudo();
    }
    return result;   // 실패·타임아웃이면 null — 제보 흐름을 막지 않음
  } catch (e) {
    console.warn('[captureViewport] failed', e);
    return null;
  }
};

Code 7. The capture entry point.

Here, I will explain pinScrollOffsets in a little more detail. This app is structured so that the inner container of main’s scrolls, rather than the entire browser window, but the scroll position is runtime state rather than CSS, so it becomes 0 in the clone. As a result, even if a user reports an issue while viewing the middle of the screen, the capture contains the top. All three libraries had the same problem, and even the two with corresponding options were ineffective with this structure.

The solution was to translate scrolling into CSS. We move the children of the scrolled container by the scroll amount using transform: translate(), while simultaneously resetting the container’s scroll position to 0. Because the two changes cancel each other out, the actual screen does not move by a single pixel, but transform is part of the computed style, so the clone copies it as-is. As a side effect, sticky/fixed elements move out of place, so we measured the offset and corrected it in the opposite direction. Before applying the correction, 96 elements were misaligned; afterward, that number dropped to 1.

What still remains

The biggest issue is that the whitelist still has to be maintained manually. Properties not included in the list are rendered with the browser defaults, without producing errors. Moreover, each host into which the library is integrated is a separate repository, and the list is inside the distributed package, so even if another team discovers a problem, they have to notify us and wait for a release. The current list was kept to a minimum, so there is room to expand it to around 300 entries by excluding only those unrelated to visual output, and we plan to find the optimal point within the one-second budget.

The second issue is that there is no mechanism to protect the validation. The coordinate comparison and pixel scan we created earlier were tools built and used ad hoc, not automated tests. The situation in which we nearly omitted content can still be reproduced in exactly the same way today. We consider adding an automated test that compares capture results to be the next task.

The third issue is the scope of validation. So far, we have measured only two types of screens: the grid list screen and the diagram screen. We do not know what CSS will be used on other screens, and the underlying structure remains unchanged: if the capture quietly becomes misaligned, no one will know.

That is why we have left open the existing path for users to upload photos they took themselves instead of automated captures. Automated capture is treated as a "convenience feature attached by default without the user having to do anything," while users can attach an image themselves when it fails to capture the scene they want. However, this is closer to a buffer than a fundamental solution. We will continue considering other possible improvements, such as narrowing the capture target from the entire screen to an area specified by the user, or using the browser's screen capture API.

Conclusion

What I realized while working on this task is that it was actually two different kinds of work. Speed was a matter of finding what was expensive, and the numbers provided the answer. Accuracy was a matter of making it possible to see what was different, and we had to build the tools before we could find the answer. Each time we moved up a level—from visual inspection to coordinates, and from coordinates to pixels—we caught another issue.

These libraries also often became difficult to work with when approached by reading only the documentation. includeStyleProperties is used only in a specific branch, so at first I incorrectly concluded that it was "ignored in Chrome." Despite its name, fontEmbedCSS was an arbitrary CSS injection point, while options whose names looked exactly right, such as restoreScrollPosition and clip: 'viewport', had no effect with this structure. When stuck, the fastest debugging method was to first find where the option was used in the source code.

There are still some regrets. We did not build the validation that would enforce the whitelist at the same time as we created it, and we did not check the accuracy issues at the same time as the performance work, discovering them only later. Still, thanks to this experience, I now understand a little more than simply how to use a library: I have a better grasp of how the browser draws the screen and what disappears when that result is transferred into an image again.

Reference documents

Owler

Site footer