Precision Edge Detection in Pure TypeScript: FocusFlow 2.5ms Morphological Filter

In complex frontend graphics workbenches (such as architecture diagram editors, infinite whiteboards, and spatial presentation engines), Click-to-Snap is one of the highest-frequency, mission-critical interactions: when a user imports a high-density 4K architectural diagram (rasterized PNG/JPEG/SVG bitmap) and holds Alt while clicking on any microservice card, database component, or cloud container, the engine must extract its pixel-perfect bounding box (Snapped Bounding Box) within a sub-2.5ms instantaneous latency, serving as the geometric anchor for 60fps cinematic camera transitions and frustum binding.

While seemingly a straightforward "click-to-box" interaction, handling real-world structured UI and modern design specifications presents formidable physical challenges to classical computer vision algorithms.

1. Architectural Considerations: Pragmatic Trade-offs for Structured Graphics

In FocusFlow's everyday workflows, the vast majority of assets users import and interact with are structured diagrams composed of regular rectangular geometries—such as microservice topologies, cloud deployment charts, system workflows, and UI screenshots.

During the initial technical exploration, we evaluated two distinct directions:

• Classical Mathematical & Morphological Operators (Current Implementation):
  Given that our current focus is squarely on structured rectangular cards and containers with predictable geometries, our priority was achieving an ultra-lightweight, zero-dependency, and fully deterministic solution. Implementing color gradient sampling, connected-component flooding, and anisotropic morphological filtering directly in pure TypeScript allows us to solve the problem cleanly without third-party weights. For high-frequency graphics interactions (such as cursor hover snapping and continuous camera targeting), this pure mathematical approach delivers immediate practical benefits: zero model loading overhead and sub-2.5ms execution times, comfortably fitting within our 60fps frame budget.

• On-Device Vision Models (e.g., MobileSAM, Architectural Roadmap Candidate):
  Visual segmentation models excel at identifying arbitrary organic contours and semantic boundaries. While we have not integrated MobileSAM into our current production pipeline, we keep it in our architectural roadmap as a promising candidate to explore when we need to handle hand-drawn sketches, irregular polygons, or more fluid natural imagery down the road.

flowchart TD
    Click["User clicks canvas coordinate (clickX, clickY)"] --> Phase1Check{"Primary Engine: Phase 1 (Morphology & Flood Fill)"}
    
    subgraph Phase1Sub ["Phase 1: Pure Frontend Morphological Closing Engine"]
        Profiler["3-Ring Flatness Profiler<br/>Top-3 Candidate Cluster Fallback"]
        FloatBFS["Floating Gradient BFS<br/>Flat Pixel Fast Path (< 2ms)"]
        AnisoKernel["Anisotropic Separable Closing<br/>rx=6 Lock Borders, ry=14 Fuse Multi-line Text"]
        Profiler --> FloatBFS --> AnisoKernel
    end

    subgraph Phase0Sub ["Phase 0: Adaptive Parallel Ray Radar (Fallback Engine)"]
        Radar1["Sample 11x11 Local Variance & Seed Color"]
        Radar2["4-Way Multi-channel Stepping Ray Detection"]
        Radar3["Color Step Delta + Edge Gradient Breaker"]
        Radar1 --> Radar2 --> Radar3
    end

    Phase1Check -->|Full Bounding Box Extracted| GuardCheck{"Geometric Consistency Guard"}
    GuardCheck -->|Verified| SubpixelSnap["Subpixel Physical Border Refinement (snapRectBounds)"]
    GuardCheck -->|Fragment / Off-center / Hole Detected| CandidateFallback{"Secondary Candidate Cluster Available?"}
    CandidateFallback -->|Yes| FloatBFS
    CandidateFallback -->|Exhausted| Phase0Run["Graceful Fallback to Phase 0 Ray Radar"]
    
    Phase1Check -->|Variance Outlier / Pure Wireframe| Phase0Run
    Phase0Run --> SubpixelSnap
    SubpixelSnap --> Output["Output Final SnappedBox (< 2.5ms)"]

2. Five Real-World CSS Edge Cases That Break Naive Sobel

Standard Sobel gradient operators assume simple high-contrast geometries with uniform solid fills. In modern production architecture diagrams, however, 5 critical physical hurdles emerge:

1. Text Hole Perforation: Card interiors are densely packed with multi-line text and monospace code snippets. High-frequency font glyph edges are easily misidentified as outer borders, causing ray probes to truncate prematurely.
2. CSS Linear Gradient Cutoff: Contemporary UIs frequently utilize subtle linear gradients spanning over 400px. Cumulative color drift causes fixed-threshold flood fills to collapse midway through a container.
3. Emoji & Colorful Badges: Clicking directly on inline icons (e.g., 🐘 PostgreSQL, ⚡ Redis) catastrophically pollutes single-point background color sampling.
4. Text Line Blockades: A continuous 24-character monospace title forms an impermeable horizontal barrier that severs flood fill propagation between upper and lower card sections.
5. Dark-on-Dark Contrast: Dark-mode cards (#0f172a) against dark canvas backgrounds (#0c1220) yield minimal perceptual contrast ($\Delta E < 25$) paired with diffuse drop shadows.

3. The Pure Math Architecture: A 3-Stage Morphological Filtering Pipeline

To attain deterministic sub-2.5ms latency without neural networks, we architected a 3-tier deterministic pipeline in pure TypeScript.

3.1 Compact 3-Ring Flatness Profiler

Abandoning single-pixel color probing, the algorithm samples 33 concentric points across four discrete rings ($r \in [1, 8, 16, 24]\text{px}$) centered around the cursor. Each sample is inversely weighted by its local Sobel gradient flatness:

$$W_i = \max(1, 35 - G_i)$$

Where the gradient magnitude $G_i$ is computed via a local $3 \times 3$ Sobel operator:

$$G_x = \begin{bmatrix} -1 & 0 & +1 \ -2 & 0 & +2 \ -1 & 0 & +1 \end{bmatrix} * I, \quad G_y = \begin{bmatrix} -1 & -2 & -1 \ 0 & 0 & 0 \ +1 & +2 & +1 \end{bmatrix} * I$$

$$G = \sqrt{G_x^2 + G_y^2}$$

High-gradient text and icon noise are aggressively suppressed, allowing flat background card pixels to dominate. Paired with a Top-3 candidate clustering fallback loop, this completely eliminates sampling drift caused by clicking directly on icons or text.

3.2 Floating Gradient BFS with Flat-Pixel Fast Path

Rather than enforcing an absolute global color threshold, we constrain local step-to-step color variation dynamically:

$$\Delta E_{\text{step}} = |R_{curr} - R_{prev}| + |G_{curr} - G_{prev}| + |B_{curr} - B_{prev}| \le 8$$

On continuous flat background pixels, the algorithm completely bypasses expensive $3 \times 3$ Sobel convolutions via a designated fast path, reducing flood-fill propagation time from 106ms down to 1.8ms (a 58x speedup).

export function floatingGradientBfs(
  pixels: Uint8ClampedArray,
  width: number,
  height: number,
  startX: number,
  startY: number,
  baseR: number,
  baseG: number,
  baseB: number
): Uint8Array {
  const mask = new Uint8Array(width * height);
  const queue = new Int32Array(width * height);
  let head = 0;
  let tail = 0;

  queue[tail++] = (startY << 16) | startX;
  mask[startY * width + startX] = 1;

  while (head < tail) {
    const pos = queue[head++];
    const x = pos & 0xffff;
    const y = pos >> 16;
    const idx = (y * width + x) * 4;

    // Fast-path step validation
    const diff =
      Math.abs(pixels[idx] - baseR) +
      Math.abs(pixels[idx + 1] - baseG) +
      Math.abs(pixels[idx + 2] - baseB);

    if (diff <= 8) {
      // Expand 4-connectivity
      // ...
    }
  }

  return mask;
}

3.3 Spatially Separable Anisotropic Morphological Closing

To seal internal text perforations while strictly maintaining crisp vertical card boundaries, we engineered an asymmetric kernel matrix:

• Horizontal Radius ($r_x = 6\text{px}$): Clamps tightly against left/right physical borders, preventing adjacent cards from coalescing.
• Vertical Radius ($r_y = 14\text{px}$): Extends 2.3x further vertically to fuse multi-line paragraph voids and long titles.

Because 2D rectangular convolution kernels are mathematically separable:

$$K_{2D}(x, y) = K_x(x) * K_y(y)$$

Algorithmic complexity plummets from $O(K^2)$ to $O(2K)$, eliminating over 90% of convolution computations compared to naive 2D dilation and erosion passes.

Sobel Raw Gradient

Morphological Closed Pipeline

◀ Prev 1 / 2 Next ▶

4. Extreme Memory Architecture: Zero-GC TypedArrays

During 60fps cursor interactions, JavaScript runtime garbage collection pauses (Minor/Major GC sweeps) are the primary culprits behind dropped frames.

• All binary masks, 4-way scanning queues, and boundary point sets reside in pre-allocated Uint8Array and Int32Array contiguous buffers.
• Transient heap allocations are strictly bounded under < 450 KB.
• Achieves 0 GC pauses across full interaction lifecycles, maintaining a rock-solid 60fps on Apple Silicon and modern PC Chrome environments.

5. Performance Benchmarks & Industrial Validation

Validated across 20 industrial-grade automated E2E test suites in FocusFlow's CI pipeline (encompassing nested cloud topologies, dark-mode wireframes, and accent-bar cards):

6. Roadmap & Architectural Takeaways

In FocusFlow, this pure TypeScript algorithm provides an ideal balance of sub-2.5ms instantaneous response, zero bundle overhead (0KB extra models), and deterministic zero-GC stability for structured UI diagrams.

Classical mathematics and deep learning models are not mutually exclusive. Deterministic morphological operators reliably guard 95% of structured rectangular cards and diagrams; for future unstructured hand-drawn sketches or organic shapes, on-device WebGPU models can be dynamically loaded on demand, yielding a resilient tiered hybrid intelligence pipeline.

In targeted problem domains, engineering directly from mathematical first principles frequently delivers unmatched lightness and performance.