Rendering & Engine

How the v2 WebGPU engine turns declarative state into pixels: the render graph, the compose-once/view-many split, density-banded tiled compositing, the pixel processing order inside the shaders, and the two-Worker IO topology.

Frame Assembly Pipeline

Frame state + Camera + volatile drafts
    │
    ▼
SceneAssembler.buildContent      flatten groups, cull, resolve masks,
    │                            plan uploads (camera-independent)
    ▼
SceneContentCache                memoizes camera-independent content
    │
    ▼
syncAssets → uploadSource        bitmaps / raw pixels / LUTs → VRAM
    │
    ▼
composeView                      overlay camera + channelMask
    │                            → immutable Scene (pure data)
    ▼
SceneCompiler.compile            separable batches vs ping-pong steps
    │
    ▼
compositeSignature               value-based dirty check —
    │                            hit = skip compositing entirely
    ▼
RenderGraph
    ├── composite()   camera-independent, document space
    └── present()     camera-dependent, screen only

Compose-Once, View-Many

Document-space compositing is completely camera-independent: the result persists in a Composited Working Texture, and the view transform is a cheap trailing ViewPass. Panning and zooming never re-run compositing — the composite signature (a value-based signature string derived from the declarative Scene) short-circuits the whole half of the graph.

Density Bands

Text (and other resolution-sensitive content) quantizes camera scale into bands {1.0, 1.5, 2.0, 3.0, 4.0}. The band is the only camera-dependent dimension in the SceneContentCache key, so any pan/zoom within a band costs zero recomposition; crossing a band recomposites once at the new density.

The RenderGraph

The graph compiles the Scene into passes:

Pass	Type	Responsibility
CompositePass	Render	source-over separable batches; adjustments inlined in the fragment shader
BlendPass	Render	15 non-separable blend modes via ping-pong
AdjustPrePass	Render	ordering only, when a layer has both adjustments and convolution filters
FilterPass	Compute	separable Gaussian / convolution
Vector Build stage	Compute/Render	SDF / stroke / text renderers produce transient textures
ViewPass	Render	camera transform, channel swizzle, terminal sRGB encode

Key properties:

Adjustments are not a pass. apply_adjustments() is a shared WGSL function inlined into the layer/blend compositing shaders. The only real pass, AdjustPrePass, exists solely to order "adjust then convolve" on the same layer — about 90% of scenes (no filters) run with zero extra passes.
Two blend paths. source-over uses hardware blend state and is batchable; the other 15 modes must read the background and compute the W3C blend in-shader, so they ping-pong (one pass per layer, unbatchable).
Vector spine, three stages. The graph is organized Build → Composite → Present. Vector renderers (sdf, stroke, text) pre-produce straight-alpha textures in the Build stage; compositing then treats them as ordinary raster layers with no special-case branches. Build must fully encode and end before CompositePass opens — WebGPU forbids nested render passes.

Pixel Processing Order

What a single pixel experiences inside a composite fragment shader, in exact order:

textureSample(layer_tex)          source pixel (encoded domain, maybe premultiplied)
    │
    ▼
srgb_to_linear                    unless flag LAYER_FLAG_SOURCE_LINEAR
    │                             (gamma-domain premultiply/blend is mathematically
    │                              wrong — the root cause of v1 dark edges)
    ▼
gamut_to_working (3×3)            per gamut_id, NO clamp — out-of-gamut
    │                             values stored as-is in the float composite buffer
    ▼
render-intent tone-map             flags bits 8..9; 0 = SDR passthrough
    ▼
apply_adjustments                 only if adjustment flags != 0
    │                             TRC-wrapped: linear → encode → curves/levels/
    │                             balance/3D LUT → decode → linear
    │                             (perceptual operators run in the encoded domain)
    ▼
mask modulation                   alpha *= bmask_alpha × vmask_alpha
    │
    ▼
opacity                           alpha *= layer opacity
    │
    ▼
premultiply                       rgb *= a (skip if source already premultiplied)
    │
    ▼
blend                             source-over → hardware blend state (batchable)
    │                             other 15 → sample bg, compute in-shader (ping-pong)
    ▼
write to the float linear-light composite buffer (rgba16float default)

The terminal sRGB encode happens exactly once, at the very end: in the view shader for screen, in unpremultiplyEncodeGamut for files. There is no second encoding point anywhere in the pipeline.

Bit-Depth Model

The rule of thumb: storage forks by source depth, compute always converges at f16, export forks again by target depth.

Source file
    ├─ 8-bit ────────► createImageBitmap → rgba8unorm texture (zero-copy, no bloat)
    ├─ 16-bit ───────► lib-vips → normalized → f16 → rgba16float
    ├─ RAW 14-bit ───► libraw → probed gamut → f16 → rgba16float
    └─ float32 ──────► Float32Array pass-through → rgba32float (full precision)
                 │
                 ▼
    ★ composite buffer is float linear light — rgba16float by default ★
                 │
        ┌────────┴─────────┐
        ▼                  ▼
   Screen (ViewPass,   Export: canvas-8 / raw-8 / raw-16
   depth-independent)  (decided by frame.bitDepth)

Invariant	Scope	Rule
A — Store	per layer	Resident texture format equals the layer's true source depth. 8-bit sources never bloat to f16; 32-bit float sources stay `rgba32float` (16 bytes/texel).
B — Compute	global	Compositing, adjustments, and filters always run in a float linear-light working format regardless of source depth: `rgba16float` by default, upgraded to `rgba32float` only when the adapter grants BOTH `float32-blendable` and `float32-filterable` (the ping-pong passes need blending and linear sampling on the target).
C — Export	per document	Export bit depth comes from `frame.bitDepth`, orthogonal to storage. Three channels are available: `canvas-8`, `raw-8`, `raw-16`.

Known edges: >8-bit AVIF is clamped to 8-bit at ingest; HEIC with an unknown gamut falls to the unsupported path; the P3-bounded working space compromise is slated for widening in the ultrawide working space plan.

Memory & Caching

Component	Policy
`TexturePool`	Tiered VRAM pool; layer textures allocated by local physical bounding box, transient intermediates pooled in size buckets — zero per-frame allocation garbage
`SourceBitmapCache`	Main-thread 8-bit source of truth
`HighDepthTextureCache`	16/32-bit naked float pixels, ~2 GB budget, recency-LRU eviction, persisted assets pinned, negative caching for failed loads
`WorkerCache`	LRU on the Engine Worker

Engine API Surface

export interface IEngine {
  init(canvas: HTMLCanvasElement): Promise<Capabilities>;
  render(scene: Scene): void;                       // compile + dispatch to swapchain
  export(scene: Scene, opts: ExportOptions): Promise<ExportResult>;
  uploadSource(assetId: string, src: UploadSource, version?: number): void;
  release(assetId: string): void;
  destroy(): void;
}

A Scene is an immutable descriptor: frame dimensions, display config (channel mask, working space, HDR flag), the view transform, and a bottom-up flat array of LayerNodes. Each node carries its source kind (raster / vector / solid / stroke), a camera-independent affine transform, opacity, one of 16 blend modes, optional mask / adjustments / filters / clip.

Feature Negotiation

At device init the engine probes and intersects the optional features: shader-f16 and subgroups (negotiated and recorded on Capabilities, currently reserved — the WGSL shaders run the f32 path), float32-blendable/float32-filterable (platform-limited; BOTH together upgrade the working format to rgba32float, otherwise it stays rgba16float), and timestamp-query (nanosecond GPU pass timing for the profiling panel). The policy is "capability adaptation ≠ fallback": a missing feature selects a more conservative WebGPU path, never a CPU branch.

Cross-Platform Presentation

The swapchain is configured as display-p3. macOS presents natively via ColorSync; Windows SDR is faithfully projected to sRGB by Chromium, Windows 11 Auto Color Management passes wide gamut through, and HDR mode (toneMapping: extended) reaches scRGB with highlights above 1.0.

IO Chain: Two Worker Topologies

PixelFacade (thin delegation)
    ├── ImageDispatcher      decode / resample / imageData / cache ─┐
    ├── CompositeDispatcher  subset composite → engine.export()     │ GPU readback
    │                        (main-thread engine, NOT a Worker)     │ (main thread)
    ├── ExportDispatcher     scene → EncodeSource → FileService.encode
    │                                                               ▼
    │                                          WorkerBridge (RPC) → Engine Worker
    │                                          (1 resident singleton, no vips:
    │                                           decoder · resample · extract-pixels ·
    │                                           histogram)
    │
    └── FileService (format routing, self-owned transcode Workers)
        ├── LibVipsWorker      shared singleton (pthreads, needs COOP/COEP)
        ├── LibRawWorker       on-demand, single-threaded WASM
        ├── VectorWorker       resvg (SVG) + Ghostscript (EPS), idle-recycled
        └── AvifWorker         jsquash AVIF codec

The two Worker families are deliberately never merged: they differ in WASM threading requirements (COOP/COEP vs. not), lifecycle (resident vs. shared-singleton/on-demand), and crash isolation (a Worker chewed by an untrusted file can be rebuilt alone).

Last updated: 2026-10-12