# Voxel Impostor system

Point-based ("voxel" / surfel) impostor inspired by the **Nanite Voxels**
representation Epic and CD Projekt RED showed in the Witcher 4 / Unreal
Engine 5 tech demo (Unreal Fest Stockholm 2025). At distance, dense
alpha-tested foliage cards stop scaling cleanly — the new technique
replaces them with a cloud of near-pixel-sized splats, each carrying a
full G-buffer slice, so the LOD never has to fall back to a flat card.

This implementation is much simpler than UE's production system (we
don't do streaming, meshlet hierarchy, or front-to-back depth binning),
but the data model and rendering approach are the same:

- Each "voxel" is a single splat with the same world-space size.
- Per-splat attributes: position, normal, albedo, ORM (occlusion,
  roughness, metalness). **No alpha.**
- Render as a camera-facing instanced quad; the splat's normal drives
  the lighting equations, not the quad's geometry.

Conceptually the system mirrors the octahedral impostor's two phases:

1. **Baking** (CPU-heavy, offline) — produce a point cloud from a
   source mesh.
2. **Drawing** (GPU, every frame) — render the points with a small set
   of shaders (unlit / lit / debug / depth caster).

---

## Baking

Implemented in [`VoxelImpostorBaker`](./VoxelImpostorBaker.js). Strategy:

1. Reuse the existing octahedral baker — it already renders the mesh
   from many directions into a 3-channel MRT atlas (albedo+alpha,
   normal+depth, ORM) with supersampling, anisotropic filtering, vertex
   colours, and normal-map evaluation. We don't reimplement any of that.

2. Read the atlas back to CPU. Every opaque pixel becomes one
   object-space sample:
   - position is reconstructed from `(D_frame, frame_local_uv, depth)`
   - normal is decoded from the atlas's view-space-encoded bytes through
     the bake camera's basis
   - albedo / ORM are taken directly from the atlas

3. Bin samples into a sparse axis-aligned voxel grid. Each cell
   averages position, sums-and-normalises normal, averages albedo and
   ORM. Empty cells are dropped — the surviving set IS the output point
   cloud.

4. The grid resolution is chosen iteratively to stay under a
   user-specified `point_budget`. Geometric shrink (halving) converges in
   `O(log resolution)` even on pathological inputs.

### Why CPU-side voxelisation

The mesh is fixed, the bake is offline, and the whole voxelisation
runs in a few hundred ms on a million samples. Doing it on CPU keeps the
code path one-shot deterministic and lets the engine's CPU path tracer
be layered in later (for high-quality baked AO replacing the O channel,
or for indirect-lighting hints in albedo) without changing anything on
the runtime rendering side.

### Why multi-view atlas sampling and not direct triangle sampling

A triangle area-weighted sampler would be cleaner in principle, but
would also re-implement everything `BakeShaderStandard` does — texture
filtering, supersampling, normal-map TBN, vertex-colour multiply,
material colour multiply, anisotropic filtering. The multi-view path
gets all of that for free, AND naturally weighs surfaces by visibility:
a surface visible from many angles gets many votes per voxel, which is
exactly what the impostor's job depends on.

---

## Drawing

One `InstancedBufferGeometry`, one instance per point, with the point
buffer wrapped directly as an `InstancedInterleavedBuffer` (zero copy
between baker and GPU). Each shader takes the per-instance attributes
and:

- expands a unit quad in view-space XY, scaled by
  `VoxelImpostorDescription.point_radius`, to keep it camera-facing;
- uses the per-instance normal (not the quad's geometry normal) for
  shading.

Shader set:

| File | Purpose |
|------|---------|
| `VoxelImpostorShaderV0` | Unlit. Writes baked albedo. Fast sanity check. |
| `VoxelImpostorShaderLitV0` | PBR-lit. Integrates three.js's standard pipeline; casts & receives shadows. |
| `VoxelImpostorShaderDepthV0` | Shadow-cast depth pass paired with the lit shader. |
| `VoxelImpostorShaderNormalsV0` | Debug — per-splat object-space normal as RGB. |
| `VoxelImpostorShaderViewportDepthV0` | Debug — viewport depth as grayscale. |

A `uDiscMask` uniform toggles between square and circular splats; the
prototype harness uses square for slot 0 (matches the Witcher 4 "voxel"
look) and disc for slot 1 (clearer view of the point distribution).

---

## Limitations

Same trade-offs as any pre-baked impostor:

- **No animation.** Position / normal / colour are frozen at bake time.
- **Static materials.** Roughness, metalness, AO, albedo are baked
  averages, not procedural.
- **View-bias.** Surfaces never visible from the bake's sphere of
  directions (eg the inside of a sealed cavity) don't get sampled.
  Increase `frames` to capture more directions, but there's a hard limit:
  if the original mesh's surface can't be hit by an external ray, the
  impostor won't show it.
- **Splat overlap at close range.** Camera-facing quads with radius
  ≈ voxel size will overlap noticeably at the edge of the visible range
  before the impostor would normally be used. Use as a distant LOD, not
  as a close-up replacement.

---

## Parameter recommendations

| Asset type | `point_budget` | `frames` | `resolution` |
|------------|---------------:|---------:|-------------:|
| Small prop (barrel, crate)   |  2 048 | 12 |  512 |
| Building / house             | 16 384 | 16 | 1024 |
| Vegetation (tree, large)     | 32 768 | 24 | 2048 |
| Hero / detailed mesh         | 65 536 | 32 | 2048 |

The budget is the **maximum** output; the baker may produce fewer
points if the bounding box is small or the surface area limited. Bump
`frames` if you see gaps in deep concavities; bump `resolution` if
per-voxel colour looks noisy (more samples per voxel → cleaner average).

---

## References

- [2025] [Large Scale Animated Foliage in The Witcher 4 UE5 Tech Demo —
  Unreal Fest Stockholm 2025](https://www.youtube.com/watch?v=EdNkm0ezP0o)
- [2025] [Nanite Foliage — Unreal Engine 5.7 docs](https://dev.epicgames.com/documentation/unreal-engine/nanite-foliage)
- [2025] [Get a glimpse of Nanite Foliage with voxel representation in
  UE 5.7 — 80.lv](https://80.lv/articles/get-a-glimpse-of-nanite-foliage-with-voxel-representation-in-ue5-7)
- Surface-splatting fundamentals: Pfister et al, "Surfels: Surface
  Elements as Rendering Primitives" (SIGGRAPH 2000)
- Octahedral encoding (used in atlas sampling): Cigolle et al, "A
  Survey of Efficient Representations for Independent Unit Vectors"
  (JCGT 2014)
