LATTICE 3D spatial data

A physical training representation for real-world intelligence

Record the physical world, not just video.

Lattice captures real interactions as synchronized dynamic 4D data. The result has full geometry, depth, motion, and the relationships between them.

GeometryKnow where objects and surfaces are in metric 3D, not just how they appear from one camera.
DynamicsCapture how people, objects, and environments move and change through time, accurately timed across every view.
InteractionsPreserve the relationships between actors, objects, trajectories, and outcomes in the same scene.
Building capture volume

Illustrative simulation · not real capture data

A working factory floor Drag to scrub · arrows step

Video is a projection of the world.

  • A video tells a model what a camera saw. It does not directly specify where objects are in 3D, what is hidden, or how the full scene changes.
  • For physical intelligence, those missing variables matter: geometry, motion, object state, occlusion, and interaction all affect what can happen next.
  • Lattice records those signals together in one synchronized spatial representation, rather than asking the model to recover all of them from pixels alone.

Capture the interaction once.

  • Record the same real interaction from synchronized calibrated viewpoints.
  • Recover RGB, depth, geometry, motion, trajectories, and scene state from one underlying event.
  • Use the resulting episode across training, evaluation, reconstruction, and simulation workflows.

One event, multiple representations.

SYNC MULTI-VIEW RGBMETRIC DEPTHCALIBRATIONDYNAMIC 3D / 4DHUMAN MOTIONOBJECT TRAJECTORIESSCENE STATEANNOTATIONSREPLAYABLE EPISODES

Every output stays tied to the same physical event, metric in space and accurate in time.

Capture
Synchronize
Reconstruct
Track
Structure
Learn

How much of the physical world should a model have to infer from pixels?

Lattice is building datasets where geometry, motion, and interaction are measured together, so models can learn from a physical record of what happened, not only from its visual projection.

Contact us

Benchmarks

What the data measures, against physical truth and against the alternatives.

LEDGER 2026-10-02

Each result is a registered entry in our benchmark ledger. Every card names what was changed, what was held fixed, the sample, and the caveat that travels with the number.

Primary

01

Measured size matches a tape measure to within 1%.

Physical accuracy

≤1.0%

error against a tape measure on two physical objects: ball radius −0.77%, bag area +1.0%

When the capture says an object is a certain size, is it?

Treatment
Lattice calibrated multi-view capture.
Control
Off-the-shelf metric monocular models (MoGe-2, UniDepthV2) on the same captures.
Truth
A ball (radius 0.1031 m) and a bag (area 0.196 m²), measured by hand with a tape.
Sample
Ball over 5 takes, found in 5 of 5; MoGe-2 placed it in 8 of 20 camera-takes with the best camera picked, UniDepthV2 in 0 of 20. Bag area on one take.

CaveatTwo reference objects in one room. The monocular models’ gap is mostly metric scale, not shape: scale-corrected as a diagnostic, they still read +24.8% and +29.0% on the bag.

Method and full caveats
  • Measured 0.1023 m against 0.1031 m for the ball and 0.198 m² against 0.196 m² for the bag.
  • The bag figure graded 13 segmentation settings against the tape; the best alternative was 16.9% worse, so no setting was tuned toward the answer.
  • Multi-view models are not ranked as a win: VGGT reaches a 0.22 mm radius error only when handed one of our scale values and the answer key; without that selection it reads 1.41–3.34 mm, and natively 49.2 mm with 0 of 20 placements. MapAnything from RGB alone reads 1.95 mm and places the ball 9 of 20 times.

Ledger led-2026-08-31-0098 · 0119 · competitor arm 0185, corrected by 0188

02

Depth at a camera the rig never saw.

Multi-view coverage

2.94×

lower depth error than the strongest multi-view model where both predicted, while covering 61.4% of the view against its 9.8%

Given three of the four cameras, how well can each method predict depth at the fourth, and how much of that view can it predict at all?

Treatment
Calibrated multi-view capture from three cameras, with the fourth held out as the evaluator.
Control
External depth models given the same three views (MapAnything with true intrinsics, UniDepthV2, MoGe-2, VGGT, Depth Anything 3), and a single camera.
Sample
5 takes × 8 frames × 4 held-out cameras: 160 cells per arm, 0 refused. An identity control reads exactly 0.

On the identical pixels every arm covers, the error is 14× lower than UniDepthV2 and MoGe-2.

CaveatRead both columns: the gain is mostly coverage. One camera to three raises coverage 29.3% to 61.4% (2.10×) but accuracy only 1.12×. Our refined output is indistinguishable from raw multi-view fusion here, so this credits calibrated multi-view capture, not the refinement layer.

Method and full caveats
  • A single calibrated camera scores 6.56 at 29.3% coverage; a constant (no information) scores 135.56.
  • Coverage is not the highest: UniDepthV2 and MoGe-2 cover about 64.7%, at roughly 10× the error.
  • Refined output vs raw fusion is a replicated null across five takes (p = 0.625 on this data), so no win is credited to refinement.
  • One room, one rig, four cameras of one model; no cross-room claim.

Ledger led-2026-09-11-0314, 0327

03

Single view vs multi-view hand tracking.

Multi-view coverage

1.72×

higher hand-joint error from a single camera than from multi-view fusion, scored on a camera neither saw

How much hand-tracking accuracy does a single RGBD camera, the common robot setup, lose against multi-view fusion?

Treatment
Multi-view fusion from the rig, with one of the four cameras held out to score every arm.
Control
One RGBD camera lifting its own pixels along its own rays.
Metric
Hand-joint reprojection error into the held-out camera, paired per frame and keypoint.
Sample
Single view was worse on 13 of 13 captures that passed both preregistered gates.

CaveatPixels, not millimetres. One operator, one rig. Most of the gain comes from the second view.

Method and full caveats
  • The gates refused 10 of 28 takes (6 on a 17 mm evidence floor, 4 on a preregistered wrong-fingertip control), including the corpus’s single highest ratio.
  • Seven captures regenerated by two independent pipelines agree to 0.030 median, 0.049 worst case.
  • This replaces an earlier 2.2× headline that came from the top two captures. Quote 1.72× with its range.
  • Whether a model trained on these hand motions is better is not yet tested.

Ledger led-2026-09-07-0163, superseding 0162

04

Measured scenes beat hand-built scenes on every capture.

Reconstruction

5/5

captures where the measured surface matched the observation better, in both directions

Does a simulation surface built from measured geometry match what the cameras saw better than one an engineer authors by hand?

Treatment
Surface built from the Lattice reconstruction.
Control
Hand-authored surface, deliberately strengthened: placed on the best-fit plane of the observed cloth at the same 3 mm lattice.
Sample
5 captures, 2 object types (bags, shirts). Lattice is 1.5–2.4× closer on every capture.

CaveatInitial geometry only. No simulator ran in this benchmark, so it does not evaluate simulated dynamics.

Method and full caveats
  • Metric: median distance between surface and observation, measured both ways through each capture’s own calibration. Preregistered; all seven gates passed.
  • Identity control reads 0.000 mm on all five captures; a 10 mm displacement reads 7.59–8.40 mm.
  • The reverse direction (observation to surface) penalises a sparser surface for being sparse; Lattice’s 1.529–1.660 mm clears its own 1.950–2.101 mm resampling floor, so the comparison holds, but that factor should not be over-read.
  • The factor is smaller than earlier single-capture work because the hand-built arm here is stronger. The consistency across five captures is the result, not any single factor.

Ledger led-2026-09-10-0292

Supporting

05

Multi-view data trains a better depth model.

Training utility

+20–34%

better depth on an unseen physical camera when a real depth model is trained on multi-camera data

Does training a depth model on multi-camera captures make it better than training on single-camera captures?

Treatment
Multi-camera training data, at equal budget.
Control
Single-camera training data.
Sample
Better on 3/3 seeds across 5 takes, replicated on 2 architectures.

A controlled follow-up held the input images fixed and changed only which surfaces were labelled: supervision covering surfaces one camera never saw cut error by 12.7% (3/3 seeds, control passes). The two results are different mechanisms and are not added.

CaveatThe +20–34% is camera invariance (training across physical cameras), not coverage. One room, one activity (cloth folding). The coverage run adapted only the model head (30.9 M of 335.3 M parameters) and its preregistered ordering check was half satisfied.

Method and full caveats
  • Model: Depth Anything V2 Metric. The camera-invariance runs were replicated on a second architecture.
  • Precision as a training advantage was tested six ways and refuted; the benefit is coverage and camera variety, not cleaner depth.

Ledger led-2026-09-10-0281, 0298 · coverage led-2026-09-11-0306

06

Gravity, recovered from a falling ball.

Motion and time

0.037%

error against true g, against 0.272% for a published smartphone slow-motion pipeline

Does the captured motion obey real physics in metric units and real time?

Treatment
Lattice capture at 60 fps, one fit per falling descent.
Control
WorldBench, iPhone 13 slow motion: the paper’s own reported figure (arXiv 2601.21282).
Sample
25 drops over 2 takes.

CaveatPoint estimate only. Our per-drop scatter is wider (SD 1.044 m/s²; if their ±0.38 is an SD, ours is 2.75× wider), and at n = 25 neither estimate is separable from true g.

Method and full caveats
  • Lattice reads 9.8030 m/s², WorldBench 9.7800, against 9.80665.
  • Throws and bounces are excluded by a preregistered rule (start speed under 0.5 m/s); takes with a broken timebase are refused outright.

Ledger led-2026-09-08-0191

Not shown: refined versus raw multi-view fusion (a replicated null across five takes), precision as a training advantage (refuted six ways), pooled single-camera accuracy figures (withdrawn because pooling hid large per-unit differences), and novel-view photometry (fails its own preregistered bar, while geometric alignment passes). The full records are available on request.

Every figure traces to a registered ledger entry Ask for the full ledger