Conventional Physical Models · Deep Learning · New Scenes
Seeing around corners — reconstructing hidden scenes from multiply-scattered light
Non-Line-of-Sight (NLOS) imaging reconstructs hidden scenes that cannot be directly observed. Light emitted by a laser hits a visible relay surface (e.g., a wall), bounces into the hidden space, reflects off the hidden object, and returns to the relay surface where a sensor captures it. Computational algorithms then invert this light transport to recover the hidden scene.
NLOS imaging spans a rich diversity of hardware platforms (streak cameras, SPADs, ToF cameras, interferometers, radar, acoustic transducers), physical models (time-of-flight ellipsoidal models, wave-based phasor field models), and reconstruction algorithms (back-projection, LCT, f-k migration, deep learning, neural implicit representations).
Applications: autonomous driving, search-and-rescue, medical imaging, robotics, surveillance — anywhere seeing beyond the direct line of sight matters.
[Laser / Detector] │ │ ① 1st bounce (relay wall) ↓ ┌─────────────────────┐ │ Visible relay wall │ │ ② 2nd bounce ──► Hidden │ │ ◄── ③ 3rd bounce Object│ └─────────────────────┘ │ ↓ Measurement τ(x, y, t) [Reconstruction Algorithm] │ ↓ Hidden scene ρ(x, y, z) FBP · LCT · f-k · Phasor Field Transformer · Mamba · GNN · Diffusion
Multi-dimensional statistics across 170+ papers (2008–2026) — all figures approximate
Stacked bars show how Active NLOS, Passive NLOS, Deep Learning, and New Scenes each grew year by year
Distribution across leading optics, vision, and AI venues (selected major venues)
Primary goal of each paper — reconstruction, detection, tracking, or recognition (papers may span multiple tasks)
Hardware diversity across surveyed papers; some papers use multiple sensor types
Among ~48 deep-learning NLOS papers — how network design has diversified since 2018
Key milestones that shaped the NLOS Imaging field
Theoretical foundation for transient NLOS imaging.
Streak camera + Filter Back Projection. Proved NLOS imaging is experimentally feasible.
Low-cost ToF camera + TV optimization for NLOS imaging.
Demonstrated SPAD as an affordable alternative to streak cameras, democratizing the field.
Reduced reconstruction from O(N⁵) to O(N³ log N). Landmark paper enabling real-time NLOS.
Reformulated NLOS imaging as a virtual LOS diffraction problem, enabling new algorithms.
Applied seismic f-k migration to NLOS. Robust wave-based reconstruction at O(N³ log N).
3D passive NLOS with an ordinary camera and partial occluder.
Extended phasor field to non-confocal NLOS settings with O(N³ log N) complexity.
Extended NLOS imaging range from meters to 1.43 km — three orders of magnitude.
First unsupervised neural implicit field for NLOS — no paired training data required.
Wavefront shaping achieves ~0.6 mm resolution at 0.55 m — 900× distance-to-resolution ratio.
Spatial-temporal self-attention captures multi-scale correlations in 3D transient volumes.
Real-time passive NLOS tracking; first large-scale dynamic passive NLOS dataset.
SDF-based neural implicit surface reconstruction for smooth 3D NLOS geometry.
Spectrum filtering + interleaved scanning achieves 4 fps NLOS video of room-sized dynamic scenes.
First Mamba model for NLOS video (temporal consistency); unsupervised reconstruction from irregular undersampled transients.
Full-color 3D passive NLOS reconstruction requiring no calibration images.
64×64 NLOS video at 10 fps from 16×16 sparse transient inputs via transfer learning.
First GNN for NLOS reconstruction; learnable physical priors for generalization across diverse scene conditions.
Full 3D NLOS scene reconstruction with a single mobile mmWave radar.
Human pose estimation (OLE), polarization single-pixel scanning-free NLOS (PRL), super-field-of-view imaging (Photonics Research).
Key papers from the survey, organized by category. Filter or search below.
Open resources for NLOS Imaging research
Active confocal SPAD dataset from O'Toole et al. (Nature 2018) and Lindell et al. (SIGGRAPH 2019). Includes transient measurements of multiple 3D hidden objects.
↗ github.com/computational-imaging/nlos-fkBenchmark dataset for time-resolved NLOS measurements, designed for systematic evaluation of reconstruction algorithms.
↗ doi.org/10.1364/OE.380140First large-scale dynamic passive NLOS tracking dataset. Thousands of video clips with trajectory labels for evaluating passive NLOS tracking methods. (Wang et al., CVPR 2023)
Link via paper authorsCode and data for the long-range 1.43 km NLOS imaging system by Wu et al. (Nature Communications 2021).
↗ github.com/quantum-inspired-lidar/…Code for acoustic NLOS imaging using microphone arrays (Lindell et al., SIGGRAPH 2019). Uses the same f-k migration framework as optical NLOS.
↗ github.com/computational-imaging/acoustic-nlosThe first dedicated NLOS transient renderer for generating realistic synthetic training data. (Royo et al., 2022)
Link via paper authorsDifferentiable transient renderer supporting auto-differentiation for end-to-end NLOS network training. (Plack et al., 2023)
Link via paper authorsReal passive NLOS measurement dataset captured with conventional cameras and ambient light, released alongside the optimal transport reconstruction paper (Geng et al., IEEE TIP 2022).
↗ github.com/ruixv/NLOS-OTOptimal transport–based passive NLOS reconstruction code. Includes the NLOS-Passive dataset, OT solver, and evaluation scripts. (Geng et al., IEEE TIP 2022)
↗ github.com/ruixv/NLOS-OTComprehensive curated list of 150+ NLOS papers with categorization, timeline, and links. Contributions welcome.
↗ github.com/ruixv/NLOS_Overview