4D Spatial Intelligence

A comprehensive survey spanning monocular depth estimation and camera pose recovery through dense 3D/4D tracking, dynamic scene reconstruction, human-centric motion capture, and physics-based simulation — encompassing the full landscape of reconstructing spatial intelligence from video.

Reconstructing the 4D world — three spatial dimensions plus time — from video has emerged as a central challenge in computer vision. The field is structured along a hierarchy of spatial complexity: from low-level depth and camera pose estimation to dense 3D point tracking, from static 3D scene reconstruction at small and large scales to full 4D dynamic scene modeling via deformable NeRFs and 4D Gaussian Splatting. Human-centric 4D capture — encompassing SMPL-based mesh recovery, egocentric motion tracking, and appearance-rich avatar modeling — enables rich understanding of people in motion, while physics-based simulation grounds these reconstructions in physically plausible dynamics for character control, human-object interaction, and scene understanding. This pillar covers 550+ papers organized across six sub-domains, drawing from the Awesome-4D-Spatial-Intelligence survey and the latest research from arXiv.

Depth & Camera Pose

Monocular and video depth estimation, visual odometry, camera pose recovery, and unified depth–pose frameworks (DUSt3R, MonST3R, VGGT).

105+ papers →

3D/4D Tracking

Dense 3D point tracking, scene flow estimation, unified depth–pose–tracking models, and long-range 4D correspondence methods.

85+ papers →

3D Scene Reconstruction

Small-scale object/scene reconstruction, large-scale scene modeling, feed-forward 3D methods, neural surface reconstruction, and multi-view stereo.

133+ papers →

4D Dynamic Scenes

General 4D scene reconstruction, deformable NeRFs, 4D Gaussian Splatting, dynamic novel view synthesis, and temporal decomposition methods.

120+ papers →

Human-Centric 4D

SMPL-based mesh recovery, egocentric motion capture, appearance-rich human avatars, and human interaction modeling (HOI, HSI, HHI).

137+ papers →

Physics-Based 4D

Physics-based character control, adversarial motion priors, physically plausible reconstruction, human simulation, and dynamic scene physics.

60+ papers →