Georgia Gkioxari Co-Leads Major 3D Perception Model Built on AI (www.caltech.edu)

Meta’s Superintelligence Labs has open‑sourced SAM 3D, a pair of models (SAM 3D Objects and SAM 3D Body) that extend the 2023 Segment Anything Model to lift single real‑world images into 3D: it can reconstruct object shapes (even occluded or small items) and estimate distance from the camera. Caltech assistant professor Georgia Gkioxari co‑led the project (with 25 paper authors including her student Ziqi Ma). Meta also published a demo and integrated the tool into Facebook Marketplace and a robotic manipulation demo, letting users try 3D reconstruction without needing GPUs. The work’s technical innovation isn’t just model scale but a scalable data pipeline: a model‑in‑the‑loop annotation engine where models propose 3D candidates and nonexpert humans select the best, creating iterative training feedback that cheaply grows 3D supervision. That approach addresses the scarcity of labeled 3D data and can generalize to other expert labeling tasks (e.g., biomedical annotation). Open‑sourcing SAM 3D accelerates research and applications across robotics, AR/VR, gaming, retail, and security by making reliable single‑image 3D perception broadly available.