Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformers (pixel-perfect-depth.github.io)

Researchers introduced Pixel-Perfect Depth, a monocular depth-estimation model that performs diffusion generation directly in pixel space to eliminate the “flying pixels” and edge artifacts caused by VAEs used in latent-space generative methods. Rather than fine-tuning Stable Diffusion’s latent pipeline, Pixel-Perfect Depth diffuses raw depth pixels and combines two key innovations to manage the huge computational burden: Semantics-Prompted Diffusion Transformers (DiT) and a Cascade DiT design. Semantics-Prompted DiT injects semantic features from large vision foundation models as prompts into the transformer diffusion process, maintaining global semantic consistency while sharpening fine-grained depth detail. The cascade design progressively increases token resolution across stages to balance efficiency and fidelity. This approach yields markedly better geometry: the model tops five public generative depth benchmarks and substantially outperforms prior methods on edge-aware point-cloud metrics, producing clean, artifact-free point clouds well-suited for downstream 3D tasks. For the AI/ML community, the work demonstrates that pixel-space diffusion—when paired with semantic guidance and progressive tokenization—can overcome VAE limitations, improving depth map fidelity and 3D reconstruction quality. The paper signals a shift toward pixel-level generative depth models that better preserve edges and fine structures critical for robotics, AR/VR, and scene understanding.