Why DETRs are replacing YOLOs for real-time object detection (blog.datameister.ai)

Transformer-based DETRs are rapidly displacing YOLO as the go-to choice for real-time object detection. The authors report they replaced older CNN detectors with D‑Fine (a DETR variant) because it delivers higher COCO accuracy while remaining competitive in inference speed. Crucially, most DETR releases use the permissive Apache‑2.0 license versus Ultralytics’ YOLO AGPL‑3.0, making DETRs far easier to integrate into commercial and proprietary systems. Benchmarking shows D‑Fine and Roboflow’s RF‑DETR outperform YOLO11 at all sizes — RF‑DETR’s nano models shine for tiny, fast deployments (benefiting from strong DINO backbones), while D‑Fine scales best (large: ~57.4 mAP). The technical reasons for the shift are practical and architectural. DETRs treat detection as set prediction using a transformer encoder–decoder with learned object queries, replacing hand‑tuned components like anchor boxes and non‑maximum suppression; Hungarian matching yields permutation‑invariant training. Advances — deformable attention, top‑k query initialization, denoising (DN‑DETR), and flash‑attention optimizations on modern GPUs — solved early drawbacks (slow convergence, weak small‑object performance). Two camps now compete: RT‑DETRs (e.g., D‑Fine, DEIMv2) focus on decoder/encoder optimization, while LW‑DETRs (e.g., RF‑DETR) lean on ViT backbones and NAS for latency/accuracy tradeoffs. For practitioners this means stronger off‑the‑shelf accuracy, simpler pipelines, and more flexible licensing for production use.