Are Foundation Models Ready for Industrial Defect Recognition? A Reality Check (arxiv.org)

Researchers evaluated multiple recent foundation models (FMs) for the practical task of industrial defect recognition and found a stark reality check: while the same FMs perform well on public benchmark datasets in zero‑shot settings, they consistently fail on custom, real‑world industrial image data. The promise that a text prompt could replace costly per‑product labeling and that a single FM could generalize across many inspection tasks did not hold up in these experiments. The paper thus directly challenges the notion that off‑the‑shelf FMs are ready to replace supervised pipelines for automated quality inspection in manufacturing. The authors point to the usual culprits behind this performance gap—domain shift, tiny and low‑contrast defects, fine‑grained texture variations, class imbalance, and real imaging conditions (lighting, viewpoints, surface reflections) that benchmarks do not capture. Technically, this implies prompt engineering and generic zero‑shot inference are insufficient; robust deployment will likely require domain‑specific data, few‑/fine‑tuning, higher‑resolution modeling, tailored augmentations or synthetic defect generation, and revised benchmarks that mirror industrial variability. For the AI/ML community, the study is a call to prioritize real‑world datasets and domain adaptation techniques before claiming FM readiness for safety‑critical industrial inspection use cases.