Amazon internal review found its AI tool Q fell 'significantly' behind rivals on accuracy in its first year (www.businessinsider.com)

An internal March review of Amazon’s Q Business — the company’s enterprise AI assistant launched at AWS re:Invent 2023 — found the product “significantly” behind rivals on accuracy and conversational experience during its first year. Customers including Accenture, Intuit and Smartsheet reported problems with connectors and non-text data: Q struggled to read embedded tables and spreadsheets, retrieve relevant metadata, pull long document passages, and maintain conversation memory, often producing “incomplete” or incorrect answers. The document also flagged staffing churn (about six product‑manager changes) and under-resourcing of engineering and data teams as contributors to mixed results; Q did, however, hit about 90% accuracy on text‑rich inputs. Amazon has responded with a formal accuracy program and a string of technical fixes — a hallucination‑mitigation feature (April), response customization (July), and an “agentic” retrieval‑augmented generation system (August) designed to improve long‑form retrieval and grounding — and says the issues are resolved for many customers (Nasdaq, Jabil, Availity). The episode highlights broader enterprise-AI challenges: robust connectors and RAG pipelines, coherent multi‑turn dialogue and evaluation metrics are decisive differentiators. Amazon’s next step — folding Q into a new agentic product called Quick — and the company’s push to shore up accuracy underscore that scalable, well‑resourced retrieval and grounding remain critical to competing in business AI.