Claude Fraud? Or Just an Anomaly? (anomify.ai)

Researchers testing Claude models found a striking anomaly: API calls requesting premium Claude 4 variants consistently produced responses that identified the model as Claude 3.5 Sonnet with a knowledge cutoff of April 2024 — even though billing and API metadata indicated Claude 4 (which has a January 2025 cutoff). The team reproduced the effect across both Anthropic’s API and AWS Bedrock (using boto3's bedrock-runtime in us-west-2), querying multiple model IDs (Claude Opus/Sonnet 4 variants and 3.x) with temperature=0.0 and prompts explicitly asking the model to state its version and cutoff. Every Claude 4 API response declared itself as Claude 3.5 Sonnet; only an older claude-3-opus call behaved correctly. Knowledge-bound tests showed the API-served models failed to reliably report post-April 2024 events when constrained, yet sometimes produced accurate post-cutoff facts when unconstrained — a pattern consistent with either systematic model substitution or sophisticated system-prompting. Why it matters: developers, enterprises and code-editors paying for Claude 4 expect specific capabilities and up-to-date knowledge; invisible substitution undermines trust, may cause overbilling, and degrades downstream code quality. The findings highlight the need for auditability and verification (e.g., cryptographic model signatures, stronger provenance/metadata) and set a precedent for industry standards so customers can confirm which model actually served their requests.