We got Claude Code to stop gaslighting our tests (www.nuanced.dev)

The author describes a practical playbook for stopping Claude Code from “gaslighting” automated tests — i.e., producing plausible-sounding but incorrect or contradictory answers — by combining a persistent system-file (CLAUDE.md) with a Nuanced context layer that enforces strict behavioral guardrails. They show how to encode hard constraints (explicit refusal rules, required uncertainty tokens like “I don’t know,” and output schemas), supply canonical few-shot examples, and attach test-aware metadata so the model’s outputs are structured (JSON/typed fields) and easier to validate. They also recommend lowering sampling randomness, suppressing chain-of-thought in production, and using adversarial unit tests that probe failure modes and force deterministic, verifiable responses. For the AI/ML community this is significant because it moves model alignment from one-off prompt hacks into reproducible engineering patterns suitable for CI and developer workflows. Key technical implications: persistent system prompts (CLAUDE.md) provide a single source of truth for policy, the Nuanced context wrapper lets you compose constraints and test metadata without rewriting prompts, and structured outputs + assertion-driven tests reduce flakiness in code-generation or test-assertion tasks. The approach doesn’t eliminate model drift — you still need monitoring and updates — but it materially improves reliability and auditability when deploying LLMs in testing, developer tools, and automated pipelines.