An alignment auditing agent capable of quickly exploring alignment hypothesis (github.com)

🤖 AI Summary
Petri is an open-source alignment auditing agent designed to let researchers rapidly test alignment hypotheses by autonomously generating environments, conducting multi‑turn audits, and scoring transcripts to surface concerning behavior. Rather than building bespoke evaluations over days or weeks, Petri can spin up parallel, realistic red‑team interactions in minutes: an “auditor” model crafts human‑like probes and simulated tool calls, a “target” model responds, and an optional “judge” model scores outputs against safety criteria. That workflow makes it much faster to explore failure modes, check robustness to instruction-following exploits, and iterate on hypotheses about model behavior. Technically, Petri is available via pip or git, configurable with role assignments (e.g., auditor/target/judge), and accepts custom special_instructions, turn limits, and transcript save paths. Example runs show it operating with Anthropic Claude variants and producing detailed token logs, and a built‑in transcript viewer helps inspection. Full documentation and local dev tooling (mkdocs preview) are provided on GitHub. Important operational notes: auditing can produce harmful content and may trigger provider blocks, so follow provider policies and handle data carefully (benchmark data must never enter training corpora). The project is MIT‑licensed and requests citation when used in research; contributions, especially new special instructions, are welcome via pull request.
Loading comments...
loading comments...