ddmitry/adversarial-review
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(skill): refresh adversarial-review под Codex 0.132

Browse Source 
- Зачем:
  - актуализация под новый Codex CLI (0.132 убрал флаг -a,
    ввёл -c approval_policy / -c approvals_reviewer);
  - переход на безопасные операторские дефолты
    (sandbox=workspace-write для эмпирической верификации,
    явные overrides sandbox:* и approvals:*);
  - наведение порядка в принятии решений по раунду ревью:
    review_quality + evaluation matrix + structural operator gate
    вместо «оператор сам разбирается».
- Что:
  - SKILL.md: override-граммар (sandbox:*, approvals:*, model:*,
    low/medium/high/xhigh), детект OPERATOR_LANGUAGE, runtime hint,
    conditional dirty-file warning в Step 2 (после захвата REPO_ROOT),
    dual-layer mutation snapshots (git status + sha256sum input-файлов),
    operation-aware dispatch table, evaluation matrix + structural
    operator gate (batch-pause), structured resume body
    (Applied / Re-scoped / Rejected / Specific asks), final operator
    summary на OPERATOR_LANGUAGE.
  - references/runner.md: переведён на Sonnet runner, добавлен
    Step R2.5 bwrap preflight, Step R4.5 review_quality + bounded
    triage, расширена 11-полевая JSON-схема результата.
  - docs/DESIGN.md: §4.14–§4.21 с обоснованиями новых решений,
    §7.8 refresh-era smoke checks, §8 version log с двумя раундами
    dogfood-а этого refresh-а (включая R2-корректировку
    обоснования residual gap для уже-грязных tracked-файлов).
  - README.md: таблица дефолтов, Safety considerations (честно
    задокументирован residual gap), Linux sandbox prerequisites
    (bwrap + AppArmor user namespaces), Operator language,
    Final operator summary, troubleshooting.
  - examples/review-output.md: модель в сэмпле обновлена на gpt-5.5.
  - docs/superpowers/specs/2026-05-20-...: сохранена спека дизайна
    с уточнениями после dogfood.
- Проверка:
  - dogfood: 2 раунда /adversarial-review code на этом refresh-е,
    R2 верифицировал R2#1 (ordering) и R2#2 (Option B, honest
    residual gap);
  - smoke: codex --version ≥ 0.132.0, bwrap preflight зелёный
    на reference WSL2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
ddadmin
2026-05-21 09:56:08 +03:00
co-authored by Claude Opus 4.7
parent 983a7a04ff
commit 1237e168c3
6 changed files with 2085 additions and 106 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/DESIGN.md
+442 -7
View File
@@ -829,6 +829,311 @@ Each decision below follows the same template:
command (codex), which matches what the skill already checks; no
`pipefail` needed.
### §4.14. Default model `gpt-5.5` (refresh 2026-05-20)
- **Decision.** The default reviewer model is `gpt-5.5`. Earlier defaults
were `gpt-5.4` and `gpt-5.3-codex`.
- **Why.** Each Codex generation reduces false-positive rate on
adversarial review and improves citation accuracy. The default tracks
the latest stable Codex model that ChatGPT+API auth can both reach.
- **Rejected alternatives.**
- *Pin to a fixed model forever.* Causes the skill to silently drift
out of date.
- *Default to the latest available regardless of stability.* Mid-loop
behavior changes invalidate the round-by-round severity trajectory
that the loop relies on.
- **Trade-offs accepted.** The version-log table in §8 needs a new row
per default bump so users can verify the smoke protocol on the chosen
model.
### §4.15. Default sandbox `workspace-write`, not `read-only` (refresh 2026-05-20)
- **Decision.** Initial / fresh-exec dispatches default to
`-s workspace-write`. `read-only` is a deliberate operator opt-in via
`sandbox:read-only`, never an automatic per-mode default.
- **Why.** The reviewer's value proposition is empirical verification —
"the reviewer ran the tests and X failed", "the reviewer built the
project and Y broke", "the reviewer queried the live API and the
assumed signature does not exist." Each of those requires write-class
operations: tests write cache and stdout files, builds emit
artifacts, CLIs touch local state. Read-only blocks all of them. The
remaining read-only-compatible verification (file inspection, `rg`,
`--help` introspection, MCP doc lookups that don't cache) is a small
fraction of the high-value path.
- **Rejected alternatives.**
- *Default `read-only` for plan reviews.* The original 2026-05
design tried this; reverted because plan reviewers also benefit
from running a test the plan relies on, building to confirm a
structural claim, or hitting an external API to validate an
assumption.
- *Sandbox-level enforcement of "no project file writes".* Codex
does not expose a "write to `/tmp` only, not `${REPO_ROOT}`"
sandbox tier — the closest is `read-only`, which blocks exec.
- *Auto-create a `git worktree` per review.* Adds setup latency to
every dispatch and complicates `RUNNER_SPEC_PATH` resolution and
`REPO_ROOT` capture (the worktree path is not the operator's
canonical repo root). Documented as an operator-side mitigation
in `README.md` instead.
- **Trade-offs accepted.** Reviewer-side writes to gitignored state
inside `REPO_ROOT` are real and not architecturally prevented; they
are mitigated through documentation (`README.md` "Safety
considerations"), the per-review runtime hint, and the
`sandbox:read-only` opt-out. Tracked-file mutation IS detected via
`git status --porcelain` snapshots — see §4.20.
### §4.16. Approval policy `on-request` + `auto_review` reviewer (refresh 2026-05-20)
- **Decision.** The default approval flow is
`-c approval_policy='"on-request"'` plus
`-c approvals_reviewer='"auto_review"'`. The top-level
`-a` / `--ask-for-approval` flag (removed in Codex CLI 0.132) is NOT
emitted by the runner.
- **Why.**
- `on-request` keeps boundary crossings explicit so the reviewer can
still ask before running a destructive operation, rather than
silently failing.
- `auto_review` for the approval reviewer avoids nested human
approval prompts inside the Codex subprocess. From the parent
Claude session, the operator cannot reliably see, answer, or even
detect those prompts — they appear as the run "hanging". The
`auto_review` policy auto-approves benign operations and rejects
obviously harmful ones, matching the boundary semantics the skill
needs.
- The config-based `-c approval_policy=...` form is the only form
supported on Codex CLI 0.132+; the runner emits it
unconditionally for forward compatibility with the unknown shape
of future flag churn.
- **Rejected alternatives.**
- *Pass `-a on-request` directly.* Removed in 0.132; would silently
fail on modern CLI versions.
- *Default to `approvals:user` (nested human prompt).* The nested
prompt is invisible from the parent Claude session and will hang
the run.
- *Default to `approvals:never` (boundary crossings fail
automatically).* Loses graceful degradation for routine operations
that just need a one-time approval.
- *Expose Codex's `untrusted` approval policy as an override.* The
skill needs predictable boundary semantics, not per-command trust
prompts.
- **Trade-offs accepted.** `auto_review` is not a security boundary —
it reviews approval requests, not actions already permitted by the
selected sandbox. The skill relies on the `<reviewer_permissions>`
prompt contract plus the workspace mutation snapshot for the real
guardrails.
### §4.17. Sonnet runner triage as a hint, not a decision (refresh 2026-05-20)
- **Decision.** The runner subagent (Sonnet) produces a compact
`triage` object — finding counts, max severity, per-severity
coverage counts, truncation flag, lead-judgment hint. This object is
returned in the result JSON and consumed by the lead as a hint. The
lead's evaluation matrix in Step 6 is built from the verbatim
review, NOT from triage. Triage is never passed back to Codex.
- **Why.**
- Codex returns a long-form review; main historically had to read
the entire review file to extract simple counts ("how many
critical findings did the reviewer raise?"). With triage, main
gets cheap structured signal without reading the file twice.
- The runner is the right layer to do triage because (a) it already
has the review file in context after Step R4 checks, and (b) the
runner's context is disposable — its 1M context can hold the
review and triage logic without polluting main.
- Keeping triage as a hint (not a decision) preserves the rule that
final `accept` / `reject` / `re-scope` outcomes are the lead's.
Triage that ranked findings or pre-emptively rejected them would
move the decision boundary into the cheaper model, which is
exactly the kind of authority drift past adversarial-review
rounds warned against.
- **Rejected alternatives.**
- *Let the runner pre-apply a heuristic accept/reject.* Drifts
final-decision authority into Sonnet. Rejected on principle.
- *Let the runner re-write the review file with a triage summary.*
Mutates the artifact main shows verbatim. Rejected to preserve
the "show review verbatim" contract.
- *Forward triage metadata to Codex on resume.* Codex would see
Sonnet's judgments and either echo them (false agreement) or
push back on them (wasted round on a triage artifact). Rejected.
- **Trade-offs accepted.** Triage may be wrong — wrong counts,
miscategorized severity, false `needs_lead_judgment` flag. The lead
re-derives counts during the evaluation matrix anyway; the hint
saves a small amount of cognitive load on round 1 but does not
load-bear on correctness.
### §4.18. Operator-language detection from recent messages (refresh 2026-05-20)
- **Decision.** At Step 1, main inspects the last few human-authored
conversation turns and captures `OPERATOR_LANGUAGE` (English by
default; non-English when detection is unambiguous). Runtime prose
uses `OPERATOR_LANGUAGE`. Repository files (this `DESIGN.md`,
`SKILL.md`, `README.md`, `references/runner.md`, specs under
`docs/superpowers/specs/`) stay in English regardless. The reviewer
is asked to produce findings in `OPERATOR_LANGUAGE` while keeping
machine-readable literals (severity tags, `VERDICT:`, section
headers) in English.
- **Why.** Operators routinely work in non-English; verbatim review
show-back, structural-gate prompts, and final summaries are more
useful in the operator's language. Repository files stay in English
because the contributor pool and AI-assistant training mass are
predominantly English, and mixed-language docs are harder to
maintain and grep.
- **Rejected alternatives.**
- *Ask the operator to declare language at invocation time.* Extra
friction; the conversation already encodes the signal.
- *Translate everything including section headers and severity
tags.* Breaks the parser regexes (`^VERDICT:`, `\[severity:\s*`,
`Summary` / `Findings` / `Verdict` section anchors).
- *Default to the operator's locale.* Locale != active language —
a Russian operator may be working through an English-language
codebase and want English review prose.
- **Trade-offs accepted.** Detection is heuristic and can be wrong on
short conversations or code-switching. Falling back to English on
ambiguity is safe; the operator can still read it, and the
machine-readable literals are stable.
### §4.19. Lead evaluation matrix + structural operator gate (refresh 2026-05-20)
- **Decision.** Step 6 builds an explicit evaluation matrix
(`finding | severity | verified? | type | action`) with three
first-class actions: `accept`, `reject with reasoning`, `re-scope`.
Step 6.3 applies a batch-pause rule: one operator prompt per round
listing structural / non-structural / rejected, only when the
structural count is ≥ 1 and an operator is reachable. Headless and
autonomous runs apply structural fixes anyway but record the fact
in the final operator summary.
- **Why.**
- Pre-refresh Step 6 used a flat "fix everything the reviewer
raised" loop. This is the failure mode the
`superpowers:receiving-code-review` skill exists to prevent — the
reviewer can be technically wrong, and applying its findings
blindly produces large structural edits based on misreadings.
The matrix forces an evaluation pass before any fix.
- `reject with reasoning` and `re-scope` are equal first-class
outcomes, not exceptions. The structured resume prompt's
"Rejected with reasoning" section gives the reviewer a chance to
contest, which keeps the loop honest.
- Structural-vs-non-structural gating exists because the highest-
leverage refactor a misread finding can trigger is "rewrite the
invocation grammar to match the reviewer's mental model." That's
the exact case where one operator pause prevents large
irrecoverable damage.
- **Rejected alternatives.**
- *Pause once per fix.* Confirmation fatigue; operator stops
reading after pause 3.
- *Apply all fixes automatically and let the operator review
post-hoc.* Reverting a misread structural fix is far more
expensive than one upfront prompt.
- *Let the runner classify structural vs non-structural and
pre-gate.* Returns to the §4.17 problem — moves final-decision
authority into the cheaper model.
- **Trade-offs accepted.** The matrix adds verbosity to the
conversation. The structural-gate prompt adds latency. Both are
small per round and saturate at low rounds because severity
declines (see §4.19 in spirit — Step 6.5 severity-decline soft
signal).
### §4.20. Workspace mutation snapshots (dual-layer, mandatory) (refresh 2026-05-20)
- **Decision.** Main captures `git status --porcelain` of `REPO_ROOT`
AND `sha256sum` of `/tmp/codex-{body,plan,resume-body}-*` BEFORE
and AFTER every Codex dispatch. Any tracked-file mutation, or any
`/tmp` review-input mutation, is a hard stop before applying fixes.
Gitignored mutation inside `REPO_ROOT` is NOT detected (deliberate
trade-off; see §4.15 and `README.md` "Safety considerations").
- **Why.** The default `workspace-write` sandbox allows the reviewer
to run commands that can mutate the workspace. The
`<reviewer_permissions>` prompt contract is the primary control,
but a prompt-only safeguard has a non-zero failure rate
(reviewer-side bug, prompt drift, ambiguous toolchain command). The
snapshot provides a secondary, after-the-fact detection layer that
catches mutation regardless of the cause.
- **Rejected alternatives.**
- *Snapshot only `git status`.* Misses reviewer mutation of the
skill's own `/tmp` prompt body — a real attack surface where a
reviewer could rewrite the prompt to suppress future findings.
- *Snapshot the entire `REPO_ROOT` tree including gitignored
files.* Too expensive per round (`node_modules`, `target/`,
`.next/`) and produces confirmation fatigue on every benign
side-effect.
- *Run the snapshot once at start-of-review only.* Misses
mid-loop mutation; mid-loop is when the round counter has
already advanced and rollback is most expensive.
- *Content-aware tracked-file snapshot (`git diff HEAD` patch +
sha256sum) on every dispatch.* Considered during round-1 dogfood
review of this refresh, and reconsidered in round-2 after a
reviewer pushed back on the original rejection rationale (see
Round-2 correction below). Would close the already-dirty
tracked-file residual risk listed in "Trade-offs accepted" below,
at the cost of an extra patch-file write + hash per dispatch, four
additional permissions entries (`Bash(git -C * diff HEAD*)`,
`Write/Read(/tmp/codex-tracked-pre-*)`,
`Write/Read(/tmp/codex-tracked-post-*)`,
`Bash(diff -q /tmp/codex-tracked-*)`), two additional cleanup
glob paths, and a third parallel snapshot layer in Step 4.
Rejected on a complexity-vs-frequency trade-off: the vector is a
legitimately-invoked verification command with unintended write
side effects on an already-dirty tracked file, with the
`<reviewer_permissions>` prompt contract as the only skill-level
protection layer and `sandbox:read-only` as the operator-side
opt-out. The skill ships the residual gap honestly documented
rather than hidden behind a third snapshot layer; if the risk
materializes in practice the design notes the implementation path
above.
- *Round-1 rejection rationale that cited the
`approval_policy=on-request` + `auto_review` gate as a protection
layer for this vector.* Withdrawn in round-2. Workspace writes are
permitted by the `workspace-write` sandbox and do not generate an
approval request, so `auto_review` never evaluates them (consistent
with §4.16's own statement that auto-review "reviews approval
requests, not actions already permitted by the selected sandbox").
The realistic vector reduces to a reviewer-invoked verification
command (formatter, snapshot updater, lockfile regen, codegen)
with unintended side effects on an already-dirty file. See
`README.md` "Residual risk: already-dirty tracked files" for the
operator-facing wording.
- **Trade-offs accepted.** Per-dispatch overhead is two `git`
commands and one `sha256sum` over three small files — negligible.
Gitignored mutation inside `REPO_ROOT` remains the documented
residual risk. **Already-dirty tracked-file content drift** is the
second documented residual risk: the porcelain snapshot tracks status
codes, not content hashes, so a reviewer that mutates a file already
marked ` M` will not trip a hard stop. The single skill-level
protection is the `<reviewer_permissions>` prompt contract; no
approval-policy gate exists for workspace-permitted writes.
Mitigated by (a) a runtime warning in `SKILL.md` Step 2 that fires
when pre-review `git status --porcelain` shows already-modified
tracked entries, (b) `README.md` "Safety considerations" explicitly
documenting the vector and the lack of an approval-policy gate, and
(c) the operator-side opt-out of `sandbox:read-only` (loses
reviewer-side verification, gains write-protection).
### §4.21. Final operator summary at every terminal state (refresh 2026-05-20)
- **Decision.** Step 8 emits an operator-facing summary in
`OPERATOR_LANGUAGE` at every terminal state — approved, max rounds
reached, not verified, aborted. The summary follows the canonical
per-state header block AND the final verbatim reviewer response;
it does not replace either. Built from per-round decision accounts
already shown in conversation; never from `/tmp` codex logs.
- **Why.** Pre-refresh Step 8 ended with the verbatim review and a
one-line status. For multi-round reviews this is insufficient —
the operator has scrolled past round 1 and 2 by the time they see
the final state, and "approved after 4 rounds" hides which
rejections and re-scopes survived to the end. The summary
consolidates the decision audit trail at the place where the
operator decides whether to merge.
- **Rejected alternatives.**
- *Show full diffs in the summary.* Too long; operator can `git
diff` for that.
- *Re-read Codex stdout/stderr to assemble the summary.* Violates
the architectural rule that main does not consume codex
transport logs (§12.x of subagent architecture).
- *Skip the summary on `Approved`.* The `Approved` summary is
actually the most useful for the merge-decision audience.
- **Trade-offs accepted.** Summary quality depends on context
compaction. The skill states this limitation explicitly when
history is incomplete rather than fabricating round-by-round
details.
---
## §5. Rejected ideas
@@ -1127,8 +1432,11 @@ End the LAST line with exactly: VERDICT: APPROVED
EOF
cat /tmp/codex-prompt-${REVIEW_ID}.md | timeout 300 codex exec --json \
-m gpt-5.4 -c model_reasoning_effort=low \
-s read-only -C "${REPO_ROOT}" \
-m gpt-5.5 -c model_reasoning_effort=low \
-s workspace-write \
-c approval_policy='"on-request"' \
-c approvals_reviewer='"auto_review"' \
-C "${REPO_ROOT}" \
-o /tmp/codex-review-${REVIEW_ID}.md \
- \
> /tmp/codex-stdout-${REVIEW_ID}.jsonl \
@@ -1223,11 +1531,17 @@ mkdir -p /tmp/smoke-cwd-a /tmp/smoke-cwd-b
( cd /tmp/smoke-cwd-b && git init -q )
# run codex in each; capture session ids
ID_A=$(cd /tmp/smoke-cwd-a && echo "ALPHA" | \
codex exec -s read-only -m gpt-5.4 \
-c model_reasoning_effort=low - 2>&1 | grep 'session id:' | awk '{print $3}')
codex exec -s workspace-write -m gpt-5.5 \
-c model_reasoning_effort=low \
-c approval_policy='"on-request"' \
-c approvals_reviewer='"auto_review"' \
- 2>&1 | grep 'session id:' | awk '{print $3}')
ID_B=$(cd /tmp/smoke-cwd-b && echo "BRAVO" | \
codex exec -s read-only -m gpt-5.4 \
-c model_reasoning_effort=low - 2>&1 | grep 'session id:' | awk '{print $3}')
codex exec -s workspace-write -m gpt-5.5 \
-c model_reasoning_effort=low \
-c approval_policy='"on-request"' \
-c approvals_reviewer='"auto_review"' \
- 2>&1 | grep 'session id:' | awk '{print $3}')
echo "A=${ID_A}"
echo "B=${ID_B}"
# resume --last from cwd-a; expect to resume ID_A, not ID_B
@@ -1264,6 +1578,125 @@ If §7.1–§7.5 do not produce the expected outputs:
before modifying `SKILL.md`. Future contributors should know which
facts they can still trust.
### §7.8. Refresh-era checks (added 2026-05-20)
These supplement §7.1–§7.6 and verify the behavior introduced by the
2026-05-20 refresh. Run them on the installed Codex CLI version and
record results in `§8`.
**Flag emission.**
- [ ] `codex exec --help | rg 'approval_policy|approvals_reviewer'` — confirm the
installed Codex CLI accepts the `-c` form.
- [ ] On a defaulted invocation, the runner subagent's launch command
contains `-s workspace-write`, `-c approval_policy='"on-request"'`,
and `-c approvals_reviewer='"auto_review"'`. Verify by reading the
runner's R3 step (the runner does NOT emit the launch command to
main, so this is checked by reading the runner spec, not by
inspecting main's logs).
- [ ] `codex exec --help` does NOT list `-a` / `--ask-for-approval` on
0.132+. Expected; the runner does not emit `-a` regardless.
- [ ] `codex exec resume --help` does NOT list `-s`, `-m`, or
approval-related `-c` overrides. The runner must not pass them
on resume.
**Sandbox preflight.**
- [ ] On a host where `bwrap --dev-bind / / --unshare-net /bin/echo ok`
fails, dispatch initial review with default sandbox. Expected:
runner returns `success + degraded_environmental` with a
`user_warning` pointing at README "Linux sandbox prerequisites";
main treats as terminal infrastructure failure; no fake review
round is shown.
- [ ] Apply the AppArmor profile per `README.md`. Re-run. Expected:
preflight passes; review proceeds normally.
- [ ] Pass `sandbox:read-only`. Expected: preflight still runs
(read-only is also bwrap-backed); review proceeds; the runtime
hint is suppressed.
- [ ] Pass `sandbox:inherit`. Expected: preflight is skipped; review
runs with whatever sandbox the user's Codex config selects.
**Review-quality classification.**
- [ ] A clean review with concrete findings → `review_quality=valid`,
`triage.status=ok`, counts populated.
- [ ] Synthesize a review file whose only content is "bwrap: setting up
uid map: Permission denied" (no findings, no real verdict) and
have the runner classify it. Expected:
`review_quality=degraded_environmental`,
`triage.status=skipped`.
- [ ] Synthesize a review with `VERDICT: REVISE` plus severity tags but
no concrete finding bodies (<80 chars each). Expected:
`review_quality=degraded_content`, `triage.status=ok`.
- [ ] Stop the triage-step Bash tool mid-run (or remove `rg` from
PATH). Expected: `triage.status=failed`,
`review_quality=valid` (the review itself is fine).
**Operation-aware dispatch.**
- [ ] Inject `success + degraded_environmental` on `OPERATION=initial`.
Expected: main emits `user_warning`, does NOT show review
verbatim, aborts with the "ABORTED — environmental failure"
Step 8 template.
- [ ] Inject `success + degraded_environmental` on `OPERATION=resume`.
Expected: main emits `user_warning`, does NOT show review
verbatim, does NOT consume a round, routes to Step 7.4 fallback
with prior round's severity.
- [ ] Inject `success + degraded_environmental` on
`OPERATION=fresh-exec`. Expected: main treats as terminal
not-verified.
- [ ] Inject `success + degraded_content` on any operation. Expected:
main emits `user_warning`, shows Step 5 verbatim, prompts the
operator (interactive) or auto-advances after a short wait
(headless).
**Workspace mutation snapshots.**
- [ ] Simulate the reviewer modifying a tracked file (touch a file in
`REPO_ROOT` during a synthetic dispatch). Expected: post-snapshot
`diff` is non-empty; main hard-stops before fixes with the
tracked-file diagnostic.
- [ ] Simulate the reviewer modifying `/tmp/codex-body-<REVIEW_ID>.md`
mid-dispatch. Expected: `sha256sum` diff is non-empty; main hard-
stops with the `/tmp` input diagnostic.
- [ ] Clean dispatch with no mutation. Expected: both diffs empty;
Step 5 proceeds.
**Backward compat for legacy runner results.**
- [ ] Manually write a runner result JSON missing `review_quality` and
`triage`. Expected: main treats as `review_quality=unknown`,
`triage.status=skipped`; no abort, no operator prompt.
**Runtime hint.**
- [ ] Default invocation: the hint appears exactly once, before Step 2.
- [ ] Default invocation, round 2/3/4/5: hint does NOT repeat.
- [ ] Invocation with `sandbox:read-only`: hint is suppressed.
- [ ] Invocation with `sandbox:workspace-write` (explicit, same as
default): hint is suppressed.
**Operator language.**
- [ ] Recent operator messages in Russian → `OPERATOR_LANGUAGE = Russian`.
Reviewer output is in Russian; severity tags, `VERDICT:`, and
section headers stay in English. Final operator summary is in
Russian.
- [ ] Recent operator messages mixed or short → `OPERATOR_LANGUAGE`
falls back to English; no language block is appended to the
prompt.
**Final operator summary.**
- [ ] Approved terminal: summary appears in `OPERATOR_LANGUAGE` after
the verbatim approved review.
- [ ] Max rounds: summary appears with remaining findings.
- [ ] Not verified: summary appears with the
`WARNING: This is NOT an approval` framing.
- [ ] Aborted environmental: summary appears with the diagnostic.
- [ ] Compacted history: summary states "round N details unavailable
due to compaction" instead of fabricating.
---
## §8. Version and verification log
@@ -1274,6 +1707,8 @@ If §7.1–§7.5 do not produce the expected outputs:
| 2026-04-17 | 0.121.0 | containerized sandbox (yantar-k8s) | external agent + lead | §7.1 `- < file` form fails EXIT=1 with empty stderr. `cat \| pipe` form works for `-o` review, but `--json` stdout is empty. Filesystem secondary session-id capture (§4.1b) verified functional: UUID extracted from rollout filename successfully resumes. Not a version issue (reproduced on 0.120.0 and 0.121.0). Root cause undiagnosed — see §6.6. Skill adapted: `§4.1` now two-tier, `§4.13` switches canonical form to `cat \| pipe`. |
| 2026-04-17 | 0.121.0 | reference env (WSL2) | live dogfood + team review | Round 6: timestamp-only secondary (§4.1b as of round 5) flagged for silent wrong-session hazard against parallel codex. Verified empirically that rollout JSONL contains prompt text (3 matches of prompt content via grep). Replaced with positive content-binding: prompt marker `<!-- ADVERSARIAL-REVIEW-SESSION: ${REVIEW_ID} -->` + `find -newer <prompt> -exec grep -l <REVIEW_ID> {} +`. All flags POSIX — GNU-find dependency of earlier §9.5 goes away. See §6.7. |
| 2026-04-17 | 0.121.0 | reference env (WSL2) | live dogfood round 2 | Round 7: review-stable marker flagged as insufficient — SKILL.md's own launch-retry flow can leave multiple rollouts matching the same `${REVIEW_ID}`, and "pick any" reintroduces silent intra-review session drift. Fixed by adding per-launch `${ATTEMPT_ID}` (6-digit random regenerated for every exec/retry/resume/fresh-exec). Marker is now `${REVIEW_ID}-${ATTEMPT_ID}`. Multi-match changed from "pick any" to fail-closed. See §6.8. |
| 2026-05-20 | TBD (refresh-era ≥ 0.132.0 required for `-c approval_policy`) | reference env (WSL2) | refresh design + implementation | Default model bump `gpt-5.4 → gpt-5.5`; default sandbox `read-only → workspace-write`; approval policy expressed via `-c approval_policy='"on-request"'` + `-c approvals_reviewer='"auto_review"'` (`-a` flag dropped per Codex 0.132 removal); new overrides `sandbox:*` and `approvals:*`. Runner result schema extended with `review_quality` + `triage` object (back-compat: legacy results treated as `unknown` / skipped). New operation-aware dispatch table in §SKILL.md Step 4. Mandatory pre/post `git status --porcelain` + `sha256sum` snapshots on every dispatch; tracked-file or `/tmp` review-input mutation is a hard stop. Step 6 rewritten with evaluation matrix + structural operator gate (batch-pause rule). Step 7 resume body restructured into Applied / Re-scoped / Rejected with reasoning / Specific asks. Step 8 adds final operator summary in `OPERATOR_LANGUAGE` at every terminal state. New rationale entries §4.14–§4.21. README adds "Safety considerations" and "Linux sandbox prerequisites" sections. Smoke protocol §7 updated for the new flags. Verification: dogfood plan + code review against this refresh (see §7.8 once added). |
| 2026-05-21 | 0.132.0 | reference env (WSL2) | dogfood rounds 1 & 2 (`/adversarial-review code` against this branch) | Round 1: applied #2 (worktree recipe loses unstaged), #3 (stale Full example permissions), #4 (bwrap diagnostic recommends `sandbox:read-only`); re-scoped #1 (porcelain misses content drift on already-dirty tracked files) to docs + runtime warning. Round 2 caught two structural issues: (a) the round-1 warning was placed in `SKILL.md` Step 1 before `REPO_ROOT` capture, making it non-functional — moved to Step 2 after capture; (b) the round-1 re-scope rationale cited `approval_policy=on-request` + `auto_review` as a protection layer for the residual gap, which is incorrect because workspace writes are sandbox-permitted and never generate approval requests — rationale withdrawn in §4.20, README "Safety considerations" rewritten to document the gap honestly. Already-dirty tracked-file content drift remains an accepted residual risk; mitigations are the `<reviewer_permissions>` prompt contract, the runtime warning, and the `sandbox:read-only` operator opt-out. |
When you re-verify (either during routine maintenance or when
triggered by §7.7), add a row. Keep the log chronological.
@@ -1484,7 +1919,7 @@ The attempt-scoped `ADVERSARIAL-REVIEW-SESSION` marker (round-7 finding) and pos
**Hypothesis to test:** Plan Mode restrictions propagate from main to any subagent main dispatches; the subagent inherits limitations on Write/Edit/Bash. If this holds:
- Main's Write to `/tmp/codex-body-*.md` may trigger a permission prompt or exit Plan Mode.
- The runner's Writes to `/tmp/codex-prompt-*.md` (Step R2, including the mtime-bump repeat Write) may also trigger prompts.
- The runner's `codex exec` (read-only sandbox) should be unaffected since it writes nothing to the user's repo.
- The runner's `codex exec` (default `workspace-write` sandbox, governed by the `<reviewer_permissions>` prompt contract per §4.15) writes to `/tmp` for review artifacts but is bound by main's mutation snapshot (§4.20) against unintended `REPO_ROOT` writes.
**What Task 7 Step 4 must determine:**
1. Does dispatching the Agent tool from Plan Mode work (is it blocked, does it prompt, does it just work)?
docs/superpowers/specs/2026-05-20-adversarial-review-refresh-design.md
+718
View File
@@ -0,0 +1,718 @@
# Adversarial Review Refresh Design
## Overview
This design updates the `adversarial-review` skill while preserving its core
shape: Claude remains the lead, Codex provides an external cross-model
adversarial review, and the Sonnet runner isolates mechanical Codex CLI work
from the expensive lead context.
The refresh makes the skill more useful to the operator, more robust on modern
Codex CLI setups, and less prone to blindly applying reviewer feedback. It does
not turn the skill into a larger framework, add mandatory custom agent
definitions, or move final decisions from the lead into the runner.
## Goals
- Use `gpt-5.5` as the default Codex reviewer model while preserving explicit
model and reasoning overrides.
- Let Codex reviews use the operator's language while keeping parsed literals
stable.
- Improve reviewer capability by avoiding overly restrictive default sandboxing.
- Avoid nested human approval prompts that the operator cannot see from the
Claude session.
- Keep runner-side work bounded and disposable so the lead context stays clean.
- Add compact Sonnet triage to reduce repeated review rounds without letting
Sonnet make final decisions.
- Require the lead to evaluate findings before applying fixes.
- Support operator sign-off for structural fixes without blocking explicitly
autonomous runs.
- Provide a final operator-facing summary of what changed across the review.
- Document Linux sandbox prerequisites clearly enough for a human or installer
agent to act on them.
## Non-Goals
- Do not require a custom Claude agent type or extra installation artifact.
- Do not use `danger-full-access` as an automatic fallback.
- Do not make the runner apply fixes, start extra review rounds, or decide
final `accept` / `reject` / `re-scope` outcomes.
- Do not move Codex stdout, stderr, or rollout contents into the main Claude
context.
- Do not add a mandatory detailed implementation plan for small instructional
changes.
- Do not mix documentation languages in repository files.
## Default Reviewer Configuration
The default Codex invocation should be optimized for non-interactive review
inside a Claude-runner child process:
```text
CODEX_MODEL = gpt-5.5
CODEX_REASONING = high
CODEX_SANDBOX = workspace-write
CODEX_APPROVAL_POLICY = on-request
CODEX_APPROVALS_REVIEWER = auto_review
```
`workspace-write` is the default for all review modes (plan, code,
code-vs-plan). The reasoning is load-bearing for this skill, so it is
captured here rather than in a comment.
The reviewer needs to actually run things to verify findings: run
tests, build the project, query the web or upstream APIs to confirm
current behavior, and exercise project CLIs end-to-end. These are
write-class operations — test runners produce output and cache files,
build commands write artifacts, and most non-trivial CLI execution
touches local state. A read-only sandbox blocks all of that.
Read-only is not a *total* verification blocker. It still allows file
inspection, `rg` / `grep` searches, MCP-backed doc lookups, and
`--help`-style CLI introspection that does not write to the workspace.
But the highest-value findings come from the path read-only blocks:
"the reviewer ran the tests and X failed", "the reviewer built the
project and Y broke", "the reviewer queried the live API and the
assumed signature does not exist." Forfeiting those to gain protection
against `.gitignored`-file side effects is a poor trade.
This applies to plan reviews as much as to code. A plan reviewer
should be able to run a test the plan relies on, build the project to
confirm a structural claim, or hit an external API to validate an
assumption — not just `rg` for cited filenames. A previous design
pass made `read-only` the default for plan mode and was reverted for
this reason.
The safety argument for forcing read-only is also weaker than it
appears. The broader `obra:superpowers` skill family demonstrates that
careful prompt-level discipline is sufficient to govern complex agent
behavior in security-relevant contexts without sandbox-level
restrictions. A concrete instance is `superpowers:receiving-code-review`:
it constrains how a lead processes adversarial feedback — read every
finding before reacting, restate the technical claim in own words,
verify empirically (especially tool-mechanic claims) before accepting,
push back with technical reasoning when wrong, no performative
agreement — and that discipline is enforced **entirely through prompt
instructions**, not through any tool-level sandboxing of the lead.
The same pattern applies to the reviewer side of this skill: a strict
instruction contract (see §Reviewer Behavior), backed by pre/post
`git status --porcelain` and the `/tmp` sha256 snapshot for
after-the-fact detection.
Read-only's specific failure modes are covered without a default
sandbox change:
- Tracked-file mutation is caught by `git status --porcelain` in
§Workspace Mutation Detection.
- `/tmp` review-input mutation is caught by the `sha256sum` snapshot
in the same section.
- Mutation of `.gitignored` state inside `REPO_ROOT` is bounded
(re-seedable dev state; tests read `.env.local`, they do not write
it) and is mitigated through documentation plus the
`sandbox:read-only` override for operators who knowingly accept the
trade-off.
Operators with a concrete reason to opt into read-only (sensitive
local state, untrusted reviewer prompt source, or any other case where
sandbox-level write protection outweighs verification capability) can
do so per-run via `sandbox:read-only`. The override stays available
exactly because the trade-off is real — for those operators.
Defaulting the rest of the user base to that mode would gut the
skill's main value.
The intended initial `codex exec` shape:
```bash
codex exec --json \
-m gpt-5.5 \
-c model_reasoning_effort=high \
-s workspace-write \
-c approval_policy='"on-request"' \
-c approvals_reviewer='"auto_review"' \
-C "<REPO_ROOT>" \
-o /tmp/codex-review-<REVIEW_ID>.md \
-
```
Codex CLI 0.132+ removed the top-level `--ask-for-approval` / `-a` flag
from `codex exec`; approval policy is now expressed only through
`-c approval_policy=<value>`. The spec uses `-c` form for both
`approval_policy` and `approvals_reviewer` so the command shape stays
self-consistent and future-proof against further short-flag churn. The
implementation should verify the supported approval-control surface
against the actual installed Codex CLI version during smoke testing.
`-o` writes only the assistant's final message. The reviewer prompt must
deliver the full structured review (findings, severity tags, VERDICT)
inside that final message; intermediate tool calls and reasoning will
not appear in the review file.
`workspace-write` gives the reviewer enough capability to run local
checks without forcing every useful command through a read-only wall.
`approval_policy="on-request"` keeps boundary crossings explicit.
`approvals_reviewer="auto_review"` avoids human approval prompts inside
the nested Codex process, which the operator cannot reliably see or
answer from the parent Claude session.
Auto-review is not a security boundary. It reviews approval requests; it does
not inspect actions already permitted by the selected sandbox. The skill still
needs strong reviewer instructions and workspace mutation checks.
Existing overrides remain:
- `model:<name>`
- `low`, `medium`, `high`, `xhigh`
New minimal overrides:
- `sandbox:read-only | workspace-write | danger-full-access | inherit`
- `approvals:user | auto_review | never`
Rules:
- Default `-s` is `workspace-write` for every review mode. Read-only is a
deliberate operator opt-in via `sandbox:read-only`, never an automatic
per-mode default.
- `sandbox:inherit` omits `-s` and relies on the user's Codex config.
Because the effective sandbox is unknown until Codex actually
launches, the runner skips bwrap preflight under `inherit`. If the
inherited config selects a bwrap-backed mode on a host where bwrap
is misconfigured, the failure surfaces as a §Runner Result Schema
`degraded_environmental` result and is treated as terminal
infrastructure failure on the initial dispatch per the dispatch table.
- `approvals:auto_review` (the default) passes
`-c approval_policy='"on-request"'` plus
`-c approvals_reviewer='"auto_review"'`.
- `approvals:user` passes `-c approval_policy='"on-request"'` only,
omitting the `approvals_reviewer` override so Codex falls back to its
default `user` reviewer. Allowed only by explicit override because
nested human approvals can hang the run.
- `approvals:never` passes `-c approval_policy='"never"'`; boundary
crossings fail instead of asking.
- `sandbox:danger-full-access` is explicit-only and should surface a warning.
- Codex's `untrusted` approval policy is intentionally not exposed as an
override; the skill needs predictable boundary semantics, not per-command
trust prompts.
- No silent fallback may change sandbox or approval semantics.
Resume commands should respect Codex CLI support for `resume`: sandbox and
approval mode are properties of the initial session unless the current CLI
explicitly supports changing them on resume. Do not pass unsupported `-s`
flags or approval-related `-c` overrides to `codex exec resume`.
## Runner Responsibilities
The runner remains a Sonnet subagent responsible for Codex CLI mechanics:
- parse the input block;
- write the prompt with the attempt-scoped session marker;
- launch exactly one Codex operation (`initial`, `resume`, or `fresh-exec`);
- own the one internal retry budget;
- validate exit code, stderr, review file, and session id;
- archive failed-resume diagnostics;
- write the authoritative result JSON;
- return the `RUNNER_RESULT_AT: <path>` line.
The runner gains bounded analysis:
- run sandbox preflight when the selected mode requires a bwrap-backed sandbox;
- detect obvious degraded reviews, such as sandbox or environment failures
disguised as reviewer output;
- extract finding count, maximum severity, and review quality;
- produce compact triage metadata.
Triage rules:
- Cover all critical and high findings.
- Cover up to 10 medium findings.
- If more medium findings remain, summarize the remainder and set an explicit
truncation flag.
- Use only cheap checks: file existence, section existence, simple `rg`, and
small read-only snippets.
- Do not run long test suites, web searches, or broad documentation lookups
during triage.
- Do not produce final `accept`, `reject`, or `re-scope` decisions.
- If uncertain, mark that lead judgment is needed.
If triage fails but the Codex review is valid, the review should continue with
a warning. If the review itself is degraded by infrastructure failure, the
runner should not count it as a normal round.
Runner instructions must include negative examples:
- Do not edit project files.
- Do not apply fixes.
- Do not run multiple Codex review rounds in one dispatch.
- Do not delete `/tmp/codex-*` files.
- Do not decide which findings the lead must accept.
- If a command unexpectedly changes project files, stop and report it.
## Runner Result Schema
The runner writes a single JSON file at `RESULT_PATH`. The schema below is
the authoritative contract between runner and lead. New fields added by
this refresh are marked `(new)`; everything else preserves the pre-refresh
shape.
```json
{
"result": "success | timeout | launch_failure | infra_error | input_error",
"verdict": "APPROVED | REVISE | null",
"review_file": "<absolute path or null>",
"codex_session_id": "<uuid or null>",
"attempt_id": "<string>",
"errors": "<string or null>",
"archived_stdout": "<path or null>",
"archived_stderr": "<path or null>",
"user_warning": "<string or null>",
"review_quality": "valid | degraded_environmental | degraded_content | unknown", // (new)
"triage": { // (new)
"status": "ok | skipped | failed",
"finding_count": "<int>",
"max_severity": "critical | high | medium | none",
"covered_critical": "<int>",
"covered_high": "<int>",
"covered_medium": "<int>",
"truncated": "<bool>",
"needs_lead_judgment": "<bool>"
}
}
```
Field semantics:
- `review_quality=valid` — review file passes R4 checks and content matches
a normal review shape.
- `review_quality=degraded_environmental` — Codex returned an exit-0
pseudo-review caused by sandbox or environment failure (bwrap-EPERM,
trust prompt, rate-limit stub, etc.). The text may contain `VERDICT:`
and severity tags but describes an inability to perform the review.
- `review_quality=degraded_content` — review parses cleanly but the
runner's cheap heuristics suggest the body is not actionable (e.g.
only a sandbox self-report, no concrete findings backing severity
tags). Conservative catch — when in doubt, mark `valid` and let the
lead decide.
- `review_quality=unknown` — triage could not classify (e.g. triage step
itself crashed). The lead should treat this as `valid` plus a warning.
- `triage.status=ok` — triage ran, fields populated.
- `triage.status=skipped` — triage skipped because Codex itself failed
(no review to triage).
- `triage.status=failed` — triage crashed; counts and severity may be
missing. Review remains usable if `review_quality=valid`.
Lead-side dispatch is **operation-aware** because `degraded_environmental`
on the first dispatch has no prior valid round to fall back on. The full
table:
| `result` | `OPERATION` | `review_quality` | Lead action |
|------------------|---------------|--------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `success` | any | `valid` | Proceed to Step 5: show review verbatim, run evaluation matrix, advance round. |
| `success` | any | `degraded_content` | Emit `user_warning`, show review verbatim, ask operator whether to advance the round. |
| `success` | any | `unknown` | Emit `user_warning`, treat as `valid` for advancement, note in final operator summary. |
| `success` | `initial` | `degraded_environmental` | Emit `user_warning`, do NOT show review verbatim, treat as terminal infrastructure failure. No prior valid review exists to recover from. |
| `success` | `resume` | `degraded_environmental` | Emit `user_warning`, do NOT show review verbatim, do NOT count as a round, route to fallback chain using prior round's severity. |
| `success` | `fresh-exec` | `degraded_environmental` | Emit `user_warning`, do NOT show review verbatim, terminal not-verified. Fresh-exec already was the fallback; a second environmental failure means the env is reliably broken. |
| `timeout` | any | n/a | Terminal at main per existing rules. |
| `launch_failure` | any | n/a | Terminal at main per existing rules (`resume` `launch_failure` routes to fresh-exec fallback per existing §7.4). |
| `infra_error` | any | n/a | Show `errors`, abort. |
| `input_error` | any | n/a | Show `errors`, abort (orchestration bug). |
Backward compatibility: a runner that omits `review_quality` or `triage`
fields entirely is treated as `review_quality=unknown` with
`triage.status=skipped`. Existing field semantics (`result`, `verdict`,
`review_file`, `codex_session_id`, `user_warning`, etc.) are unchanged.
## Reviewer Behavior
The Codex reviewer is an auditor, not a contributor. The prompt should allow
useful verification while prohibiting project mutation:
```text
You may run commands to verify findings when useful.
Do not create, edit, delete, commit, or apply fixes to project files.
Prefer commands that do not mutate the working tree.
Do not run commands likely to rewrite generated files, snapshots, migrations,
lockfiles, or configs.
If verification would require mutation, report that limitation instead.
If a command unexpectedly changes files, stop and report it.
```
This is a deliberate compromise. A reviewer locked into a strict read-only
sandbox often cannot run the checks needed to validate tool behavior, tests, or
documentation-dependent claims. The primary safeguard is the instruction
contract plus post-dispatch mutation detection, not a hard read-only shell.
## Lead Responsibilities
The lead owns all product decisions. After each valid Codex review:
1. Show the Codex review to the operator verbatim before applying fixes.
2. Use runner triage only as a hint.
3. Build a compact evaluation matrix:
```text
finding | severity | evidence | action | reason
```
4. Choose one of three first-class actions for each finding:
`accept`, `re-scope`, or `reject with reasoning`.
5. Apply only accepted or re-scoped fixes.
6. Send the reviewer a structured re-review prompt with `Applied`,
`Re-scoped`, and `Rejected with reasoning` sections.
Reviewer findings are suggestions to evaluate, not orders to follow. The lead
should verify findings proportionally to risk. Critical and high findings need
more scrutiny; medium findings can often be accepted, narrowed, or rejected
based on local checks and reasoning.
For plan reviews, the lead and reviewer must judge the plan at its declared
level of abstraction. Missing implementation details are findings only when
their absence blocks feasibility, safety, rollback, verification, or a public
contract.
## Operator Language
The skill should detect the operator language from recent user messages and
ask Codex to respond in that language. If detection is unclear, use English.
Prompt block:
```text
<language>
Respond in the operator's language: <detected language>.
Keep these machine-readable literals unchanged in English:
- [severity: critical|high|medium]
- VERDICT: APPROVED
- VERDICT: REVISE
</language>
```
Runtime prose shown to the operator should use the operator's language when
practical. Repository documentation and skill files remain English.
## Prompt Changes
Plan review prompts should add abstraction-level calibration:
```text
Judge the plan at its declared level of abstraction.
Do not demand implementation details unless their absence blocks feasibility,
safety, rollback, verification, or a public contract.
If a detail can reasonably be decided during implementation, do not count it
as a finding.
```
Re-review prompts should replace the current "I've revised based on your
feedback" shape with:
```text
I've evaluated the findings.
## Applied
- ...
## Re-scoped
- ...
## Rejected with reasoning
- ...
## Specific asks for re-review
1. Are my rejections technically valid?
2. Did the applied/re-scoped fixes resolve the original findings?
3. Did the fixes introduce new issues?
```
Sonnet triage metadata should not be passed to Codex automatically. Codex sees
verbatim findings and the lead's decisions, not the runner's notes.
## Structural Operator Gate
Before applying structural fixes, the lead should pause once and ask the
operator to approve the batch unless the operator explicitly requested
autonomous work.
Structural changes include:
- invocation grammar or argument semantics;
- output format or parsed literals;
- workflow steps, fallback semantics, or terminal states;
- sandbox, approval, or security guarantees;
- public configuration semantics;
- schema, migration, or data format changes;
- broad architectural rewrites;
- any fix whose scope the lead is uncertain about.
Non-structural fixes include wording, factual clarifications, examples, and
local changes that do not alter external behavior.
If the operator explicitly requested autonomous mode, the lead may apply
structural fixes without pausing, but the final summary must state that
structural changes were applied without operator sign-off due to autonomous
mode.
## Workspace Mutation Detection
Mutation detection runs at two layers: the repo tree (git-tracked + new
untracked files) and the skill's own `/tmp` review inputs. A third class
of mutation — gitignored files already inside `REPO_ROOT` — is documented
as a known, operator-mitigated risk rather than detected automatically.
**Repo tree.** The main thread should snapshot workspace state with
`git status --porcelain` before and after each runner dispatch, before
Step 5 applies any fixes.
If tracked files changed during the runner dispatch, the skill must hard
stop before applying fixes and show an operator diagnostic. This catches
reviewer or runner mutation without requiring a custom agent type.
If only untracked generated artifacts appeared, the skill should warn and
gate continuation. Some tools leave local artifacts, but the skill must
not silently fold them into fixes.
**Review inputs in `/tmp`.** The materialized plan file
(`/tmp/codex-plan-<REVIEW_ID>.md` when present), the prompt body
(`/tmp/codex-body-<REVIEW_ID>.md`), and the resume body
(`/tmp/codex-resume-body-<REVIEW_ID>.md`) sit outside `REPO_ROOT` and are
not covered by `git status`. The main thread must hash these files before
dispatch and re-hash after the runner returns:
```bash
sha256sum /tmp/codex-{body,plan,resume-body}-<REVIEW_ID>.md 2>/dev/null \
> /tmp/codex-inputs-pre-<REVIEW_ID>.sha
# ... runner dispatch ...
sha256sum /tmp/codex-{body,plan,resume-body}-<REVIEW_ID>.md 2>/dev/null \
> /tmp/codex-inputs-post-<REVIEW_ID>.sha
diff -q /tmp/codex-inputs-pre-<REVIEW_ID>.sha \
/tmp/codex-inputs-post-<REVIEW_ID>.sha
```
A non-empty diff is treated identically to a tracked-file mutation: hard
stop before applying fixes and surface the operator diagnostic.
**Out-of-scope: gitignored files inside `REPO_ROOT`.** `.gitignored` files
that already exist inside `REPO_ROOT` (local SQLite DBs, `.env.local`,
service-state directories, build caches) are NOT detected by either
layer. Full-tree snapshotting would be too expensive to run every round,
and a pre-dispatch confirmation about "ignored files present" would
trigger on essentially every repo (`node_modules`, `target/`, `.next/`)
and produce confirmation fatigue — security theater rather than
protection.
The realistic mutation vector for these files is the reviewer running a
project test or build command that side-effects on the ignored file —
for example `pytest` triggering an unintended migration on `dev.sqlite`
because the test settings point at it. The damage is bounded (the state
is re-seedable; tests generally read `.env.local`, they do not write to
it), but it is real on default workspace-write reviews.
The skill mitigates this with three operator-facing affordances rather
than architectural force:
1. The `sandbox:read-only` override is always available — operators who
know they have sensitive ignored state can opt out of test execution
at the cost of empirical verification (the reviewer loses the
ability to run tests, linters, and most MCP-backed verification).
2. The README must include a "Safety considerations" section with the
concrete pytest → `dev.sqlite` vector, the opt-out guidance (with
its explicit verification trade-off), and the worktree-isolation
pattern (`git worktree add /tmp/review-worktree <ref>`) for sensitive
repos.
3. At the start of every review, the skill emits a one-line runtime
hint: `workspace-write in effect; pass sandbox:read-only if
sensitive ignored state lives under REPO_ROOT`. The hint appears
exactly once per review (not per round) and is suppressed when the
operator passed an explicit `sandbox:*` override.
This is a deliberate trade-off. Defaulting reviews to `read-only` or to
an isolated worktree would gut the reviewer's empirical verification
capability (tests, linters, builds, MCP doc lookups, web searches all
require execute) — and empirical verification is precisely what makes
adversarial review more valuable than a same-model self-check. The
mitigation level is documentation and operator awareness rather than
architectural force.
## Linux Sandbox Prerequisites
The README should document sandbox prerequisites for Linux users and installer
agents. With the default `workspace-write` mode, Codex may rely on bubblewrap
and unprivileged user namespaces.
Diagnostic probe:
```bash
bwrap --dev-bind / / --unshare-net /bin/echo ok
```
If the probe fails on Ubuntu 24.04+ due to AppArmor user namespace
restrictions, recommend the official bwrap AppArmor profile:
```bash
sudo apt install -y apparmor-profiles apparmor-utils
sudo install -m 0644 /usr/share/apparmor/extra-profiles/bwrap-userns-restrict \
/etc/apparmor.d/bwrap-userns-restrict
sudo apparmor_parser -r /etc/apparmor.d/bwrap-userns-restrict
```
Then verify:
```bash
bwrap --dev-bind / / --unshare-net /bin/echo ok
```
The README should link to the official OpenAI Codex sandboxing documentation:
`https://developers.openai.com/codex/concepts/sandboxing`.
Installer agents should not change AppArmor policy silently. They should probe,
show the exact commands, request explicit permission, apply the profile only
after approval, and re-run the probe.
## Final Operator Summary
At every terminal state, the skill must provide an operator-facing summary in
the operator's language. This summary comes after the final verbatim reviewer
response and does not replace it.
Include:
- final status: approved, maximum rounds reached, not verified, or aborted;
- what changed across all review rounds;
- findings applied, re-scoped, and rejected;
- structural changes and whether operator sign-off was obtained;
- verification performed and verification not performed;
- remaining findings or risks;
- a short explanation of what the status means for non-approved terminal
states.
Constraints:
- Do not include full diffs.
- Do not repeat full reviewer findings unless unresolved findings matter.
- Keep it concise and useful to the operator.
- Build it from per-round decision summaries already shown in conversation.
- Do not read Codex stdout, stderr, or rollout files from the main thread.
- If context compaction makes history incomplete, state that limitation
explicitly instead of inventing details.
## Documentation Language Rule
Repository documentation and skill files stay in English:
- `SKILL.md`
- `references/runner.md`
- `README.md`
- `docs/DESIGN.md`
- design specs under `docs/superpowers/specs/`
Runtime output may use the operator's language. Do not mix Russian and English
inside repository documentation paragraphs except for machine-readable literals,
CLI flags, config keys, JSON keys, or quoted runtime examples.
## Files To Update
Expected implementation touches:
- `SKILL.md`
- default model, sandbox, approval parsing (note: approval policy is
expressed via `-c approval_policy=...`, not the removed `-a` flag);
- language detection and language block;
- runner input schema extensions;
- runner result schema consumption (`review_quality`, `triage.*`) and the
operation-aware lead-side dispatch table from §Runner Result Schema;
- evaluation matrix;
- structural operator gate;
- structured resume prompt;
- workspace mutation snapshots at both layers (`git status --porcelain`
AND `sha256sum` of `/tmp/codex-{body,plan,resume-body}-*`);
- one-line runtime hint at the start of every review about the
`workspace-write` default and the `sandbox:read-only` opt-out
(suppressed when an explicit `sandbox:*` override is passed);
- final operator summary.
- `references/runner.md`
- default Codex command flags (config-based approval policy via `-c`);
- bwrap preflight before every dispatch that selects a bwrap-backed
sandbox mode;
- degraded-review detection;
- bounded triage metadata;
- emit the full runner result schema (§Runner Result Schema), including
`review_quality` and the `triage` object;
- stronger mandate and negative examples.
- `docs/DESIGN.md`
- rationale for reviewer permissions;
- default model update;
- nested approval reasoning;
- Sonnet triage compromise;
- operator-language behavior;
- final summary rationale;
- version and verification log update after smoke testing.
- `README.md`
- updated defaults (including config-based approval policy via `-c`,
not `-a`);
- sandbox/approval behavior;
- Safety considerations section: the gitignored-file mutation vector
(pytest → `dev.sqlite` migration), residual risk acknowledgment, the
`sandbox:read-only` opt-out (with the explicit caveat that read-only
blocks empirical verification — operators trade verification for
write-protection), and the `git worktree`-based isolation pattern for
sensitive repos;
- Linux bwrap/AppArmor setup;
- operator language behavior;
- final summary behavior.
Do not renumber existing `docs/DESIGN.md` sections.
## Verification
Use the existing smoke protocol in `docs/DESIGN.md §7`, updated for the new
model and flags where needed.
Additional manual smoke checks:
- `codex exec` launches with `-s workspace-write`,
`-c approval_policy='"on-request"'`, and
`-c approvals_reviewer='"auto_review"'` on the target Codex CLI version
(verify against the actual installed version — `codex exec --help`
must list `-s` and `-c`; the absence of `-a/--ask-for-approval` is
expected on 0.132+).
- `codex exec resume` does not receive unsupported sandbox or approval
flags.
- bwrap-backed sandbox failure for `workspace-write` produces an early
diagnostic before Codex launches, instead of a fake review round.
- `sandbox:read-only` override still works and the runner skips approval
prompts inside the nested process (the operator can verify this by
running a review against a known-bwrap-failing host with the override).
- language block preserves parseable `[severity:]` and `VERDICT` literals.
- runner result JSON contains `review_quality` and `triage` fields when
triage runs; legacy result JSON (no `review_quality`, no `triage`) is
accepted by the lead as `review_quality=unknown` /
`triage.status=skipped`.
- triage failure does not fail an otherwise valid review.
- a synthetic `success + degraded_environmental` on `OPERATION=initial` is
treated as terminal infrastructure failure (no prior round to recover
from); the verbatim review is NOT shown.
- a synthetic `success + degraded_environmental` on `OPERATION=resume`
emits the warning, does not consume a round, and routes to the existing
fallback chain using the prior round's severity.
- a synthetic `success + degraded_environmental` on `OPERATION=fresh-exec`
is treated as terminal not-verified.
- a synthetic `success + degraded_content` (on any operation) emits the
warning, shows Step 5 verbatim, and prompts the operator for explicit
advancement.
- pre/post `git status --porcelain` catches workspace-tree mutation.
- pre/post `sha256sum` of `/tmp/codex-{body,plan,resume-body}-<REVIEW_ID>.md`
catches reviewer mutation of `/tmp` review inputs.
- the runtime hint about `workspace-write` and the `sandbox:read-only`
opt-out appears exactly once per review (regardless of mode) and is
suppressed when the operator passed an explicit `sandbox:*` override.
- final operator summary appears at approved, max-rounds, not-verified,
and aborted terminal states.
Dogfood the result with:
- plan review of this design;
- code review after implementation.
No automated CI is required for this refresh.