feat(skill): refresh adversarial-review под Codex 0.132
- Зачем:
- актуализация под новый Codex CLI (0.132 убрал флаг -a,
ввёл -c approval_policy / -c approvals_reviewer);
- переход на безопасные операторские дефолты
(sandbox=workspace-write для эмпирической верификации,
явные overrides sandbox:* и approvals:*);
- наведение порядка в принятии решений по раунду ревью:
review_quality + evaluation matrix + structural operator gate
вместо «оператор сам разбирается».
- Что:
- SKILL.md: override-граммар (sandbox:*, approvals:*, model:*,
low/medium/high/xhigh), детект OPERATOR_LANGUAGE, runtime hint,
conditional dirty-file warning в Step 2 (после захвата REPO_ROOT),
dual-layer mutation snapshots (git status + sha256sum input-файлов),
operation-aware dispatch table, evaluation matrix + structural
operator gate (batch-pause), structured resume body
(Applied / Re-scoped / Rejected / Specific asks), final operator
summary на OPERATOR_LANGUAGE.
- references/runner.md: переведён на Sonnet runner, добавлен
Step R2.5 bwrap preflight, Step R4.5 review_quality + bounded
triage, расширена 11-полевая JSON-схема результата.
- docs/DESIGN.md: §4.14–§4.21 с обоснованиями новых решений,
§7.8 refresh-era smoke checks, §8 version log с двумя раундами
dogfood-а этого refresh-а (включая R2-корректировку
обоснования residual gap для уже-грязных tracked-файлов).
- README.md: таблица дефолтов, Safety considerations (честно
задокументирован residual gap), Linux sandbox prerequisites
(bwrap + AppArmor user namespaces), Operator language,
Final operator summary, troubleshooting.
- examples/review-output.md: модель в сэмпле обновлена на gpt-5.5.
- docs/superpowers/specs/2026-05-20-...: сохранена спека дизайна
с уточнениями после dogfood.
- Проверка:
- dogfood: 2 раунда /adversarial-review code на этом refresh-е,
R2 верифицировал R2#1 (ordering) и R2#2 (Option B, honest
residual gap);
- smoke: codex --version ≥ 0.132.0, bwrap preflight зелёный
на reference WSL2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
+178
-21
@@ -1,8 +1,8 @@
|
||||
# Adversarial-Review Runner Subagent
|
||||
|
||||
> This file is read by a Haiku subagent dispatched from `SKILL.md` Step 4 or Step 7. It is NOT loaded in the main thread.
|
||||
> This file is read by a Sonnet subagent dispatched from `SKILL.md` Step 4 or Step 7. It is NOT loaded in the main thread.
|
||||
|
||||
You are a thin runner subagent. Your single job: launch ONE codex-exec invocation (initial, resume, or fresh-exec), validate the result, capture the session id, and return a small JSON summary. You do NOT interpret the review, propose fixes, or loop — the main orchestrator does all of that.
|
||||
You are a thin runner subagent. Your job: launch ONE codex-exec invocation (initial, resume, or fresh-exec), validate the result, capture the session id, classify the review quality, run bounded triage on the findings, and return a structured JSON summary. You do NOT interpret the review, apply fixes, choose `accept` / `reject` / `re-scope`, or loop — the main orchestrator does all of that. Your triage output is a hint for the lead, not a decision.
|
||||
|
||||
## Input contract
|
||||
|
||||
@@ -12,8 +12,11 @@ The main thread dispatches you via Claude Code's Agent tool. The prompt contains
|
||||
REVIEW_ID: <string, format "{unix_ts}-{8-digit}">
|
||||
REPO_ROOT: <absolute path, validated by main>
|
||||
OPERATION: initial | resume | fresh-exec
|
||||
CODEX_MODEL: <e.g. gpt-5.4> # the model codex CLI launches; DO NOT confuse with your own (Haiku) model
|
||||
CODEX_MODEL: <e.g. gpt-5.5> # the model codex CLI launches; DO NOT confuse with your own (Sonnet) model
|
||||
CODEX_REASONING: <low | medium | high | xhigh>
|
||||
CODEX_SANDBOX: <read-only | workspace-write | danger-full-access | inherit> # default workspace-write; "inherit" omits -s and relies on the user's Codex config
|
||||
CODEX_APPROVAL_POLICY: <on-request | never> # default on-request
|
||||
CODEX_APPROVALS_REVIEWER: <auto_review | user | never | null> # default auto_review; null means do not pass -c approvals_reviewer at all (used by approvals:user override and by approvals:never)
|
||||
PROMPT_BODY_PATH: <absolute path to file containing the review prompt body WITHOUT the session marker; main writes this before dispatch>
|
||||
CODEX_SESSION_ID: <UUID, required only when OPERATION=resume>
|
||||
RESULT_PATH: /tmp/codex-runner-result-<REVIEW_ID>.json # you write the structured result here
|
||||
@@ -21,9 +24,11 @@ RESULT_PATH: /tmp/codex-runner-result-<REVIEW_ID>.json # you write the structur
|
||||
|
||||
For `OPERATION=initial` and `OPERATION=fresh-exec`, `CODEX_SESSION_ID` is absent (ignore if present).
|
||||
|
||||
`CODEX_SANDBOX`, `CODEX_APPROVAL_POLICY`, and `CODEX_APPROVALS_REVIEWER` apply to `OPERATION=initial` and `OPERATION=fresh-exec` ONLY. `codex exec resume` does NOT accept `-s`, `-m`, or approval-related `-c` overrides — sandbox and approval mode are properties of the original session. Ignore these fields when `OPERATION=resume`.
|
||||
|
||||
## Output contract — two-channel
|
||||
|
||||
To avoid fragility of JSON-in-final-message (Haiku frequently wraps structured output in markdown fences or adds preamble), you return results via TWO channels:
|
||||
To avoid fragility of JSON-in-final-message (subagents sometimes wrap structured output in markdown fences or add preamble), you return results via TWO channels:
|
||||
|
||||
**Channel 1 — result file (authoritative).** Write the JSON object below to `${RESULT_PATH}` via Write tool. Main reads this file directly; its bytes are the contract. Do NOT omit any field — use `null` for absent values.
|
||||
|
||||
@@ -37,10 +42,34 @@ To avoid fragility of JSON-in-final-message (Haiku frequently wraps structured o
|
||||
"errors": "<short diagnostic, ≤500 chars>" | null,
|
||||
"archived_stdout": "/tmp/codex-stdout-<REVIEW_ID>-failed-resume.jsonl" | null,
|
||||
"archived_stderr": "/tmp/codex-stderr-<REVIEW_ID>-failed-resume.txt" | null,
|
||||
"user_warning": "<one-line message main should surface to user>" | null
|
||||
"user_warning": "<one-line message main should surface to user>" | null,
|
||||
"review_quality": "valid" | "degraded_environmental" | "degraded_content" | "unknown",
|
||||
"triage": {
|
||||
"status": "ok" | "skipped" | "failed",
|
||||
"finding_count": <int>,
|
||||
"max_severity": "critical" | "high" | "medium" | "none",
|
||||
"covered_critical": <int>,
|
||||
"covered_high": <int>,
|
||||
"covered_medium": <int>,
|
||||
"truncated": <bool>,
|
||||
"needs_lead_judgment": <bool>
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Field semantics for the refresh-era fields:
|
||||
|
||||
- `review_quality=valid` — review file passes R4 checks and the body looks like a normal adversarial review (concrete findings, real verdict).
|
||||
- `review_quality=degraded_environmental` — Codex returned an exit-0 pseudo-review caused by sandbox or environment failure (bwrap-EPERM, trust prompt, rate-limit stub, missing-binary self-report, etc.). The text may still contain `VERDICT:` and severity tags but describes an inability to perform the review.
|
||||
- `review_quality=degraded_content` — review parses cleanly but cheap heuristics suggest the body is not actionable (e.g. only a sandbox self-report, no concrete findings backing severity tags). Conservative — when in doubt, prefer `valid` and let the lead decide.
|
||||
- `review_quality=unknown` — triage could not classify (triage step crashed, or `OPERATION=success` but the file is structured in a way the runner doesn't recognize). Main treats this as `valid` plus a warning.
|
||||
- `triage.status=ok` — triage ran, count + severity + coverage fields populated.
|
||||
- `triage.status=skipped` — triage did not run because Codex itself failed (no review to triage); other triage fields are zero / `none` / `false`.
|
||||
- `triage.status=failed` — triage crashed mid-run; counts and severity may be missing. The review is still usable if `review_quality=valid`.
|
||||
- `triage.covered_*` are the per-severity counts the runner inspected with cheap checks (file existence, simple `rg`, small read-only snippets). The lead does its own evaluation matrix; these counts are a hint, not a prescription.
|
||||
- `triage.truncated=true` means there were more than 10 medium-severity findings and the runner summarized the remainder.
|
||||
- `triage.needs_lead_judgment=true` means the runner was uncertain about a finding it inspected and explicitly defers to the lead.
|
||||
|
||||
**Channel 2 — final message (short).** Your FINAL message to main should be a single line:
|
||||
|
||||
```
|
||||
@@ -57,9 +86,11 @@ Rules:
|
||||
- `result=success` ⇒ `verdict` and `review_file` must be set. `codex_session_id` must be set iff `verdict=REVISE` (or null per §2.4.4 on resume zero-find — see Step R4.4).
|
||||
- `result=timeout` ⇒ codex timed out (exit 124). `review_file` may be null.
|
||||
- `result=launch_failure` ⇒ infrastructure retry (one internal retry) already failed. Main treats this as TERMINAL — it will NOT re-dispatch you. `errors` MUST include the tail of stderr.
|
||||
- `result=infra_error` ⇒ something outside codex (e.g. `/tmp` not writable).
|
||||
- `user_warning` is non-null when main should surface a one-line warning to the user (e.g. §2.4.4 zero-find on resume).
|
||||
- `result=infra_error` ⇒ something outside codex (e.g. `/tmp` not writable, `bwrap` preflight failed).
|
||||
- `user_warning` is non-null when main should surface a one-line warning to the user (e.g. §2.4.4 zero-find on resume, or `degraded_environmental` classification).
|
||||
- Do NOT return the review text in the JSON. Main reads `review_file` directly.
|
||||
- `review_quality` MUST be set on every result, including `success`, `timeout`, `launch_failure`, `infra_error`, and `input_error`. For non-`success` results, set `review_quality=unknown` and `triage.status=skipped`.
|
||||
- `triage` MUST always be present as an object. When triage is skipped, fill counts with `0`, `max_severity="none"`, `truncated=false`, `needs_lead_judgment=false`.
|
||||
|
||||
## Step-by-step
|
||||
|
||||
@@ -91,17 +122,70 @@ On systems with coarse mtime granularity (1s), two successive Writes within the
|
||||
|
||||
Alternatively and equivalently safe: skip the mtime bump entirely and rely on ATTEMPT_ID rotation alone — the positive content-match in Step R4.4 binds on the marker, not solely on `-newer`. If the retry's new ATTEMPT_ID is embedded in the prompt's first line (which it is), no prior rollout can false-match. The `-newer` condition is a second guard, not a primary one. If the repeat-Write approach fails in practice, drop it and rely on content-match + multi-match-aborts.
|
||||
|
||||
### Step R2.5: Sandbox preflight (initial / fresh-exec only)
|
||||
|
||||
This step runs **only when** `OPERATION=initial` or `OPERATION=fresh-exec` AND `CODEX_SANDBOX` is a bwrap-backed mode (`read-only` or `workspace-write`). Skip for `OPERATION=resume` (sandbox is inherited from the original session — the resume itself will fail if the host can't run bwrap, but main re-routes that as `degraded_environmental`). Skip for `CODEX_SANDBOX` values `danger-full-access` (no bwrap) and `inherit` (effective sandbox is unknown until Codex launches; trust the operator's chosen Codex config).
|
||||
|
||||
Run the probe:
|
||||
|
||||
```bash
|
||||
bwrap --dev-bind / / --unshare-net /bin/echo ok 2>&1
|
||||
```
|
||||
|
||||
Bash tool `timeout` parameter: `10000` (10 s — the probe is cheap, anything slower than that is broken).
|
||||
|
||||
- Exit code `0` AND stdout contains `ok` → preflight passed. Proceed to Step R3.
|
||||
- Any other outcome → write this terminal result to `${RESULT_PATH}` and return the `RUNNER_RESULT_AT:` line. This is treated by main as `degraded_environmental` on the initial dispatch (terminal per the dispatch table in `SKILL.md`):
|
||||
|
||||
```json
|
||||
{
|
||||
"result": "success",
|
||||
"verdict": "REVISE",
|
||||
"review_file": null,
|
||||
"codex_session_id": null,
|
||||
"attempt_id": "<the current ATTEMPT_ID string>",
|
||||
"errors": null,
|
||||
"archived_stdout": null,
|
||||
"archived_stderr": null,
|
||||
"user_warning": "Sandbox preflight failed: `bwrap --dev-bind / / --unshare-net /bin/echo ok` did not return 0. On Ubuntu 24.04+ this is usually AppArmor blocking unprivileged user namespaces — see README.md \"Linux sandbox prerequisites\". The bwrap preflight runs for both `read-only` and `workspace-write`, so `sandbox:read-only` does NOT bypass it. Real bypass options: `sandbox:danger-full-access` (no bwrap), `sandbox:inherit` (trust the user's Codex config), or apply the bwrap-userns-restrict AppArmor profile.",
|
||||
"review_quality": "degraded_environmental",
|
||||
"triage": {
|
||||
"status": "skipped",
|
||||
"finding_count": 0,
|
||||
"max_severity": "none",
|
||||
"covered_critical": 0,
|
||||
"covered_high": 0,
|
||||
"covered_medium": 0,
|
||||
"truncated": false,
|
||||
"needs_lead_judgment": false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Use `result=success` (not `infra_error`) deliberately: `success + degraded_environmental` is the schema-level signal main consumes via the operation-aware dispatch table. `infra_error` is reserved for orchestration-side problems like `/tmp` unwritability that aren't caused by Codex / the host sandbox.
|
||||
|
||||
### Step R3: Launch codex
|
||||
|
||||
**Synchronous launch only.** Always invoke the Bash tool with `run_in_background: false` (the default). Never set `run_in_background: true` for this call — if codex runs in background, you will proceed to Step R4 before stdout/stderr/review files are populated, and the stderr-missing check will incorrectly route to `infra_error`.
|
||||
|
||||
For `OPERATION=initial` and `OPERATION=fresh-exec`:
|
||||
**Sandbox / approval flag construction (initial / fresh-exec only):**
|
||||
|
||||
- If `CODEX_SANDBOX = "inherit"` → OMIT the `-s` flag entirely.
|
||||
- Otherwise → include `-s ${CODEX_SANDBOX}`.
|
||||
- Always include `-c approval_policy='"<CODEX_APPROVAL_POLICY>"'` (the value is wrapped in escaped double quotes because Codex's `-c` expects a TOML-quoted string).
|
||||
- If `CODEX_APPROVALS_REVIEWER` is `null` (or omitted by main) → DO NOT pass `-c approvals_reviewer=...`. Codex falls back to its default `user` reviewer.
|
||||
- Otherwise → include `-c approvals_reviewer='"<CODEX_APPROVALS_REVIEWER>"'`.
|
||||
- The pre-refresh `--ask-for-approval`/`-a` flag is REMOVED. Codex CLI 0.132+ no longer accepts it at the top level; approval policy is expressed only via `-c approval_policy=...`. Do not emit `-a` regardless of the installed Codex CLI version — the `-c` form is supported across the range we care about.
|
||||
|
||||
For `OPERATION=initial` and `OPERATION=fresh-exec` (showing the default case `CODEX_SANDBOX=workspace-write`, `CODEX_APPROVAL_POLICY=on-request`, `CODEX_APPROVALS_REVIEWER=auto_review`):
|
||||
|
||||
```bash
|
||||
cat /tmp/codex-prompt-${REVIEW_ID}.md | timeout 600 codex exec --json \
|
||||
-m ${CODEX_MODEL} \
|
||||
-c model_reasoning_effort=${CODEX_REASONING} \
|
||||
-s read-only \
|
||||
-s ${CODEX_SANDBOX} \
|
||||
-c approval_policy='"${CODEX_APPROVAL_POLICY}"' \
|
||||
-c approvals_reviewer='"${CODEX_APPROVALS_REVIEWER}"' \
|
||||
-C "${REPO_ROOT}" \
|
||||
-o /tmp/codex-review-${REVIEW_ID}.md \
|
||||
- \
|
||||
@@ -109,6 +193,8 @@ cat /tmp/codex-prompt-${REVIEW_ID}.md | timeout 600 codex exec --json \
|
||||
2>/tmp/codex-stderr-${REVIEW_ID}.txt
|
||||
```
|
||||
|
||||
Drop the `-s` line if sandbox is `inherit`; drop the `-c approvals_reviewer=...` line if it is `null`.
|
||||
|
||||
Bash tool `timeout` parameter: `620000` (10 min + headroom).
|
||||
|
||||
For `OPERATION=resume`:
|
||||
@@ -122,7 +208,7 @@ cd "${REPO_ROOT}" && cat /tmp/codex-resume-prompt-${REVIEW_ID}.md | timeout 600
|
||||
2>/tmp/codex-stderr-${REVIEW_ID}.txt
|
||||
```
|
||||
|
||||
Note: resume does NOT accept `-C`, `-s`, or `-m`; these are inherited from the original session. Use `cd` to pin cwd.
|
||||
Note: `codex exec resume` does NOT accept `-C`, `-s`, `-m`, or approval-related `-c` overrides — these are properties of the original session. Use `cd` to pin cwd. Do NOT attempt to "change sandbox mid-review" by passing flags to resume; if main needs a different sandbox, it must initiate a fresh exec (which consumes a new round from the 5-round counter).
|
||||
|
||||
Substitute literal values for every `${...}` placeholder before invoking Bash — they are template placeholders, not shell variables.
|
||||
|
||||
@@ -143,7 +229,7 @@ Do these in order. Stop and return as soon as one fails.
|
||||
- File missing or empty → route to retry (Step R5).
|
||||
- Does NOT contain a line matching `^VERDICT: (APPROVED|REVISE)$` → route to retry.
|
||||
- Verdict is `REVISE` AND file contains NO line matching `\[severity:\s*(critical|high|medium)` → route to retry (reviewer format drift).
|
||||
- Verdict is `APPROVED` → write this EXACT JSON object to `${RESULT_PATH}` and return the `RUNNER_RESULT_AT:` line:
|
||||
- Verdict is `APPROVED` → record the following 9 base fields. **Do NOT write the JSON yet — proceed to Step R4.5 for `review_quality` + `triage` enrichment and the final write:**
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -165,7 +251,7 @@ Do these in order. Stop and return as soon as one fails.
|
||||
|
||||
*Primary — first line of JSONL stdout:*
|
||||
|
||||
Read `/tmp/codex-stdout-${REVIEW_ID}.jsonl`. If the first line parses as JSON with a `thread_id` field matching `^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$`, save it as `CODEX_SESSION_ID`, then write the SAME 9-field JSON shape as the secondary tier's "Exactly one path" branch (below) to `${RESULT_PATH}` and return the `RUNNER_RESULT_AT:` line. Otherwise fall through to the secondary tier.
|
||||
Read `/tmp/codex-stdout-${REVIEW_ID}.jsonl`. If the first line parses as JSON with a `thread_id` field matching `^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$`, save it as `CODEX_SESSION_ID`, then record the SAME 9 base fields as the secondary tier's "Exactly one path" branch (below). **Do NOT write the JSON yet — proceed to Step R4.5 for `review_quality` + `triage` enrichment and the final write.** Otherwise fall through to the secondary tier.
|
||||
|
||||
*Secondary — rollout content-match:*
|
||||
|
||||
@@ -177,7 +263,7 @@ find ~/.codex/sessions -name 'rollout-*.jsonl' -newer <anchor> -exec grep -l 'AD
|
||||
|
||||
**Interpret the result by STDOUT, not exit code.** Split the command's stdout on newlines; count non-empty lines. Empty stdout means ZERO paths regardless of the pipeline's exit status (`find` returning no matches and `grep -l` matching nothing in found files both yield empty stdout with different exit codes; treat both as zero).
|
||||
|
||||
- **Exactly one path** → extract the trailing UUID from the filename (pattern `rollout-<ISO-timestamp>-<UUID>.jsonl`; UUID is the 36-char hex-and-dashes segment before `.jsonl`), then write this EXACT JSON object to `${RESULT_PATH}` (all 9 fields explicitly; do NOT leave any omitted or as literal placeholder like `"<uuid>"`):
|
||||
- **Exactly one path** → extract the trailing UUID from the filename (pattern `rollout-<ISO-timestamp>-<UUID>.jsonl`; UUID is the 36-char hex-and-dashes segment before `.jsonl`), then record the following 9 base fields. **Do NOT write the JSON yet — proceed to Step R4.5 for `review_quality` + `triage` enrichment and the final write:**
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -193,7 +279,7 @@ find ~/.codex/sessions -name 'rollout-*.jsonl' -newer <anchor> -exec grep -l 'AD
|
||||
}
|
||||
```
|
||||
|
||||
- **Zero paths for resume** → write this EXACT JSON object (9 fields, `codex_session_id` is null, `user_warning` carries the §2.4.4 diagnostic):
|
||||
- **Zero paths for resume** → record the following 9 base fields with `codex_session_id=null` and `user_warning` carrying the §2.4.4 diagnostic. **Do NOT write the JSON yet — proceed to Step R4.5:**
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -210,9 +296,61 @@ find ~/.codex/sessions -name 'rollout-*.jsonl' -newer <anchor> -exec grep -l 'AD
|
||||
```
|
||||
|
||||
- **Zero paths for initial/fresh-exec** → route to retry (Step R5). Main needs the id to launch next round. Set `errors: "session-id capture failed: both tiers empty on initial/fresh-exec"`.
|
||||
- **Multiple paths** → write `launch_failure` result with `errors: "multiple rollouts matched marker — aborting to avoid wrong-session bind"`. Do NOT pick one.
|
||||
- **Multiple paths** → write a terminal `launch_failure` result with `errors: "multiple rollouts matched marker — aborting to avoid wrong-session bind"` and `review_quality="unknown"` + the skipped-triage object (`triage.status="skipped"`, counts `0`, `max_severity="none"`, `truncated=false`, `needs_lead_judgment=false`). Do NOT pick one rollout. Do NOT route through Step R5 — this is an "aborting to avoid silent drift" terminal result, not a retryable failure. The result file MUST be 11-field complete.
|
||||
|
||||
(The two `success` JSON shapes are inlined above per branch. Every success path through R4.4 MUST emit a complete 9-field JSON object — never rely on implicit defaults, never leave a field omitted, never write a literal placeholder like `"<uuid>"` in the output.)
|
||||
(Every success path through R4.4 records 9 base fields. The final 11-field write — base + `review_quality` + `triage` — happens in Step R4.5.)
|
||||
|
||||
### Step R4.5: Classify review quality and triage findings
|
||||
|
||||
This step runs once per success result, after R4.3/R4.4 have recorded the 9 base fields. It produces the `review_quality` classification and the bounded `triage` object, then writes the final 11-field JSON to `${RESULT_PATH}` and returns the `RUNNER_RESULT_AT:` line.
|
||||
|
||||
**Step R4.5.1: Classify `review_quality`.**
|
||||
|
||||
Default to `valid`. Re-classify based on cheap content checks against `/tmp/codex-review-${REVIEW_ID}.md`:
|
||||
|
||||
- **`degraded_environmental`** when the review body matches any of these patterns (case-insensitive, anchored at line start where applicable):
|
||||
- `bwrap: ` (bubblewrap diagnostic leaking into review)
|
||||
- `^Sandbox(ing)?( error| failure)?:` or `sandbox setup failed`
|
||||
- `permission denied .* (sandbox|exec|bind)`
|
||||
- `cannot run (commands|tests|tools) in (this )?sandbox`
|
||||
- `rate limit (exceeded|hit)` AS THE ONLY substantive content (review body < 500 chars total)
|
||||
- `trust prompt` / `requires (interactive )?approval` AS THE ONLY substantive content
|
||||
- `unable to perform (the )?review` / `failed to start (the )?review` AS THE ONLY substantive content
|
||||
Use `rg` (`-i -F` for literal substrings, `-i` for short regex). If two or more independent patterns match, classify as `degraded_environmental` even if the body is long — the reviewer likely produced a meta-explanation rather than a review.
|
||||
|
||||
- **`degraded_content`** (conservative catch) when:
|
||||
- `VERDICT: REVISE` is present BUT the file contains zero `[severity: critical|high|medium]` tags (already caught by R4.3, but if a previous attempt slipped through, this is a backstop), OR
|
||||
- Severity tags exist but each finding body is < 80 chars (no concrete scenario, no file references) — heuristic for placeholder findings.
|
||||
When in doubt, prefer `valid`. False positives here force the lead through an unnecessary user-prompt; false negatives at worst surface a noisy review the lead will downgrade in the evaluation matrix.
|
||||
|
||||
- **`unknown`** if classification itself errors (file disappeared between R4.3 and here, `rg` not on PATH, etc.). Do NOT retry classification — emit `unknown` and let the lead handle it.
|
||||
|
||||
- **`valid`** otherwise.
|
||||
|
||||
**Step R4.5.2: Run bounded triage.**
|
||||
|
||||
If `review_quality = degraded_environmental` → set `triage.status = skipped`; fill numeric fields with `0`, `max_severity = "none"`, `truncated = false`, `needs_lead_judgment = false`. Skip to R4.5.3.
|
||||
|
||||
Otherwise:
|
||||
|
||||
1. Count severity tags: `rg -c -i '^\[severity:\s*(critical|high|medium)' /tmp/codex-review-${REVIEW_ID}.md` (or equivalent — match the actual finding-header form used by the prompt template). Populate `finding_count`.
|
||||
2. Compute `max_severity` from the highest tier present (`critical > high > medium > none`).
|
||||
3. For each `[severity: critical]` finding (up to all of them) and each `[severity: high]` finding (up to all of them), do a cheap inspection: if the finding cites a file path or line range, verify with `ls`/`rg` that the path exists and the citation is plausible. Increment `covered_critical` / `covered_high` accordingly. **Do NOT run tests, builds, doc lookups, or web searches in this step** — those belong to the main thread / the lead's evaluation matrix.
|
||||
4. For medium findings: inspect up to 10. If more remain, set `truncated = true`. Increment `covered_medium` for each inspected.
|
||||
5. If any inspected finding is ambiguous (cited path doesn't exist, severity feels miscategorized, or the inspection produced no signal), set `needs_lead_judgment = true`. This is a hint, not a gate.
|
||||
6. If any rg / ls / read call errors out, set `triage.status = failed`, leave fields at whatever was populated so far (zero-initialize anything not yet set), and proceed. A failed triage does NOT downgrade `review_quality` — main treats triage as a hint.
|
||||
|
||||
On success, set `triage.status = ok`.
|
||||
|
||||
**Hard ceiling.** Total wall time for R4.5.2 SHOULD NOT exceed 30 seconds. If you're approaching that, stop, mark `triage.status = failed`, and proceed. The lead does the real evaluation; triage is a hint.
|
||||
|
||||
**Step R4.5.3: Write the final 11-field JSON.**
|
||||
|
||||
Combine the 9 base fields recorded in R4.3/R4.4 with `review_quality` (R4.5.1) and the `triage` object (R4.5.2). Use the Write tool to overwrite `${RESULT_PATH}`. All 11 top-level fields MUST be present; never leave a field omitted, never write a literal `"<uuid>"` placeholder.
|
||||
|
||||
If `review_quality = degraded_environmental` and `user_warning` is currently `null`, replace `user_warning` with a one-line operator diagnostic describing the most likely cause (e.g. "Reviewer returned an environmentally-degraded response — content matched sandbox/permission-failure patterns. Inspect /tmp/codex-review-<REVIEW_ID>.md and consider `sandbox:read-only` or fresh-exec.").
|
||||
|
||||
Return the `RUNNER_RESULT_AT: ${RESULT_PATH}` line.
|
||||
|
||||
### Step R5: Retry once on any failure (TERMINAL — main will not re-dispatch)
|
||||
|
||||
@@ -237,9 +375,16 @@ Then write the result with `archived_stdout` and `archived_stderr` set to the `-
|
||||
- For `OPERATION=initial` or `OPERATION=fresh-exec`: no archival needed (there is no next attempt within this REVIEW_ID to collide). Leave files at their normal paths for main's diagnostic read (main is allowed to `mv`/`rm` by path; it just doesn't read content).
|
||||
|
||||
Write the appropriate terminal result and return the `RUNNER_RESULT_AT: ...` line:
|
||||
- Second attempt exit was 124 → write `{"result":"timeout","errors":"codex exceeded 600s on both attempts", ...}` (9 fields, all others null as applicable).
|
||||
- Second attempt exit was 124 → write `{"result":"timeout","errors":"codex exceeded 600s on both attempts", ...}`.
|
||||
- Any other failure mode → write `launch_failure` with stderr tail (≤500 chars) in `errors`.
|
||||
In both cases, fill all 9 fields (set `archived_stdout`/`archived_stderr` only when the archival mv in the OPERATION=resume branch ran, else null; set `user_warning` null; set `verdict` null; set `review_file` to `/tmp/codex-review-${REVIEW_ID}.md` only if that file contains a valid VERDICT line, else null).
|
||||
|
||||
In both cases, fill all 11 top-level fields:
|
||||
|
||||
- Base 9 fields: set `verdict = null`, `review_file = /tmp/codex-review-${REVIEW_ID}.md` only if that file contains a valid VERDICT line (else `null`), `codex_session_id = null`, `attempt_id = <current>`, `errors` per above, `archived_stdout`/`archived_stderr` set only when the archival mv in the OPERATION=resume branch ran (else `null`), `user_warning = null`, plus the chosen `result`.
|
||||
- `review_quality = "unknown"` (no successful review to classify).
|
||||
- `triage = { "status": "skipped", "finding_count": 0, "max_severity": "none", "covered_critical": 0, "covered_high": 0, "covered_medium": 0, "truncated": false, "needs_lead_judgment": false }`.
|
||||
|
||||
The same 11-field rule applies to the `infra_error` and `input_error` paths elsewhere in this spec: every result file MUST be 11-field complete.
|
||||
|
||||
### Step R6: Cleanup
|
||||
|
||||
@@ -252,7 +397,19 @@ The one exception is the `mv` in Step R5 above — this is NOT cleanup (files ar
|
||||
|
||||
## Notes
|
||||
|
||||
- You run as a Haiku subagent. Your 250K context is disposed when you return. Anything you read (stderr files, rollout paths, JSONL streams) does NOT reach the main thread — that is the whole point.
|
||||
- Do NOT ask the main thread clarifying questions. If input is missing or malformed, write an `input_error` result to `${RESULT_PATH}` and return the `RUNNER_RESULT_AT:` line.
|
||||
- Do NOT attempt to apply fixes, interpret severity, or re-run more than one retry. The 5-round orchestration loop lives in main.
|
||||
- You run as a Sonnet subagent. Your 1M context is disposed when you return. Anything you read (stderr files, rollout paths, JSONL streams, the review file during triage) does NOT reach the main thread — that is the whole point of the runner layer.
|
||||
- Do NOT ask the main thread clarifying questions. If input is missing or malformed, write an `input_error` result to `${RESULT_PATH}` (11-field complete: base + `review_quality="unknown"` + skipped triage) and return the `RUNNER_RESULT_AT:` line.
|
||||
- Do NOT attempt to apply fixes, interpret severity beyond the bounded triage in R4.5, or run more than one retry. The 5-round orchestration loop and ALL final `accept` / `reject with reasoning` / `re-scope` decisions live in main.
|
||||
- The final line of your message is ONLY `RUNNER_RESULT_AT: <path>` — nothing before, nothing after, no markdown fence. Main's regex tolerates minor wrapping, but adhering to the spec eliminates edge cases entirely.
|
||||
|
||||
## Negative examples — what the runner MUST NOT do
|
||||
|
||||
These are common ways to drift outside the runner's mandate. If you find yourself doing any of these, stop and re-read this spec.
|
||||
|
||||
- **Do not edit, create, or delete any file inside `${REPO_ROOT}`.** The runner is read-only with respect to the workspace tree. The reviewer (Codex) may or may not be sandboxed depending on `CODEX_SANDBOX`, but the runner itself never modifies project files.
|
||||
- **Do not apply fixes to the artifact under review.** Even if you can see exactly what the reviewer is asking for, "fix and re-launch" is not the runner's job. Return the result; main applies fixes.
|
||||
- **Do not start a second Codex review round.** Each dispatch is exactly one Codex operation (initial, resume, or fresh-exec) plus at most one internal retry. Multi-round orchestration is in `SKILL.md` Steps 5–7.
|
||||
- **Do not delete or rename `/tmp/codex-*` files** except for the explicit archival `mv` in Step R5's `OPERATION=resume` branch. Main owns the cleanup glob at `SKILL.md` Step 9.
|
||||
- **Do not pick which findings the lead should accept, reject, or re-scope.** Triage emits counts and a `needs_lead_judgment` hint; nothing else. If you wrote any per-finding "accept" or "reject" annotation into the JSON, you've overstepped — remove it.
|
||||
- **Do not decide that a review is "good enough" to terminate the loop.** `verdict=APPROVED` comes from Codex; the runner only passes it through. Never emit `verdict=APPROVED` based on your own judgment.
|
||||
- **If a command unexpectedly modifies project files (e.g. a probe command had a side effect on `${REPO_ROOT}`), STOP immediately and report.** Write a result with `review_quality="degraded_environmental"`, `user_warning` describing what was modified, and the `errors` field naming the offending command. Do NOT attempt to undo the modification — main does the post-dispatch `git status --porcelain` snapshot and will detect the drift on its own.
|
||||
|
||||
Reference in New Issue
Block a user