diff --git a/README.md b/README.md index e2b43fd..248ad33 100644 --- a/README.md +++ b/README.md @@ -118,6 +118,8 @@ chosen config file: "Bash(cd * && cat /tmp/codex-resume-prompt-* | timeout 600 codex exec resume *)", // Session-id filesystem fallback (newest rollout file in ~/.codex/sessions/) "Bash(find ~/.codex/sessions*)", +// Diagnostic aid when filesystem fallback finds nothing +"Bash(ls -t ~/.codex/sessions*)", // Temp files: prompts (initial + resume), plans, review output, JSONL stdout, stderr "Write(/tmp/codex-plan-*)", "Write(/tmp/codex-prompt-*)", @@ -145,6 +147,7 @@ chosen config file: "Bash(cat /tmp/codex-prompt-* | timeout 600 codex exec *)", "Bash(cd * && cat /tmp/codex-resume-prompt-* | timeout 600 codex exec resume *)", "Bash(find ~/.codex/sessions*)", + "Bash(ls -t ~/.codex/sessions*)", "Write(/tmp/codex-plan-*)", "Write(/tmp/codex-prompt-*)", "Write(/tmp/codex-resume-prompt-*)", diff --git a/SKILL.md b/SKILL.md index f386f55..a68afea 100644 --- a/SKILL.md +++ b/SKILL.md @@ -23,7 +23,7 @@ Sends current work for adversarial review through an external AI model (OpenAI C ## Instructions -> **Placeholders:** `${REVIEW_ID}`, `${CODEX_SESSION_ID}`, `${REPO_ROOT}`, and `${BASE_BRANCH}` in the steps below are template placeholders, NOT shell variables. Substitute literal values directly into each tool call. In particular, `${REPO_ROOT}` is ALWAYS an absolute path captured at Step 2; never replace it with `$(pwd)`. +> **Placeholders:** `${REVIEW_ID}`, `${CODEX_SESSION_ID}`, `${CODEX_SESSIONS_BEFORE}`, `${REPO_ROOT}`, and `${BASE_BRANCH}` in the steps below are template placeholders, NOT shell variables. Substitute literal values directly into each tool call. In particular, `${REPO_ROOT}` is ALWAYS an absolute path captured at Step 2 (never `$(pwd)`); `${CODEX_SESSIONS_BEFORE}` is a unix-timestamp integer captured immediately before every `codex exec` / `codex exec resume` (see Steps 4 and 7), used by the filesystem session-id fallback — substitute the integer verbatim into `find -newermt "@"`, never leave `${CODEX_SESSIONS_BEFORE}` as a shell variable reference. ### Step 1: Determine review mode @@ -308,7 +308,7 @@ And the following items are added to ``: **Launching Codex — command template:** Flags: -- `--json` — stdout becomes JSONL events (primary path for session-ID capture). In some sandbox configurations this stream ends up empty; the filesystem fallback in check 3 below handles that case. +- `--json` — stdout becomes JSONL events (primary path for session-ID capture). In some sandbox configurations this stream ends up empty; the filesystem fallback in check 4 below handles that case. - `-m gpt-5.4` — model (overridden by `model:...` argument) - `-c model_reasoning_effort=high` — reasoning depth (overridden by `xhigh`, `low`, etc.) - `-s read-only` — reviewer only reads, does not write @@ -319,13 +319,13 @@ Flags: **Plan Mode note:** Writing to `/tmp` via Write tool may trigger a permission prompt or exit Plan Mode. This is a known Claude Code limitation — Plan Mode restricts edits to the plan file only. If this happens, it does not affect review correctness: the review mode is already determined, and the skill only edits the plan file and `/tmp` temp files. -**Capture pre-exec timestamp** (for filesystem fallback of session-id; see check 3 below). Substitute the value literally: +**Capture pre-exec timestamp** (for filesystem fallback of session-id; see the secondary-path check below). Substitute the value literally: ```bash -date +%s +echo $(($(date +%s) - 1)) ``` -Save as `CODEX_SESSIONS_BEFORE` (a template placeholder — a Unix timestamp as an integer). +Save as `CODEX_SESSIONS_BEFORE` (a template placeholder — a Unix timestamp as an integer). The `- 1` shifts the window back one second to avoid a same-epoch race: `find -newermt "@N"` treats mtime **strictly greater** than N, so if codex finishes in the same epoch-second as the capture (fast path, cached response), the rollout file would be missed without this shift. Cost: the lookup window widens by 1 second, which is irrelevant against the codex exec duration (seconds to minutes). ```bash cat /tmp/codex-prompt-${REVIEW_ID}.md | timeout 600 codex exec --json \ @@ -359,7 +359,12 @@ cat /tmp/codex-prompt-${REVIEW_ID}.md | timeout 600 codex exec --json \ - If file contains a line matching `^Error:` or `Failed to write` → codex reported an infrastructure failure despite exit 0. Show stderr to user, route to launch-failure retry (max 1 per round; after retry failure → hard abort). - Otherwise → proceed. -3. **Capture `CODEX_SESSION_ID` — two-tier.** +3. **Review file sanity.** Read `/tmp/codex-review-${REVIEW_ID}.md`. It must exist and contain a line matching `^VERDICT: (APPROVED|REVISE)$`; if REVISE, it must also contain at least one line matching `\[severity:\s*(critical|high|medium)` (a structured finding). The full semantic-check logic is in Step 5; do the same thing here. + - Fails → route to launch-failure retry (max 1 per round; after retry failure → hard abort). Do NOT capture `CODEX_SESSION_ID` — if the review itself is broken, the session is of no use. + - Passes with `VERDICT: APPROVED` → `CODEX_SESSION_ID` is not needed (no Step 7 resume will happen). Skip the capture below entirely and proceed to Step 5. + - Passes with `VERDICT: REVISE` → capture `CODEX_SESSION_ID` next (check 4). + +4. **Capture `CODEX_SESSION_ID` — two-tier.** Only reached when the review was valid AND the verdict is REVISE (Step 7 resume is about to happen). **What you are looking for.** A UUID string (format `[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}`) that identifies the codex session, so `codex exec resume ` in Step 7 can continue this conversation. Try the cheap source first, fall back to the filesystem only if needed. @@ -367,7 +372,7 @@ cat /tmp/codex-prompt-${REVIEW_ID}.md | timeout 600 codex exec --json \ ```json {"type":"thread.started","thread_id":"","...":...} ``` - Parse it as JSON, take `thread_id`, save as `CODEX_SESSION_ID`, proceed to check 4. + Parse it as JSON, take `thread_id`, save as `CODEX_SESSION_ID`, proceed to Step 5. **Secondary: rollout filename.** In some Claude Code sandbox configurations the `--json` stdout file is empty (0 bytes) even when the review completes successfully (`-o` is populated, exit 0, stderr clean). In that case the session is still recoverable from disk: every `codex exec` writes a rollout file named `rollout--.jsonl` under `~/.codex/sessions/YYYY/MM/DD/` (see `DESIGN.md §2.3`). The trailing UUID in the filename is the session id. @@ -379,16 +384,19 @@ cat /tmp/codex-prompt-${REVIEW_ID}.md | timeout 600 codex exec --json \ This prints zero or more lines of ` ` for rollout files created after the pre-exec timestamp. From the result: - - **Zero lines** → no rollout file was created; treat as launch failure, show stderr, retry once, then abort. + - **Zero lines** → no rollout file found. Before aborting, surface useful diagnostic context to the user: the contents of `/tmp/codex-stdout-${REVIEW_ID}.jsonl` (if non-empty), `/tmp/codex-stderr-${REVIEW_ID}.txt`, and the 3 most-recent rollout filenames (`ls -t ~/.codex/sessions/*/*/*/rollout-*.jsonl 2>/dev/null | head -3`). Then treat as launch failure, retry once, then abort. - **One or more lines** → pick the line with the largest epoch-mtime (there is usually just one; multiple lines indicate a parallel codex invocation in the same second). Extract the trailing UUID from that filename (the 36-char hex-and-dashes pattern above) and save as `CODEX_SESSION_ID`. A single `find` invocation keeps the permission rule simple (`Bash(find ~/.codex/sessions*)`) and the parsing stays in your head — no shell pipeline needed. **Platform note.** `-newermt "@"` and `-printf` are GNU extensions. On macOS (BSD find) they are unsupported — substitute an equivalent that achieves the same goal: "list rollout files modified since `${CODEX_SESSIONS_BEFORE}`, newest first". For example, `find ~/.codex/sessions -name 'rollout-*.jsonl' -type f` plus `stat -f '%m %N' ` per result, or `ls -t ~/.codex/sessions/*/*/*/rollout-*.jsonl` and filter by a reference file's mtime. The goal is what matters, not the exact flags. - **Parallel-codex caveat:** if you happened to pick a rollout from a parallel codex invocation, Step 7's resume will either succeed against the wrong session (detected later via VERDICT / severity checks) or fail at the stderr/exit-code check and route to the standard fallback (§4.11). Either outcome is recoverable. + **Parallel-codex caveat — real silent-corruption risk, not benign.** If the user runs `codex` in parallel in the same cwd during the pre-exec timestamp window, the newest rollout may belong to that other invocation. Step 7's resume against the wrong session returns a normally-shaped response (VERDICT + severity markers), so the skill's post-resume checks in Step 7 will NOT detect the mismatch. The skill then applies "fixes" guided by a review of the wrong artifact. Two mitigations in effect: -4. **Review file sanity.** (Performed again in Step 5, but note upfront.) `/tmp/codex-review-${REVIEW_ID}.md` must exist and contain a line matching `^VERDICT: (APPROVED|REVISE)$`. If not → Step 5 will handle it via retry/abort. + 1. The `CODEX_SESSIONS_BEFORE` timestamp is captured immediately before the exec, so the window is narrow (seconds). + 2. `--last` is never used (`§4.5`) — explicit UUID is always passed to resume. + + Neither eliminates the race. If you see suspicious behavior mid-review (reviewer mentions files or sections not in this work), halt and tell the user before applying any fix. **Where `thread_id` / session id is NOT:** @@ -483,10 +491,10 @@ End with VERDICT: APPROVED or VERDICT: REVISE **2. Run resume.** Resume does NOT accept `-C`, so prefix the command with an explicit `cd` to `${REPO_ROOT}` (captured at Step 2). Use single quotes around `${REPO_ROOT}` — the path was validated at Step 2 to contain no single quotes. -Capture a pre-resume timestamp for the secondary session-id fallback (same pattern as Step 4): +Capture a pre-resume timestamp for the secondary session-id fallback (same pattern as Step 4, with the `-1` shift against same-epoch race): ```bash -date +%s +echo $(($(date +%s) - 1)) ``` Save as `CODEX_SESSIONS_BEFORE`. Then launch resume via the same `cat | ... -` pattern: @@ -518,7 +526,7 @@ Use `timeout: 620000` in Bash tool parameters. 3. **Review file sanity.** Read `/tmp/codex-review-${REVIEW_ID}.md` and apply the same checks as Step 5.2: - Missing / empty / no `^VERDICT: (APPROVED|REVISE)$` line / REVISE without `[severity:` lines → route to fallback. Do NOT update `CODEX_SESSION_ID`. -**4. Only if all three checks pass** → refresh `CODEX_SESSION_ID` with the same two-tier approach from Step 4 check 3 (primary = first JSONL line of `/tmp/codex-stdout-${REVIEW_ID}.jsonl`; secondary = newest rollout filename with mtime > `CODEX_SESSIONS_BEFORE`, UUID extracted from the basename). +**4. Only if all three checks pass AND the verdict is REVISE** → refresh `CODEX_SESSION_ID` with the same two-tier approach from Step 4 check 4 (primary = first JSONL line of `/tmp/codex-stdout-${REVIEW_ID}.jsonl`; secondary = newest rollout filename with mtime > `CODEX_SESSIONS_BEFORE`, UUID extracted from the basename). On APPROVED verdict, skip the refresh — there is no round N+1. Per `DESIGN.md §2.4.4`, successful resume does not rotate the thread id — the new value equals the previous one, so this refresh is defensive. If both tiers yield nothing but the three checks passed → the resume itself was fine; keep the previous `CODEX_SESSION_ID` unchanged and continue. @@ -669,7 +677,7 @@ Do NOT delete plan files that existed before the review (only temp files created - **`REPO_ROOT` is captured at Step 2** via `git rev-parse --show-toplevel` and substituted as an absolute literal path into every codex command. Never use `$(pwd)` inside codex commands — cwd drift between Bash calls makes it unreliable. - **Resume requires `cd '${REPO_ROOT}' && ...`** because `codex exec resume` has no `-C` flag; cwd is inherited from the shell. The initial exec uses `-C "${REPO_ROOT}"` instead. - **`CODEX_SESSION_ID` is updated only on full success** — ALL of (exit=0 AND stderr has no `Error:`/`thread/resume failed` line AND review file contains a valid `VERDICT:` line with findings on REVISE). On any failure, leave it unchanged and route to the fallback. -- **Session ID capture is two-tier.** Primary: `thread_id` from the first JSONL line of stdout. Secondary (when stdout is empty — env-specific): UUID from the trailing component of the newest `~/.codex/sessions/**/rollout-*.jsonl` filename with mtime > `CODEX_SESSIONS_BEFORE`. Capture `CODEX_SESSIONS_BEFORE=$(date +%s)` **before** every `codex exec` / `codex exec resume` call. +- **Session ID capture is two-tier.** Primary: `thread_id` from the first JSONL line of stdout. Secondary (when stdout is empty — env-specific): UUID from the trailing component of the newest `~/.codex/sessions/**/rollout-*.jsonl` filename with mtime > `CODEX_SESSIONS_BEFORE`. Capture `CODEX_SESSIONS_BEFORE=$(($(date +%s) - 1))` **before** every `codex exec` / `codex exec resume` call (the `-1` shift prevents a same-epoch race against `-newermt`'s strict-greater semantics). - **Prompt delivery is `cat file | codex exec ... -`.** The `- < file` stdin-redirect form is accepted by codex but exits 1 with empty stderr in some Claude Code sandbox configurations. Pipe is portable across both envs observed. - **The `--json` stdout stream is never human-readable review text** — JSONL events when populated, empty when suppressed by sandbox. Never treat Bash result as review content; the review lives exclusively in `/tmp/codex-review-*.md`. - **Launch-failure retry** is capped at 1 per round and does NOT consume the 5-round counter. The retry counter is per-round; it resets at the start of every new round and is tracked only in that round's reasoning. diff --git a/docs/DESIGN.md b/docs/DESIGN.md index e8ef52b..1f72839 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -460,9 +460,15 @@ Each decision below follows the same template: - *Use only the filesystem path as the single source.* Rejected: relies on a filesystem race window against any parallel codex invocation in the same second. As a secondary (only consulted - when stdout is empty) the race is rare, and when it does hit, - one of the Step 7 post-resume checks (`§4.8`) catches the wrong - session and routes to the standard fallback (`§4.11`). + when stdout is empty) the race is rare, but — honestly — NOT + auto-detectable by the skill: a wrong-session resume returns a + normally-shaped review (VERDICT + `[severity:` markers), so the + Step 7 post-resume checks (`§4.8`) pass and the skill applies + fixes based on an unrelated artifact. The `CODEX_SESSIONS_BEFORE` + timestamp narrows the race window to seconds, but does not + eliminate it. Risk is acknowledged in `SKILL.md` Step 4 check 4 + ("Parallel-codex caveat") as a silent-corruption hazard, not a + fallback-handled hazard. - *Drop `--json` entirely and use plain-text stdout.* Rejected: `--json` makes stdout machine-readable only, which is *load- bearing* for the show-review gate (`§4.9`). Plain-text stdout @@ -477,10 +483,18 @@ Each decision below follows the same template: - Human-readable review is no longer in stdout (it went to `-o` only) — load-bearing for `§4.9`. - Secondary path introduces a filesystem race against parallel - codex invocations (§9.1 scope). Mitigated by the pre-exec - timestamp (`CODEX_SESSIONS_BEFORE`) narrowing the window to - "files created between the two timestamps" rather than "newest - anywhere". + codex invocations (§9.1 scope). Mitigated (not eliminated) by + the pre-exec timestamp (`CODEX_SESSIONS_BEFORE=$(($(date +%s) - + 1))`) narrowing the window to "files created within ~1-2 seconds + of the exec start". The `-1` shift against `-newermt`'s strict- + greater semantics prevents same-epoch miss; the race window is + one second wider as a result, still negligible compared to a + real codex exec duration. + - Session-id capture happens only after review-file sanity passes + AND only when verdict is `REVISE` (Step 4 check order in + `SKILL.md`). This avoids aborting a valid round-1 APPROVED over + a secondary-capture failure: APPROVED means no resume, no + session-id needed. - Extra permission surface: `Bash(find ...)` is now in the recommended permissions list. @@ -615,16 +629,30 @@ Each decision below follows the same template: prompt wording; prompt drift would be a separate failure mode (see §6). -### §4.8. Strict check order: exit → stderr → review file +### §4.8. Strict check order: exit → stderr → review → (session-id when needed) - **Decision.** After every `codex exec` / `codex exec resume` call, - checks are run in a fixed order: exit code first, then stderr file - (even on exit 0), then the `-o` review file. + checks run in a fixed order: + 1. Exit code. + 2. Stderr file (even on exit 0). + 3. `-o` review file semantic sanity. + 4. Session-id capture (two-tier, `§4.1`). In Step 4 this is skipped + on `VERDICT: APPROVED` — no resume will happen, so no session + is needed. In Step 7 it is a defensive refresh (thread id + doesn't rotate per `§2.4.4`) and is skipped identically on + APPROVED. - **Where in SKILL.md.** Steps 4 and 7 post-launch. - **Context.** Non-zero exit implies `-o` may not exist; reading it would crash. Exit 0 does not imply everything is fine (e.g., `-o` - unwritable case, §2.5). A defined order routes every failure mode to - the right diagnostic without the lead having to improvise. + unwritable case, §2.5). Review-sanity before session-id keeps the + two concerns orthogonal: a broken review aborts on its own merits, + and a valid APPROVED review completes without depending on + session-id capture. An earlier draft ordered session-id *before* + review-sanity, which meant a secondary-capture failure (e.g., + empty `~/.codex/sessions/` on a first-ever codex run, or a super- + fast codex exec hitting the `-newermt` same-epoch edge) would + abort an otherwise-successful APPROVED round. The current order + avoids that. - **Alternatives considered.** - *Ad-hoc checks in whatever order.* Rejected: invites null-pointer- style crashes on missing files.