ddadmin
f4971e94ca
- Зачем: - зафиксировать состояние конвейера #5 перед долгой живой приёмкой, чтобы новая сессия продолжила без потери контекста. - Что: - в handoff добавлена дельта 23:10: обе находки FIXED (фрагменты ID с SHA-256-цепочкой), перепроверка линией B — APPROVED; - обновлено состояние стенда: им владеет сценарий приёмки; - в .scratch/issue5-run добавлены свежие отчёты, перепроверка и сценарий приёмки acceptance-issue5.sh. - Проверка: - git show --stat; лог приёмки — до строки SCRIPT_EXIT_CODE=.
71 lines
3.4 KiB
Markdown
71 lines
3.4 KiB
Markdown
# Narrow recheck: issue #5 fix round (findings CODE-1, CODE-2 only)
|
|
|
|
You are review line B (code quality) doing a NARROW recheck after a fix
|
|
round. Fresh session, no executor history. Do NOT modify sources, git
|
|
state, or permanent environment. You may read anything and run focused
|
|
tests (pytest of specific files is fine).
|
|
|
|
Repo: /home/dementev/sources/clickstream-ch-kafka-superset-demo, branch
|
|
feature/mentee-path. Scope: ALL uncommitted working-tree changes
|
|
(git status / git diff vs HEAD), but ONLY re-verify the two findings below.
|
|
This is NOT a full re-review — do not open new lines of critique unless
|
|
you find a defect introduced BY the fix itself.
|
|
|
|
Context files (same directory as this file):
|
|
- review-issue5-code.md — your line's original full report (CODE-1..CODE-2)
|
|
- report-issue5.md — executor's report; section "## Triage round 1"
|
|
describes the fix design and measurements
|
|
- mandate-issue5.md — original executor mandate (invariants)
|
|
|
|
## Finding CODE-1 (was High, == TASK-1): manifest payload ceiling
|
|
|
|
Original defect: cumulative manifest embedded exact click_id/uid sets,
|
|
~0.81 MB already on the reference world, Kafka 1 MiB default message limit
|
|
not overridden → save fails after ~1 more next-day; storage O(history) in
|
|
a SINGLE message.
|
|
|
|
Executor's claimed fix: exact ID sets moved out of the manifest into
|
|
content-addressed chunks (≤10,000 IDs each, SHA-256 chain, separate compact
|
|
topic generator_startup_history_counter_chunks); main manifest keeps totals
|
|
+ rolling checksums + chain head/count; next-day appends only new-ID
|
|
chunks, never rewrites or scans history; explicit 900,000-byte pre-send
|
|
guard with Russian error on both manifest and chunks.
|
|
|
|
Verify:
|
|
1. The single-message ceiling is genuinely gone: no code path serializes
|
|
the full ID history into one Kafka message (check import, backfill AND
|
|
next-day paths).
|
|
2. Chunk chain integrity: hash chain actually validated on read; chain
|
|
head/count in manifest actually checked; a broken/missing chunk fails
|
|
loudly (Russian error), not silently.
|
|
3. next-day is still O(new day): no full chunk-chain scan or historical
|
|
chunk rewrite on the incremental path.
|
|
4. Exactness preserved: uniqExact comparison against ClickHouse still uses
|
|
exact counts; the required incremental-vs-full-recompute equality test
|
|
still genuinely compares incremental vs full recompute after the
|
|
restructure (not weakened to pass).
|
|
5. The 900,000-byte guard is real (applied before send on every producer
|
|
path for these payloads) and its failure mode is explicit.
|
|
6. Boundary-ID sourcing from generator state (active visits + users with
|
|
visits, pre-created unused users excluded) does not break determinism
|
|
or the day-boundary duplicate handling the tests claim.
|
|
|
|
## Finding CODE-2 (was Low): write ordering
|
|
|
|
Original defect: next-day saved state BEFORE manifest (import did reverse),
|
|
widening the half-advanced window.
|
|
|
|
Verify: order is now chunks → manifest → state on ALL three paths (import,
|
|
backfill, next-day), and the two-day test actually asserts it.
|
|
|
|
## Report
|
|
|
|
Write full report (English) to recheck-issue5-code.md in this same
|
|
directory. Structure: per finding — RESOLVED / NOT_RESOLVED / REGRESSION
|
|
(new defect introduced by the fix, with severity, file:line, scenario).
|
|
End with VERDICT: APPROVED (both resolved, no regressions) or
|
|
CHANGES_REQUESTED.
|
|
|
|
Final message: VERDICT, one line per finding status, one line per
|
|
regression if any, report path.
|