ddmitry/local-transcriber
1
0 Fork 0
Code Issues 5 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(cuda): добавлен необязательный runtime для Linux и Windows #57

Merged
ddmitry merged 4 commits from feat/optional-cuda-windows into master 2026-09-21 23:31:31 +03:00
Conversation 1 Commits 4 Files Changed 21
+613 -184
ddmitry commented 2026-09-21 23:30:27 +03:00
Owner
Copy link
Copy source

Обычная установка больше не включает NVIDIA cuBLAS. CUDA подключается через extra cuda на Linux/WSL x86_64 и Windows x64 и выбирается явно через CLI или TOML; auto всегда использует ONNX CPU. Для ASR и VAD закреплён CPU provider.

На Windows добавлены поиск DLL в окружении приложения и их загрузка перед CTranslate2. Сохранён Linux bootstrap; CPU-путь не загружает CUDA. Зафиксированы проверенные версии CTranslate2 4.8.1 и cuBLAS 12.9.1.4. Ошибки библиотек, несовместимости GPU, типа вычислений и нехватки памяти получают соответствующие подсказки без молчаливого переключения явно выбранной CUDA. Обновлены инструкции установки, ADR и спецификация.

Проверки: 524 теста прошли, 2 пропущены; метаданные wheel и разрешение зависимостей для Linux/Windows base/extra проверены; независимый Opus подтвердил установку через uv tool и Linux bootstrap. Ревью версии a8b8a8c: APPROVED, обе находки исправлены и перепроверены, новых замечаний нет.

Ограничение: реальная транскрипция на Windows x64 с NVIDIA без системного Toolkit пока не выполнена. Статический аудит DLL и тесты с подменами её не заменяют. В ADR отдельно отмечена динамическая ссылка cuBLAS Lt на NVRTC, которую следует проверить при аппаратном прогоне.

Closes #2

Обычная установка больше не включает NVIDIA cuBLAS. CUDA подключается через extra `cuda` на Linux/WSL x86_64 и Windows x64 и выбирается явно через CLI или TOML; `auto` всегда использует ONNX CPU. Для ASR и VAD закреплён CPU provider. На Windows добавлены поиск DLL в окружении приложения и их загрузка перед CTranslate2. Сохранён Linux bootstrap; CPU-путь не загружает CUDA. Зафиксированы проверенные версии CTranslate2 4.8.1 и cuBLAS 12.9.1.4. Ошибки библиотек, несовместимости GPU, типа вычислений и нехватки памяти получают соответствующие подсказки без молчаливого переключения явно выбранной CUDA. Обновлены инструкции установки, ADR и спецификация. Проверки: 524 теста прошли, 2 пропущены; метаданные wheel и разрешение зависимостей для Linux/Windows base/extra проверены; независимый Opus подтвердил установку через uv tool и Linux bootstrap. Ревью версии `a8b8a8c`: APPROVED, обе находки исправлены и перепроверены, новых замечаний нет. Ограничение: реальная транскрипция на Windows x64 с NVIDIA без системного Toolkit пока не выполнена. Статический аудит DLL и тесты с подменами её не заменяют. В ADR отдельно отмечена динамическая ссылка cuBLAS Lt на NVRTC, которую следует проверить при аппаратном прогоне. Closes #2
ddmitry added 4 commits 2026-09-21 23:30:29 +03:00
fix(device): закреплён ONNX CPU для автоматического выбора 937eafe8db
feat(cuda): добавлен optional runtime для Linux и Windows 99ed5ed92f
docs(cuda): описаны установка extra и проверка Windows DLL 453b5d9dfa
fix(cuda): учтён отказ по неподдерживаемому типу вычислений a8b8a8c0c9
ddmitry commented 2026-09-21 23:31:13 +03:00
Author
Owner
Copy link
Copy source

Независимое ревью Claude Opus: APPROVED после исправления двух замечаний. Первичное ревью — 23 мин 12 с, перепроверка в той же сессии — 3 мин 56 с. Ниже сохранены отчёты перед очисткой локальных временных файлов.

Первичное ревью 453b5d9

Independent review: optional CUDA on Linux and Windows (issue #2)

  • Reviewed version: branch feat/optional-cuda-windows, HEAD 453b5d9dfa38aa5487edf4e8ab2d3e74dfe1fce0, diff since base 2b54f5b.
  • Reviewer: Opus 5, independent (no shared context with the implementer). Repository and Git untouched; all experiments under $CLAUDE_JOB_DIR/tmp and /tmp.
  • Task reference: /tmp/local-transcriber-opus-review/issue-2.md; base spec docs/specs/2026-09-21-transcription-execution.md at 2b54f5b.
  • Host: Linux x86_64, Quadro M3000M (CC 5.2, driver 580.178.04), nvidia-smi present, nvidia_uvm not loaded → CUDA runtime init fails (cudaErrorUnknown). A Windows x64 GPU machine is not available. Real GPU transcription was therefore not possible on any platform; see "Limits".

Verdict

VERDICT: CHANGES_REQUESTED

Counters (scale from the review line: BLOCKER | MAJOR | MINOR | NIT): BLOCKER 0, MAJOR 0, MINOR 1, NIT 1. Total findings: 2.

Findings

OPUS-CUDA-1 | MINOR | src/local_transcriber/transcriber.py:198-235 (cuda_error_hint), src/local_transcriber/config.py:51 (cuda default float16) | criterion: issue #2 scope "различать … обнаруженную несовместимость GPU/runtime … давать подходящую подсказку"; acceptance "ошибки … несовместимого GPU различаются … включая пользовательский вывод"; spec §"Выбор исполнения и ошибки" ("при несовместимости железа объясняется отказ выбранного пути")

  • Failure scenario: user with a Maxwell/Pascal GPU (CC < 7.0 — the class this repo documents as incompatible, including the owner's Quadro M3000M) installs the extra and runs transcribe f.wav --device cuda with the default profile. CTranslate2 rejects the requested float16 before any kernel runs and raises ValueError("Requested float16 compute type, but the target device or backend do not support efficient float16 computation."). cuda_error_hint returns None (no cuda/cublas/no kernel image keyword), so the CLI prints only Ошибка: Requested float16 compute type, …. The user output does not say this is a GPU/runtime incompatibility, that the extra cannot fix it, or to switch to --device onnx/cpu. The same happens for --compute-type int8 (CC ≤ 6.0). Only a second attempt with --compute-type float32 reaches no kernel image is available and gets the intended hint.
  • Evidence:
    • CT2 4.8.1 source: src/types.cc resolve_compute_type → unsupported_compute_type("float16") throws std::invalid_argument when !support_float16 and enable_fallback == false (default in include/ctranslate2/types.h); mayiuse_float16(CUDA) = cuda::gpu_has_fp16_tensor_cores (CC ≥ 7.0). pybind11 maps invalid_argument → ValueError.
    • Reproduced the exact message with real CT2 4.8.1 in a temp tool env (create_model(path, "cpu", "float16") on a CPU without fp16 uses the same code path): ValueError | Requested float16 compute type, …; cuda_error_hint(e) is None, _is_cuda_error(e) is False.
    • docs/gpu.md §"Совместимость GPU" already states "float16 требует CC ≥ 7.0, float32 падает с ошибкой" for this GPU class, so the case is known to the project, but neither the hint keywords nor tests/test_cli.py::test_explicit_cuda_errors_preserve_cause_without_fallback cover it.
  • Minimal verification: cuda_error_hint(ValueError("Requested float16 compute type, but the target device or backend do not support efficient float16 computation.")) → expected an incompatibility hint, actual None.
  • Note on a fix: the hint function has no device context; the same CT2 text is produced for --device cpu --compute-type float16, so matching the text alone would mis-hint CPU users. Any fix needs the resolved device (or the CLI to only consult the hint when the requested device is cuda). Cost is small; risk removed is wrong/no guidance for the most common real-world "old GPU" first contact.

OPUS-CUDA-2 | NIT | tests/test_cli.py:800-821 (test_default_cli_stays_on_onnx_with_nvidia_driver) | criterion: testing claims must match evidence (brief FOCUS)

  • Failure scenario: the test patches local_transcriber._cuda_bootstrap.ensure_cublas_loadable and asserts bootstrap.assert_not_called(). backends/faster_whisper.py binds the name at import (from local_transcriber._cuda_bootstrap import ensure_cublas_loadable), so that patch can never observe a call from the backend; the assertion is vacuous. If auto regressed to cuda the test would still fail, but only through get_backend.assert_called_once_with("onnx", …), not through the bootstrap assertion the docstring implies.
  • Evidence: src/local_transcriber/backends/faster_whisper.py:12 (import-time binding); tests/test_backend_faster_whisper.py::test_bootstrap_only_for_explicit_cuda patches the correct target (local_transcriber.backends.faster_whisper.ensure_cublas_loadable).
  • Minimal verification: temporarily make detect_device("auto") return "cuda" in a scratch copy and run the test with get_backend un-patched for the assertion — the bootstrap mock still records zero calls.

Non-gating considerations

  • Static audit gap for the outstanding Windows validation: cublasLt64_12.dll 12.9.1.4 (and Linux libcublasLt.so.12 12.9.1.4) contain a dynamic-load reference to NVRTC (nvrtc64_120_0.dll / libnvrtc.so.12) that ADR-001 does not record; PE/ELF imports do not include it, and Linux worked historically with 12.9.1.4 without an NVRTC package, so it is presumably optional for the Whisper GEMM path. NVIDIA made it a hard dependency in 12.9.2.10 (verified via PyPI JSON). The first real Windows run should watch for an NVRTC load failure.
  • CUDA failed with error unknown error (observed on the owner's own machine, nvidia_uvm not loaded) yields no hint at all by design ("unknown → no advice"). A one-line generic fallback (check driver / use --device onnx) would cost nothing; deferring is defensible.
  • In batch mode the multi-line install hint is printed once per failing file; with many files and a missing library this repeats the same 5 lines N times. Cosmetic.
  • ctranslate2==4.8.1 is now an exact pin in the main dependencies. Justified by the DLL audit, but every CT2 bump (including security fixes) now requires the re-audit ADR-001 describes; uv lock --upgrade will not move it.
  • README §"Подключение CUDA" says explicit CUDA fails "с исходной причиной и подсказкой"; the hint is only emitted for recognised causes (ADR-001 says so). Minor doc precision; folds into OPUS-CUDA-1 if that is fixed.
  • cuda_error_hint text suggests uv tool install --force '.[cuda]' without --python 3.13 while README uses it. Works (uv resolves an interpreter for requires-python >=3.13), but inconsistent.

Checks performed and outcomes

Packaging metadata and platform markers

  • Built the wheel from git archive 453b5d9 in a temp dir (uv build --wheel). METADATA: no nvidia-* in unconditional Requires-Dist; Provides-Extra: cuda; nvidia-cublas-cu12==12.9.1.4; ((linux and x86_64) or (win32 and (AMD64 or x86_64))) and extra == 'cuda'; ctranslate2==4.8.1 pinned. ✓ Same metadata as the author's /tmp/local-transcriber-issue2-dist wheel.
  • uv pip compile --python-platform … of the built wheel: x86_64-unknown-linux-gnu and x86_64-pc-windows-msvc base → no nvidia-*; with [cuda] → exactly nvidia-cublas-cu12==12.9.1.4 added. aarch64-unknown-linux-gnu with [cuda] → no nvidia-* (marker excludes ARM). ✓ aarch64-pc-windows-msvc and aarch64-apple-darwin do not resolve at all (no ctranslate2 win_arm64 wheel; onnxruntime macOS tag) — pre-existing, unrelated to this change.
  • uv lock --check passes; uv sync --locked --dry-run → "Would make no changes" (cuBLAS already absent); uv sync --locked --extra cuda --dry-run → "+ nvidia-cublas-cu12==12.9.1.4". ✓ Lock lists the win_amd64 cuBLAS wheel with hash matching PyPI.
  • uv tool install --python 3.13 --force ".[cuda]" from a clone copy into a temp UV_TOOL_DIR: installs ctranslate2 4.8.1, nvidia-cublas-cu12 12.9.1.4, transcribe entry point works. ✓
  • uv tool install --python 3.13 --force "local-transcriber[cuda] @ git+file:///…@feat/optional-cuda-windows" into a second temp tool dir: extras applied, cuBLAS installed. ✓ (README's HTTPS URL not exercised; until merged, master has no cuda extra — transient.)

Windows wheel runtime dependencies (static, primary artifacts)

  • /tmp/ct2-windows-audit-ci15wlre/ct2.whl SHA-256 d52499f0…f827 and cublas.whl SHA-256 1e5fee10…40bf recomputed locally and matched against PyPI JSON for ctranslate2 4.8.1 (cp313 win_amd64) and nvidia-cublas-cu12 12.9.1.4 (win_amd64). ✓
  • objdump -p ctranslate2.dll imports: libiomp5md.dll, KERNEL32, MSVCP140, VCRUNTIME140, VCRUNTIME140_1, UCRT api-sets — no static cuBLAS/cuDNN/cudart import. Strings: cublas64_12.dll, nvcuda.dll; zero cudnn strings (ASCII and UTF-16). _ext.cp313-win_amd64.pyd imports ctranslate2.dll, python313.dll, MSVC runtime. cudnn64_9.dll in the wheel is a 266 KB shim importing only KERNEL32; nothing references it. ✓ ADR-001 statements match (VC++ runtime required; cuDNN not used; Conv1D is im2col+GEMM — symbols im2col_transposed_kernel present).
  • CT2 4.8.1 src/cuda/cublas_stub.cc: cuBLAS is loaded lazily via LoadLibraryA("cublas64_12.dll") (Windows) / dlopen("libcublas.so.12") (Linux); failure text Library <name> is not found or cannot be loaded. python/ctranslate2/__init__.py adds only its own package dir via os.add_dll_directory and ctypes.CDLLs every DLL in it. ✓ Matches ADR-001 and the hint keyword not found/cannot be loaded.
  • cuBLAS wheel layout nvidia/cublas/bin/{cublas64_12,cublasLt64_12,nvblas64_12}.dll, no __init__.py → PEP 420 namespace import nvidia.cublas works; __path__ points at the package dir. cublas64_12.dll imports cublasLt64_12.dll + KERNEL32; cublasLt64_12.dll imports KERNEL32 only; nvblas64_12.dll imports cublas64_12.dll (unused). ✓ Additional dynamic string nvrtc64_120_0.dll in cublasLt — see considerations.
  • Microsoft "Dynamic-Link Library Search Order" (fetched): the loader first consults the loaded-module list by base name; AddDllDirectory directories are only searched when LOAD_LIBRARY_SEARCH_* flags are used (per call or via SetDefaultDllDirectories). CPython ctypes/__init__.py _load_library: default winmode = LOAD_LIBRARY_SEARCH_DEFAULT_DIRS, plus LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR when a path is given. Conclusion: the bootstrap's full-path WinDLL preload (Lt first, then cuBLAS) is what makes CT2's plain LoadLibraryA succeed; os.add_dll_directory alone would not. The code comment states this correctly. ✓ (Reasoned from primary docs; not executed on Windows.)

Runtime library loading (Linux, real libraries)

  • In the temp tool env with the extra: is_cublas_available() False → ensure_cublas_loadable() → True (libcublas resolvable by soname via RTLD_GLOBAL preload). ✓
  • Bootstrap is only called for device == "cuda" in FasterWhisperBackend.create_model, before the local from faster_whisper import WhisperModel; module import of the backend no longer triggers it. ✓ (tests/test_backend_faster_whisper.py::test_bootstrap_only_for_explicit_cuda)
  • Without optional package: import nvidia.cublas → ImportError guard; Windows and Linux both return silently. ✓ Windows DLL load failure propagates (OSError), keys stored so retries are idempotent (tests/test_cuda_bootstrap.py, both branches).
  • Real CLI, explicit CPU with the extra installed: transcribe tone.wav --device cpu --model tiny → transcript written, Устройство: cpu Compute: float32. ✓ (tiny model downloaded to a temp HF_HOME; no download in the ordinary test suite.)

Automatic / explicit device policy

  • detect_device("auto") → "onnx" unconditionally; shutil import removed from utils.py. ✓
  • Real CLI on this host (nvidia-smi present, extra installed, HF_HUB_OFFLINE=1): default run prints Инициализирую модель на onnx... and fails on the offline model fetch with no CUDA attempt and no FasterWhisper CPU fallback. ✓
  • Explicit --device cuda (CLI) and device = "cuda" in .transcriber.toml (batch of 2) with real CT2 on this host: Инициализирую модель на cuda... → Ошибка: CUDA failed with error unknown error, exit 1, no Переключение на CPU, --verbose traceback shows faster_whisper.py:98 → WhisperModel.__init__. ✓ Original cause preserved; no hint (unknown cause, by design).
  • strict = requested_device != "auto" in both _run_single and _run_batch; TOML device counts as explicit. ✓

ONNX ASR / VAD providers

  • onnx_asr 0.12.0 installed: load_model(..., providers=...) and load_vad(model, *, providers=...) exist; default without providers is rt.get_available_providers() (minus Azure) — i.e. CoreML/CUDA would be picked automatically if present. Backend now passes ["CPUExecutionProvider"] for both. With CPU first, use_numpy_preprocessors stays True as before, so Linux behaviour is unchanged. ✓ tests/test_onnx_asr.py::test_cpu_provider_is_explicit_for_asr_and_vad.

Error preservation and hints (single and batch)

  • cuda_error_hint classification probed with real/likely CT2 texts: no kernel image… → incompatibility hint; Library libcublas.so.12 is not found or cannot be loaded → install hint; CUDA failed with error out of memory → OOM hint; CUDA failed with error unknown error, CUBLAS_STATUS_NOT_INITIALIZED → none; [WinError 126] Could not find module 'cublas64_12.dll' → install hint. Gap: CT2's unsupported-compute-type ValueError → none (OPUS-CUDA-1).
  • CLI paths: main() prints the hint in ValueError, FileNotFoundError and generic handlers; _run_batch per-file handler prints the hint and counts the failure; load-phase errors in batch propagate to main() once (no duplicate). Old unconditional Windows choco/winget Toolkit advice removed. ✓ tests/test_cli.py::test_explicit_cuda_errors_preserve_cause_without_fallback (30 parametrisations: CLI/TOML × load/single/batch × 5 messages).

Tests and lint

  • .venv/bin/python -m pytest -q: 500 passed, 2 skipped (skips: Windows OEM codec; Linux bootstrap integration because cuBLAS is no longer installed) — matches ADR-001's claim. Note: with this shell's FORCE_COLOR=3 one pre-existing test (test_cli_renders_runtime_warning_without_python_details) fails on ANSI codes; unrelated to the diff and passes with the variable unset.
  • uvx ruff check src tests: 34 issues at HEAD vs 38 at base — the change removes 4 (noqa: E402 no longer needed) and adds none. ruff format --check: same single pre-existing file at both revisions.

Documentation vs evidence

  • README, ADR-001/003/006, docs/gpu.md, PRD, spec: no remaining winget/choco/"ставится автоматически"/"auto → CUDA" statements (grep). ADR-001's corrected claim about the Windows cuBLAS wheel, the SHA-256 values, the import lists, the NVRTC note for 12.9.2.10, test counts and the "Windows run not performed" disclaimers all check out against the evidence above. ✓ docs/plan.md still describes the old nvidia-smi auto rule but is a historical planning document outside the issue's list.

Limits

  • No Windows machine: the Windows bootstrap (add_dll_directory + WinDLL preload) is verified by reading CPython/CT2 sources and Microsoft loader documentation plus the mocked unit tests, not by execution. The acceptance criterion "реальная транскрипция на Windows x64 с совместимым GPU, драйвером и extra, без Toolkit" remains outstanding, exactly as README/ADR-001 state.
  • No working CUDA on this Linux host (nvidia_uvm absent; loading it needs root, which is out of bounds). Linux CUDA was verified up to library resolution and CT2 initialisation failure with preserved cause; GPU inference and the cuBLAS/NVRTC runtime behaviour were not exercised.
  • Scope exclusions honoured: model-ownership refactor, thread budgeting, new execution providers, diarization, pre-existing platform resolution failures (win_arm64, macOS tag) and the pre-existing programmatic transcribe(device="auto") routing were not assessed.
Разбор замечаний

Triage of Opus review for issue #2

Original immutable report: opus-453b5d9-review.md
Original version: 453b5d9dfa
Resolution commit: a8b8a8c0c9
Triage owner and implementer: /root (original implementation context)
Disposition counters: FIXED 2; REJECTED 0; OUT_OF_SCOPE 0; NEEDS_DECISION 0.

OPUS-CUDA-1 | FIXED

Confirmed against the documented older-GPU scenario and the precise CT2
unsupported-compute-type message. A new CLI regression test failed before the
production change (float16/CUDA/load/from-CLI: expected a compatibility hint,
actual only the original ValueError text).

The fix adds an optional device context to cuda_error_hint. The CT2 message
about unsupported efficient computation is recognized only when device is
cuda. The main CLI error handlers receive resolved_device (initialized to None
before resolution so configuration errors remain safe); the batch handler
receives actual_device. The message describes an incompatible compute profile,
not an assertion that all execution on that GPU is impossible. It explains
that installing the extra does not add hardware support and suggests choosing
a supported compute type or an explicit CPU path. The original exception
message and strict-device policy remain intact.

Tests cover 24 combinations: CLI/TOML x load/single/batch x CUDA/CPU x
float16/int8, including absence of a CUDA-specific hint on the identical CPU
error, original-cause output, and absence of installation/fallback advice.
This changes only diagnostics, without a new exception hierarchy or execution
policy refactor. README now correctly promises hints only for recognized causes.

OPUS-CUDA-2 | FIXED

Confirmed the wrong patch target and additionally that the mocked get_backend
means this CLI test cannot exercise the real FasterWhisper bootstrap at all.
Changing only the patch target would preserve a vacuous assertion. Removed
that redundant patch/assertion instead. The CLI test continues to assert ONNX
selection and model/compute defaults with nvidia-smi present. Bootstrap
execution is covered separately by test_bootstrap_only_for_explicit_cuda,
which calls the real backend method and patches its actual lookup location.

Non-gating considerations

  • Independently confirmed nvrtc64_120_0.dll in strings of the audited cuBLAS Lt
    DLL. Added this limitation to ADR-001; no speculative extra dependency was
    introduced. Windows GPU validation remains outstanding and must exercise
    this path. No GPU inference claim was added.
  • Unknown CUDA causes remain unclassified intentionally; original cause is
    preserved. No heuristic generic-driver diagnosis was added.
  • Repeated per-file hints in batches and optional Python-version spelling in
    the installation hint remain non-gating UI details.
  • Runtime pins and their re-audit obligation remain the explicitly documented
    implementation tradeoff; no unrelated dependency work was introduced.

Checks actually run

  • Before the production fix: .venv/bin/python -m pytest tests/test_cli.py -q
    -k unsupported_compute_type_hint --maxfail=1 => expected regression failure.
  • After fix: .venv/bin/python -m pytest tests/test_cli.py
    tests/test_transcriber.py tests/test_backend_faster_whisper.py -q => 175 passed.
  • Full suite: .venv/bin/python -m pytest -q => 524 passed, 2 skipped in 2.77s.
    Skips remain Windows OEM encoding and absent optional Linux cuBLAS.
  • strings /tmp/ct2-windows-audit-ci15wlre/cublasLt64_12.dll |
    rg -m 3 nvrtc64_120_0.dll => matching dynamic-library name.
  • git diff --check => clean.

No source change outside the five files in the resolution commit. No merge,
push, tracker change, or real Windows GPU run.

Перепроверка a8b8a8c

Narrow re-check: OPUS-CUDA-1 and OPUS-CUDA-2 (issue #2)

  • Original report: /tmp/local-transcriber-opus-review/opus-453b5d9-review.md (version 453b5d9).
  • Resolutions: /tmp/local-transcriber-opus-review/triage-a8b8a8c.md (FIXED 2; REJECTED 0; OUT_OF_SCOPE 0; NEEDS_DECISION 0).
  • Re-checked version: a8b8a8c0c9c1689a7ed43fb974f74af3795fd54e, one commit on top of 453b5d9, five files: README.md, docs/adr/001-cuda-bootstrap.md, src/local_transcriber/cli.py, src/local_transcriber/transcriber.py, tests/test_cli.py.
  • Scope: only the two original findings and their adjacent diagnostic/test/doc changes. Packaging, binary audit, network reads, installs, model downloads and the outstanding Windows GPU validation were not repeated and are unchanged.
  • Repository and Git untouched (git status clean before and after); scratch copies under $CLAUDE_JOB_DIR/tmp.

Verdict

VERDICT: APPROVED

Status counters: CLOSED 2 (both PROPORTIONATE); NOT_CLOSED 0; REJECTION_VALID 0; REJECTION_INVALID 0; OUT_OF_SCOPE 0; NEEDS_DECISION 0; new findings 0.

Index

  • OPUS-CUDA-1 | FIXED → CLOSED, PROPORTIONATE | CT2 unsupported-compute-type refusal now gets a CUDA-only hint; original cause preserved; CPU negative cases hold.
  • OPUS-CUDA-2 | FIXED → CLOSED, PROPORTIONATE | vacuous bootstrap assertion removed; bootstrap coverage lives in the backend test that patches the real lookup site.

OPUS-CUDA-1 — CLOSED, PROPORTIONATE

Change (transcriber.py:198-206, cli.py:68-73, 351, 409-417, 763):

  • cuda_error_hint(exc, *, device=None) gains a keyword-only device context. A new first branch matches requested + compute type + do not support efficient only when device == "cuda" and returns a hint saying the compute type is incompatible with the selected GPU/CUDA runtime, that the extra adds no hardware support, and to pick a supported --compute-type or --device onnx / --device cpu. No install advice.
  • main() initialises resolved_device = None before the try, assigns it from detect_device(defaults["device"]) inside, and passes it to _print_cuda_hint in all three handlers (ValueError, FileNotFoundError, generic). _run_batch's per-file handler passes actual_device. Load-phase errors in batch propagate to main() as before (single hint, from resolved_device).

Evidence:

  • Direct probe of the function: the exact CT2 texts for float16 and int8 return a hint for device="cuda" and None for "cpu", "onnx", "openvino-cpu" and None. The pre-existing branches (missing library, no kernel image, OOM, unknown) are unchanged and still device-agnostic; CUDA failed with error unknown error still yields None, as designed.
  • CLI paths: with --device cuda the ValueError from WhisperModel is not a _is_cuda_error, so load_model re-raises it unchanged; main() catches it in except ValueError with resolved_device == "cuda" (CLI or TOML). Single-file transcribe-phase errors take the same route; batch transcribe-phase errors take the per-file handler with actual_device == "cuda" (strict, no fallback, so it cannot have drifted to cpu). Config-validation errors occur before resolved_device is assigned and get None → no CUDA-specific hint. All four CLI error handlers are covered by the new device argument; none was left on the old signature.
  • Tests: tests/test_cli.py::test_unsupported_compute_type_hint_requires_cuda_context — 24 parametrisations (CLI/TOML × load/single/batch × cuda/cpu × float16/int8). Asserts exit 1, original message present in output, hint present iff device is cuda, no install hint, no Переключение на CPU. Mandated selection -k 'unsupported_compute_type_hint or default_cli_stays_on_onnx': 25 passed. tests/test_backend_faster_whisper.py + tests/test_transcriber.py: 41 passed. Full suite: 524 passed, 2 skipped (matches the updated ADR-001 count; skips unchanged).
  • Regression-test teeth: the new test file run against a scratch copy of the 453b5d9 sources (PYTHONPATH to the old src, old cli.py) → 12 failed (all cuda cases), 12 passed (all cpu cases). So the test detects the original defect and does not depend on the fix for its CPU negatives.
  • Docs: README now says a hint is shown "для распознанных причин" (closes the doc-precision consideration); ADR-001 records the device-aware branch and the CPU non-hint rationale.
  • Lint/format: ruff check/format --check on the three code files show the same pre-existing issues before and after (line numbers shifted by one); no new issues. git diff --check clean.

Proportionality: the fix is the minimum shape that avoids mis-hinting CPU users — one keyword-only parameter, one guarded branch, threading the already-known device through four call sites. No exception hierarchy, no device-policy change, no change to strict/fallback semantics. My original prescription ("needs the resolved device") was necessary, not an over-ask: the probe confirms the same CT2 text on --device cpu --compute-type float16, and the 12 CPU parametrisations would fail with a text-only match. The hint's suggestion to "choose a supported --compute-type" can lead a Maxwell/Pascal user to float32, which then fails with no kernel image and receives the existing incompatibility hint — a two-step path, but every step is truthful and non-misleading; not a defect.

OPUS-CUDA-2 — CLOSED, PROPORTIONATE

Change (tests/test_cli.py:800-813): the patch("local_transcriber._cuda_bootstrap.ensure_cublas_loadable") context and bootstrap.assert_not_called() are removed from test_default_cli_stays_on_onnx_with_nvidia_driver. The test keeps get_backend.assert_called_once_with("onnx", compute_type_explicit=False) and the ONNX model/compute default assertion with nvidia-smi present.

Evidence: the remaining assertions are the ones that actually gate the auto → onnx policy (a regression to cuda fails get_backend.assert_called_once_with("onnx", …)). Bootstrap invocation is covered by tests/test_backend_faster_whisper.py::test_bootstrap_only_for_explicit_cuda, which calls the real create_model and patches local_transcriber.backends.faster_whisper.ensure_cublas_loadable (the import-time binding site) — passes. Test passes at a8b8a8c.

Proportionality: removing the vacuous assertion is smaller and more honest than my suggested re-targeting, which — as the triage notes — would still be vacuous because get_backend is mocked in that CLI test. The docstring no longer implies a bootstrap check. No overreach.

Non-gating considerations (carried, unchanged status)

  • ADR-001 now records the nvrtc64_120_0.dll dynamic reference in cuBLAS Lt and defers its assessment to the first real Windows run. Accurate; no speculative dependency added.
  • Unknown CUDA causes intentionally unclassified; per-file hint repetition in batch; --python 3.13 spelling in the install hint; exact runtime pins — all remain non-gating and were not re-judged.
  • Windows x64 GPU transcription without a Toolkit remains not performed; nothing in this commit claims otherwise.

Limits

No real CT2 execution of the new hint path (the temp tool env from the original review holds the 453b5d9 package and no installs were permitted); the CT2 message text used in the tests is the one reproduced from real CT2 4.8.1 during the original review, so the mocked coverage is anchored to the real string.

Независимое ревью Claude Opus: APPROVED после исправления двух замечаний. Первичное ревью — 23 мин 12 с, перепроверка в той же сессии — 3 мин 56 с. Ниже сохранены отчёты перед очисткой локальных временных файлов. <details> <summary>Первичное ревью 453b5d9</summary> # Independent review: optional CUDA on Linux and Windows (issue #2) - Reviewed version: branch `feat/optional-cuda-windows`, HEAD `453b5d9dfa38aa5487edf4e8ab2d3e74dfe1fce0`, diff since base `2b54f5b`. - Reviewer: Opus 5, independent (no shared context with the implementer). Repository and Git untouched; all experiments under `$CLAUDE_JOB_DIR/tmp` and `/tmp`. - Task reference: `/tmp/local-transcriber-opus-review/issue-2.md`; base spec `docs/specs/2026-09-21-transcription-execution.md` at `2b54f5b`. - Host: Linux x86_64, Quadro M3000M (CC 5.2, driver 580.178.04), `nvidia-smi` present, `nvidia_uvm` not loaded → CUDA runtime init fails (`cudaErrorUnknown`). A Windows x64 GPU machine is not available. Real GPU transcription was therefore **not** possible on any platform; see "Limits". ## Verdict `VERDICT: CHANGES_REQUESTED` Counters (scale from the review line: BLOCKER | MAJOR | MINOR | NIT): BLOCKER 0, MAJOR 0, MINOR 1, NIT 1. Total findings: 2. ## Findings ### OPUS-CUDA-1 | MINOR | `src/local_transcriber/transcriber.py:198-235` (`cuda_error_hint`), `src/local_transcriber/config.py:51` (`cuda` default `float16`) | criterion: issue #2 scope "различать … обнаруженную несовместимость GPU/runtime … давать подходящую подсказку"; acceptance "ошибки … несовместимого GPU различаются … включая пользовательский вывод"; spec §"Выбор исполнения и ошибки" ("при несовместимости железа объясняется отказ выбранного пути") - Failure scenario: user with a Maxwell/Pascal GPU (CC < 7.0 — the class this repo documents as incompatible, including the owner's Quadro M3000M) installs the extra and runs `transcribe f.wav --device cuda` with the default profile. CTranslate2 rejects the requested `float16` before any kernel runs and raises `ValueError("Requested float16 compute type, but the target device or backend do not support efficient float16 computation.")`. `cuda_error_hint` returns `None` (no `cuda`/`cublas`/`no kernel image` keyword), so the CLI prints only `Ошибка: Requested float16 compute type, …`. The user output does not say this is a GPU/runtime incompatibility, that the extra cannot fix it, or to switch to `--device onnx`/`cpu`. The same happens for `--compute-type int8` (CC ≤ 6.0). Only a second attempt with `--compute-type float32` reaches `no kernel image is available` and gets the intended hint. - Evidence: - CT2 4.8.1 source: `src/types.cc` `resolve_compute_type` → `unsupported_compute_type("float16")` throws `std::invalid_argument` when `!support_float16` and `enable_fallback == false` (default in `include/ctranslate2/types.h`); `mayiuse_float16(CUDA)` = `cuda::gpu_has_fp16_tensor_cores` (CC ≥ 7.0). pybind11 maps `invalid_argument` → `ValueError`. - Reproduced the exact message with real CT2 4.8.1 in a temp tool env (`create_model(path, "cpu", "float16")` on a CPU without fp16 uses the same code path): `ValueError | Requested float16 compute type, …`; `cuda_error_hint(e) is None`, `_is_cuda_error(e) is False`. - `docs/gpu.md` §"Совместимость GPU" already states "float16 требует CC ≥ 7.0, float32 падает с ошибкой" for this GPU class, so the case is known to the project, but neither the hint keywords nor `tests/test_cli.py::test_explicit_cuda_errors_preserve_cause_without_fallback` cover it. - Minimal verification: `cuda_error_hint(ValueError("Requested float16 compute type, but the target device or backend do not support efficient float16 computation."))` → expected an incompatibility hint, actual `None`. - Note on a fix: the hint function has no device context; the same CT2 text is produced for `--device cpu --compute-type float16`, so matching the text alone would mis-hint CPU users. Any fix needs the resolved device (or the CLI to only consult the hint when the requested device is `cuda`). Cost is small; risk removed is wrong/no guidance for the most common real-world "old GPU" first contact. ### OPUS-CUDA-2 | NIT | `tests/test_cli.py:800-821` (`test_default_cli_stays_on_onnx_with_nvidia_driver`) | criterion: testing claims must match evidence (brief FOCUS) - Failure scenario: the test patches `local_transcriber._cuda_bootstrap.ensure_cublas_loadable` and asserts `bootstrap.assert_not_called()`. `backends/faster_whisper.py` binds the name at import (`from local_transcriber._cuda_bootstrap import ensure_cublas_loadable`), so that patch can never observe a call from the backend; the assertion is vacuous. If `auto` regressed to `cuda` the test would still fail, but only through `get_backend.assert_called_once_with("onnx", …)`, not through the bootstrap assertion the docstring implies. - Evidence: `src/local_transcriber/backends/faster_whisper.py:12` (import-time binding); `tests/test_backend_faster_whisper.py::test_bootstrap_only_for_explicit_cuda` patches the correct target (`local_transcriber.backends.faster_whisper.ensure_cublas_loadable`). - Minimal verification: temporarily make `detect_device("auto")` return `"cuda"` in a scratch copy and run the test with `get_backend` un-patched for the assertion — the `bootstrap` mock still records zero calls. ## Non-gating considerations - Static audit gap for the outstanding Windows validation: `cublasLt64_12.dll` 12.9.1.4 (and Linux `libcublasLt.so.12` 12.9.1.4) contain a dynamic-load reference to NVRTC (`nvrtc64_120_0.dll` / `libnvrtc.so.12`) that ADR-001 does not record; PE/ELF imports do not include it, and Linux worked historically with 12.9.1.4 without an NVRTC package, so it is presumably optional for the Whisper GEMM path. NVIDIA made it a hard dependency in 12.9.2.10 (verified via PyPI JSON). The first real Windows run should watch for an NVRTC load failure. - `CUDA failed with error unknown error` (observed on the owner's own machine, `nvidia_uvm` not loaded) yields no hint at all by design ("unknown → no advice"). A one-line generic fallback (check driver / use `--device onnx`) would cost nothing; deferring is defensible. - In batch mode the multi-line install hint is printed once per failing file; with many files and a missing library this repeats the same 5 lines N times. Cosmetic. - `ctranslate2==4.8.1` is now an exact pin in the main dependencies. Justified by the DLL audit, but every CT2 bump (including security fixes) now requires the re-audit ADR-001 describes; `uv lock --upgrade` will not move it. - README §"Подключение CUDA" says explicit CUDA fails "с исходной причиной и подсказкой"; the hint is only emitted for recognised causes (ADR-001 says so). Minor doc precision; folds into OPUS-CUDA-1 if that is fixed. - `cuda_error_hint` text suggests `uv tool install --force '.[cuda]'` without `--python 3.13` while README uses it. Works (uv resolves an interpreter for `requires-python >=3.13`), but inconsistent. ## Checks performed and outcomes ### Packaging metadata and platform markers - Built the wheel from `git archive 453b5d9` in a temp dir (`uv build --wheel`). `METADATA`: no `nvidia-*` in unconditional `Requires-Dist`; `Provides-Extra: cuda`; `nvidia-cublas-cu12==12.9.1.4; ((linux and x86_64) or (win32 and (AMD64 or x86_64))) and extra == 'cuda'`; `ctranslate2==4.8.1` pinned. ✓ Same metadata as the author's `/tmp/local-transcriber-issue2-dist` wheel. - `uv pip compile --python-platform …` of the built wheel: `x86_64-unknown-linux-gnu` and `x86_64-pc-windows-msvc` base → no `nvidia-*`; with `[cuda]` → exactly `nvidia-cublas-cu12==12.9.1.4` added. `aarch64-unknown-linux-gnu` with `[cuda]` → no `nvidia-*` (marker excludes ARM). ✓ `aarch64-pc-windows-msvc` and `aarch64-apple-darwin` do not resolve at all (no `ctranslate2` win_arm64 wheel; `onnxruntime` macOS tag) — pre-existing, unrelated to this change. - `uv lock --check` passes; `uv sync --locked --dry-run` → "Would make no changes" (cuBLAS already absent); `uv sync --locked --extra cuda --dry-run` → "+ nvidia-cublas-cu12==12.9.1.4". ✓ Lock lists the win_amd64 cuBLAS wheel with hash matching PyPI. - `uv tool install --python 3.13 --force ".[cuda]"` from a clone copy into a temp `UV_TOOL_DIR`: installs `ctranslate2 4.8.1`, `nvidia-cublas-cu12 12.9.1.4`, `transcribe` entry point works. ✓ - `uv tool install --python 3.13 --force "local-transcriber[cuda] @ git+file:///…@feat/optional-cuda-windows"` into a second temp tool dir: extras applied, cuBLAS installed. ✓ (README's HTTPS URL not exercised; until merged, master has no `cuda` extra — transient.) ### Windows wheel runtime dependencies (static, primary artifacts) - `/tmp/ct2-windows-audit-ci15wlre/ct2.whl` SHA-256 `d52499f0…f827` and `cublas.whl` SHA-256 `1e5fee10…40bf` recomputed locally and matched against PyPI JSON for `ctranslate2 4.8.1` (cp313 win_amd64) and `nvidia-cublas-cu12 12.9.1.4` (win_amd64). ✓ - `objdump -p ctranslate2.dll` imports: `libiomp5md.dll`, `KERNEL32`, `MSVCP140`, `VCRUNTIME140`, `VCRUNTIME140_1`, UCRT api-sets — no static cuBLAS/cuDNN/cudart import. Strings: `cublas64_12.dll`, `nvcuda.dll`; zero `cudnn` strings (ASCII and UTF-16). `_ext.cp313-win_amd64.pyd` imports `ctranslate2.dll`, `python313.dll`, MSVC runtime. `cudnn64_9.dll` in the wheel is a 266 KB shim importing only KERNEL32; nothing references it. ✓ ADR-001 statements match (VC++ runtime required; cuDNN not used; Conv1D is im2col+GEMM — symbols `im2col_transposed_kernel` present). - CT2 4.8.1 `src/cuda/cublas_stub.cc`: cuBLAS is loaded lazily via `LoadLibraryA("cublas64_12.dll")` (Windows) / `dlopen("libcublas.so.12")` (Linux); failure text `Library <name> is not found or cannot be loaded`. `python/ctranslate2/__init__.py` adds only its own package dir via `os.add_dll_directory` and `ctypes.CDLL`s every DLL in it. ✓ Matches ADR-001 and the hint keyword `not found`/`cannot be loaded`. - cuBLAS wheel layout `nvidia/cublas/bin/{cublas64_12,cublasLt64_12,nvblas64_12}.dll`, no `__init__.py` → PEP 420 namespace import `nvidia.cublas` works; `__path__` points at the package dir. `cublas64_12.dll` imports `cublasLt64_12.dll` + KERNEL32; `cublasLt64_12.dll` imports KERNEL32 only; `nvblas64_12.dll` imports `cublas64_12.dll` (unused). ✓ Additional dynamic string `nvrtc64_120_0.dll` in cublasLt — see considerations. - Microsoft "Dynamic-Link Library Search Order" (fetched): the loader first consults the loaded-module list by base name; `AddDllDirectory` directories are only searched when `LOAD_LIBRARY_SEARCH_*` flags are used (per call or via `SetDefaultDllDirectories`). CPython `ctypes/__init__.py` `_load_library`: default `winmode = LOAD_LIBRARY_SEARCH_DEFAULT_DIRS`, plus `LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR` when a path is given. Conclusion: the bootstrap's full-path `WinDLL` preload (Lt first, then cuBLAS) is what makes CT2's plain `LoadLibraryA` succeed; `os.add_dll_directory` alone would not. The code comment states this correctly. ✓ (Reasoned from primary docs; not executed on Windows.) ### Runtime library loading (Linux, real libraries) - In the temp tool env with the extra: `is_cublas_available()` False → `ensure_cublas_loadable()` → True (libcublas resolvable by soname via `RTLD_GLOBAL` preload). ✓ - Bootstrap is only called for `device == "cuda"` in `FasterWhisperBackend.create_model`, before the local `from faster_whisper import WhisperModel`; module import of the backend no longer triggers it. ✓ (`tests/test_backend_faster_whisper.py::test_bootstrap_only_for_explicit_cuda`) - Without optional package: `import nvidia.cublas` → `ImportError` guard; Windows and Linux both return silently. ✓ Windows DLL load failure propagates (`OSError`), keys stored so retries are idempotent (`tests/test_cuda_bootstrap.py`, both branches). - Real CLI, explicit CPU with the extra installed: `transcribe tone.wav --device cpu --model tiny` → transcript written, `Устройство: cpu Compute: float32`. ✓ (tiny model downloaded to a temp `HF_HOME`; no download in the ordinary test suite.) ### Automatic / explicit device policy - `detect_device("auto")` → `"onnx"` unconditionally; `shutil` import removed from `utils.py`. ✓ - Real CLI on this host (`nvidia-smi` present, extra installed, `HF_HUB_OFFLINE=1`): default run prints `Инициализирую модель на onnx...` and fails on the offline model fetch with no CUDA attempt and no FasterWhisper CPU fallback. ✓ - Explicit `--device cuda` (CLI) and `device = "cuda"` in `.transcriber.toml` (batch of 2) with real CT2 on this host: `Инициализирую модель на cuda...` → `Ошибка: CUDA failed with error unknown error`, exit 1, no `Переключение на CPU`, `--verbose` traceback shows `faster_whisper.py:98` → `WhisperModel.__init__`. ✓ Original cause preserved; no hint (unknown cause, by design). - `strict = requested_device != "auto"` in both `_run_single` and `_run_batch`; TOML device counts as explicit. ✓ ### ONNX ASR / VAD providers - `onnx_asr 0.12.0` installed: `load_model(..., providers=...)` and `load_vad(model, *, providers=...)` exist; default without `providers` is `rt.get_available_providers()` (minus Azure) — i.e. CoreML/CUDA would be picked automatically if present. Backend now passes `["CPUExecutionProvider"]` for both. With CPU first, `use_numpy_preprocessors` stays True as before, so Linux behaviour is unchanged. ✓ `tests/test_onnx_asr.py::test_cpu_provider_is_explicit_for_asr_and_vad`. ### Error preservation and hints (single and batch) - `cuda_error_hint` classification probed with real/likely CT2 texts: `no kernel image…` → incompatibility hint; `Library libcublas.so.12 is not found or cannot be loaded` → install hint; `CUDA failed with error out of memory` → OOM hint; `CUDA failed with error unknown error`, `CUBLAS_STATUS_NOT_INITIALIZED` → none; `[WinError 126] Could not find module 'cublas64_12.dll'` → install hint. Gap: CT2's unsupported-compute-type `ValueError` → none (OPUS-CUDA-1). - CLI paths: `main()` prints the hint in `ValueError`, `FileNotFoundError` and generic handlers; `_run_batch` per-file handler prints the hint and counts the failure; load-phase errors in batch propagate to `main()` once (no duplicate). Old unconditional Windows `choco/winget` Toolkit advice removed. ✓ `tests/test_cli.py::test_explicit_cuda_errors_preserve_cause_without_fallback` (30 parametrisations: CLI/TOML × load/single/batch × 5 messages). ### Tests and lint - `.venv/bin/python -m pytest -q`: **500 passed, 2 skipped** (skips: Windows OEM codec; Linux bootstrap integration because cuBLAS is no longer installed) — matches ADR-001's claim. Note: with this shell's `FORCE_COLOR=3` one pre-existing test (`test_cli_renders_runtime_warning_without_python_details`) fails on ANSI codes; unrelated to the diff and passes with the variable unset. - `uvx ruff check src tests`: 34 issues at HEAD vs 38 at base — the change removes 4 (`noqa: E402` no longer needed) and adds none. `ruff format --check`: same single pre-existing file at both revisions. ### Documentation vs evidence - README, ADR-001/003/006, `docs/gpu.md`, PRD, spec: no remaining `winget`/`choco`/"ставится автоматически"/"auto → CUDA" statements (grep). ADR-001's corrected claim about the Windows cuBLAS wheel, the SHA-256 values, the import lists, the NVRTC note for 12.9.2.10, test counts and the "Windows run not performed" disclaimers all check out against the evidence above. ✓ `docs/plan.md` still describes the old `nvidia-smi` auto rule but is a historical planning document outside the issue's list. ## Limits - No Windows machine: the Windows bootstrap (`add_dll_directory` + `WinDLL` preload) is verified by reading CPython/CT2 sources and Microsoft loader documentation plus the mocked unit tests, not by execution. The acceptance criterion "реальная транскрипция на Windows x64 с совместимым GPU, драйвером и extra, без Toolkit" remains **outstanding**, exactly as README/ADR-001 state. - No working CUDA on this Linux host (`nvidia_uvm` absent; loading it needs root, which is out of bounds). Linux CUDA was verified up to library resolution and CT2 initialisation failure with preserved cause; GPU inference and the cuBLAS/NVRTC runtime behaviour were not exercised. - Scope exclusions honoured: model-ownership refactor, thread budgeting, new execution providers, diarization, pre-existing platform resolution failures (win_arm64, macOS tag) and the pre-existing programmatic `transcribe(device="auto")` routing were not assessed. </details> <details> <summary>Разбор замечаний</summary> # Triage of Opus review for issue #2 Original immutable report: opus-453b5d9-review.md Original version: 453b5d9dfa38aa5487edf4e8ab2d3e74dfe1fce0 Resolution commit: a8b8a8c0c9c1689a7ed43fb974f74af3795fd54e Triage owner and implementer: /root (original implementation context) Disposition counters: FIXED 2; REJECTED 0; OUT_OF_SCOPE 0; NEEDS_DECISION 0. ## OPUS-CUDA-1 | FIXED Confirmed against the documented older-GPU scenario and the precise CT2 unsupported-compute-type message. A new CLI regression test failed before the production change (float16/CUDA/load/from-CLI: expected a compatibility hint, actual only the original ValueError text). The fix adds an optional device context to cuda_error_hint. The CT2 message about unsupported efficient computation is recognized only when device is cuda. The main CLI error handlers receive resolved_device (initialized to None before resolution so configuration errors remain safe); the batch handler receives actual_device. The message describes an incompatible compute profile, not an assertion that all execution on that GPU is impossible. It explains that installing the extra does not add hardware support and suggests choosing a supported compute type or an explicit CPU path. The original exception message and strict-device policy remain intact. Tests cover 24 combinations: CLI/TOML x load/single/batch x CUDA/CPU x float16/int8, including absence of a CUDA-specific hint on the identical CPU error, original-cause output, and absence of installation/fallback advice. This changes only diagnostics, without a new exception hierarchy or execution policy refactor. README now correctly promises hints only for recognized causes. ## OPUS-CUDA-2 | FIXED Confirmed the wrong patch target and additionally that the mocked get_backend means this CLI test cannot exercise the real FasterWhisper bootstrap at all. Changing only the patch target would preserve a vacuous assertion. Removed that redundant patch/assertion instead. The CLI test continues to assert ONNX selection and model/compute defaults with nvidia-smi present. Bootstrap execution is covered separately by test_bootstrap_only_for_explicit_cuda, which calls the real backend method and patches its actual lookup location. ## Non-gating considerations - Independently confirmed nvrtc64_120_0.dll in strings of the audited cuBLAS Lt DLL. Added this limitation to ADR-001; no speculative extra dependency was introduced. Windows GPU validation remains outstanding and must exercise this path. No GPU inference claim was added. - Unknown CUDA causes remain unclassified intentionally; original cause is preserved. No heuristic generic-driver diagnosis was added. - Repeated per-file hints in batches and optional Python-version spelling in the installation hint remain non-gating UI details. - Runtime pins and their re-audit obligation remain the explicitly documented implementation tradeoff; no unrelated dependency work was introduced. ## Checks actually run - Before the production fix: .venv/bin/python -m pytest tests/test_cli.py -q -k unsupported_compute_type_hint --maxfail=1 => expected regression failure. - After fix: .venv/bin/python -m pytest tests/test_cli.py tests/test_transcriber.py tests/test_backend_faster_whisper.py -q => 175 passed. - Full suite: .venv/bin/python -m pytest -q => 524 passed, 2 skipped in 2.77s. Skips remain Windows OEM encoding and absent optional Linux cuBLAS. - strings /tmp/ct2-windows-audit-ci15wlre/cublasLt64_12.dll | rg -m 3 nvrtc64_120_0.dll => matching dynamic-library name. - git diff --check => clean. No source change outside the five files in the resolution commit. No merge, push, tracker change, or real Windows GPU run. </details> <details> <summary>Перепроверка a8b8a8c</summary> # Narrow re-check: OPUS-CUDA-1 and OPUS-CUDA-2 (issue #2) - Original report: `/tmp/local-transcriber-opus-review/opus-453b5d9-review.md` (version `453b5d9`). - Resolutions: `/tmp/local-transcriber-opus-review/triage-a8b8a8c.md` (FIXED 2; REJECTED 0; OUT_OF_SCOPE 0; NEEDS_DECISION 0). - Re-checked version: `a8b8a8c0c9c1689a7ed43fb974f74af3795fd54e`, one commit on top of `453b5d9`, five files: `README.md`, `docs/adr/001-cuda-bootstrap.md`, `src/local_transcriber/cli.py`, `src/local_transcriber/transcriber.py`, `tests/test_cli.py`. - Scope: only the two original findings and their adjacent diagnostic/test/doc changes. Packaging, binary audit, network reads, installs, model downloads and the outstanding Windows GPU validation were not repeated and are unchanged. - Repository and Git untouched (`git status` clean before and after); scratch copies under `$CLAUDE_JOB_DIR/tmp`. ## Verdict `VERDICT: APPROVED` Status counters: CLOSED 2 (both PROPORTIONATE); NOT_CLOSED 0; REJECTION_VALID 0; REJECTION_INVALID 0; OUT_OF_SCOPE 0; NEEDS_DECISION 0; new findings 0. ## Index - OPUS-CUDA-1 | FIXED → `CLOSED`, `PROPORTIONATE` | CT2 unsupported-compute-type refusal now gets a CUDA-only hint; original cause preserved; CPU negative cases hold. - OPUS-CUDA-2 | FIXED → `CLOSED`, `PROPORTIONATE` | vacuous bootstrap assertion removed; bootstrap coverage lives in the backend test that patches the real lookup site. ## OPUS-CUDA-1 — CLOSED, PROPORTIONATE Change (`transcriber.py:198-206`, `cli.py:68-73, 351, 409-417, 763`): - `cuda_error_hint(exc, *, device=None)` gains a keyword-only device context. A new first branch matches `requested ` + `compute type` + `do not support efficient` **only when `device == "cuda"`** and returns a hint saying the compute type is incompatible with the selected GPU/CUDA runtime, that the extra adds no hardware support, and to pick a supported `--compute-type` or `--device onnx` / `--device cpu`. No install advice. - `main()` initialises `resolved_device = None` before the `try`, assigns it from `detect_device(defaults["device"])` inside, and passes it to `_print_cuda_hint` in all three handlers (`ValueError`, `FileNotFoundError`, generic). `_run_batch`'s per-file handler passes `actual_device`. Load-phase errors in batch propagate to `main()` as before (single hint, from `resolved_device`). Evidence: - Direct probe of the function: the exact CT2 texts for `float16` and `int8` return a hint for `device="cuda"` and `None` for `"cpu"`, `"onnx"`, `"openvino-cpu"` and `None`. The pre-existing branches (missing library, `no kernel image`, OOM, unknown) are unchanged and still device-agnostic; `CUDA failed with error unknown error` still yields `None`, as designed. - CLI paths: with `--device cuda` the `ValueError` from `WhisperModel` is not a `_is_cuda_error`, so `load_model` re-raises it unchanged; `main()` catches it in `except ValueError` with `resolved_device == "cuda"` (CLI or TOML). Single-file transcribe-phase errors take the same route; batch transcribe-phase errors take the per-file handler with `actual_device == "cuda"` (strict, no fallback, so it cannot have drifted to `cpu`). Config-validation errors occur before `resolved_device` is assigned and get `None` → no CUDA-specific hint. All four CLI error handlers are covered by the new device argument; none was left on the old signature. - Tests: `tests/test_cli.py::test_unsupported_compute_type_hint_requires_cuda_context` — 24 parametrisations (CLI/TOML × load/single/batch × cuda/cpu × float16/int8). Asserts exit 1, original message present in output, hint present **iff** device is `cuda`, no install hint, no `Переключение на CPU`. Mandated selection `-k 'unsupported_compute_type_hint or default_cli_stays_on_onnx'`: 25 passed. `tests/test_backend_faster_whisper.py` + `tests/test_transcriber.py`: 41 passed. Full suite: **524 passed, 2 skipped** (matches the updated ADR-001 count; skips unchanged). - Regression-test teeth: the new test file run against a scratch copy of the `453b5d9` sources (`PYTHONPATH` to the old `src`, old `cli.py`) → 12 failed (all `cuda` cases), 12 passed (all `cpu` cases). So the test detects the original defect and does not depend on the fix for its CPU negatives. - Docs: README now says a hint is shown "для распознанных причин" (closes the doc-precision consideration); ADR-001 records the device-aware branch and the CPU non-hint rationale. - Lint/format: `ruff check`/`format --check` on the three code files show the same pre-existing issues before and after (line numbers shifted by one); no new issues. `git diff --check` clean. Proportionality: the fix is the minimum shape that avoids mis-hinting CPU users — one keyword-only parameter, one guarded branch, threading the already-known device through four call sites. No exception hierarchy, no device-policy change, no change to strict/fallback semantics. My original prescription ("needs the resolved device") was necessary, not an over-ask: the probe confirms the same CT2 text on `--device cpu --compute-type float16`, and the 12 CPU parametrisations would fail with a text-only match. The hint's suggestion to "choose a supported `--compute-type`" can lead a Maxwell/Pascal user to `float32`, which then fails with `no kernel image` and receives the existing incompatibility hint — a two-step path, but every step is truthful and non-misleading; not a defect. ## OPUS-CUDA-2 — CLOSED, PROPORTIONATE Change (`tests/test_cli.py:800-813`): the `patch("local_transcriber._cuda_bootstrap.ensure_cublas_loadable")` context and `bootstrap.assert_not_called()` are removed from `test_default_cli_stays_on_onnx_with_nvidia_driver`. The test keeps `get_backend.assert_called_once_with("onnx", compute_type_explicit=False)` and the ONNX model/compute default assertion with `nvidia-smi` present. Evidence: the remaining assertions are the ones that actually gate the `auto → onnx` policy (a regression to `cuda` fails `get_backend.assert_called_once_with("onnx", …)`). Bootstrap invocation is covered by `tests/test_backend_faster_whisper.py::test_bootstrap_only_for_explicit_cuda`, which calls the real `create_model` and patches `local_transcriber.backends.faster_whisper.ensure_cublas_loadable` (the import-time binding site) — passes. Test passes at `a8b8a8c`. Proportionality: removing the vacuous assertion is smaller and more honest than my suggested re-targeting, which — as the triage notes — would still be vacuous because `get_backend` is mocked in that CLI test. The docstring no longer implies a bootstrap check. No overreach. ## Non-gating considerations (carried, unchanged status) - ADR-001 now records the `nvrtc64_120_0.dll` dynamic reference in cuBLAS Lt and defers its assessment to the first real Windows run. Accurate; no speculative dependency added. - Unknown CUDA causes intentionally unclassified; per-file hint repetition in batch; `--python 3.13` spelling in the install hint; exact runtime pins — all remain non-gating and were not re-judged. - Windows x64 GPU transcription without a Toolkit remains **not performed**; nothing in this commit claims otherwise. ## Limits No real CT2 execution of the new hint path (the temp tool env from the original review holds the `453b5d9` package and no installs were permitted); the CT2 message text used in the tests is the one reproduced from real CT2 4.8.1 during the original review, so the mocked coverage is anchored to the real string. </details>
ddmitry merged commit 343a31b35b into master 2026-09-21 23:31:31 +03:00
ddmitry deleted branch feat/optional-cuda-windows 2026-09-21 23:31:31 +03:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
needs-info
Ожидает дополнительной информации от автора
 needs-triage
Требует оценки сопровождающим
 ready-for-agent
Полностью описана и готова для автономного агента
 ready-for-human
Требует реализации человеком
 wayfinder:grilling
Тикет карты: решение через разговор по одному вопросу (HITL)
 wayfinder:map
Карта wayfinder: индекс решений и точка входа
 wayfinder:prototype
Тикет карты: дешёвый черновик, чтобы было на что реагировать (HITL)
 wayfinder:research
Тикет карты: чтение источников, итог — markdown-сводка (AFK)
 wayfinder:task
Тикет карты: ручная работа, разблокирующая решение
 wontfix
Выполняться не будет
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
ddmitry (Dmitry Dementiev)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: ddmitry/local-transcriber#57