- Зачем:
- развести документацию по времени жизни: транзиентные спеки отдельно от
долговечных решений (ADR) и доменного словаря (CONTEXT.md), вместо одного
громоздкого spec-документа.
- Что:
- добавлен блок Agent skills в AGENTS.md (issue tracker / triage / domain docs).
- созданы docs/agents/{issue-tracker,triage-labels,domain}.md: локальный
markdown-трекер в .scratch/, дефолтные triage-метки, single-context раскладка.
- зафиксировано решение как docs/adr/0001-spec-adr-issue-layout.md.
- Проверка:
- git show --stat HEAD; прочитать AGENTS.md и docs/adr/0001-spec-adr-issue-layout.md.
- Зачем:
- превратить стенд в самостоятельный учебный материал (трек «со звёздочкой») для продвинутых менти.
- Что:
- docs/course/: PRD, LEARNING_PLAN, LESSON_STANDARD и README-индекс.
- AGENTS.md: ссылка на курс в разделе навигации.
- CLAUDE.md: @-include AGENTS.md для контекста агента.
- Проверка:
- открыть docs/course/README.md и пройти по ссылкам на PRD/план/стандарт.
- Why:\n - AGENTS.md became too large and mixed policy with operational details\n - context7 requirement was easy to miss in long text\n- What:\n - reduce AGENTS.md to a compact contributor contract\n - add explicit mandatory MCP Context7 workflow block\n - move runbook details to docs/OPERATIONS.md\n - move artifact map to docs/REPO_MAP.md\n- Check:\n - reviewed links and content after split\n - ensured only documentation files are included in commit
- Why:
- keep Airflow artifacts under a single airflow/ directory
- align repository layout with intended project structure
- What:
- move dags/ to airflow/dags/ and update compose mounts
- make SQL root resolution work in container and local runs
- update DAG path references in README, AGENTS, ARCHITECTURE, and plans
- remove tracked Python cache artifacts from old DAG location
- Check:
- airflow dags list
- airflow dags list-import-errors
- e2e success: ddl_init, kafka_load(limit=50), etl_pipeline
- Why:
- For DE task we only need full ingest or limit-based sample.
- load_* and full_load params were redundant and unclear in current flow.
- What:
- Remove full_load and load_* params from kafka_load DAG contract.
- Simplify kafka helpers (validate/check files) to fixed 4-stream ingest.
- Sync AGENTS, README, runbook, architecture and airflow plan docs.
- Check:
- python3 -m py_compile dags/kafka_load_dag.py dags/utils/kafka_helpers.py
- Airflow smoke/full runs: ddl_init -> kafka_load -> etl_pipeline (all success).
- Legacy path: make data && make transform (success).
- Зачем:
- унифицировать стиль коммитов для всех участников проекта
- Что сделано:
- добавлен документ docs/COMMIT_RULES.md с форматом и примерами
- добавлена ссылка на правила в AGENTS.md
- Проверка:
- проверен staged diff перед коммитом
Move DDL files from flat ddl/ directory to sql/ddl/ with layer-based
subdirectories (stg, ods, dds, dm). Move batch transformation SQL from
jobs/ to sql/ layer directories. Update scripts and documentation to
reflect new paths for improved organization and Airflow integration.
Update Airflow configuration to integrate with ClickHouse DWH instead of
PostgreSQL training database. Changes include:
- Switch Airflow dependencies from PostgreSQL to ClickHouse connector
- Update docker-compose to use ClickHouse connection and correct Dockerfile
- Refactor airflow/requirements.txt to include only essential packages
- Add DAGs directory for ETL pipeline orchestration
- Update documentation to reflect Airflow integration and access credentials
- Adjust service dependencies to wait for ClickHouse startup
ddadmin