Commit Graph
3 Commits
Author SHA1 Message Date
ddadmin d7cff5ad1e docs: add russian comments to pipeline files 2026-02-06 22:45:39 +03:00
ddadmin e44b988d76 feat(data): add error handling for ODS layer and improve partial data support
Add materialized views to capture parsing errors from browser, location,
device, and geo raw staging tables and route them to dedicated error
tables in the ODS layer. Refactor DDS refresh logic to handle partial
data arrivals where device and geo events may arrive independently by
using a unified click_id source with LEFT JOINs. Add TRUNCATE command
to prevent duplicate data accumulation in DQ summary table.
2026-02-06 22:10:51 +03:00
ddadmin 6bbb26b9b3 feat(infra): implement batch transformation layer and comprehensive documentation
Add batch ETL pipeline with ODS→DDS→DM transformation jobs and scripts.
Create DDL infrastructure with automated database schema application.
Update Makefile with transform target for executing batch processes.
Rewrite README with complete Russian documentation including architecture
diagrams, quick start guide, and data flow visualization.
2026-02-06 21:58:17 +03:00