feat(data): add error handling for ODS layer and improve partial data support

Add materialized views to capture parsing errors from browser, location,
device, and geo raw staging tables and route them to dedicated error
tables in the ODS layer. Refactor DDS refresh logic to handle partial
data arrivals where device and geo events may arrive independently by
using a unified click_id source with LEFT JOINs. Add TRUNCATE command
to prevent duplicate data accumulation in DQ summary table.
This commit is contained in:
2026-02-06 22:10:51 +03:00
parent 6bbb26b9b3
commit e44b988d76
3 changed files with 128 additions and 6 deletions
+2
View File
@@ -40,6 +40,8 @@ PARTITION BY toYYYYMM(check_date)
ORDER BY (check_date, layer, table_name, check_name);
-- Truncate and refill DQ summary
TRUNCATE TABLE dm.dq_summary;
INSERT INTO dm.dq_summary
SELECT
today() AS check_date,