feat(monitoring): add Airflow monitoring via statsd-exporter

- Add statsd-exporter service to docker-compose.yml (prom/statsd-exporter:v0.27.1)
- Add StatsD env vars to airflow-default-env for metrics export
- Add airflow job to prometheus.yml scrape configs
- Add Airflow Overview dashboard (Grafana provisioning)
- Add Airflow alert rules: scheduler down, queue backlog, failures, parse time
- Add configs/statsd_mapping.yml for StatsD → Prometheus conversion
- Use Prometheus naming convention (_total for counters, _seconds for timers)
- Add monitoring plan at plans/monitoring_airflow_plan.md
- Update OPERATIONS.md and Makefile for airflow monitoring

Tested: all 3 jobs (airflow, clickhouse, kafka) showing UP in Prometheus,
metrics flowing (dagbag_size=3, executor slots, heartbeats with _total suffix),
all 4 alert rules loaded in Grafana
This commit is contained in:
2026-02-08 22:27:32 +03:00
parent 8e31b06241
commit 4917a9a6ad
8 changed files with 1701 additions and 8 deletions
+23
View File
@@ -5,6 +5,11 @@ x-airflow-env: &airflow-default-env
AIRFLOW__WEBSERVER__SECRET_KEY: ${AIRFLOW_SECRET_KEY:-replace-me-with-random-string}
# ClickHouse connection для ETL
AIRFLOW_CONN_CLICKHOUSE_DEFAULT: clickhouse://default:123456@clickhouse:9000/default
# StatsD для мониторинга (метрики → statsd-exporter → Prometheus)
AIRFLOW__METRICS__STATSD_ON: "True"
AIRFLOW__METRICS__STATSD_HOST: "statsd-exporter"
AIRFLOW__METRICS__STATSD_PORT: "8125"
AIRFLOW__METRICS__STATSD_PREFIX: "airflow"
services:
@@ -242,6 +247,24 @@ services:
- kafka
restart: unless-stopped
# StatsD Exporter для Airflow метрик (StatsD → Prometheus)
statsd-exporter:
image: prom/statsd-exporter:v0.27.1
command: [
"--statsd.listen-udp=:8125",
"--statsd.listen-tcp=",
"--web.listen-address=:9102",
"--statsd.mapping-config=/tmp/statsd_mapping.yml"
]
ports:
- "9102:9102" # Prometheus metrics endpoint
- "8125:8125/udp" # StatsD receive port
volumes:
- ./configs/statsd_mapping.yml:/tmp/statsd_mapping.yml:ro
networks:
- cs_dwh
restart: unless-stopped
networks:
cs_dwh:
driver: bridge