Update Airflow configuration to integrate with ClickHouse DWH instead of
PostgreSQL training database. Changes include:
- Switch Airflow dependencies from PostgreSQL to ClickHouse connector
- Update docker-compose to use ClickHouse connection and correct Dockerfile
- Refactor airflow/requirements.txt to include only essential packages
- Add DAGs directory for ETL pipeline orchestration
- Update documentation to reflect Airflow integration and access credentials
- Adjust service dependencies to wait for ClickHouse startup
Add Apache Airflow infrastructure with webserver, scheduler, and metadata
database to enable DAG-based pipeline orchestration. Includes optimized
requirements file and Docker configuration for Airflow 2.9.3.
ddadmin