ddmitry/clickstream-ch-kafka-superset-demo
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(airflow): configure ClickHouse connection and update infrastructure

Browse Source 
Update Airflow configuration to integrate with ClickHouse DWH instead of
PostgreSQL training database. Changes include:

- Switch Airflow dependencies from PostgreSQL to ClickHouse connector
- Update docker-compose to use ClickHouse connection and correct Dockerfile
- Refactor airflow/requirements.txt to include only essential packages
- Add DAGs directory for ETL pipeline orchestration
- Update documentation to reflect Airflow integration and access credentials
- Adjust service dependencies to wait for ClickHouse startup
This commit is contained in:
ddadmin
2026-02-06 23:34:45 +03:00
parent d33cdb3fb0
commit fe9c15c0fe
9 changed files with 102 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Dockerfile.airflow
+1 -1
View File
@@ -5,7 +5,7 @@ FROM apache/airflow:2.9.3
ENV AIRFLOW_HOME=/opt/airflow
# Copy the project requirements file
COPY --chown=airflow:0 requirements.txt ${AIRFLOW_HOME}/requirements.txt
COPY --chown=airflow:0 airflow/requirements.txt ${AIRFLOW_HOME}/requirements.txt
# Install the required Python packages
RUN pip install --no-cache-dir -r ${AIRFLOW_HOME}/requirements.txt