ddadmin
fe9c15c0fe
Update Airflow configuration to integrate with ClickHouse DWH instead of PostgreSQL training database. Changes include: - Switch Airflow dependencies from PostgreSQL to ClickHouse connector - Update docker-compose to use ClickHouse connection and correct Dockerfile - Refactor airflow/requirements.txt to include only essential packages - Add DAGs directory for ETL pipeline orchestration - Update documentation to reflect Airflow integration and access credentials - Adjust service dependencies to wait for ClickHouse startup
24 lines
649 B
Docker
24 lines
649 B
Docker
# Use the official Airflow image as base
|
|
FROM apache/airflow:2.9.3
|
|
|
|
# Set environment variables
|
|
ENV AIRFLOW_HOME=/opt/airflow
|
|
|
|
# Copy the project requirements file
|
|
COPY --chown=airflow:0 airflow/requirements.txt ${AIRFLOW_HOME}/requirements.txt
|
|
|
|
# Install the required Python packages
|
|
RUN pip install --no-cache-dir -r ${AIRFLOW_HOME}/requirements.txt
|
|
|
|
# Create necessary directories
|
|
RUN mkdir -p /opt/airflow/dags /opt/airflow/logs /opt/airflow/config /opt/airflow/data
|
|
|
|
# Set working directory
|
|
WORKDIR ${AIRFLOW_HOME}
|
|
|
|
# Expose the webserver port
|
|
EXPOSE 8080
|
|
|
|
# The image will use Airflow's default entrypoint
|
|
# Commands will be passed at runtime
|