ddadmin
d33cdb3fb0
Add Apache Airflow infrastructure with webserver, scheduler, and metadata database to enable DAG-based pipeline orchestration. Includes optimized requirements file and Docker configuration for Airflow 2.9.3.
15 lines
443 B
Plaintext
15 lines
443 B
Plaintext
# Optimized requirements for Airflow Docker setup
|
|
# Only includes packages actually used in DAGs
|
|
|
|
# Core database connector
|
|
psycopg2-binary==2.9.9
|
|
|
|
# Data processing (used in data_processing_dag.py and file_operations_dag.py)
|
|
pandas==2.1.4
|
|
|
|
# Data generation library (used in file_operations_dag.py)
|
|
mimesis==15.1.0
|
|
|
|
# Airflow PostgreSQL provider (used in sql_basic_dag.py and data_processing_dag.py)
|
|
apache-airflow-providers-postgres==5.11.1
|