Airflow Security: The DAG That Leaked DB Credentials

Airflow holds the keys to your kingdom. Every warehouse password, API token, and cloud credential flows through its connections, and one leaked DAG file hands attackers the whole map.

One team learned this when a database password sat in plain text inside a DAG. It shipped to git, got copied into three branches, and worked perfectly until the day it didn't. The cleanup took longer than the breach.

You'll lock this down with Fernet rotation, a real secrets backend, and RBAC that gives each team the smallest key that works. Least privilege. Zero drama.

The Threat Model for a Workflow Platform

Airflow is a credential concentrator. Connections hold warehouse passwords, cloud keys, and API tokens, Variables hold the rest, and every task execution touches them. Your threat model starts there, not at the login page.

Attackers don't need exploits when secrets sit in git. A read-only repo viewer, a forgotten fork, or a DB backup without its key discipline each hand over the kingdom. You'll defend all three surfaces or you'll defend none.

Scope the model per environment. Prod gets unique keys, SSO-backed roles, and backend-managed secrets. Dev gets the same shapes with throwaway values so nobody learns bad habits from a lax playground.

Fernet: Encrypt, Store, and Rotate the Key

Fernet encrypts connection passwords and sensitive Variables at rest in the metadata DB. It protects backups and disk images, not runtime memory or git history. Know what it covers before trusting it.

The key lives in AIRFLOW__CORE__FERNET_KEY and must be unique per environment. The documented example key is public knowledge, so shipping it means encrypting with a password printed in the manual. Generate a real one and store it in your secret manager.

Rotate append-first. New key decrypts first while old ciphertext stays readable, you re-save connections to re-encrypt them, then you drop the old key. You'll practice on staging because the one-step shortcut orphans every secret at once.

Store the key like it matters. AIRFLOW__CORE__FERNET_KEY overrides airflow.cfg, and AIRFLOW__CORE__FERNET_KEY_CMD reads it from a file or KMS wrapper so it never sits in env dumps. Generate before creating users, back it up somewhere only the Airflow processes can read, and rotate on a calendar — a leaked default sample key decrypts every connection for anyone with DB read access.

# 1. generate a fresh key (cryptography must be installed)
NEW_KEY=$(python3 -c 'from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())')
echo "new key staged (prefix): ${NEW_KEY:0:6}..."

# 2. append-first: new key decrypts, old ciphertext still readable
OLD_KEY=$(airflow config get-value core fernet_key)
AIRFLOW__CORE__FERNET_KEY="$NEW_KEY,$OLD_KEY" airflow connections list >/dev/null && echo OK

# 3. re-save each connection so it re-encrypts under the new key
# airflow connections add --conn-id prod_warehouse --conn-type postgres ... (repeat per conn)
# 4. verify a decrypt, then drop the old key from the setting

Secrets Backends Versus Connection Encryption

Secrets backends move credentials out of Airflow's database into Vault or a cloud manager. Airflow looks up connections at runtime, rotation happens in one place, and no deploy is needed to roll a password. You'll feel the difference the first time security asks for an emergency rotation.

Connection encryption and backends complement each other. Fernet still seals whatever Airflow stores locally, while the backend holds the crown jewels with its own KMS keys and rotation schedules. Use both; neither replaces the other.

Migrate team by team. Point the backend at Airflow, move one team's connections, verify their DAGs resolve secrets, then move the next. You'll avoid a flag-day migration and each team learns the new lookup pattern with support nearby.

Know the lookup order cold: secrets backend first, then environment variables (AIRFLOW_CONN_ / AIRFLOW_VAR_), then the metastore last — and it's not configurable. That order is why Vault-first works: set [secrets] backend to VaultBackend (or SecretsManagerBackend / CloudSecretManagerBackend) with connections_path and variables_path, and DAG code never changes when rotation happens. On Airflow 3 you can also scope a separate backend just for workers, so execution nodes resolve only what they need instead of inheriting the scheduler's full secret surface.

import pendulum
from airflow.sdk import dag, task
from airflow.providers.postgres.hooks.postgres import PostgresHook

@dag(
    dag_id="billing_reconcile",
    schedule="0 5 * * *",
    start_date=pendulum.datetime(2026, 1, 1, tz="UTC"),
    catchup=False,
    tags=["tier-1", "billing"],
)
def billing_reconcile():
    @task
    def reconcile() -> str:
        # secret comes from the backend-backed connection, never code
        hook = PostgresHook(postgres_conn_id="prod_billing_ro")
        rows = hook.get_records("SELECT count(*) FROM charges WHERE ds = '{{ ds }}'")
        return f"charges={rows[0][0]}"

    reconcile()

billing_reconcile()

RBAC Roles and DAG-Level Access

RBAC turns one shared admin login into named humans with scoped power. Built-in roles cover the ladder: Viewer reads, User runs, Op manages runs and connections in scope, Admin configures. DAG-level grants slice horizontally per team.

Design for the incident you fear. Analysts get read on dev DAGs so curiosity can't break prod. On-call gets Op on their team's DAGs so 3 AM clears don't wait for platform. Platform keeps Admin behind SSO group membership that revokes with HR offboarding.

Review quarterly. Roles accrete like sediment: every exception becomes permanent unless someone prunes. You'll pull the user-role export, ask each manager to justify prod grants, and delete the unjustified ones the same day.

Harden the surfaces around RBAC too. Set expose_config = False so the UI never leaks full config to curious roles, and give the API its own auth backend rather than inheriting the webserver's. Authentication itself lives in webserver_config.py (AUTH_DB by default; OAuth, OpenID, LDAP, REMOTE_USER supported) — password auth is day one, LDAP/OAuth is production.

Authentication: Default Versus OAuth and LDAP

Default auth proves the UI works, not that your org is secure. Production needs OAuth or LDAP so credentials live in the identity provider, MFA applies, and deprovisioning happens in one place when someone leaves.

Map identity groups to Airflow roles at login. Data-eng lands as Op on their DAGs, analytics as Viewer, platform as Admin. You'll sync on login so role changes propagate without manual UI edits.

Test the offboarding path, not just login. Disable a test user in the IdP and confirm their Airflow session dies and API tokens fail. Joiners get celebrated; leavers get verified.

auth_manager: SimpleAuthManager  # dev only
# --- production: OAuth-backed login with group mapping ---
# AIRFLOW__CORE__AUTH_MANAGER=airflow.providers.fab.auth_manager.fab_auth_manager.FabAuthManager
# AIRFLOW__FAB__AUTH_TYPE=AUTH_OAUTH
# AIRFLOW__FAB__AUTH_ROLES_SYNC_AT_LOGIN=True
# AIRFLOW__FAB__AUTH_USER_REGISTRATION=True
# map IdP groups to Airflow roles:
# data-eng -> Op on team DAGs, analytics -> Viewer, platform -> Admin

airflow users list -o json 2>/dev/null | head -c 1500

Audit Logs and the API Surface

Audit logs turn who changed what from archaeology into a query. API and webserver logs record connection edits, role grants, and variable changes with actor and timestamp. Ship them to immutable storage where even admins can't rewrite history.

Alert on the sensitive three. Connection edits, role grants, and Variable changes each deserve a Slack ping to the platform channel. You'll catch the well-meaning analyst editing prod before the blast, not after.

Read the log monthly. Patterns emerge: Friday-afternoon connection tweaks, repeated role escalations before launches, Variables edited instead of connections. Each pattern becomes a guardrail or a training note.

Ship audit logs somewhere you'll actually read them. Airflow's event logs record who touched what; forwarding them to your SIEM turns a post-leak shrug into a timeline. Pair that with TLS on the webserver (web_server_ssl_cert/key) and least-privilege service accounts for the scheduler, and each layer shrinks what one leaked credential can reach.