Secrets in Git History — Purge Leaks and Rotate Keys

It takes five seconds: export AWS_SECRET_ACCESS_KEY=... pasted into the wrong terminal, committed with -a, pushed before coffee. The key now lives in your repo — not just in the current files, but baked into a commit object that git will faithfully preserve, copy to every clone, and serve to every fork forever. Deleting the line in the next commit changes nothing about the copies already distributed.

Attackers know this better than most developers. Bots watch public push events and test fresh credentials within seconds — studies of honeytoken leakage have repeatedly shown exploitation starting under a minute after exposure. Private repos aren't safe either: contractors' clones, CI logs, error trackers, and backup snapshots all carry the secret beyond the repo's access controls.

Recovery has a strict order that teams keep getting backwards. Rotation comes first, because a purged-but-still-valid key is still a breach. History rewriting comes second, with git filter-repo or BFG, followed by cache expiry and clone notification. Prevention comes third and lasts: pre-commit hooks, push protection, and secret managers that keep credentials out of files entirely. This guide walks all three in that order.

How Secrets End Up Committed (and Why Delete Isn't Enough)

Secrets enter repos through boring, hurried paths: a .env file added with git add -A, a debug line printing a token left in a commit, an AWS key pasted into a config example, a private key vendored 'temporarily' for a deploy script. The commit that carries them usually isn't malicious or even careless in intent — it's a Friday-evening -a flag or a tutorial-following afternoon. The damage comes from git's memory, not the author's morals.

Git remembers everything by design. Each commit snapshots the full tree, objects are content-addressed and deduplicated, and history is the product. Deleting the secret in a later commit adds a new snapshot without it; every prior snapshot keeps it, fetchable by hash, copied into every clone and fork, cached by forges and CI systems. 'But I deleted it Saturday' is the sentence every incident review hears — and it's always followed by the log line showing exploitation started Friday.

Internalize the mental model: pushing a secret is publishing it, with a delay before you notice. Private repos only narrow the audience to everyone with past, present, or indirect access — contractors, CI vendors, error-tracking services, backup operators. The recovery chapters below assume the worst audience and work backward to safety; this chapter's only job is killing the comforting fiction that deletion equals recall.

Rotate First: Why Rewriting Alone Is Never Enough

Rotation means killing the exposed credential at the provider and issuing a replacement: deactivate the IAM key (deactivation beats deletion for audit trails), revoke the OAuth token, roll the password, reissue the certificate. Do this before any git surgery, because every minute the old secret stays valid is a minute someone's clone can spend it. The exposure window runs from commit time to revocation time — measure it, record it, and scope your audit to it.

Rotation alone isn't enough either, and that's the point of the sequence. Revoking stops future abuse but leaves the secret readable in history for anyone who finds the old commit — a future attacker with repo access, a leaked backup, a forgotten fork. So purge after rotating: history rewriting removes the readable copy so the incident can't recur from the same artifact, even though rotation already defanged it.

Audit what the old credential touched between exposure and revocation. Provider audit logs (CloudTrail, Azure Activity, token-use logs) show every action by the exposed identity: instances launched, data read, permissions changed. Review each action as potentially hostile, destroy what it created, and rotate anything it could have read — because a key that listed your other secrets compromises them transitively. Rotation first, purge second, audit always.

Rewriting History with git filter-repo and BFG

git filter-repo is the modern rewriting tool: it purges files or patterns from every commit across all branches and tags, strips the secret from history, and leaves you with a clean repo to force-push. BFG Repo-Cleaner fills the same role with a simpler interface that's faster on enormous monorepos. Both rewrite commit hashes (unavoidably — history is the hashes), so coordinate the force-push: freeze merges, land in-flight PRs or rebase them after, and announce the hash change so nobody 'repairs' it by pushing the old history back.

The mechanics matter less than the completeness. Purge every branch, every tag, and stashed refs — secrets hide in release tags and ancient feature branches that HEAD-focused cleanup misses. After force-pushing, expire the forge's cached views and prune reflogs; GitHub and peers keep unreachable objects around until garbage collection, and cached file views can serve the secret for hours. Then tell every collaborator to re-clone fresh: a pull into an old clone keeps the tainted objects locally.

The snippet below shows the canonical filter-repo purge for a file, plus the verification scan that must follow. Treat forks as unrecoverable — contact their owners, rotate anyway, and accept that copies beyond your control are why rotation came first. A rewritten canonical repo plus revoked credentials is the complete recovery; either half alone is theater.

# Purge a secret file from ALL history (coordinate freeze + force-push first)
git filter-repo --invert-paths --path .env --force

# Or strip a secret pattern from every file in every commit:
# git filter-repo --replace-text <(echo 'AKIAIOSFODNN7EXAMPLE==>REDACTED') --force

# Push the rewritten history and expire the forge cache
git push origin --force --all && git push origin --force --tags

# Verify with a FRESH clone: no trace of the pattern anywhere
cd /tmp && git clone <repo-url> verify-clone && cd verify-clone
gitleaks detect --source . --no-git -v

Pre-commit Hooks That Block the Next Leak

Prevention beats response by orders of magnitude, and pre-commit hooks are the cheapest prevention available. Tools like gitleaks and detect-secrets scan staged changes before a commit object exists: a match blocks the commit with the offending line highlighted, and the secret never enters history at all. Installation is one config file plus one hook-install command; the whole team inherits it from the repo template.

Layer server-side protection behind the local hook, because local hooks can be skipped with --no-verify and contractors may never install them. Forge push protection (GitHub's secret scanning push protection, GitLab's secret push protection) rejects pushes containing known secret formats even when local gates were bypassed. Treat the two as belt and suspenders: hooks catch mistakes instantly at the keyboard, push protection catches everything else at the door.

Tune the signal to survive contact with developers. Baseline existing repos so historical secrets don't block every commit, allowlist test fixtures and documented examples explicitly (with review), and keep scans fast — a hook slower than three seconds gets uninstalled by lunch. The config below wires both scanners into pre-commit: gitleaks for broad pattern coverage, detect-secrets for entropy-based finds. Review hook bypasses (--no-verify usage) in CI to catch the bypassers.

repos:
  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.24.2
    hooks:
      - id: gitleaks  # blocks commits carrying known secret patterns
  - repo: https://github.com/Yelp/detect-secrets
    rev: v1.5.0
    hooks:
      - id: detect-secrets
        args: ['--baseline', '.secrets.baseline']
        # Update the baseline after reviewed rotations:
        # detect-secrets scan --baseline .secrets.baseline

Environment Variables and Secret Managers Done Right

The deepest fix is structural: keep secrets out of the working tree so there's nothing to commit. Applications read credentials from environment variables injected at runtime; the values live in a secret manager (HashiCorp Vault, AWS Secrets Manager, GCP Secret Manager, Azure Key Vault) and reach processes through the orchestrator — never through files, chat, or tickets. A repo that contains only secret names (DB_PASSWORD_REF, not the password) can't leak credentials it never held.

Do the boring parts rigorously. .env files stay gitignored from project birth (add them to global gitignore templates, not just each repo), example files ship with placeholder values that fail loudly if used, and CI injects secrets from the manager into job environments rather than checking in service keys. Prefer short-lived, dynamically generated credentials — database passwords minted per deploy, cloud sessions lasting hours — so any single leak expires on its own.

Mind the new leak vectors this creates. CI logs must mask secret values (most platforms do this only for marked variables — mark them), error trackers must scrub config dumps, and docker build args carrying secrets end up in image layers (use secret mounts, not args). The snippet below shows the safe application side: read from the environment, fail fast when unset, never log the value. Structure doesn't eliminate carelessness, but it moves mistakes from 'published forever' to 'failed locally.'

import os

def required_secret(name: str) -> str:
    # Secret arrives via environment from the manager, never from files.
    value = os.environ.get(name)
    if not value:
        raise RuntimeError(f"Missing required secret: {name} (check env injection)")
    return value

DB_PASSWORD = required_secret("DB_PASSWORD")  # name in repo, value in manager

# Logging the VALUE is a leak; logging presence is safe:
# logger.info("db password configured: %s", bool(DB_PASSWORD))

Auditing Repos and CI Logs for Leftovers

Gates protect the future; audits clean the past. Run a historical scan across every repo in the org — all branches, tags, and stashes — because the next committed-secret incident is usually an old one the new hooks would have caught. Prioritize by exposure: public repos first, then repos with broad collaborator lists, ex-contractor access, or mirroring to other forges. Every find triggers the same rotate-purge-verify sequence; there are no minor committed secrets, only unexploited ones.

Extend the audit beyond git objects. Search CI logs for secret fingerprints (key prefixes, not full values), inspect container layers and build caches for baked-in files, review error-tracker payloads for config dumps, and check chat archives where developers paste 'temporary' credentials. Each of these is a copy the history rewrite won't reach — they need their own rotation and deletion pass.

Make auditing continuous, not episodic. Schedule organization-wide secret scanning weekly, alert on new findings like production incidents, and track mean-time-to-rotate as a security metric. The goal isn't a clean scan once — it's a system where a newly committed secret pages someone within the hour. Between pre-commit hooks, push protection, managers, and scheduled scans, leaked secrets become a brief, loud, self-healing event instead of a quiet permanent one.

# Org-wide historical scan: all refs, then CI log fingerprints
for repo in $(gh repo list my-org --limit 200 --json name -q '.[].name'); do
  echo "=== $repo ==="
  gh repo clone "my-org/$repo" "/tmp/audit/$repo" -- --bare --quiet
  gitleaks detect --source "/tmp/audit/$repo" --no-git -v
 done

# Never print full secrets in logs: search fingerprints only (e.g. key IDs)