Ansible Execution Environments: Stop Fighting Dependencies, Ship Faster

Every team I've worked with has a horror story about Ansible breaking because someone's laptop had a different Python version or a collection conflicted with a system package. I've seen a pip install inside a playbook silently upgrade a library and take down a production deployment at 2 AM. The root cause? No isolation. Ansible Execution Environments are the fix — they containerize your automation stack so you never have to debug a 'but it works on my machine' again. By the end of this, you'll be able to build, test, and debug EEs in production, and you'll know exactly when they're overkill.

Why Your Ansible Setup Is Broken (And EEs Fix It)

Before EEs, every Ansible control node was a snowflake. You'd have a Jenkins server with Python 3.6, a developer's Mac with Python 3.9, and a production bastion with Python 3.8. Collections installed via ansible-galaxy would conflict with system packages. Someone would run pip install --upgrade and break everything. EEs solve this by bundling Ansible, Python, collections, and system dependencies into a single container image. Now every run — local, CI, production — uses the exact same environment. No more 'works on my machine'.

// io.thecodeforge — DevOps tutorial

---
version: 1

build_arg_defaults:
  EE_BASE_IMAGE: 'quay.io/ansible/ansible-runner:stable-2.12-devel'

ansible_config: 'ansible.cfg'

dependencies:
  galaxy: requirements.yml
  python: requirements.txt
  system: bindep.txt

additional_build_steps:
  prepend: |
    RUN pip3 install --upgrade pip setuptools
  append: |
    RUN echo "EE built on $(date)" > /image_build_info.txt

Building Your First EE: The Right Way

You don't build EEs by hand — you use ansible-builder. It reads a definition file (execution-environment.yml) and generates a Containerfile (or Dockerfile) for you. The key is to keep it minimal. Start with a base image from quay.io/ansible/ansible-runner. Then add only the collections and Python packages your playbook needs. Use bindep.txt for system-level dependencies (like libffi-dev for cryptography). The build process is: ansible-builder build -f execution-environment.yml -t my-ee:1.0.0. This creates a container image you can push to any registry.

// io.thecodeforge — DevOps tutorial

---
collections:
  - name: ansible.posix
    version: '>=1.5.0,<2.0.0'
  - name: community.general
    version: '>=5.0.0,<6.0.0'
  - name: amazon.aws
    version: '>=3.0.0,<4.0.0'

Running Playbooks with ansible-navigator

ansible-navigator is the CLI tool that runs your playbook inside the EE. It handles mounting inventory, playbooks, and SSH keys into the container. The basic command: ansible-navigator run playbook.yml --eei my-ee:1.0.0 --mode stdout. The --mode stdout flag makes it behave like traditional ansible-playbook — otherwise you get an interactive TUI. For CI, always use --mode stdout and --pull-policy missing to avoid pulling the image every time if it's already cached.

// io.thecodeforge — DevOps tutorial

#!/bin/bash
# Run a playbook inside an EE with production settings
ansible-navigator run site.yml \
  --eei registry.example.com/ansible-ee:1.0.0 \
  --mode stdout \
  --pull-policy missing \
  --timeout 300 \
  --container-options '--memory=2g --cpus=2' \
  --pass-environment-variable AWS_ACCESS_KEY_ID \
  --pass-environment-variable AWS_SECRET_ACCESS_KEY

Debugging EE Failures: The TUI and Logs

When a playbook fails inside an EE, you can't just SSH into the container — it's ephemeral. Use ansible-navigator's interactive mode (--mode interactive) to explore the EE after a failure. Or, for CI, capture the container logs by running the EE manually: podman run --rm -it my-ee:1.0.0 /bin/bash. Inside, you can check Python versions, collection paths, and test imports. Common failure: a collection's Python dependency is missing. Fix it by adding to requirements.txt and rebuilding.

// io.thecodeforge — DevOps tutorial

#!/bin/bash
# Debug an EE by running it interactively
podman run --rm -it \
  -v $(pwd):/runner/project:z \
  -v ~/.ssh:/home/runner/.ssh:ro,z \
  my-ee:1.0.0 /bin/bash

# Inside the container, check:
# ansible --version
# python3 -c "import boto3"  # test import
# ansible-galaxy collection list

CI/CD Integration: The Pipeline That Doesn't Lie

EEs shine in CI. Your pipeline builds the EE once, pushes it to a registry, then every job pulls the same image. No more 'but it passed on Jenkins'. Use a multi-stage build: first stage installs dependencies, second stage copies only what's needed. Tag with the commit SHA for traceability. In your pipeline, run ansible-navigator run with --pull-policy always to ensure you're using the exact image from the registry. Example GitLab CI job: image: $CI_REGISTRY/ansible-ee:$CI_COMMIT_SHA.

// io.thecodeforge — DevOps tutorial

stages:
  - build
  - test
  - deploy

build-ee:
  stage: build
  script:
    - ansible-builder build -f execution-environment.yml -t $CI_REGISTRY/ansible-ee:$CI_COMMIT_SHA
    - podman push $CI_REGISTRY/ansible-ee:$CI_COMMIT_SHA

deploy:
  stage: deploy
  image: $CI_REGISTRY/ansible-ee:$CI_COMMIT_SHA
  script:
    - ansible-navigator run deploy.yml --mode stdout --pull-policy never

When EEs Are Overkill (And What to Use Instead)

EEs add complexity. You need a container runtime, a registry, and the build pipeline. For a single playbook run weekly on a laptop, a virtual environment (python3 -m venv) with pip install ansible is simpler. For teams with fewer than 5 people and no CI, EEs are overkill. But if you have multiple projects with conflicting dependencies, or you're running automation in production, EEs are the only sane choice. The rule: if you've ever said 'but it works on my machine', you need EEs.