Ansible state:latest — One Task Broke Payments for 47 Min

Every cloud infrastructure beyond a certain size hits the same wall. Someone on the team is spending their Friday afternoon manually SSH-ing into 30 servers, running the same five commands in sequence, and quietly hoping they didn't typo on server 24. It's slow. It's completely unauditable. And it doesn't scale — not to 100 servers, not to three environments, certainly not to a team of ten engineers who all have slightly different opinions about how to run that one sed command.

Scale that manual process to hundreds of EC2 instances or GCP VMs and the problem stops being annoying and starts being a business risk. An outage caused by configuration drift — two servers out of forty that silently diverged from the others — is nearly impossible to diagnose if you have no record of what was changed, when, and by whom.

That's the exact world Ansible was built to fix. It enforces a declared, version-controlled state across every machine in your fleet simultaneously, starting from nothing more than an SSH key and a YAML file checked into Git.

But Ansible has traps that aren't obvious from the documentation. state: latest looks safe until it upgrades Nginx on a Monday morning and changes a default TLS cipher suite. Handlers look optional until you realize your playbook has been restarting your app server on every run for three months. Roles look like bureaucratic overhead until your playbook hits 300 lines and two engineers are editing conflicting sections.

By the end of this article you'll understand not just how to write playbooks, but why they're structured the way they are. You'll know how inventory files map to real cloud environments, how roles package automation that other teams can actually reuse, and how handlers restart services only when config genuinely changed — not on every run. You'll leave with the mental model and the production lessons that take most engineers two or three incidents to learn the hard way.

Step-by-Step Ansible Installation on Ubuntu 22.04 — Control Node Setup That Lasts

Before you write a single playbook, you need a control node. This is the machine from which you'll run all Ansible commands. It can be your laptop, a dedicated jump box, or a CI runner. The installation method you choose has operational consequences for upgrade cycles and environment consistency.

Three installation methods compete for your attention. The Python package manager (pip) is the most flexible and lets you pin exact versions. The distribution's apt repository gives you system integration and automatic updates. The newer pipx method isolates Ansible in its own virtual environment and is the official Python Packaging Authority (PyPA) recommendation for installing CLI tools.

For production control nodes — dedicated VMs or CI runners — pip installation inside a Python virtual environment is the standard. It gives you version pinning (critical for consistency), isolation from the system Python, and easy upgrades via requirements files. The following sequence sets up Ansible in a virtual environment under /opt/ansible, with a symlink in /usr/local/bin for global access.

After installation, you configure the control node's SSH access. Ansible needs to reach every managed server via SSH with key-based authentication. The common failure point is SSH host key checking. When you connect to a server for the first time, Ansible's default behavior is to verify the host key against ~/.ssh/known_hosts. In dynamic cloud environments where IPs are recycled, this causes prompt blocks. The production fix is to manage known_hosts via a pre-seeded file or use host_key_checking=False in ansible.cfg with an understanding of the security trade-off.

The inventory file is your first configuration file. It lists your managed nodes and groups them logically. For testing, a one-line inventory with a single server is enough. In production, you'll use dynamic inventory plugins that query cloud APIs.

To verify installation, run ansible all -i 'localhost,' -m ping -c local. This pings the control node itself without SSH, confirming the Ansible engine works.

# ── Option A: pip in a virtual environment (recommended for dedicated control nodes) ──
sudo apt update && sudo apt install python3-venv python3-pip -y
sudo python3 -m venv /opt/ansible
/opt/ansible/bin/pip install --upgrade pip
/opt/ansible/bin/pip install ansible-core==2.17.4
sudo ln -s /opt/ansible/bin/ansible* /usr/local/bin/

# Verify
ansible --version

# ── Option B: apt (simple but version lags) ──
sudo apt update
sudo apt install software-properties-common -y
sudo add-apt-repository --yes --update ppa:ansible/ansible
sudo apt install ansible -y

# ── Option C: pipx (isolated CLI tool, good for laptops) ──
sudo apt install pipx -y
pipx ensurepath
pipx install ansible-core==2.17.4

# ── After any method: configure ansible.cfg ──
mkdir -p ~/ansible
cat > ~/ansible/ansible.cfg << 'EOF'
[defaults]
inventory = inventory
host_key_checking = False
pipelining = True
forks = 25
EOF

# ── Create a test inventory ──
echo 'localhost ansible_connection=local' > ~/ansible/inventory

# ── Test the ping module ──
cd ~/ansible
ansible all -i inventory -m ping

Ansible Architecture — How the Control Node, Inventory, and Managed Nodes Interact

Understanding Ansible's architecture is the foundation for debugging connection issues, scaling automation, and choosing the right deployment model. The architecture is deceptively simple: a control node runs Ansible, reads an inventory, and connects to managed nodes via SSH. But the simplicity hides a few sharp edges that only show up in production at scale.

The control node is any machine with Ansible installed — your laptop, a build server, a dedicated jump host. It's the single point of failure in the architecture. If your control node goes down, you cannot run any automation until it's restored. This is why production setups use multiple control nodes in a load-balanced fashion or rely on a CI/CD platform that can re-run jobs from any agent.

The inventory is the source of truth for which nodes exist and how they're grouped. Static inventory files map hostnames to IP addresses. Dynamic inventory plugins query cloud provider APIs and build the host list at runtime. The inventory also holds variables that travel with hosts into playbooks.

Managed nodes are the target servers. They need SSH access from the control node and Python 3 installed. That's it. No agent, no daemon, no open ports beyond SSH. This is the biggest architectural advantage over agent-based tools: you can manage any server that's reachable over SSH, including on-premise machines, cloud VMs, containers (via Docker exec), and even Windows via WinRM.

Ansible's execution model is push-based. You run a command on the control node, Ansible opens SSH connections to each managed node in parallel (controlled by the forks setting), copies the Python module code over, executes it, collects JSON results, and closes the connection. There is no persistent connection. This simplicity means Ansible is stateless from the managed node's perspective, but it also means every playbook run pays the SSH connection overhead.

The following diagram visualizes the flow during a typical playbook run. The control node reads the playbook and inventory, resolves variables, then fans out tasks to each managed node group sequentially (play by play) but within a play, tasks run on all hosts in parallel up to forks concurrent connections.

┌─────────────────────────────────────────────────────────────────────┐
│                     CONTROL NODE (Ansible CLI)                      │
│  ┌─────────────┐   ┌──────────────┐   ┌────────────────────────┐   │
│  │  Playbook    │   │  Inventory    │   │  ansible.cfg          │   │
│  │  (YAML)      │   │  (static/    │   │  forks=25             │   │
│  │  - hosts     │──▶│   dynamic)   │   │  pipelining=True      │   │
│  │  - tasks     │   │  - groups    │   │  host_key_checking=no │   │
│  │  - handlers  │   │  - variables │   └────────────────────────┘   │
│  └──────┬───────┘   └──────────────┘                                │
│         │                                                            │
│         │  Ansible compiles task list per host                       │
│         ▼                                                            │
│  ┌──────────────────────────────────┐                                │
│  │  SSH Connection Pool (forks=25) │                                │
│  │  (Manages up to 25 parallel SSH)│                                │
│  └────────────┬─────────────────────┘                                │
└────────────────┼─────────────────────────────────────────────────────┘
                 │
                 │  SSH (port 22) + Python execution
                 ▼
┌─────────────────────────────────────────────────────────────────────┐
│                      MANAGED NODES                                  │
│                                                                     │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐     ┌──────────┐        │
│  │ Web-01   │  │ Web-02   │  │ DB-01    │     │ DB-02    │        │
│  │ 10.0.1.1 │  │ 10.0.1.2 │  │ 10.0.2.1 │     │ 10.0.2.2 │        │
│  │ Python 3 │  │ Python 3 │  │ Python 3 │     │ Python 3 │        │
│  │ SSH key  │  │ SSH key  │  │ SSH key  │     │ SSH key  │        │
│  └──────────┘  └──────────┘  └──────────┘     └──────────┘        │
│                                                                     │
│  On each node:                                                      │
│  1. Ansible copies Python script (module)                           │
│  2. Executes with given arguments                                   │
│  3. Returns JSON result (changed, ok, failed)                       │
│  4. Temporary files are cleaned up                                  │
└─────────────────────────────────────────────────────────────────────┘

Ad-Hoc Ansible Commands — Quick Operations Without Writing a Playbook

Not every task deserves a playbook. Sometimes you need to check the uptime on 50 servers, copy a configuration file to a specific server, or restart a service immediately during an incident. Ad-hoc commands are single-module operations you run directly from the command line without a playbook file. They're ideal for read-only queries, one-off changes, and emergency responses.

The pattern is always: ansible <host-pattern> -m <module> -a '<module arguments>'. The host pattern matches inventory groups, wildcards, or specific hostnames. The module name is the Ansible module to use. The -a argument string depends on the module.

Three modules dominate ad-hoc usage. The ping module tests SSH connectivity and Python availability — it's the first command you run after setting up a new inventory. The command module runs any shell command with arguments directly, but it always reports 'changed' and is not idempotent. In ad-hoc mode, that's usually fine because you're making a one-off change. The shell module is similar but runs through /bin/sh and supports shell operators like pipes and redirects.

For copying files, the copy module is idempotent even in ad-hoc mode: it only transfers the file if the source and destination differ. This makes it safe to use for emergency configuration pushes without worrying about overwriting an identical file unnecessarily.

Ad-hoc commands are powerful but leave no audit trail unless you log them. Every ad-hoc change should be logged with script or tee and followed up with a permanent playbook change. If you find yourself running the same ad-hoc command twice, it's a sign that operation should be a playbook.

A practical production use case: a security vulnerability requires updating a package version across the fleet immediately. You cannot wait for the CI pipeline. ansible all -m ansible.builtin.apt -a 'name=openssl state=latest update_cache=yes' --become patches all servers in one command. After the emergency, you pin the intended version in your main playbook and remove the state=latest usage.

# ── Ping all servers in the 'production' group ─────────────────────
ansible production -i inventory -m ping

# ── Run uptime on web servers ───────────────────────────────────────
ansible web_servers -i inventory -m command -a 'uptime'

# ── Check disk usage on all servers ─────────────────────────────────
ansible all -i inventory -m shell -a 'df -h / | tail -1'

# ── Copy a configuration file to one server ─────────────────────────
ansible app-01 -i inventory -m copy -a 'src=./nginx.conf dest=/etc/nginx/nginx.conf owner=root group=root mode=0644'

# ── Restart Nginx service on all web servers ────────────────────────
ansible web_servers -i inventory -m service -a 'name=nginx state=restarted' --become

# ── Gather facts for a specific host ─────────────────────────────────
ansible db-primary -i inventory -m setup -a 'gather_subset=network'

# ── Install a package everywhere (emergency) ─────────────────────────
ansible all -i inventory -m apt -a 'name=openssl state=latest update_cache=yes' --become

# ── Check if a service is running with piped shell command ───────────
ansible all -i inventory -m shell -a 'systemctl is-active nginx && echo "active" || echo "inactive"'

Your First Ansible Playbook — Install Apache on a Web Server Cluster

A playbook is a YAML file that describes the desired state of a set of hosts. It's the core unit of automation in Ansible. Writing your first playbook reinforces the mental model of declaring 'what should be true' rather than scripting 'what commands to run'.

The canonical first playbook installs and configures Apache on a group of web servers. It exercises the three most common modules: apt for package management, template for configuration files, and service for daemon management. It also introduces handlers by restarting Apache only when the configuration file actually changes.

Create an inventory file with one or two test servers, or use localhost with ansible_connection=local for a safe first run. The playbook below assumes an inventory group called web_servers that you define. It becomes root via become: true because installing packages and starting services requires superuser privileges.

The playbook has four tasks plus a handler. The first task installs Apache at the version provided by the distribution's default repositories. Using state: present ensures it's installed but won't upgrade it unexpectedly. The second task creates a custom index.html using the copy module with content directly — avoids a template file for this simple example. The third task copies an Apache virtualhost configuration from a file on the control node. The fourth task enables the site and ensures Apache runs on boot. The handler restarts Apache only when the virtualhost config changes.

Running the playbook the first time changes state (installs, writes, restarts). Running it again reports 'ok' for all tasks because the desired state is already in place — this is idempotency in action.

Use --check --diff before the first real run to see what would change without actually changing anything. This is covered in the next section.

# ── First playbook: install and configure Apache ──────────────────
# Run with: ansible-playbook -i inventory apache.yml
# Run dry-run: ansible-playbook -i inventory apache.yml --check --diff

- name: Install and configure Apache on web servers
  hosts: web_servers
  become: true
  gather_facts: false

  vars:
    # Default index.html content
    welcome_message: "Welcome to TheCodeForge demo!"
    # Apache virtualhost port (can be overridden per host via inventory)
    apache_port: 80

  handlers:
    - name: Restart Apache
      ansible.builtin.service:
        name: apache2
        state: restarted
      listen: "apache config changed"

  tasks:
    - name: Install Apache2 package
      ansible.builtin.apt:
        name: apache2
        state: present
        update_cache: true
        cache_valid_time: 3600

    - name: Create custom index.html
      ansible.builtin.copy:
        content: |
          <html>
          <head><title>TheCodeForge Demo</title></head>
          <body><h1>{{ welcome_message }}</h1></body>
          </html>
        dest: /var/www/html/index.html
        owner: www-data
        group: www-data
        mode: '0644'

    - name: Deploy virtualhost configuration
      ansible.builtin.copy:
        src: files/apache-site.conf
        dest: /etc/apache2/sites-available/001-webapp.conf
        owner: root
        group: root
        mode: '0644'
      notify: "apache config changed"
      # The handler only fires if this task reports 'changed'

    - name: Ensure Apache is enabled and running
      ansible.builtin.service:
        name: apache2
        state: started
        enabled: true

---
# files/apache-site.conf
# Place this file alongside the playbook:
# <VirtualHost *:{{ apache_port }}>
#     DocumentRoot /var/www/html
#     ServerName localhost
# </VirtualHost>

# ── Sample output on first run ───────────────────────────────────────
# PLAY [Install and configure Apache on web servers] ********************
# TASK [Install Apache2 package] ***************************************
# changed: [web-01]
# TASK [Create custom index.html] ***************************************
# changed: [web-01]
# TASK [Deploy virtualhost configuration] *******************************
# changed: [web-01]
# TASK [Ensure Apache is enabled and running] ***************************
# changed: [web-01]
# RUNNING HANDLER [Restart Apache] **************************************
# changed: [web-01]
# PLAY RECAP ************************************************************
# web-01   : ok=5   changed=4   unreachable=0   failed=0   skipped=0

Playbook Check Mode and Diff — Validate Before You Change

The --check flag runs a playbook in 'dry-run' mode: Ansible evaluates every task's condition and reports what would change without actually making any changes. Combined with --diff, it shows the exact content differences for template, copy, and other modules that manage file content. This combination is the closest thing Ansible has to a pre-deployment validation step.

Check mode is not a simulation. Modules that support it (most built-in modules) check their current state and report 'changed' or 'ok' based on whether the task would alter the system. Modules that don't support check mode run partially or report that they would change, reducing confidence. The shell and command modules, for example, always report 'changed' in check mode because they cannot predict their outcome. This is another reason to prefer dedicated modules.

The --diff flag shows the before-and-after content for files managed by copy, template, file (with content), and others. It also shows which lines in configuration files would be added, removed, or modified. You review this output to catch mistakes like a typo in a template variable or an incorrect file permission before they reach production.

In production CI pipelines, every playbook run that targets staging or production should first execute a check-diff run. If the playbook would change more than expected (e.g., 200 files changed when you only expected 2), the pipeline should halt and alert a human. This is a classic 'change validation' pattern.

One caveat: check mode does not execute handlers, even if they would be notified. It also does not run command or shell tasks, so if your playbook relies on those for idempotency, check mode gives less reliable output. The rule of thumb: the more dedicated modules you use, the more accurate your dry-run results will be.

# ── Dry-run a playbook against staging ─────────────────────────────
ansible-playbook -i staging site.yml --check --diff

# ── Dry-run against a single host ────────────────────────────────────
ansible-playbook -i production site.yml --limit app-01 --check --diff

# ── Capture diff output for review ───────────────────────────────────
ansible-playbook -i staging site.yml --check --diff 2>&1 | tee /tmp/dry-run-$(date +%Y%m%d-%H%M).log

# ── Grep for any tasks that would change in dry-run ───────────────────
grep -E '(changed|TASK)' /tmp/dry-run-*.log | grep -v 'ok:'

# ── Run in check mode with increased verbosity for detailed output ───
ansible-playbook -i production site.yml --check -vv

# ── In a CI pipeline, fail if check mode shows unexpected changes ─────
# After dry-run, if the output contains 'changed:', and the expected
# change count is > X, the pipeline should fail.
ansible-playbook -i staging site.yml --check --diff 2>&1 | \n  awk '/^TASK/ { task=$0 } /changed/ { print task, $0 }' | \n  wc -l
# (examine count, set threshold in pipeline logic)

How Ansible Connects to Your Cloud Servers — Inventory Files Without the Confusion

Before Ansible can do anything, it needs to know what it's talking to. That's the inventory file's job. Think of it as the contacts list for your infrastructure — it maps human-readable group names like web_servers or database_servers to the actual IP addresses or DNS hostnames Ansible will SSH into.

In cloud environments, hard-coding IP addresses into an inventory file is a short-term convenience that becomes a maintenance problem fast. EC2 instances get recycled during deployments. Autoscaling groups add and remove instances based on load. Elastic IPs get reassigned when infrastructure changes. An inventory file with hard-coded IPs becomes stale within days in a dynamic cloud environment. That's why Ansible supports dynamic inventory — plugins that query the AWS EC2, GCP Compute, or Azure APIs at runtime and return a fresh list of running instances, grouped however you want.

For learning the mental model, static inventory is clearer. Once the model clicks, the dynamic inventory plugin is just a YAML config file that points at your cloud API instead of listing hostnames manually.

Groups are where the power is. You can target the web_servers group for an app deployment, the database_servers group for a schema migration, and the production parent group for a security patch that applies to all tiers simultaneously. The group hierarchy is resolved at runtime — you don't repeat hostnames across multiple groups.

Variables attached to groups and hosts in the inventory travel with them into every playbook that targets them. A deploy_user variable defined at the web_servers group level is automatically available in every task running against those servers. Host variables override group variables, and group variables override the global all group. This precedence order means you can set sensible defaults at the group level and override exceptions at the host level — without writing a single conditional in your playbook.

# io/thecodeforge/ansible/cloud_inventory.ini
#
# Static inventory for a three-tier cloud application.
# In a production environment with more than ~20 dynamic instances,
# replace this file with a dynamic inventory plugin.
#
# For AWS, install the collection and point to a plugin config:
#   ansible-galaxy collection install amazon.aws
#   ansible-playbook -i aws_ec2.yml site.yml
#
# The aws_ec2 plugin queries EC2 at runtime and groups instances
# by tags like Environment=production automatically.

# ── TIER 1: Load Balancers ──────────────────────────────────────────
[load_balancers]
# Format: alias  ansible_host=<IP>  connection_variable=value
nginx-lb-01 ansible_host=54.210.100.11 ansible_user=ubuntu

# ── TIER 2: Application Servers ─────────────────────────────────────
[web_servers]
app-server-01 ansible_host=10.0.1.10 ansible_user=ubuntu
app-server-02 ansible_host=10.0.1.11 ansible_user=ubuntu
# Canary server runs on a different port — host variable overrides group variable below
app-server-03 ansible_host=10.0.1.12 ansible_user=ubuntu app_port=8090

# ── TIER 3: Databases ───────────────────────────────────────────────
[database_servers]
# Primary and replica are targeted separately in playbooks
# (schema migrations only run against primary, never replica)
db-primary-01 ansible_host=10.0.2.10 ansible_user=ubuntu
db-replica-01 ansible_host=10.0.2.11 ansible_user=ubuntu

# ── GROUP OF GROUPS ─────────────────────────────────────────────────
# 'production' contains all three tiers.
# Running a playbook against 'production' hits all servers.
# Running it against 'web_servers' hits only the app layer.
[production:children]
load_balancers
web_servers
database_servers

# ── GROUP VARIABLES ─────────────────────────────────────────────────
# These variables are available in every task targeting web_servers.
# Host variables (like app-server-03's app_port above) override these.
[web_servers:vars]
app_port=8080              # Default port — app-server-03 overrides this to 8090
deploy_user=apprunner      # The OS user that owns application files
max_connections=1000       # Used in Nginx and app server config templates

[database_servers:vars]
db_port=5432
db_replication_user=replicator

Writing Playbooks That Reflect Production Reality — Not Just What You Hope Is True

A playbook is where your intent lives as code. Every playbook answers three questions: which servers (hosts), with which privileges (become), and what should be true about them (tasks). That phrase 'what should be true' is deliberate and important — Ansible tasks are declarative. You're not writing a script that says 'run apt-get install nginx'. You're asserting 'Nginx must be present at version 1.24'. Ansible figures out whether any action is needed to make that assertion true.

This is idempotency in practice, and it's what separates a playbook you can run repeatedly as a drift-correction job from a script you run once and never touch again out of fear. Run an idempotent playbook ten times: the first run installs and configures everything from scratch. Runs two through ten touch nothing because the declared state already exists on disk. A CI pipeline that runs your playbook against staging on every merge becomes a continuous configuration validation test rather than a deployment risk.

But idempotency is not automatic. It depends entirely on using modules that understand current state. The apt module checks whether the package is already installed and at the correct version before touching the package manager. The template module compares the rendered output to the file on disk before writing. The service module checks whether the service is in the desired state before restarting. The shell and command modules do none of this — they execute unconditionally and report 'changed' every time, which is exactly how you end up with a playbook output full of false positives that everyone ignores.

Handlers are Ansible's mechanism for 'only react to real changes'. Instead of adding a service: state=restarted task after every config change, you declare a handler and notify it from the tasks that might change configuration. The handler only fires if at least one notifying task reported an actual change during that play run. If Nginx's config file was already correct and the template task reported 'ok', the handler never runs — no restart, no dropped connections, no unnecessary downtime.

Variables from the inventory flow directly into playbook tasks through Jinja2 template syntax — the double-curly-brace {{ variable_name }} notation. app_port defined in group_vars/web_servers.yml becomes available in every task and template targeting those servers. This is how one playbook serves multiple environments without branching: staging has app_port: 8080, production has app_port: 443. The playbook and the role don't change. The inventory and group_vars do.

---
# io/thecodeforge/ansible/deploy_web_app.yml
#
# Configures Nginx as a reverse proxy and deploys our Node.js application
# to all servers in the web_servers inventory group.
#
# Usage:
#   ansible-playbook -i inventories/production/hosts.ini deploy_web_app.yml
#   ansible-playbook -i inventories/staging/hosts.ini deploy_web_app.yml --check
#
# Add --check for a dry-run that shows what would change without changing it.
# Add --diff to see file content differences alongside the change report.

- name: Configure Nginx reverse proxy and deploy Node.js application
  hosts: web_servers          # Matches group name from inventory — targets all servers in that group
  become: true                # Escalate to root via sudo — required for apt and systemd
  gather_facts: true          # Collect OS info (distro, arch, IP) — used in 'when:' conditionals below

  vars:
    # Pinned to a specific minor version — patch releases are allowed, major changes are not.
    # Upgrade intentionally by changing this value and running through staging first.
    node_version: "20"
    app_directory: "/opt/webapp"
    nginx_config_path: "/etc/nginx/sites-available/webapp.conf"

  # Handlers: only execute when explicitly notified by a task that reported 'changed'.
  # Key behavior: a handler notified 5 times in one play still runs exactly once,
  # at the end of the play — Ansible deduplicates handler triggers automatically.
  handlers:
    - name: Reload Nginx
      ansible.builtin.service:
        name: nginx
        state: reloaded     # Graceful reload: re-reads config without dropping connections.
                            # Use 'restarted' only when you need a full process restart
                            # (e.g., after a binary upgrade, not a config change).

    - name: Restart application service
      ansible.builtin.service:
        name: webapp
        state: restarted

  tasks:

    # ── Task 1: Install Nginx ──────────────────────────────────────────
    # 'state: present' means: ensure it's installed at the pinned version.
    # 'state: latest' would mean: upgrade to whatever the mirror serves today.
    # We use 'present' with an explicit version. Never 'latest' in production.
    - name: Install Nginx at pinned version
      ansible.builtin.apt:
        name: "nginx=1.24.*"     # Wildcard allows patch releases within 1.24.x
        state: present
        update_cache: true
        cache_valid_time: 3600   # Only refresh apt cache if older than 1 hour.
                                 # Without this, every playbook run pays a 10-second
                                 # apt-get update cost per server, per play.

    # ── Task 2: Deploy Nginx config from Jinja2 template ──────────────
    # The template module renders nginx_webapp.conf.j2 with current variables
    # and compares the result to what's on disk. If they differ, it writes
    # the file and reports 'changed' — triggering the Reload Nginx handler.
    # If they're identical, it reports 'ok' — handler never runs.
    - name: Deploy Nginx reverse proxy configuration
      ansible.builtin.template:
        src: templates/nginx_webapp.conf.j2
        dest: "{{ nginx_config_path }}"
        owner: root
        group: root
        mode: '0644'
        validate: 'nginx -t -c %s'   # Runs nginx config syntax check BEFORE writing.
                                      # A broken config never reaches disk.
      notify: Reload Nginx

    # ── Task 3: Create application directory ──────────────────────────
    # 'state: directory' is idempotent — does nothing if directory exists.
    # 'deploy_user' and 'app_directory' come from inventory group_vars.
    - name: Ensure application directory exists with correct ownership
      ansible.builtin.file:
        path: "{{ app_directory }}"
        state: directory
        owner: "{{ deploy_user }}"
        group: "{{ deploy_user }}"
        mode: '0755'

    # ── Task 4: Install Node.js — conditional on OS family ────────────
    # gather_facts: true above populates ansible_facts['os_family'].
    # The 'creates:' argument makes this task idempotent:
    # if /usr/bin/node already exists, skip execution entirely.
    # Without 'creates:', this shell task would run and report 'changed' every time.
    - name: Install Node.js via NodeSource setup script (Debian/Ubuntu only)
      ansible.builtin.shell: |
        curl -fsSL https://deb.nodesource.com/setup_{{ node_version }}.x | bash -
        apt-get install -y nodejs
      args:
        creates: /usr/bin/node    # Skip if Node.js is already installed
      when: ansible_facts['os_family'] == 'Debian'

    # ── Task 5: Ensure webapp service is running and enabled ──────────
    # 'state: started' does nothing if service is already running.
    # 'enabled: true' ensures it restarts on server reboot.
    - name: Enable and start webapp systemd service
      ansible.builtin.service:
        name: webapp
        state: started
        enabled: true
      notify: Restart application service