Ansible Network Automation: Surviving Idempotency with network_cli & NAPALM

It's 2 AM on a Saturday. I'm on-call, and our network automation pipeline just pushed a bad VLAN config to 200 Cisco switches. The symptom: all trunk ports flapped, causing a 15-minute outage for the entire east coast data center. The root cause? A playbook using ios_command with commands: 'vlan 100' instead of ios_config with lines: 'vlan 100'. The ios_command module doesn't enforce idempotency—it blindly sends the command, and the switch accepted it even though the VLAN existed, causing a momentary interface reset. That night, I learned the hard way that network automation is not server automation. You can't just SSH in and run commands; you need structured modules that understand the device's state.

Historically, Ansible started as a server automation tool. The command and shell modules worked fine for Linux, but network devices are state machines with fragile CLI parsers. Early adopters used connection: local and raw SSH, which led to brittle playbooks. The Ansible network team responded with connection plugins like network_cli and vendor-specific modules. But even today, many engineers fall into the trap of treating network devices like Linux boxes.

This article covers the production patterns I've developed over 5 years of automating Cisco, Juniper, Arista, and Nexus gear. We'll dive into the network_cli connection plugin, the difference between ios_command and ios_config, network-agnostic modules, NAPALM integration, and how to avoid the idempotency landmines that will take down your network. I'll also show you how to secure credentials with ansible-vault and debug when things go wrong.

By the end, you'll know exactly which module to use for which task, how to structure your playbooks for reliability, and what to do when a switch doesn't respond as expected. Let's fix your automation before it fixes you.

1. The network_cli Connection Plugin: Why local is Dead

The network_cli connection plugin (ansible.netcommon.network_cli) is the standard for SSH-based network automation in Ansible 2.9+. It replaces the deprecated local connection and provides persistent SSH sessions, privilege escalation (enable mode), and automatic prompt handling.

Key differences from local: - network_cli maintains a single SSH connection for the entire playbook, reducing overhead. - It handles enable mode automatically when ansible_become: yes and ansible_become_method: enable are set. - It parses device prompts and waits for the correct prompt before sending commands.

Production setup: ``yaml # group_vars/network.yml ansible_connection: ansible.netcommon.network_cli ansible_network_os: cisco.ios.ios ansible_user: admin ansible_password: "{{ vault_ansible_password }}" ansible_become: yes ansible_become_method: enable ansible_become_password: "{{ vault_enable_password }}" ansible_command_timeout: 30 ``

Gotcha: If you set ansible_connection: ssh (the default), network modules will fail because they expect network_cli. Always explicitly set it.

Verification: ``bash ansible-inventory --host switch1 --yaml | grep ansible_connection # Should output: ansible_connection: ansible.netcommon.network_cli ``

---
- name: Use network_cli for Cisco IOS
  hosts: cisco
  gather_facts: no
  connection: ansible.netcommon.network_cli
  tasks:
    - name: Show version
      ansible.netcommon.cli_command:
        command: show version
      register: output
    - name: Display output
      ansible.builtin.debug:
        var: output.stdout_lines

2. Cisco IOS: ios_command vs ios_config

The two most common Cisco IOS modules are cisco.ios.ios_command and cisco.ios.ios_config. They serve different purposes:

• ios_command: Sends arbitrary CLI commands and returns output. Not idempotent. Use for show commands only.
• ios_config: Manages configuration sections idempotently. It reads the running config, compares it with the desired state, and applies only the necessary changes.

Example: Correct usage ``yaml - name: Configure VLAN 100 on interface cisco.ios.ios_config: lines: - vlan 100 parents: - interface GigabitEthernet0/1 ` This will only apply vlan 100` if it's not already present under the interface.

Example: Wrong usage (causes flap) ``yaml - name: BAD - using ios_command for config cisco.ios.ios_command: commands: - interface GigabitEthernet0/1 - vlan 100 `` This sends the commands blindly, causing the switch to re-apply VLAN 100, which resets the interface.

Idempotency check: Use --diff flag to see what changes Ansible will make: ``bash ansible-playbook -i inventory playbook.yml --diff --check ``

The --check mode with ios_config will simulate the change without applying it.

---
- name: IOS commands
  hosts: cisco
  gather_facts: no
  connection: ansible.netcommon.network_cli
  tasks:
    - name: Execute show command
      cisco.ios.ios_command:
        commands: show ip interface brief
      register: result
    - name: Configure hostname
      cisco.ios.ios_config:
        lines:
          - hostname R1
        match: exact
        replace: line
      register: config_result
    - name: Display results
      ansible.builtin.debug:
        msg: "{{ result.stdout_lines }}"

3. Network-Agnostic Modules: cli_config and cli_command

For multi-vendor environments, Ansible provides network-agnostic modules: ansible.netcommon.cli_config and ansible.netcommon.cli_command. These modules rely on the ansible_network_os variable to determine the correct CLI syntax.

Example: cli_config ``yaml - name: Set hostname using agnostic module ansible.netcommon.cli_config: config: "hostname {{ inventory_hostname }}" ` This works on Cisco IOS, Junos, EOS, and NXOS as long as ansible_network_os` is set correctly.

Example: cli_command ``yaml - name: Show version ansible.netcommon.cli_command: command: show version register: version_output ``

Limitations: - cli_config does not support structured config; it sends raw text. For idempotent structured config, use vendor-specific modules. - cli_command does not handle privilege escalation automatically; you may need to include enable in the command string.

When to use: - Quick ad-hoc commands across vendors. - When you don't have vendor-specific collections installed. - For read-only operations (show commands).

Gotcha: The cli_config module uses the configure terminal command on Cisco devices. If you send a command that requires enable mode, you must set ansible_become: yes.

---
- name: Use cli_config and cli_command
  hosts: all
  gather_facts: no
  connection: ansible.netcommon.network_cli
  tasks:
    - name: Run show command
      ansible.netcommon.cli_command:
        command: show running-config | include hostname
      register: output
    - name: Configure banner
      ansible.netcommon.cli_config:
        config: |
          banner motd ^
          Unauthorized access prohibited
          ^
      register: result
    - name: Display config result
      ansible.builtin.debug:
        var: result

4. NAPALM Integration via community.network

NAPALM (Network Automation and Programmability Abstraction Layer with Multivendor support) provides a unified API for network devices. The community.network.napalm_* modules (e.g., napalm_cli, napalm_config, napalm_get_facts) use NAPALM under the hood.

Setup: ``bash pip install napalm==4.1.0 ansible-galaxy collection install community.network ``

Inventory variables: ``yaml ansible_connection: community.network.napalm ansible_network_os: cisco.ios.ios # or junipernetworks.junos.junos, etc. napalm_platform: ios # must match napalm's platform name ``

Example: Get facts ``yaml - name: Gather facts via NAPALM community.network.napalm_get_facts: filter: ['facts', 'interfaces'] register: napalm_facts ``

Example: Configure using NAPALM ``yaml - name: Configure using NAPALM community.network.napalm_config: replace: yes config: | interface GigabitEthernet0/1 description NAPALM managed ``

Advantages: - Works with devices that have API access (NETCONF, RESTCONF) – more reliable than CLI scraping. - napalm_get_facts returns structured data (JSON) instead of CLI text. - napalm_config supports replace and commit operations (important for Junos).

Disadvantages: - Requires NAPALM on the control node, not on the device. - Some platforms require additional libraries (e.g., junos-eznc for Junos). - Not all NAPALM methods are idempotent; napalm_config with replace is, but napalm_cli is not.

---
- name: NAPALM integration
  hosts: all
  gather_facts: no
  tasks:
    - name: Get facts via NAPALM
      community.network.napalm_get_facts:
        hostname: "{{ ansible_host }}"
        username: "{{ ansible_user }}"
        password: "{{ ansible_ssh_pass }}"
        dev_os: "{{ napalm_os }}"
        filter:
          - interfaces
      register: facts
    - name: Display interfaces
      ansible.builtin.debug:
        var: facts.ansible_facts.napalm_interfaces

5. Managing Junos: junos_config and junos_command

Juniper Junos devices use a different paradigm: candidate config and commit. The junipernetworks.junos.junos_config module handles this.

Example: Configure an interface ``yaml - name: Configure interface ge-0/0/0 junipernetworks.junos.junos_config: lines: - set interfaces ge-0/0/0 description "Ansible managed" - set interfaces ge-0/0/0 unit 0 family inet address 10.0.0.1/24 comment: "Updated by Ansible" ``

Commit options: ``yaml - name: Commit with confirm junipernetworks.junos.junos_config: lines: - set system hostname new-hostname commit: yes confirm: 5 # confirm in 5 minutes ``

Rollback: ``yaml - name: Rollback to previous config junipernetworks.junos.junos_config: rollback: 1 ``

Gotcha: Junos modules require ansible_network_os: junipernetworks.junos.junos and the junipernetworks.junos collection. Also, you need junos-eznc Python library on the control node: pip install junos-eznc.

Command module: ``yaml - name: Show interface status junipernetworks.junos.junos_command: commands: - show interfaces terse register: output ``

Idempotency: junos_config only applies changes that differ from the candidate config. Use --diff to see what will change.

---
- name: Manage Junos
  hosts: juniper
  gather_facts: no
  connection: ansible.netcommon.network_cli
  tasks:
    - name: Show interface
      junipernetworks.junos.junos_command:
        commands: show interfaces terse
      register: result
    - name: Configure hostname
      junipernetworks.junos.junos_config:
        lines:
          - set system host-name J1
        comment: Update hostname
      register: config_result
    - name: Display config result
      ansible.builtin.debug:
        var: config_result

6. Managing Arista EOS: eos_config and eos_command

Arista EOS is similar to Cisco IOS but with some differences. The arista.eos.eos_config module is the primary config module.

Example: Configure VLAN ``yaml - name: Configure VLAN 200 arista.eos.eos_config: lines: - vlan 200 - name WEB_VLAN parents: - vlan 200 ``

Example: Configure interface ``yaml - name: Set interface description arista.eos.eos_config: lines: - description "Ansible managed" parents: - interface Ethernet1 ``

Command module: ``yaml - name: Show running config arista.eos.eos_command: commands: - show running-config register: run_cfg ``

Gotcha: Arista EOS uses enable mode like Cisco. Set ansible_become: yes and ansible_become_method: enable.

Idempotency: eos_config is idempotent. It compares the desired config with the running config. Use --diff to verify.

Collection: Install with ansible-galaxy collection install arista.eos.

---
- name: Manage Arista EOS
  hosts: arista
  gather_facts: no
  connection: ansible.netcommon.network_cli
  tasks:
    - name: Show version
      arista.eos.eos_command:
        commands: show version
      register: result
    - name: Configure hostname
      arista.eos.eos_config:
        lines:
          - hostname A1
        match: exact
        replace: line
      register: config_result
    - name: Display config result
      ansible.builtin.debug:
        var: config_result

7. Managing Cisco NXOS: nxos_config and nxos_command

Cisco NX-OS (Nexus) uses a syntax similar to IOS but with differences. The cisco.nxos.nxos_config module is the config module.

Example: Configure VLAN ``yaml - name: Configure VLAN 300 cisco.nxos.nxos_config: lines: - vlan 300 - name PROD_VLAN ``

Example: Configure interface ``yaml - name: Set interface description cisco.nxos.nxos_config: lines: - description "Ansible managed" parents: - interface Ethernet1/1 ``

Command module: ``yaml - name: Show version cisco.nxos.nxos_command: commands: - show version register: version ``

Gotcha: NX-OS requires feature commands for certain features (e.g., feature interface-vlan). You must ensure these are enabled before configuring related features.

Idempotency: nxos_config is idempotent. Use --diff to see changes.

Collection: ansible-galaxy collection install cisco.nxos.

---
- name: Manage Cisco NXOS
  hosts: nxos
  gather_facts: no
  connection: ansible.netcommon.network_cli
  tasks:
    - name: Show interface
      cisco.nxos.nxos_command:
        commands: show interface brief
      register: result
    - name: Configure hostname
      cisco.nxos.nxos_config:
        lines:
          - hostname N1
        match: exact
        replace: line
      register: config_result
    - name: Display config result
      ansible.builtin.debug:
        var: config_result

8. Idempotency Challenges in Network Automation

Idempotency is the property that running a playbook multiple times produces the same result. In network automation, achieving idempotency is harder than on servers because:

1. Config modules compare text, not structured data. A slight difference in whitespace or ordering can cause false positives.
2. Stateful devices: Some commands have side effects (e.g., no shutdown on an already up interface).
3. Commit models: Junos requires explicit commit; if the module doesn't commit, the config is not applied.
4. CLI drift: If someone manually changes the config, Ansible may not detect it if the module uses a cached version.

Best practices: - Always use config modules (*_config) not command modules. - Use --diff to verify what will change. - Run playbooks with --check first. - For critical changes, use --diff and manual review. - Use ansible_network_os correctly to ensure the right module is used.

Example: Idempotent VLAN config ``yaml - name: Ensure VLAN 100 exists cisco.ios.ios_config: lines: - vlan 100 - name TEST_VLAN parents: - vlan 100 `` This will only create VLAN 100 if it doesn't exist. If it exists, no change.

Non-idempotent example: ``yaml - name: BAD - always sets hostname cisco.ios.ios_command: commands: - configure terminal - hostname {{ inventory_hostname }} `` This sets the hostname every run, even if it's already correct.

---
- name: Idempotency challenge
  hosts: cisco
  gather_facts: no
  connection: ansible.netcommon.network_cli
  tasks:
    - name: Ensure hostname is set
      cisco.ios.ios_config:
        lines:
          - hostname R1
        match: exact
        replace: line
      register: result
    - name: Check if changed
      ansible.builtin.debug:
        msg: "{{ 'Changed' if result.changed else 'Idempotent' }}"

9. Using ansible-vault for Device Credentials

Storing device credentials in plaintext is a security risk. ansible-vault encrypts sensitive data so it can be safely stored in version control.

Setup: ``bash ansible-vault create group_vars/all/vault.yml ` Inside the vault file: `yaml vault_ansible_password: cisco123 vault_enable_password: cisco ``

Reference in group_vars: ``yaml # group_vars/all/main.yml ansible_user: admin ansible_password: "{{ vault_ansible_password }}" ansible_become: yes ansible_become_method: enable ansible_become_password: "{{ vault_enable_password }}" ``

Running playbook: ``bash ansible-playbook -i inventory playbook.yml --ask-vault-pass ` Or use a vault password file: `bash echo 'my_vault_pass' > .vault_pass ansible-playbook -i inventory playbook.yml --vault-password-file .vault_pass ``

Best practices: - Use separate vault files for different environments (dev, prod). - Never commit the vault password to version control. - Use ansible-vault rekey to change passwords. - For automation (CI/CD), use a vault password file stored in a secrets manager (e.g., HashiCorp Vault, AWS Secrets Manager).

Gotcha: If you use ansible-vault encrypt on a file that already contains variables, ensure the variables are referenced correctly. Use {{ }} syntax.

# Create encrypted vault file
ansible-vault create group_vars/all/vault.yml
# Inside vault.yml:
# vault_ansible_user: admin
# vault_ansible_ssh_pass: secret123

# Use in playbook
ansible-playbook -i inventory site.yml --ask-vault-pass

10. Multi-Vendor Playbook Structure

Managing different vendors in one playbook requires careful structuring. Use group_vars per vendor type.

Inventory structure: `` production/ hosts.yml group_vars/ all.yml cisco_ios.yml juniper_junos.yml arista_eos.yml cisco_nxos.yml ``

hosts.yml example: ``yaml all: children: cisco_ios: hosts: switch1: ansible_host: 10.0.0.1 juniper_junos: hosts: switch2: ansible_host: 10.0.0.2 arista_eos: hosts: switch3: ansible_host: 10.0.0.3 cisco_nxos: hosts: switch4: ansible_host: 10.0.0.4 ``

group_vars/cisco_ios.yml: ``yaml ansible_network_os: cisco.ios.ios ansible_connection: ansible.netcommon.network_cli ``

Playbook example: ``yaml - name: Configure NTP on all devices hosts: all gather_facts: no tasks: - name: Set NTP server ansible.builtin.include_role: name: ntp_config ``

The role ntp_config would have tasks for each vendor using when: ansible_network_os == 'cisco.ios.ios' etc.

---
- name: Multi-vendor playbook
  hosts: all
  gather_facts: no
  connection: ansible.netcommon.network_cli
  tasks:
    - name: Show version (Cisco)
      cisco.ios.ios_command:
        commands: show version
      when: ansible_network_os == 'cisco.ios.ios'
      register: version
    - name: Show version (Juniper)
      junipernetworks.junos.junos_command:
        commands: show version
      when: ansible_network_os == 'junipernetworks.junos.junos'
      register: version
    - name: Display version
      ansible.builtin.debug:
        var: version.stdout_lines

11. Error Handling and Rollback Strategies

Network changes can break connectivity. Implement rollback strategies.

Using any_errors_fatal: ``yaml - hosts: all any_errors_fatal: true tasks: - name: Critical change cisco.ios.ios_config: lines: - interface GigabitEthernet0/1 - shutdown `` If this task fails, the playbook stops immediately.

Using block and rescue: ``yaml - name: Change with rollback block: - name: Apply new config cisco.ios.ios_config: lines: - interface GigabitEthernet0/1 - shutdown backup: yes - name: Test connectivity wait_for: host: "{{ ansible_host }}" port: 22 timeout: 10 register: result rescue: - name: Rollback cisco.ios.ios_config: src: "{{ playbook_dir }}/backup/{{ inventory_hostname }}_config.cfg" ``

Using NAPALM rollback: ``yaml - name: Apply config with rollback community.network.napalm_config: replace: yes config: "{{ desired_config }}" commit_changes: yes commit_confirm: 5 # confirm in 5 minutes register: napalm_result - name: If connectivity lost, rollback community.network.napalm_config: rollback: "{{ napalm_result.candidate_cfg }}" when: connectivity_lost ``

Backup config: ``yaml - name: Backup running config cisco.ios.ios_config: backup: yes backup_options: filename: "{{ inventory_hostname }}_config.cfg" dir_path: /backup/ ``

Gotcha: The backup option only saves the config before changes; it doesn't automatically rollback on failure.

---
- name: Error handling and rollback
  hosts: cisco
  gather_facts: no
  connection: ansible.netcommon.network_cli
  tasks:
    - name: Backup running config
      cisco.ios.ios_config:
        backup: yes
      register: backup
    - name: Apply risky config
      cisco.ios.ios_config:
        lines:
          - no ip routing
      register: result
      ignore_errors: yes
    - name: Rollback on failure
      cisco.ios.ios_config:
        src: "{{ backup.backup_path }}"
      when: result.failed
      register: rollback_result
    - name: Notify
      ansible.builtin.debug:
        msg: "Rollback performed"
      when: rollback_result.changed

12. Debugging and Logging Best Practices

When things go wrong, you need detailed logs.

Enable verbose output: ``bash ansible-playbook -i inventory playbook.yml -vvv ` - -v: basic info - -vv: more details - -vvv: connection details - -vvvv`: SSH debugging

Enable logging: In ansible.cfg: ``ini [defaults] log_path = /var/log/ansible/network.log ``

Use register and debug: ```yaml - name: Show version cisco.ios.ios_command: commands: - show version register: version_output

• name: Print version
• ansible.builtin.debug:
• var: version_output.stdout_lines
• ```

Check module return values: ```yaml - name: Configure VLAN cisco.ios.ios_config: lines: - vlan 100 register: config_result

• name: Debug config result
• ansible.builtin.debug:
• var: config_result
• ```

Use --syntax-check: ``bash ansible-playbook -i inventory playbook.yml --syntax-check ``

Test with --check and --diff: ``bash ansible-playbook -i inventory playbook.yml --check --diff ``

Gotcha: --check may not work with all network modules; some will still apply changes. Test in a lab first.

---
- name: Debugging and logging
  hosts: cisco
  gather_facts: no
  connection: ansible.netcommon.network_cli
  vars:
    ansible_network_debug: true
  tasks:
    - name: Show running-config
      cisco.ios.ios_command:
        commands: show running-config | include hostname
      register: result
    - name: Debug output
      ansible.builtin.debug:
        var: result
      when: ansible_network_debug | bool
    - name: Log to file
      ansible.builtin.copy:
        content: "{{ result.stdout }}"
        dest: /tmp/debug_output.txt
      delegate_to: localhost