Ansible Network Automation: Surviving Idempotency with network_cli & NAPALM

It's 2 AM on a Saturday. I'm on-call, and our network automation pipeline just pushed a bad VLAN config to 200 Cisco switches. The symptom: all trunk ports flapped, causing a 15-minute outage for the entire east coast data center. The root cause? A playbook using ios_command with commands: 'vlan 100' instead of ios_config with lines: 'vlan 100'. The ios_command module doesn't enforce idempotency—it blindly sends the command, and the switch accepted it even though the VLAN existed, causing a momentary interface reset. That night, I learned the hard way that network automation is not server automation. You can't just SSH in and run commands; you need structured modules that understand the device's state.

Historically, Ansible started as a server automation tool. The command and shell modules worked fine for Linux, but network devices are state machines with fragile CLI parsers. Early adopters used connection: local and raw SSH, which led to brittle playbooks. The Ansible network team responded with connection plugins like network_cli and vendor-specific modules. But even today, many engineers fall into the trap of treating network devices like Linux boxes.

This article covers the production patterns I've developed over 5 years of automating Cisco, Juniper, Arista, and Nexus gear. We'll dive into the network_cli connection plugin, the difference between ios_command and ios_config, network-agnostic modules, NAPALM integration, and how to avoid the idempotency landmines that will take down your network. I'll also show you how to secure credentials with ansible-vault and debug when things go wrong.

By the end, you'll know exactly which module to use for which task, how to structure your playbooks for reliability, and what to do when a switch doesn't respond as expected. Let's fix your automation before it fixes you.

1. The network_cli Connection Plugin: Why local is Dead

The network_cli connection plugin (ansible.netcommon.network_cli) is the standard for SSH-based network automation in Ansible 2.9+. It replaces the deprecated local connection and provides persistent SSH sessions, privilege escalation (enable mode), and automatic prompt handling.

Key differences from local: - network_cli maintains a single SSH connection for the entire playbook, reducing overhead. - It handles enable mode automatically when ansible_become: yes and ansible_become_method: enable are set. - It parses device prompts and waits for the correct prompt before sending commands.

Production setup: ``yaml # group_vars/network.yml ansible_connection: ansible.netcommon.network_cli ansible_network_os: cisco.ios.ios ansible_user: admin ansible_password: "{{ vault_ansible_password }}" ansible_become: yes ansible_become_method: enable ansible_become_password: "{{ vault_enable_password }}" ansible_command_timeout: 30 ``

Gotcha: If you set ansible_connection: ssh (the default), network modules will fail because they expect network_cli. Always explicitly set it.

Verification: ``bash ansible-inventory --host switch1 --yaml | grep ansible_connection # Should output: ansible_connection: ansible.netcommon.network_cli ``

2. Cisco IOS: ios_command vs ios_config

The two most common Cisco IOS modules are cisco.ios.ios_command and cisco.ios.ios_config. They serve different purposes:

• ios_command: Sends arbitrary CLI commands and returns output. Not idempotent. Use for show commands only.
• ios_config: Manages configuration sections idempotently. It reads the running config, compares it with the desired state, and applies only the necessary changes.

Example: Correct usage ``yaml - name: Configure VLAN 100 on interface cisco.ios.ios_config: lines: - vlan 100 parents: - interface GigabitEthernet0/1 ` This will only apply vlan 100` if it's not already present under the interface.

Example: Wrong usage (causes flap) ``yaml - name: BAD - using ios_command for config cisco.ios.ios_command: commands: - interface GigabitEthernet0/1 - vlan 100 `` This sends the commands blindly, causing the switch to re-apply VLAN 100, which resets the interface.

Idempotency check: Use --diff flag to see what changes Ansible will make: ``bash ansible-playbook -i inventory playbook.yml --diff --check ``

The --check mode with ios_config will simulate the change without applying it.

3. Network-Agnostic Modules: cli_config and cli_command

For multi-vendor environments, Ansible provides network-agnostic modules: ansible.netcommon.cli_config and ansible.netcommon.cli_command. These modules rely on the ansible_network_os variable to determine the correct CLI syntax.

Example: cli_config ``yaml - name: Set hostname using agnostic module ansible.netcommon.cli_config: config: "hostname {{ inventory_hostname }}" ` This works on Cisco IOS, Junos, EOS, and NXOS as long as ansible_network_os` is set correctly.

Example: cli_command ``yaml - name: Show version ansible.netcommon.cli_command: command: show version register: version_output ``

Limitations: - cli_config does not support structured config; it sends raw text. For idempotent structured config, use vendor-specific modules. - cli_command does not handle privilege escalation automatically; you may need to include enable in the command string.

When to use: - Quick ad-hoc commands across vendors. - When you don't have vendor-specific collections installed. - For read-only operations (show commands).

Gotcha: The cli_config module uses the configure terminal command on Cisco devices. If you send a command that requires enable mode, you must set ansible_become: yes.

4. NAPALM Integration via community.network

NAPALM (Network Automation and Programmability Abstraction Layer with Multivendor support) provides a unified API for network devices. The community.network.napalm_* modules (e.g., napalm_cli, napalm_config, napalm_get_facts) use NAPALM under the hood.

Setup: ``bash pip install napalm==4.1.0 ansible-galaxy collection install community.network ``

Inventory variables: ``yaml ansible_connection: community.network.napalm ansible_network_os: cisco.ios.ios # or junipernetworks.junos.junos, etc. napalm_platform: ios # must match napalm's platform name ``

Example: Get facts ``yaml - name: Gather facts via NAPALM community.network.napalm_get_facts: filter: ['facts', 'interfaces'] register: napalm_facts ``

Example: Configure using NAPALM ``yaml - name: Configure using NAPALM community.network.napalm_config: replace: yes config: | interface GigabitEthernet0/1 description NAPALM managed ``

Advantages: - Works with devices that have API access (NETCONF, RESTCONF) – more reliable than CLI scraping. - napalm_get_facts returns structured data (JSON) instead of CLI text. - napalm_config supports replace and commit operations (important for Junos).

Disadvantages: - Requires NAPALM on the control node, not on the device. - Some platforms require additional libraries (e.g., junos-eznc for Junos). - Not all NAPALM methods are idempotent; napalm_config with replace is, but napalm_cli is not.

5. Managing Junos: junos_config and junos_command

Juniper Junos devices use a different paradigm: candidate config and commit. The junipernetworks.junos.junos_config module handles this.

Example: Configure an interface ``yaml - name: Configure interface ge-0/0/0 junipernetworks.junos.junos_config: lines: - set interfaces ge-0/0/0 description "Ansible managed" - set interfaces ge-0/0/0 unit 0 family inet address 10.0.0.1/24 comment: "Updated by Ansible" ``

Commit options: ``yaml - name: Commit with confirm junipernetworks.junos.junos_config: lines: - set system hostname new-hostname commit: yes confirm: 5 # confirm in 5 minutes ``

Rollback: ``yaml - name: Rollback to previous config junipernetworks.junos.junos_config: rollback: 1 ``

Gotcha: Junos modules require ansible_network_os: junipernetworks.junos.junos and the junipernetworks.junos collection. Also, you need junos-eznc Python library on the control node: pip install junos-eznc.

Command module: ``yaml - name: Show interface status junipernetworks.junos.junos_command: commands: - show interfaces terse register: output ``

Idempotency: junos_config only applies changes that differ from the candidate config. Use --diff to see what will change.

6. Managing Arista EOS: eos_config and eos_command

Arista EOS is similar to Cisco IOS but with some differences. The arista.eos.eos_config module is the primary config module.

Example: Configure VLAN ``yaml - name: Configure VLAN 200 arista.eos.eos_config: lines: - vlan 200 - name WEB_VLAN parents: - vlan 200 ``

Example: Configure interface ``yaml - name: Set interface description arista.eos.eos_config: lines: - description "Ansible managed" parents: - interface Ethernet1 ``

Command module: ``yaml - name: Show running config arista.eos.eos_command: commands: - show running-config register: run_cfg ``

Gotcha: Arista EOS uses enable mode like Cisco. Set ansible_become: yes and ansible_become_method: enable.

Idempotency: eos_config is idempotent. It compares the desired config with the running config. Use --diff to verify.

Collection: Install with ansible-galaxy collection install arista.eos.

7. Managing Cisco NXOS: nxos_config and nxos_command

Cisco NX-OS (Nexus) uses a syntax similar to IOS but with differences. The cisco.nxos.nxos_config module is the config module.

Example: Configure VLAN ``yaml - name: Configure VLAN 300 cisco.nxos.nxos_config: lines: - vlan 300 - name PROD_VLAN ``

Example: Configure interface ``yaml - name: Set interface description cisco.nxos.nxos_config: lines: - description "Ansible managed" parents: - interface Ethernet1/1 ``

Command module: ``yaml - name: Show version cisco.nxos.nxos_command: commands: - show version register: version ``

Gotcha: NX-OS requires feature commands for certain features (e.g., feature interface-vlan). You must ensure these are enabled before configuring related features.

Idempotency: nxos_config is idempotent. Use --diff to see changes.

Collection: ansible-galaxy collection install cisco.nxos.

8. Idempotency Challenges in Network Automation

Idempotency is the property that running a playbook multiple times produces the same result. In network automation, achieving idempotency is harder than on servers because:

1. Config modules compare text, not structured data. A slight difference in whitespace or ordering can cause false positives.
2. Stateful devices: Some commands have side effects (e.g., no shutdown on an already up interface).
3. Commit models: Junos requires explicit commit; if the module doesn't commit, the config is not applied.
4. CLI drift: If someone manually changes the config, Ansible may not detect it if the module uses a cached version.

Best practices: - Always use config modules (*_config) not command modules. - Use --diff to verify what will change. - Run playbooks with --check first. - For critical changes, use --diff and manual review. - Use ansible_network_os correctly to ensure the right module is used.

Example: Idempotent VLAN config ``yaml - name: Ensure VLAN 100 exists cisco.ios.ios_config: lines: - vlan 100 - name TEST_VLAN parents: - vlan 100 `` This will only create VLAN 100 if it doesn't exist. If it exists, no change.

Non-idempotent example: ``yaml - name: BAD - always sets hostname cisco.ios.ios_command: commands: - configure terminal - hostname {{ inventory_hostname }} `` This sets the hostname every run, even if it's already correct.

9. Using ansible-vault for Device Credentials

Storing device credentials in plaintext is a security risk. ansible-vault encrypts sensitive data so it can be safely stored in version control.

Setup: ``bash ansible-vault create group_vars/all/vault.yml ` Inside the vault file: `yaml vault_ansible_password: cisco123 vault_enable_password: cisco ``

Reference in group_vars: ``yaml # group_vars/all/main.yml ansible_user: admin ansible_password: "{{ vault_ansible_password }}" ansible_become: yes ansible_become_method: enable ansible_become_password: "{{ vault_enable_password }}" ``

Running playbook: ``bash ansible-playbook -i inventory playbook.yml --ask-vault-pass ` Or use a vault password file: `bash echo 'my_vault_pass' > .vault_pass ansible-playbook -i inventory playbook.yml --vault-password-file .vault_pass ``

Best practices: - Use separate vault files for different environments (dev, prod). - Never commit the vault password to version control. - Use ansible-vault rekey to change passwords. - For automation (CI/CD), use a vault password file stored in a secrets manager (e.g., HashiCorp Vault, AWS Secrets Manager).

Gotcha: If you use ansible-vault encrypt on a file that already contains variables, ensure the variables are referenced correctly. Use {{ }} syntax.

10. Multi-Vendor Playbook Structure

Managing different vendors in one playbook requires careful structuring. Use group_vars per vendor type.

Inventory structure: `` production/ hosts.yml group_vars/ all.yml cisco_ios.yml juniper_junos.yml arista_eos.yml cisco_nxos.yml ``

hosts.yml example: ``yaml all: children: cisco_ios: hosts: switch1: ansible_host: 10.0.0.1 juniper_junos: hosts: switch2: ansible_host: 10.0.0.2 arista_eos: hosts: switch3: ansible_host: 10.0.0.3 cisco_nxos: hosts: switch4: ansible_host: 10.0.0.4 ``

group_vars/cisco_ios.yml: ``yaml ansible_network_os: cisco.ios.ios ansible_connection: ansible.netcommon.network_cli ``

Playbook example: ``yaml - name: Configure NTP on all devices hosts: all gather_facts: no tasks: - name: Set NTP server ansible.builtin.include_role: name: ntp_config ``

The role ntp_config would have tasks for each vendor using when: ansible_network_os == 'cisco.ios.ios' etc.

11. Error Handling and Rollback Strategies

Network changes can break connectivity. Implement rollback strategies.

Using any_errors_fatal: ``yaml - hosts: all any_errors_fatal: true tasks: - name: Critical change cisco.ios.ios_config: lines: - interface GigabitEthernet0/1 - shutdown `` If this task fails, the playbook stops immediately.

Using block and rescue: ``yaml - name: Change with rollback block: - name: Apply new config cisco.ios.ios_config: lines: - interface GigabitEthernet0/1 - shutdown backup: yes - name: Test connectivity wait_for: host: "{{ ansible_host }}" port: 22 timeout: 10 register: result rescue: - name: Rollback cisco.ios.ios_config: src: "{{ playbook_dir }}/backup/{{ inventory_hostname }}_config.cfg" ``

Using NAPALM rollback: ``yaml - name: Apply config with rollback community.network.napalm_config: replace: yes config: "{{ desired_config }}" commit_changes: yes commit_confirm: 5 # confirm in 5 minutes register: napalm_result - name: If connectivity lost, rollback community.network.napalm_config: rollback: "{{ napalm_result.candidate_cfg }}" when: connectivity_lost ``

Backup config: ``yaml - name: Backup running config cisco.ios.ios_config: backup: yes backup_options: filename: "{{ inventory_hostname }}_config.cfg" dir_path: /backup/ ``

Gotcha: The backup option only saves the config before changes; it doesn't automatically rollback on failure.

12. Debugging and Logging Best Practices

When things go wrong, you need detailed logs.

Enable verbose output: ``bash ansible-playbook -i inventory playbook.yml -vvv ` - -v: basic info - -vv: more details - -vvv: connection details - -vvvv`: SSH debugging

Enable logging: In ansible.cfg: ``ini [defaults] log_path = /var/log/ansible/network.log ``

Use register and debug: ```yaml - name: Show version cisco.ios.ios_command: commands: - show version register: version_output

• name: Print version
• ansible.builtin.debug:
• var: version_output.stdout_lines
• ```

Check module return values: ```yaml - name: Configure VLAN cisco.ios.ios_config: lines: - vlan 100 register: config_result

• name: Debug config result
• ansible.builtin.debug:
• var: config_result
• ```

Use --syntax-check: ``bash ansible-playbook -i inventory playbook.yml --syntax-check ``

Test with --check and --diff: ``bash ansible-playbook -i inventory playbook.yml --check --diff ``

Gotcha: --check may not work with all network modules; some will still apply changes. Test in a lab first.