Developing Advanced Automation with Red Hat Ansible Automation Platform

Each release of automation execution environments adds enhancements and improvements. Running the latest version of Red Hat Ansible Automation Platform might help increase the speed of your playbooks as the core components of Ansible and especially the modules provided with it are optimized over time.

An architectural optimization that you can make is to keep your control node close to the managed nodes from a networking perspective. Ansible relies heavily on network communication and the transfer of data. High latency connections or low bandwidth between the control node and its managed hosts degrade the execution time of playbooks.

Each play has a hidden task which runs first, using the ansible.builtin.setup module to collect facts from each host. Those facts provide information about the nodes that plays can use through the ansible_facts variable.

Collecting the facts on each remote host takes time. If you do not use those facts in your play, skip the fact gathering task by setting the gather_facts directive to false.

The following play disables fact gathering:

---
- name: Demonstrate disabling the facts gathering
  hosts: web_servers
  gather_facts: false

  tasks:
    - ansible.builtin.debug:
        msg: "gather_facts is set to False"

The following example uses the Linux time command to compare the execution time of the previous playbook when fact gathering is enabled, and when it is disabled.

[user@host ~]$ time ansible-navigator run \
> -m stdout  speed_facts.yml -i inventory

PLAY [Demonstrate activating the facts gathering] ************************

TASK [Gathering Facts] ***************************************************
ok: [www1.example.com]
ok: [www2.example.com]
ok: [www3.example.com]

TASK [debug] *************************************************************
ok: [www1.example.com] => {
    "msg": "gather_facts is set to True"
}
ok: [www2.example.com] => {
    "msg": "gather_facts is set to True"
}
ok: [www3.example.com] => {
    "msg": "gather_facts is set to True"
}

PLAY RECAP ***************************************************************
www1.example.com    : ok=1    changed=0    unreachable=0  ...
www2.example.com    : ok=1    changed=0    unreachable=0  ...
www3.example.com    : ok=1    changed=0    unreachable=0  ...

real 0m6.171s
user	0m2.146s
sys	0m1.118s
[user@host ~]$ time ansible-navigator run \
> -m stdout  speed_nofacts.yml -i inventory

PLAY [Demonstrate disabling the facts gathering] *************************

TASK [debug] *************************************************************
ok: [www1.example.com] => {
    "msg": "gather_facts is set to False"
}
ok: [www2.example.com] => {
    "msg": "gather_facts is set to False"
}
ok: [www3.example.com] => {
    "msg": "gather_facts is set to False"
}

PLAY RECAP ***************************************************************
www1.example.com    : ok=1    changed=0    unreachable=0  ...
www2.example.com    : ok=1    changed=0    unreachable=0  ...
www3.example.com    : ok=1    changed=0    unreachable=0  ...

real 0m1.336s
user	0m1.116s
sys	0m0.246s

Playbooks often use the ansible_facts['hostname'], ansible_hostname, ansible_facts['nodename'], or ansible_nodename variables to refer to the host currently being processed. Those variables come from the fact gathering task, but you can usually replace them with the inventory_hostname and inventory_hostname_short magic variables.

Even if you disable fact gathering, you can choose to collect facts manually at any point in a play by running the ansible.builtin.setup module as a task, and the collected facts are then available for subsequent plays in the playbook.

Ansible uses cache plug-ins to store gathered facts or inventory source data gathered by a play. You can take advantage of the fact cache to limit how many times you need to gather facts by reusing facts gathered earlier.

Fact caching is always enabled. You can only use one cache plug-in at a time. The memory cache plug-in is enabled by default if you do not change the ansible-navigator configuration. This plug-in caches facts gathered during the current Ansible run.

You can take advantage of this to improve performance for playbooks that contain multiple plays. The first play can gather facts for all the hosts for which you need facts in the playbook. Subsequent plays can then disable fact gathering and use the cached facts from the first play, improving performance.

The following playbook contains two simple plays and illustrates how this works.

- name: Gather facts for everyone
  hosts: all
  gather_facts: true

  # any tasks we might want for the first play
  # if you do not have tasks, "setup" will still run

- name: The next play, does not gather facts
  hosts: all
  gather_facts: false

  tasks:
    - name: Show that we still know the facts
      ansible.builtin.debug:
        var: ansible_facts

[defaults]
gathering=smart

Fact caching also works on automation controller by default. In addition, when you edit a job template in automation controller, you can select the Enable Fact Storage checkbox. This changes the fact caching plug-in to one that stores facts gathered by jobs launched by that template, so that they can be reused between multiple playbook runs. (You need to periodically run a job that gathers facts to update the automation controller fact storage if you use this feature.)

If that checkbox is not selected, automation controller uses the default memory fact caching plug-in, which only caches facts during a particular job run.

You can also selectively limit fact gathering if you disable automatic fact gathering and run the ansible.builtin.setup module as an explicit task with its gather_subset option. This is generally quicker than gathering all available facts.

The possible subsets include all, min, hardware, network, virtual, ohai, and facter. If you exclude the all subset, then you still get the min subset.

For example, if you only want to retrieve facts in the network subset, you can include it but exclude all and min:

- name: A play that gathers some facts
  hosts: all
  gather_facts: false

  tasks:
    - name: Collect only network-related facts
      ansible.builtin.setup:
        gather_subset:
          - '!all'
          - '!min'
          - network

When Ansible is running a play, it runs the first task on every host in the current batch, and then runs the second task on every host in the current batch, and so on until the play completes. The forks parameter controls how many connections Ansible can have active at the same time. By default, this is set to 5, which means that even if there are 100 hosts to process on the current task, Ansible only communicates with them in groups of five. After it has communicated with all 100 hosts, Ansible moves to the next task.

By increasing the forks value, Ansible runs each task simultaneously on more hosts, and the playbook usually completes in less time. For example, if you set forks to 100, Ansible can attempt to open connections to all 100 hosts in the previous example simultaneously. This places more load on the control node, which still needs enough time to communicate with each of the hosts.

You can specify the number of forks to use in the Ansible configuration file, or you can use the -f option with the ansible-navigator command.

The following example shows the forks parameter set to 100 under the [defaults] section of the ansible.cfg configuration file.

[defaults]
forks=100

Because the forks value specifies how many worker processes Ansible starts, a number that is too high might slow down your control node and your network. Try first with a conservative value, such as 20 or 50, and increase that number step by step, each time monitoring your system resources.

Some modules accept a list of items to work on and do not require the use of a loop. This approach can increase efficiency, because the module is only called one time.

The modules for managing operating system packages work this way. The following example uses the ansible.builtin.yum module to install several packages in a single transaction, which is the most efficient way to install a group of packages.

---
- name: Install the packages on the web servers
  hosts: web_servers
  become: true
  gather_facts: false

  tasks:
    - name: Ensure the packages are installed
      ansible.builtin.yum:
        name:
          - httpd
          - mod_ssl
          - httpd-tools
          - mariadb-server
          - mariadb
          - php
          - php-mysqlnd
        state: present

The preceding playbook would be equivalent to running the following command from the shell prompt:

[root@host ~]# yum install httpd mod_ssl httpd-tools \
> mariadb-server mariadb php php-mysqlnd

The following example is not efficient. It uses a loop to install the packages one at a time:

---
- name: Install the packages on the web servers
  hosts: web_servers
  become: true
  gather_facts: false

  tasks:
    - name: Ensure the packages are installed
      ansible.builtin.yum:
        name: "{{ item }}"
        state: present
      loop:
        - httpd
        - mod_ssl
        - httpd-tools
        - mariadb-server
        - mariadb
        - php
        - php-mysqlnd

The second example is equivalent to running multiple yum commands:

[root@host ~]# yum install httpd
[root@host ~]# yum install mod_ssl
[root@host ~]# yum install httpd-tools
[root@host ~]# yum install mariadb-server
[root@host ~]# yum install mariadb
[root@host ~]# yum install php
[root@host ~]# yum install php-mysqlnd

The second example is slower and less efficient because Ansible runs the ansible.builtin.yum module seven times, starting a process for the module seven times, and doing dependency resolution seven times.

Not all Ansible modules accept a list for the name parameter. For example, the ansible.builtin.service module only accepts a single value for its name parameter, and you need a loop to operate on multiple items:

- name: Starting the services on the web servers
  hosts: web_servers
  become: true
  gather_facts: false

  tasks:
    - name: Ensure the services are started
      ansible.builtin.service:
        name: "{{ item }}"
        state: started
        enabled: true
      loop:
        - httpd
        - mariadb

Use the ansible-navigator doc command to get information about what types of values different module arguments can accept:

[user@host ~]$ ansible-navigator doc ansible.builtin.yum -m stdout
...output omitted...
- name
        A package name or package specifier with version, like
        `name-1.0'.
        If a previous version is specified, the task also needs to
        turn `allow_downgrade' on. See the `allow_downgrade'
        documentation for caveats with downgrading packages.
        When using state=latest, this can be `'*'' which means run
        `yum -y update'.
        You can also pass a url or a local path to a rpm file (using
        state=present). To operate on several packages this can accept
        a comma separated string of packages or (as of 2.0) a list of
        packages.
        (Aliases: pkg)[Default: (null)]
...output omitted...
[user@host ~]$ ansible-navigator doc ansible.builtin.service -m stdout
...output omitted...
= name
        Name of the service.

        type: str
...output omitted...

The ansible.builtin.copy module recursively copies files and directories to managed hosts. When the directory is large, with many files, the copy can take a long time. If you run the playbook multiple times, subsequent copies take less time because the module only copies the files that are different.

However, it is generally more efficient to use the ansible.posix.synchronize module to copy large numbers of files to managed hosts. This module uses rsync in the background and is usually faster than the ansible.builtin.copy module. By setting the delete option to true, the module can also remove files on the target that no longer exist on the source.

The following playbook uses the ansible.posix.synchronize module to recursively copy the web_content directory to the web servers.

---
- name: Deploy the web content on the web servers
  hosts: web_servers
  become: true
  gather_facts: false

  tasks:
    - name: Ensure web content is updated
      ansible.posix.synchronize:
        src: web_content/
        dest: /var/www/html

The ansible.builtin.lineinfile module inserts or removes lines in a file, such as configuration directives in a configuration file. The following playbook updates the Apache HTTP Server configuration file by replacing several lines.

---
- name: Configure the Apache HTTP Server
  hosts: web_servers
  become: true
  gather_facts: false

  tasks:

    - name: Ensure proper configuration of the Apache HTTP Server
      ansible.builtin.lineinfile:
        dest: /etc/httpd/conf/httpd.conf
        regexp: "{{ item.regexp }}"
        line: "{{ item.line }}"
        state: present
      loop:
       - regexp: '^Listen 80$'
         line: 'Listen 8181'
       - regexp: '^ServerAdmin root@localhost'
         line: 'ServerAdmin support@example.com'
       - regexp: '^DocumentRoot "/var/www/html"'
         line: 'DocumentRoot "/var/www/web"'
       - regexp: '^<Directory "/var/www/html">'
         line: '<Directory "/var/www/web">'

When used with a loop, the ansible.builtin.lineinfile module is inefficient (and can be error-prone). In this situation, use either the ansible.builtin.template or the ansible.builtin.copy module instead.

---
- name: Configure the Apache HTTP Server
  hosts: web_servers
  become: true
  gather_facts: false

  tasks:
    - name: Ensure proper configuration of the Apache HTTP Server
      ansible.builtin.template:
        src: httpd.conf.j2
        dest: /etc/httpd/conf/httpd.conf

The httpd.conf.j2 template file in the previous example is the customized version of the httpd.conf file.

To run a task on a remote node, Ansible performs several SSH operations to copy the module and all its data to the remote node and to run the module. To increase the performance of your playbook, you can activate the pipelining feature. With pipelining, Ansible establishes fewer SSH connections.

To activate pipelining, set the ANSIBLE_PIPELINING environment variable to true in the execution-environment section of the ansible-navigator.yml configuration file.

---
ansible-navigator:
  ansible:
    config: ./ansible.cfg

  execution-environment:
    image: ee-supported-rhel8:latest
    pull-policy: missing
    environment-variables:
      set:
        ANSIBLE_PIPELINING: true

Ansible does not use pipelining by default because the feature requires that the requiretty sudo option on all the remote nodes be disabled. On Red Hat Enterprise Linux 8, that sudo option is disabled by default, but it might be active on other systems.

To disable the option, use the visudo command to edit the /etc/sudoers file on your managed nodes and disable the requiretty option:

[root@host ~]# visudo
...output omitted...
Defaults !requiretty
...output omitted...

You can use the timer, profile_tasks, and profile_roles callback plug-ins to help identify slow tasks and roles.

The timer plug-in displays the duration of playbook execution.

The profile_tasks plug-in displays the start time of each task, and the time spent on each task, sorted in descending order, at the end of the playbook execution.

The profile_roles plug-in displays the time spent on each role at the end of the output, sorted in descending order.

To activate these plug-ins, add or update the callbacks_enabled directive in the ansible.cfg file.

[defaults]
callbacks_enabled=timer, profile_tasks, profile_roles

You do not have to enable all three plug-ins; select the ones that you need.

The following example shows the output of the ansible-navigator command when you activate the three plug-ins.

[user@host ~]$ ansible-navigator  run \
> -m stdout deploy_webservers.yml
...output omitted...
PLAY RECAP ***************************************************************
www1.example.com    : ok=9    changed=7    unreachable=0 ...
www2.example.com    : ok=10   changed=9    unreachable=0 ...
www3.example.com    : ok=10   changed=9    unreachable=0 ...

Playbook run took 0 days, 0 hours, 0 minutes, 21 seconds
Wednesday 08 September 2021  19:12:31 +0000 (0:00:00.858)       0:00:21.325 ***
===============================================================================
apache : Ensure httpd packages are installed ---------------------------- 7.14s
haproxy : Ensure haproxy packages are present --------------------------- 1.78s
apache : Ensure SELinux allows httpd connections to a remote database --- 1.72s
Gathering Facts --------------------------------------------------------- 3.89s
haproxy : Ensure haproxy configuration is set --------------------------- 1.12s
haproxy : Ensure haproxy is started and enabled ------------------------- 0.92s
webapp : Ensure stub web content is deployed ---------------------------- 0.86s
firewall : Ensure Firewall Sources Configuration ------------------------ 0.85s
firewall : Ensure Firewall Port Configuration --------------------------- 0.84s
firewall : Ensure Firewall Service Configuration ------------------------ 0.79s
apache : Ensure httpd service is started and enabled -------------------- 0.71s
firewall : Firewall Port Configuration ---------------------------------- 0.70s
Wednesday 08 September 2021  19:12:31 +0000 (0:00:00.867)       0:00:21.333 ***
===============================================================================
apache ------------------------------------------------------------------ 9.57s
gather_facts ------------------------------------------------------------ 3.89s
haproxy ----------------------------------------------------------------- 3.81s
firewall ---------------------------------------------------------------- 3.18s
webapp ------------------------------------------------------------------ 0.86s
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
total ------------------------------------------------------------------ 21.31s