Bookmark this page

Transforming Data with Filters

Objectives

  • Format, parse, manipulate, and define the values of variables by using filters.

Ansible Filters

Ansible applies variable values to playbooks and templates by using Jinja2 expressions. For example, the following Jinja2 expression replaces the name of the variable inside the double braces with its value:

{{ variable }}

Jinja2 expressions also support filters. Filters are used to modify or process the value from the variable that is placed in the playbook or template. Some filters are provided by the Jinja2 language; others are included with Red Hat Ansible Automation Platform as plug-ins. You can also create custom filters. Filters can be extremely useful for preparing data for use in your playbook or template.

To use a filter in a Jinja2 expression, add a pipe (|) character after the variable name or value and then write the filter or filter expression. You can specify multiple filters in a pipeline, separating each filter or filter expression with a pipe character.

{{ variable | filter }}

Gathering Facts about Network Devices

A number of filters and lookup plug-ins are available to collect and process network information for Ansible automation. These filters and plug-ins can be useful with fact gathering when you configure the network managed nodes in your environment.

Network Information Filters

You can validate and manipulate networking data stored in variables and facts with the ansible.utils.ipaddr filter. The ansible.utils.ipaddr filter can validate the syntax of IP addresses, filter out bad data, convert from Variable Length Subnet Mask (VLSM) to Classless Inter-domain Routing (CIDR) subnet prefix notation, perform subnet math, or find the next usable address in a network range.

This section explores some uses of the ansible.utils.ipaddr filter.

Testing IP Addresses

In its simplest form, the ansible.utils.ipaddr filter with no arguments accepts a single value. If the value is an IP address, then the filter expression returns the IP address. If the value is not an IP address, then the filter expression returns false.

If the value is a list, then the filter expression returns the valid IP addresses but not the invalid ones. If all the items are invalid, then the filter expression returns an empty list.

{{ my_hosts_list | ansible.utils.ipaddr }}

You can also pass arguments to the ansible.utils.ipaddr filter. These arguments can make the filter return different information based on the data input.

Filtering Data

If you pass a list of items to the ansible.utils.ipaddr filter, then the filter attempts to convert the items that appear to be valid addresses and filter out items that are not. Use the following options to the ansible.utils.ipaddr filter to restrict results:

OptionDescription
host The host option in the ansible.utils.ipaddr('host') filter expression returns valid individual IP addresses (in the correct CIDR prefix format). This filter does not return networks.
net The net option in the ansible.utils.ipaddr('net') filter expression returns valid network specifications and converts the netmask into a CIDR prefix if necessary. This filter does not return individual IP addresses.
private The private option in the ansible.utils.ipaddr('private') filter expression returns IP addresses or network ranges that are in ranges reserved by Internet Assigned Numbers Authority (IANA) to be private. These ranges include RFC 1918 address spaces and IPv6 site-local addresses at a minimum. This filter does not return public addresses and networks.
public The public option in the ansible.utils.ipaddr('public') filter expression returns addresses and networks in public (globally routable) address spaces, according to IANA. This filter does not return private addresses and networks.

Manipulating IP Addresses

You can use a number of the ansible.utils.ipaddr options to format networking data from an IPv4 or IPv6 address.

OptionDescription
address The address option in the ansible.utils.ipaddr('address') filter expression returns the address without a subnet mask or prefix, such as 192.0.2.1 for the 192.0.2.1/24 address.
netmask The netmask option in the ansible.utils.ipaddr('netmask') filter expression returns the variable length subnet mask, such as 255.255.255.0 for the 192.0.2.1/24 address.
prefix The prefix option in the ansible.utils.ipaddr('prefix') filter expression returns the CIDR prefix, such as 24 for the 192.0.2.1/24 address.
revdns The revdns option in the ansible.utils.ipaddr('revdns') filter expression returns the IPv4 or IPv6 address in DNS PTR record format, such as 1.2.0.192.in-addr.arpa. for the 192.0.2.1/24 address.
network The network option in the ansible.utils.ipaddr('network') filter expression returns the network's network address, such as 192.0.2.0 for the 192.0.2.1/24 address.
broadcast The broadcast option in the ansible.utils.ipaddr('broadcast') filter expression returns the network's broadcast address, such as 192.0.2.255 for the 192.0.2.1/24 address.

Important

The broadcast option should not be used with an IPv6 address/prefix because that protocol does not assign a broadcast address. Other options should work with IPv6 addresses as expected.

Calculating, Validating, and Reformatting Network Information

You can specify options to the ansible.utils.ipaddr filter to generate an address and identify valid addresses in a network.

OptionDescription
integer You can display an IP address on a given network by passing an integer to the ansible.utils.ipaddr filter. For example, the {{ '192.0.2.0/24' | ansible.utils.ipaddr(5) }} expression returns the fifth address (192.0.2.5/24) on the 192.0.2.0/24 network.
range_usable The range_usable option in the ansible.utils.ipaddr('range_usable') filter expression returns the range of usable host addresses for a network, such as 192.0.2.1-192.0.2.254 for the 192.0.2.0/24 network.
next_usable If you provide an existing address on a network, then the next_usable option in the ansible.utils.ipaddr('next_usable') filter expression returns the next address or an empty value if the end of the network has been reached. For example, the {{ '192.0.2.5/24' | ansible.utils.ipaddr('next_usable') }} expression returns the 192.0.2.6 address.

The following filters also belong to the ansible.utils collection:

ansible.utils.network_in_usable

If you pass a network specification to the ansible.utils.network_in_usable filter, then you can specify a particular address as an option to determine if that address is usable for hosts on that network. You might use this filter to verify that an address is not reserved for broadcasts or other uses. The following expression returns the true Boolean value:

{{ '192.0.2.0/24' | ansible.utils.network_in_usable('192.0.2.5') }}

If you replace 192.0.2.5 with 192.0.2.255 (the reserved broadcast address) or 192.0.3.2 (an address on a different network), then the expression returns the false Boolean value.

ansible.utils.cidr_merge

You can use the ansible.utils.cidr_merge filter to merge a list of subnets and individual hosts into the minimal representation. For example, consider the following variables:

vars:
  my_networks:
    - 10.0.1.0/24
    - 10.0.2.0/24
    - 10.0.3.0/24
    - 10.0.100.0/24
    - 2001:db8::/64
    - 2001:db8:0:1::/64
  merged_networks: "{{ my_networks | ansible.utils.cidr_merge }}"

The merged_networks variable reduces the networks from the my_networks variable into the following networks:

    - 10.0.1.0/24
    - 10.0.2.0/23
    - 10.0.100.0/24
    - 2001:db8::/63
ansible.utils.ipwrap

The ansible.utils.ipwrap filter puts brackets around the address part of items that appear to be IPv6 addresses. The filter passes all other values unchanged. This is useful in templates for certain configuration files that expect this syntax for IPv6 addresses. Consider the following variables:

vars:
  provided:
    - 192.168.2.1
    - 10.0.0.128/25
    - 172.24.10.0/255.255.255.0
    - ff02::1
    - ::1
    - 2001::1/64
    - learn.spidernet.pl
  brackets: "{{ provided | ansible.utils.ipwrap }}"

The brackets variable contains all the entries in the provided variable and formats the IPv6 addresses and networks so that they contain brackets:

    - 192.168.2.1
    - 10.0.0.128/25
    - 172.24.10.0/255.255.255.0
    - [ff02::1]
    - [::1]
    - [2001::1]/64
    - learn.spidernet.pl

Note

If desired, then you can use the ansible.utils.ipwrap filter to exclude the learn.spidernet.pl list item. The following expression for the previous example does not add the learn.spidernet.pl list item to the brackets variable.

  brackets: "{{ provided | ansible.utils.ipaddr | ansible.utils.ipwrap }}"

General Purpose Filters

Several general purpose filters are available for you to use in your playbooks. The following table briefly describes some of these filters:

FilterDescription
bool Generates a Boolean value. This filter can help avoid problems where a developer assigns the "true" or "false" (quoted) value to a variable rather than the true or false (unquoted) value to a variable.
default Provides a value if the variable is undefined.
dict2items Converts a dictionary to a list.
from_json Indicates that the data in the variable comes from the JSON format.
from_yaml Indicates that the data in the variable comes from the YAML format.
map Generates a list of values from array items that contain a specific attribute key.
password_hash Transforms a text string to a hashed password.
regex_replace Removes or replaces a text pattern.
selectattr Selects items from an array that match a particular value.
split Generates an array that is delimited by the pattern used in the split filter expression.
to_json Converts data into formatted JSON. Use the to_nice_json filter to convert data into nicely formatted JSON.
to_yaml Converts data into formatted YAML. Use the to_nice_yaml filter to convert data into nicely formatted YAML.

Several of the filters in the previous table are available as part of the ansible-core RPM package. If the ansible-core package is installed on your local machine, then you can list all the files in that package and search for a specific plug-in.

[user@host ~]$ rpm -ql ansible-core | grep to_nice_yaml
/usr/lib/python3.9/site-packages/ansible/plugins/filter/to_nice_yaml.yml

After you identify the location of one plug-in, you can find additional plug-ins in the same directory:

[user@host ~]$ ls /usr/lib/python3.9/site-packages/ansible/plugins/filter/
__init__.py      flatten.yml        random.yml               to_nice_json.yml
__pycache__      from_json.yml      realpath.yml             to_nice_yaml.yml
b64decode.yml    from_yaml.yml      regex_escape.yml         to_uuid.yml
b64encode.yml    from_yaml_all.yml  regex_findall.yml        to_yaml.yml
basename.yml     hash.yml           regex_replace.yml        type_debug.yml
bool.yml         human_readable.yml regex_search.yml         union.yml
checksum.yml     human_to_bytes.yml rekey_on_member.yml      unique.yml
combinations.yml intersect.yml      relpath.yml              unvault.yml
combine.yml      items2dict.yml     root.yml                 urldecode.yml
comment.yml      log.yml            sha1.yml                 urls.py
core.py          mandatory.yml      shuffle.yml              urlsplit.py
dict2items.yml   mathstuff.py       split.yml                vault.yml
difference.yml   md5.yml            splitext.yml             win_basename.yml
dirname.yml      password_hash.yml  strftime.yml             win_dirname.yml
encryption.py    path_join.yml      subelements.yml          win_splitdrive.yml
expanduser.yml   permutations.yml   symmetric_difference.yml zip.yml
expandvars.yml   pow.yml            ternary.yml              zip_longest.yml
extract.yml      product.yml        to_datetime.yml
fileglob.yml     quote.yml          to_json.yml

The other filters from the table are Jinja2 built-in filters. You can find more information about both types of filters using the links in the References section.

Revision: do457-2.3-7cfa22a