Abstract
| Goal |
Populate, manipulate, and manage data in variables using filters and plug-ins. |
| Objectives |
|
| Sections |
|
| Lab |
|
Ansible applies variable values to playbooks and templates by using Jinja2 expressions. For example, the following Jinja2 expression replaces the name of the variable enclosed in double braces with its value:
{{ variable }}Jinja2 expressions also support filters. Filters are used to modify or process the value from the variable that is placed in the playbook or template. Some filters are provided by the Jinja2 language; others are included with Red Hat Ansible Automation Platform as plug-ins. You can also create custom filters, but that is beyond the scope of this course. Filters can be extremely useful for preparing data for use in your playbook or template.
To use a filter in a Jinja2 expression, add a pipe (|) character after the variable name or value and then write the filter or filter expression.
You can specify multiple filters in a pipeline, separating each filter or filter expression with a pipe character.
{{ variable | filter }}You can use the default filter to ignore an undefined variable or to provide a value to an undefined variable.
Both use cases prevent a playbook from generating an error.
The following portion of an error message indicates that a task uses the shell variable, but that the variable is undefined:
FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'shell'You an use the default filter to prevent this error.
The following generic task uses the ansible.builtin.user module to manage users defined in the user_list variable.
Each user in the list must define the name key.
Additionally, each user in the list can optionally define the groups, system, shell, state, and remove keys.
Use the default filter with each optional key.
Consult the module documentation to identify the keys required by the module and the default values for keys.
- name: Manage user
ansible.builtin.user:
name: "{{ item['name'] }}"
groups: "{{ item['groups'] | default(omit) }}"
system: "{{ item['system'] | default(false) }}"
shell: "{{ item['shell'] | default('/bin/bash') }}"
state: "{{ item['state'] | default('present') }}"
remove: "{{ item['remove'] | default(false) }}"
loop: "{{ user_list }}"If the | |
If the | |
If the | |
If the | |
If the |
Whenever you use a module, you must decide which module keys to use in the task.
You can manually specify values for the keys or you can configure the keys to set their values based on variables passed to the task.
Because undefined variables produce errors, you can use the default filter to either ignore an undefined variable or supply a value for the undefined variable.
Typically, the default filter only provides a value if a variable is not defined.
In some situations, you might want to provide a value if the variable passed to the default filter is an empty string or evaluates to the false Boolean value.
You can do this by adding true to the default filter.
Consider the following playbook:
---
- name: Default filter examples
hosts: localhost
tasks:
- name: Default filter examples
vars:
pattern: "some text"
ansible.builtin.debug:
msg: "{{ item }}"
loop:
- "{{ pattern | regex_search('test') | default('MESSAGE') }}"
- "{{ pattern | regex_search('test') | default('MESSAGE', true) }}"
- "{{ pattern | bool | default('MESSAGE') }}"
- "{{ pattern | bool | default('MESSAGE', true) }}" 
Because the regular expression is not found in the variable, the | |
Although the | |
Because the string evaluates to the | |
Although the string evaluates to the |
Where appropriate, you can use the default filter before passing a variable to additional filters.
To understand filters, you must first know more about how variable values are handled by Ansible.
Ansible stores runtime data in variables. The YAML structure or the content of the value defines the exact type of data. The following table lists some value types:
| Type | Description |
|---|---|
| Strings | A sequence of characters. |
| Numbers | A numeric value. |
| Booleans | True or false values. |
| Dates | ISO-8601 calendar date. |
| Null | The variable becomes undefined. |
| Lists or Arrays | A sorted collection of values. |
| Dictionaries | A collection of key-value pairs. |
Sometimes, you might first need to convert the value to an integer with the int filter, or to a float with the float filter.
For example, the following Jinja2 expression increments the current hour value, which is collected as a fact and stored as a string, not an integer:
{{ ( ansible_facts['date_time']['hour'] | int ) + 1 }}Various filters are available that can perform mathematical operations on numbers, such as log, pow, root, abs, and round.
The root filter, for example, takes the square root of the variable or value.
{{ 1764 | root }}Many filters are available to analyze and manipulate lists.
If the list consists of numbers, you can use the max, min, or sum filters to find the largest number, the smallest number, or the sum of all list items.
{{ [2, 4, 6, 8, 10, 12] | sum }}You can obtain information about the contents of lists, such as the first or last elements, or the length of a list:
- name: All three of these assertions are true
ansible.builtin.assert:
that:
- "{{ [ 2, 4, 6, 8, 10, 12 ] | length }} is eq( 6 )"
- "{{ [ 2, 4, 6, 8, 10, 12 ] | first }} is eq( 2 )"
- "{{ [ 2, 4, 6, 8, 10, 12 ] | last }} is eq( 12 )"The random filter returns a random element from the list:
{{ ['Douglas', 'Marvin', 'Arthur'] | random }}You can use any of the following methods to reorder a list.
The sort filter returns a list that is sorted by the natural order of its elements.
The reverse filter returns a list where the order is the opposite of the original order.
The shuffle filter returns a list with the same elements, but in a random order.
- name: reversing and sorting lists
ansible.builtin.assert:
that:
- "{{ [ 2, 4, 6, 8, 10 ] | reverse }} is eq( [ 10, 8, 6, 4, 2] )"
- "{{ [ 4, 8, 10, 6, 2 ] | sort }} is eq( [ 2, 4, 6, 8, 10 ] )"Sometimes it is useful to merge several lists into a single list to simplify iteration.
The flatten filter recursively takes any inner list in the input list value, and adds the inner values to the outer list.
- name: Flatten turns nested lists on the left to list on the right
ansible.builtin.assert:
that:
- "{{ [ 2, [4, [6, 8]], 10 ] | flatten }} is eq( [ 2, 4, 6, 8, 10] )"Use the flatten filter to merge list values that come from iterating a parent list.
Use the unique filter to ensure that a list has no duplicate elements.
This filter is useful if you are operating on a list of facts that you have collected, such as usernames or hostnames that might have duplicate entries.
- name: The 'unique' filter leaves unique elements
ansible.builtin.assert:
that:
- "{{ [ 1, 1, 2, 2, 2, 3, 4, 4 ] | unique }} is eq( [ 1, 2, 3, 4 ] )"If two lists have no duplicate elements, then you can use set theory operations on them.
The union filter returns a set with elements from both input sets.
The intersect filter returns a set with elements common to both sets.
The difference filter returns a set with elements from the first set that are not present in the second set.
- name: The 'difference' filter provides elements not in specified set
ansible.builtin.assert:
that:
- "{{ [2, 4, 6, 8, 10] | difference([2, 4, 6, 16]) }} is eq( [8, 10] )"Unlike lists, dictionaries are not ordered in any way, but rather are just a collection of key-value pairs. You can use filters to construct dictionaries and you can convert those dictionaries into lists, and vice versa.
Use the combine filter to join two dictionaries.
Entries from the second dictionary have higher priority than entries from the first dictionary, as seen in the following task:
- name: The 'combine' filter combines two dictionaries into one
vars:
expected:
A: 1
B: 4
C: 5
ansible.builtin.assert:
that:
- "{{ {'A':1,'B':2} | combine({'B':4,'C':5}) }} is eq( expected )"Use the dict2items filter to convert a dictionary to a list.
Use the items2dict filter to convert a list to a dictionary.
- name: converting between dictionaries and lists
vars:
characters_dict:
Douglas: Human
Marvin: Robot
Arthur: Human
characters_items:
- key: Douglas
value: Human
- key: Marvin
value: Robot
- key: Arthur
value: Human
ansible.builtin.assert:
that:
- "{{ characters_dict | dict2items }} is eq( characters_items )"
- "{{ characters_items | items2dict }} is eq( characters_dict )"Various filters are available to manipulate the text of a value. You can compute checksums, create password hashes, and convert text to and from Base64 encoding, as used by a number of applications.
The hash filter returns the hash value of the input string, using the provided hashing algorithm:
- name: the string's SHA-1 hash
vars:
expected: '8bae3f7d0a461488ced07b3e10ab80d018eb1d8c'
ansible.builtin.assert:
that:
- "'{{ 'Arthur' | hash('sha1') }}' is eq( expected )"Use the password_hash filter to generate password hashes:
{{ 'secret_password' | password_hash('sha512') }}Use the b64encode filter to translate binary data to Base64, or translate Base64 encoded data back to binary data with the b64decode filter:
- name: Base64 encoding and decoding of values
ansible.builtin.assert:
that:
- "'{{ 'âÉïôú' | b64encode }}' is eq( 'w6LDicOvw7TDug==' )"
- "'{{ 'w6LDicOvw7TDug==' | b64decode }}' is eq( 'âÉïôú' )"Before sending strings to the underlying shell, and to avoid parsing or code injection issues, it is a good practice to sanitize the string by using the quote filter:
- name: Put quotes around 'my_string'
shell: echo {{ my_string | quote }}Use the lower, upper, or capitalize filters to enforce the case of an input string:
- name: Change case of characters
ansible.builtin.assert:
that:
- "'{{ 'Marvin' | lower }}' is eq( 'marvin' )"
- "'{{ 'Marvin' | upper }}' is eq( 'MARVIN' )"
- "'{{ 'marvin' | capitalize }}' is eq( 'Marvin' )"Use the replace filter to replace all occurrences of a substring inside the input string:
- name: Replace 'ar' with asterisks
ansible.builtin.assert:
that:
- "'{{ 'marvin, arthur' | replace('ar','**') }}' is eq( 'm**vin, **thur' )"Use the regex_search and regex_replace filters to perform more complex searches and replacements by using regular expressions.
- name: Test results of regex search and search-and-replace
ansible.builtin.assert:
that:
- "'{{ 'marvin, arthur' | regex_search('ar\S*r') }}' is eq( 'arthur' )"
- "'{{ 'arthur up' | regex_replace('ar(\S*)r','\\1mb') }}' is eq( 'thumb up' )"Many data structures used by Ansible are in JSON format. JSON and YAML are closely related, and Ansible data structures can be processed as JSON. Likewise, many APIs that Ansible Playbooks might interact with consume or provide information in JSON format. Because this format is widely used, JSON filters are particularly useful.
You can combine selectattr with the map filter to extract information from Ansible data structures.
Use the selectattr filter to select a sequence of objects based on attributes of the objects in the list.
Use the map filter to turn a list of dictionaries into a simple list based on a given attribute.
Although the community.general collection provides the json_query filter, you can usually achieve the same functionality using the selectattr and map filters.
Red Hat does not support the community.general collection.
Consider the following playbook, which queries the /api/v2/execution_environments/ API endpoint and displays the ID for the Control Plane Execution Environment automation controller resource:
---
- name: Query automation controller execution environments
hosts: localhost
gather_facts: false
tasks:
- name: Query EEs
vars:
username_password: "admin:redhat"
ansible.builtin.uri:
url: https://controller.lab.example.com/api/v2/execution_environments/
method: GET
headers:
Authorization: Basic {{ username_password | string | b64encode }}
validate_certs: false
register: query_results
- name: Show execution environment ID
ansible.builtin.debug:
msg: "{{ query_results['json']['results'] | selectattr('name', '==', 'Control Plane Execution Environment') | map(attribute='id') | first }}"The query_results['json']['results'] variable is a list containing entries for each execution environment resource.
Because each entry defines a name key, you can select the entry for the Control Plane Execution Environment resource by using the selectattr filter.
After selecting the correct entry, you can use the map filter to display the value of any key in that entry, such as the value of the id key.
Because the selectattr filter combined with the map filter frequently creates a list consisting of one item, you can use the first filter to select that item.
The playbook might produce the following output to indicate that the ID for the Control Plane Execution Environment resource is 4:
PLAY [Query automation controller EEs] *****************************************
TASK [Query projects] **********************************************************
ok: [localhost]
TASK [Show execution environment ID] *******************************************
ok: [localhost] => {
"msg": "4"
}
PLAY RECAP *********************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0 ...Similarly, you might list all the execution environment names by sending the query_results['json']['results'] variable to the map filter:
- name: Show execution environment names
ansible.builtin.debug:
msg: "{{ query_results['json']['results'] | map(attribute='name') }}"Adding this task to the previous playbook generates this additional output:
TASK [Show execution environment names] ****************************************
ok: [localhost] => {
"msg": [
"Control Plane Execution Environment",
"Automation Hub Default execution environment",
"Automation Hub Ansible Engine 2.9 execution environment",
"Automation Hub Minimal execution environment"
]
}You can use the selectattr and map filters with both JSON and YAML data structures.
Transforming data structures to and from text is useful for debugging and communication.
Data structures serialize to JSON or YAML format with the to_json and to_yaml filters.
Use to_nice_json and to_nice_yaml filters to obtain a formatted, human-readable output.
- name: Convert between JSON and YAML format
vars:
hosts:
- name: bastion
ip:
- 172.25.250.254
- 172.25.252.1
hosts_json: '[{"name": "bastion", "ip": ["172.25.250.254", "172.25.252.1"]}]'
ansible.builtin.assert:
that:
- "'{{ hosts | to_json }}' is eq( hosts_json )"Other filters are introduced in the following sections and chapters that are suited to specific scenarios. Review the official Ansible and Jinja2 documentation to discover more useful filters to meet your needs.