Bookmark this page

Lab: Transforming Data with Filters and Plug-ins

  • Use filters and plug-ins to transform data in variables and facts in order to use it for other tasks in the same play.

Outcomes

  • Use filters and lookup plug-ins to manipulate variables in playbook and role tasks.

As the student user on the workstation machine, use the lab command to prepare your system for this exercise.

This command ensures that the remote Git repository https://git.lab.example.com/student/data-review.git is initialized. The Git repository contains playbooks that configure a front-end load balancer and a pool of back-end web servers. You can push changes to this repository using Student@123 as the Git password.

[student@workstation ~]$ lab start data-review

Procedure 7.5. Instructions

  1. Clone the https://git.lab.example.com/student/data-review.git repository to the /home/student/git-repos directory and then create the exercise branch.

    1. From a terminal, create the /home/student/git-repos directory if it does not exist, and then change into it.

      [student@workstation ~]$ mkdir -p ~/git-repos/
      [student@workstation ~]$ cd ~/git-repos/
    2. Clone the https://git.lab.example.com/student/data-review.git repository and then change into the cloned repository:

      [student@workstation git-repos]$ git clone \
      > https://git.lab.example.com/student/data-review.git
      Cloning into 'data-review'...
      ...output omitted...
      [student@workstation git-repos]$ cd data-review
    3. Create the exercise branch and check it out.

      [student@workstation data-review]$ git checkout -b exercise
      Switched to a new branch 'exercise'
  2. Examine the firewall role in the ~/git-repos/data-review project directory.

    The tasks file, roles/firewall/tasks/main.yml, contains three tasks that use the firewalld module to configure firewall rules defined by the firewall_rules variable.

    Edit the tasks file so that it uses filters to set default values for variables in each of the three tasks if they are not set, as follows:

    • For the state option, if item['state'] is not set, then set it to enabled by default.

    • For the zone option, if item['zone'] is not set, then omit the zone option.

    • In the "Ensure Firewall Port Configuration" task, in the port option to the firewalld module, if item['protocol'] is not set then set it to tcp. Also use the lower filter to ensure that the value of the item['protocol'] option is in lowercase.

    1. Edit the roles/firewall/tasks/main.yml file. For the state option of all three firewalld tasks, add the default('enabled') filter to the item['state'] Jinja2 expression.

      The resulting file contains the following content:

      - name: Ensure Firewall Sources Configuration
        ansible.posix.firewalld:
          source: "{{ item['source'] }}"
          zone: "{{ item['zone'] }}"
          immediate: true
          permanent: true
          state: "{{ item['state'] | default('enabled') }}"
        loop: "{{ firewall_rules }}"
        when: item['source'] is defined
      
      - name: Ensure Firewall Service Configuration
        ansible.posix.firewalld:
          service: "{{ item['service'] }}"
          zone: "{{ item['zone'] }}"
          immediate: true
          permanent: true
          state: "{{ item['state'] | default('enabled') }}"
        loop: "{{ firewall_rules }}"
        when: item['service'] is defined
      
      - name: Ensure Firewall Port Configuration
        ansible.posix.firewalld:
          port: "{{ item['port'] }}/{{ item['protocol'] }}"
          zone: "{{ item['zone'] }}"
          immediate: true
          permanent: true
          state: "{{ item['state'] | default('enabled') }}"
        loop: "{{ firewall_rules }}"
        when: item['port'] is defined
    2. Continue editing the tasks file. For the zone option of all three firewalld tasks, add the default(omit) filter to the item['zone'] Jinja2 expression.

      Note

      If desired, you can use the default(omit, true) filter instead. Adding true to the default filter means that the filter is also used if the variable is either an empty string or is set to the false Boolean value.

      The default filter is always used if the variable is undefined.

      After this step, the task file contains the following content:

      - name: Ensure Firewall Sources Configuration
        ansible.posix.firewalld:
          source: "{{ item['source'] }}"
          zone: "{{ item['zone'] | default(omit) }}"
          immediate: true
          permanent: true
          state: "{{ item['state'] | default('enabled') }}"
        loop: "{{ firewall_rules }}"
        when: item['source'] is defined
      
      - name: Ensure Firewall Service Configuration
        ansible.posix.firewalld:
          service: "{{ item['service'] }}"
          zone: "{{ item['zone'] | default(omit) }}"
          immediate: true
          permanent: true
          state: "{{ item['state'] | default('enabled') }}"
        loop: "{{ firewall_rules }}"
        when: item['service'] is defined
      
      - name: Ensure Firewall Port Configuration
        ansible.posix.firewalld:
          port: "{{ item['port'] }}/{{ item['protocol'] }}"
          zone: "{{ item['zone'] | default(omit) }}"
          immediate: true
          permanent: true
          state: "{{ item['state'] | default('enabled') }}"
        loop: "{{ firewall_rules }}"
        when: item['port'] is defined
    3. Edit the task file again. In the third firewalld task, "Ensure Firewall Port Configuration", replace {{ item['protocol'] }} with {{ item['protocol'] | default('tcp') | lower }}. Save your work.

      The completed task file contains the following content:

      - name: Ensure Firewall Sources Configuration
        ansible.posix.firewalld:
          source: "{{ item['source'] }}"
          zone: "{{ item['zone'] | default(omit) }}"
          immediate: true
          permanent: true
          state: "{{ item['state'] | default('enabled') }}"
        loop: "{{ firewall_rules }}"
        when: item['source'] is defined
      
      - name: Ensure Firewall Service Configuration
        ansible.posix.firewalld:
          service: "{{ item['service'] }}"
          zone: "{{ item['zone'] | default(omit) }}"
          immediate: true
          permanent: true
          state: "{{ item['state'] | default('enabled') }}"
        loop: "{{ firewall_rules }}"
        when: item['service'] is defined
      
      - name: Ensure Firewall Port Configuration
        ansible.posix.firewalld:
          port: "{{ item['port'] }}/{{ item['protocol'] | default('tcp') | lower }}"
          zone: "{{ item['zone'] | default(omit) }}"
          immediate: true
          permanent: true
          state: "{{ item['state'] | default('enabled') }}"
        loop: "{{ firewall_rules }}"
        when: item['port'] is defined
  3. Test the changes you made to the roles/firewall/tasks/main.yml file in the preceding step by running the test_firewall_role.yml playbook.

    If you completed the preceding step successfully, the playbook runs without errors.

    Run the test_firewall_role.yml playbook to test your changes:

    [student@workstation data-review]$ ansible-navigator run \
    > -m stdout test_firewall_role.yml
    
    PLAY [Test Firewall Role] ****************************************************
    ...output omitted...
    
    PLAY RECAP *******************************************************************
    serverb.lab.example.com  : ok=4  changed=2  unreachable=0  failed=0  ...
    serverc.lab.example.com  : ok=4  changed=2  unreachable=0  failed=0  ...
  4. Examine the deploy_apache.yml playbook. It calls the apache role to ensure that Apache HTTP Server is deployed, which itself calls the firewall role. The playbook also defines the firewall_rules variable to make sure that the http service for firewalld and the IPv4 address of the load balancer (172.25.250.10) are both enabled in the internal zone for firewalld.

    Edit the playbook. You need to change the value of the firewall_rules variable in the deploy_apache.yml playbook to a Jinja2 expression that uses the template lookup plug-in to dynamically generate the variable's setting from the apache_firewall_rules.yml.j2 Jinja2 template. You must use the from_yaml filter in the Jinja2 expression to convert the resulting value from a text string into a YAML data structure that can be interpreted by Ansible.

    Note

    In the preceding example, ensure you enter the expression for the firewall_rules variable on a single line.

    If you use linting rules that indicate that lines should not exceed 80 characters, then you might define the Jinja2 expression over multiple lines by defining the firewall_rules variable using a greater than sign. For example:

          firewall_rules: >
            {{ lookup('ansible.builtin.template', 'apache_firewall_rules.yml.j2')
            | from_yaml }}

    One correct solution results in the following contents in the deploy_apache.yml Ansible Playbook:

    - name: Ensure Apache is deployed
      hosts: web_servers
      force_handlers: true
      gather_facts: false
    
      roles:
        # Use the apache_firewall_rules.yml.j2 template to
        # generate the firewall rules.
        - role: apache
          firewall_rules: "{{ lookup('ansible.builtin.template', 'apache_firewall_rules.yml.j2') | from_yaml }}"
  5. Examine the templates/apache_firewall_rules.yml.j2 template that your playbook now uses to set the firewall_rules variable.

    For the first rule, the apache_port variable is set by files in the group_vars directory.

    The template contains a Jinja2 for loop that creates a rule for each host in the lb_servers group, setting the source IP address from the value of the load_balancer_ip_addr variable.

    This is not the best solution. It requires you to set load_balancer_ip_addr as a host variable for each load balancer in the lb_servers group. To remove this manual maintenance, use gathered facts from these hosts to set that value instead.

    Edit the templates/apache_firewall_rules.yml.j2 template to replace the load_balancer_ip_addr host variable in the Jinja2 for loop with the hostvars[server]['ansible_facts']['default_ipv4']['address'] fact.

    The completed templates/apache_firewall_rules.yml.j2 template should contain the following content:

    - port: {{ apache_port }}
      protocol: TCP
      zone: internal
    {% for server in groups['lb_servers'] %}
    - zone: internal
      source: "{{ hostvars[server]['ansible_facts']['default_ipv4']['address'] }}"
    {% endfor %}

    Save the template.

  6. Run the site.yml playbook to test your changes. If you successfully completed the preceding steps, then the playbook runs without errors.

    [student@workstation data-review]$ ansible-navigator run site.yml -m stdout
    ...output omitted...
    [student@workstation data-review]$ echo $?
    0
  7. Verify that web browser requests from workstation to the load balancer on servera succeed, and that direct requests from workstation to either TCP port 80 or TCP port 8008 on the back-end web servers, serverb and serverc, are denied.

    [student@workstation data-review]$ curl servera
    This is serverb. (version v1.0)
    [student@workstation data-review]$ curl servera
    This is serverc. (version v1.0)
    [student@workstation data-review]$ curl serverb; curl serverb:8008
    curl: (7) Failed to connect to serverb port 80: No route to host
    curl: (7) Failed to connect to serverb port 8008: No route to host
    [student@workstation data-review]$ curl serverc; curl serverc:8008
    curl: (7) Failed to connect to serverc port 80: No route to host
    curl: (7) Failed to connect to serverc port 8008: No route to host
  8. Save your work, and then use Git to commit your changes and push them to the remote Git repository. If prompted, use Student@123 as the Git password.

    [student@workstation data-review]$ git status
    On branch exercise
    Changes not staged for commit:
      (use "git add <file>..." to update what will be committed)
      (use "git restore <file>..." to discard changes in working directory)
            modified:   deploy_apache.yml
            modified:   roles/firewall/tasks/main.yml
            modified:   templates/apache_firewall_rules.yml.j2
    
    no changes added to commit (use "git add" and/or "git commit -a")
    [student@workstation data-review]$ git add deploy_apache.yml \
    > roles/firewall/tasks/main.yml templates/apache_firewall_rules.yml.j2
    [student@workstation data-review]$ git commit \
    > -m "Added Filters and Plug-ins"
    ...output omitted...
    [student@workstation data-review]$ git push -u origin exercise
    Password for 'https://student@git.lab.example.com': Student@123
    ...output omitted...

Evaluation

On the workstation machine, change to the student user home directory and use the lab command to grade your work. Correct any reported failures and rerun the command until successful. You must add, commit, and push any additional changes to the remote Git repository before grading.

[student@workstation ~]$ lab grade data-review

Finish

On the workstation machine, change to the student user home directory and use the lab command to complete this exercise. This step is important to ensure that resources from previous exercises do not impact upcoming exercises.

[student@workstation ~]$ lab finish data-review

This concludes the section.

Revision: do374-2.2-82dc0d7