Bookmark this page

Guided Exercise: Event-Driven Ansible and Automated Notifications

Use Event-Driven Ansible to notify administrators monitoring a chat room when a network event triggers a rulebook.

Outcomes

  • Identify event keys when running an activation rulebook.

  • Create and configure an incoming webhook in Mattermost.

  • Modify a rulebook to use the post_event action to set variables for event keys.

  • Create a rule in the rulebook that triggers a job template in automation controller with extra variables.

  • Modify a playbook to use the extra variables from the rulebook.

As the student user on the workstation machine, use the lab command to prepare your environment for this exercise, and to ensure that all required resources are available:

[student@workstation ~]$ lab start example-notifications

Instructions

  1. Examine the automation controller and Event-Driven Ansible (EDA) controller resources created by the lab command.

    1. Navigate to https://controller.lab.example.com and log in as admin using redhat as the password. If necessary, click the main menu icon at the upper left of the page to display the main menu.

    2. Go to ResourcesProjects and then click the link for the Notifications project. This project uses the git@git.lab.example.com:student/notifications Git repository. Notice that the Notifications project has the Update revision on job launch option enabled.

    3. Go to ResourcesTemplates and click the link for the Send a message to mattermost job template. This job template uses the mattermost_message.yml playbook from the Notifications project.

    4. In a new browser tab, navigate to https://eda-controller.lab.example.com and log in as admin using redhat as the password. If necessary, click the main menu icon at the upper left of the page to display the main menu.

    5. Go to Projects and then click the link for the Notifications project. This EDA controller project uses the same Git repository as the automation controller project.

    6. Go to Rulebook Activations and click the link for the Monitor admins rulebook activation. This rulebook activation uses the new_admins.yml rulebook from the Notifications project. Currently the rulebook activation is stopped.

  2. Using the GitLab web UI, examine the playbook and rulebook created by the lab command.

    1. In a new browser tab, navigate to https://git.lab.example.com and log in as student using Stud3nt123 as the password.

    2. Click the notifications project. This project contains a rulebooks directory with the new_admins.yml rulebook:

      ---
      - name: Identifying new admin users
        hosts: eos 1
        sources:
          - name: Collect events from kafka
            ansible.eda.kafka: 2
              host: utility.lab.example.com
              port: 9092
              topic: users 3
      
        rules:
          - name: New admin user  4
            condition: >
              event.body.fields.role is search("SYSTEM_ROLE_ADMIN")
              and event.body.tags.username != "admin"
              and event.body.tags.username != "student"
            action:
              debug:
                msg: "The {{ event.body.tags.username }} admin user was created"

      1

      The rulebook has a unique ruleset that runs against the eos group of managed nodes. In the current lab environment, that group only contains the arista1 managed node.

      2

      The ruleset uses the ansible.eda.kafka event source plug-in to collect events from the utility.lab.example.com host.

      3

      The users topic was configured by the lab command to receive information when a user is created.

      4

      The ruleset has a unique rule that displays a message. The message includes the username of a new user if the user is created with the administration role, and the username is not student or admin.

    3. The notifications project contains a playbooks directory with the mattermost_message.yml playbook:

      ---
      - name: Send a message to the chat group
        hosts: eos 1
        gather_facts: false
        tasks:
          - name: Send the message
            ansible.builtin.uri: 2
              url: <put your url here> 3
              method: POST
              headers:
                Content-Type: application/json
              body_format: json
              body:
                text: |
                  # Alert
                  #### A network admin user was created
            delegate_to: workstation.lab.example.com

      1

      The playbook has a unique play that runs against the eos group of managed nodes.

      2

      The task in the play uses the ansible.builtin.uri module to post a message to Mattermost. The message alerts you about the creation of an admin user.

      3

      You enter the value for the url parameter in a later step.

  3. Test the Monitor admins rulebook activation in EDA controller. Identify the event keys for the created admin user and the managed node where that user was created.

    1. On the EDA controller web browser tab, go to Rulebook Activations and enable the Monitor admins rulebook activation. Notice that the rulebook activation status changes from Stopped to Pending and then to Running.

    2. In a terminal window, connect to the arista1 managed node. Enable privilege escalation and switch to the configuration mode for the terminal:

      [student@workstation ~]$ ssh arista1
      ...output omitted...
      arista1.lab.example.com>enable
      arista1.lab.example.com#configure terminal
      arista1.lab.example.com(config)#
    3. Create a regular user named frank and an admin user named steve, both with S3cr3t as the password:

      arista1.lab.example.com(config)#username frank secret S3cr3t
      arista1.lab.example.com(config)#username steve role network-admin secret S3cr3t
    4. On the EDA controller web browser tab, wait for the number in the Fire count column to change from 0 to 1 for the Monitor admins rulebook activation. The Fire count increases when it detects a new admin user.

    5. Click the Monitor admins rulebook activation and then click the History tab. To see the details for the rulebook activation, click the name that has a status of Running, and scroll down to the end of the output.

      The debug action prints the following message: The steve admin user was created.

      ** 2024-03-14 17:11:29.786195 [debug] ******************************************
      The steve admin user was created
      ********************************************************************************
      ********************************************************************************
    6. Go to Rule Audit and then click the New admin user rule. If you see more than one New admin user rule in the rule audit list, then click the most recent one, at the top of the list.

    7. Click the Events tab, and then click the Collect events from kafka link to examine the event log.

      The following event keys are related to the creation of the new admin user.

      Event keyValue
      body.tags.source arista1.lab.example.com
      body.tags.username steve
      body.fields.role openconfig-aaa-types:SYSTEM_ROLE_ADMIN
    8. Click Close to close the Event details window.

    9. In the terminal window connected to the arista1 managed node, remove the frank and steve users:

      arista1.lab.example.com(config)#no username frank
      arista1.lab.example.com(config)#no username steve
  4. This exercise uses Mattermost for the chat room. Configure Mattermost to integrate with incoming webhooks.

    1. In a new browser tab, navigate to http://mattermost.lab.example.com:8065 and click View in Browser.

    2. Create your account with the information in the table and then click Create Account.

      FieldValue
      Email address student@lab.example.com
      Username student
      Password Stud3nt123
    3. In the Organization name field, enter lab.example.com and then click Continue. Select GitLab as the tool to work with and then click Continue. To complete the setup process, click Finish setup.

      Note

      If you do not see the Continue button, then you might need to increase the display resolution on the workstation machine, decrease the web browser zoom, or both.

    4. Close the Welcome to Mattermost window. Click the main menu icon at the upper left of the page and then click Integrations. Click Incoming Webhooks.

    5. Click Add Incoming Webhook, enter the information from the following table in the appropriate fields, and then click Save to create an incoming webhook.

      FieldValue
      Title EDA message
      Channel Town Square
    6. Copy the URL and click Done.

    7. Click Back to Mattermost at the upper left of the page to go to the Town Square channel.

  5. Update the mattermost_message.yml playbook with the URL you that you previously copied. Verify that the Send a message to mattermost job template sends the message to the chat room.

    1. In a new terminal window, change to the ~/git-repos/notifications directory. This directory is a clone of the git@git.lab.example.com:student/notifications Git repository.

      [student@workstation ~]$ cd ~/git-repos/notifications
    2. Use a text editor, such as VS Code, to update the playbooks/mattermost_message.yml playbook. Enter the value for the url parameter that you copied in the previous step:

      ...output omitted...
          - name: Send the message
            ansible.builtin.uri:
              url: http://mattermost.lab.example.com:8065/hooks/ummq5nihktgcxezdwik4ctpi
              method: POST
      ...output omitted...

      Important

      Your URL is different from the URL displayed in this example.

    3. Add and commit the local changes, and then push the changes to the remote Git repository:

      [student@workstation notifications]$ git commit -a -m 'Updating URL'
      [main 627785a] Updating URL
       1 file changed, 1 insertion(+), 1 deletion(-)
      [student@workstation notifications]$ git push
      ...output omitted...
      [student@workstation notifications]
    4. On the automation controller web browser tab, go to ResourcesTemplates, and then click the Launch Template icon to launch the Send a message to mattermost job template.

    5. On the Mattermost web browser tab, verify that you receive the message.

  6. Modify the new_admins.yml rulebook and the mattermost_message.yml playbook.

    • Instead of displaying a message, the rulebook must launch the Send a message to mattermost job template in automation controller.

    • The rulebook must also add the admin user that was created and the managed node in which the admin user was created as extra variables in the job template.

    • The playbook must use the extra variables in the message sent to the Mattermost chat room.

    Note

    The ~/example-notifications/new_admins.yml file contains the correct updates. You can use that file for comparison or copy it to the ~/git-repos/notifications/rulebooks directory.

    1. Modify the rulebooks/new_admins.yml rulebook in the ~/git-repos/notifications directory.

      • Replace the debug action with the post_event action.

      • Use the new_user and managed_node variables to refer to event data for the event.body.tags.username and event.body.tags.source event keys.

      ...output omitted...
        rules:
          - name: New admin user
            condition: >
              event.body.fields.role is search("SYSTEM_ROLE_ADMIN")
              and event.body.tags.username != "admin"
              and event.body.tags.username != "student"
            action:
              post_event:
                event:
                  new_user: "{{ event.body.tags.username }}"
                  managed_node: "{{ event.body.tags.source }}"
    2. Add a second rule to the rulebook that runs the Send a message to mattermost job template in automation controller with the extra variables given as additional job arguments:

      ...output omitted...
                  new_user: "{{ event.body.tags.username }}"
                  managed_node: "{{ event.body.tags.source }}"
      
          - name: Send a chat message
            condition: event.new_user is defined
            action:
              run_job_template:
                name: Send a message to mattermost
                organization: Default
                post_events: true
                job_args:
                  extra_vars:
                    admin: "{{ event.new_user }}"
                    node: "{{ event.managed_node }}"

      Note

      The ~/example-notifications/new_admins.yml file contains the correct updates. You can use that file for comparison or copy it to the ~/git-repos/notifications/rulebooks directory.

    3. Modify the playbooks/mattermost_message.yml playbook. Use the node variable added as an extra variable in the rulebook to update the hosts key. In this way, the play only targets the managed node defined by the event.

      ---
      - name: Send a message to the chat group
        hosts: "{{ node | default('eos') }}"
        gather_facts: false
      ...output omitted...

      Note

      Although in this exercise the eos group contains a single managed node, in a production environment it would be useful to restrict the managed node targeted by the play.

      By providing the default eos value for the targeted hosts, you prevent the job template from failing when it is run directly from automation controller and the node variable is not defined.

    4. Update the message defined in the body argument for the ansible.builtin.uri module. Specify the managed node for which the alert applies and the username of the admin user.

      Use the node and admin variables added as extra variables in the rulebook:

      ...output omitted...
              body:
                text: |
                  # Alert: {{ node }}
                  #### A network admin user was created: {{ admin }}
            delegate_to: workstation.lab.example.com
            when:
              - node is defined
              - admin is defined

      Important

      Adding the when condition to the Send the message task can prevent the playbook from failing. Ansible skips the task unless both the node and admin variables are defined.

    5. Add and commit the local changes, and then push the changes to the remote Git repository in the terminal where you changed to the ~/git-repos/notifications directory:

      [student@workstation notifications]$ git commit -a -m 'Automation completed'
      [main 20d8ad4] Automation completed
       2 files changed, 22 insertions(+), 5 deletions(-)
      [student@workstation notifications]$ git push
      ...output omitted...
      [student@workstation notifications]
  7. Update automation controller and EDA controller resources to use the Git repository changes.

    1. On the automation controller web browser tab, go to ResourcesTemplates, and then click the Edit Template icon for the Send a message to mattermost job template.

    2. Select the Prompt on launch checkbox for the Variables field and then click Save.

    3. On the EDA controller web browser tab, go to Projects and then click the Sync project icon for the Notifications project. Wait until the project displays a status of Completed.

    4. Go to Rulebook Activations. Click the vertical ellipsis icon for the Monitor admins rulebook activation and then click Restart rulebook activation.

    5. Select the Yes, I confirm that I want to restart these 1 rulebook activations checkbox and then click Restart rulebook activations.

    6. When the status changes to Success, click Close to close the Restart rulebook activations window. Wait until the number of rules for the Monitor admins rulebook activation increases from 1 to 2.

  8. Create an admin user in the arista1 managed node and verify that EDA controller triggers the sending of a message to the Mattermost chat room. The message includes the username of the admin user and the managed node where that user was created.

    1. From the terminal window connected to the arista1 managed node, create the steve admin user:

      arista1.lab.example.com(config)#username steve role network-admin secret S3cr3t
    2. Disconnect from the arista1 managed node:

      arista1.lab.example.com(config)#logout
      Connection to arista1 closed.
      [student@workstation ~]$
    3. On the Mattermost web browser tab, wait for the message about the creation of the network admin user. Verify that the message indicates the username and the managed node where the admin user was created.

Finish

On the workstation machine, use the lab command to complete this exercise. This step is important to ensure that resources from previous exercises do not impact upcoming exercises.

[student@workstation ~]$ lab finish example-notifications

Revision: do274-2.4-65daa25