The first Event-driven Ansible plugin

During the Red Hat Summit in May this year, an interesting extension of the popular Infrastructure as Code solution was presented with Event-driven Ansible. With EDA, Ansible is extended by a proactive component that allows it to react dynamically to events. This focuses on the previous unique selling point of SaltStack (Event-driven infrastructure).

The central component of the extension are so-called rulebooks, in which events to be monitored are defined. If such an event occurs, a corresponding countermeasure can be defined - for example, the execution of playbooks. For monitoring, corresponding Event Source Plugins are required. With a manageable number of plug-ins, the variety of possibilities is currently still somewhat limited and cannot quite compete with the more mature SaltStack EDI. Infrastructure-related plug-ins, for example for local services or hard disk utilisation, are still in vain.

Event-driven Ansible has found its way into Ansible Automation Platform (AAP) with version 2.4. The execution logic is encapsulated in a Decision Environment (DEs) - analogous to the similarly structured Execution Environments (EEs). These are Podman containers based on RHEL or CentOS Stream with Ansible installed as well as the required collections. These containers are then started when Ansible logic is executed via the Automation Controller (EEs) or EDA Controller (DEs).

Installation

According to the documentation, Python packages must be installed in addition to Java - for example, under Fedora:

1# dnf install java-17-openjdk python3-pip
2$ pip3 install --user ansible-rulebook ansible-runner

In my case, the Python module watchdog was also missing:

1$ pip3 install --user watchdog

The associated Ansible Collection is also important:

1$ ansible-galaxy collection install ansible.eda

As a first test, you can quickly start a local web server with Python and monitor it with a rulebook.

The following call is sufficient to start the server:

1$ python3 -m http.server
2Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...

This then listens on port 8080. A simple rulebook could look like this:

local_webserver_rulebook.yml

 1---
 2- name: Listen for events on a web service
 3  hosts: localhost
 4  # define source plugin
 5  sources:
 6     - ansible.eda.url_check:
 7        urls:
 8          - http://localhost:8000
 9        delay: 10
10
11  rules:
12    # define condition
13    - name: Website is up
14      condition: event.url_check.status == "up"
15    # define action
16      action:
17        run_module:
18          name: ansible.builtin.debug
19          module_args:
20            msg: "Website is up, all good!"
21
22    - name: Website is down
23      condition: event.url_check.status == "down"
24      action:
25        run_module:
26          name: ansible.builtin.debug
27          module_args:
28            msg: "Oh noes, website is down :("

An inventory is still missing - for simplicity, only debug output is done here and no playbooks are executed. Therefore, a simple localhost inventory will do:

localhost-inventory.ini

1localhost

The whole thing is now executed in another terminal as follows:

1$ ansible-rulebook -i localhost-inventory.ini --rulebook local_webserver_rulebook.yml

Every 10 seconds, the accessibility of the web server is checked - if you cancel the web server after a few seconds with CTRL+C, it looks like this, for example:

 1...
 2PLAY [wrapper] *****************************************************************
 3
 4TASK [Module wrapper] **********************************************************
 5ok: [localhost] => {
 6    "msg": "Website is up, all good!"
 7}
 8
 9TASK [save result] *************************************************************
10ok: [localhost]
11
12...
13
14PLAY [wrapper] *****************************************************************
15
16TASK [Module wrapper] **********************************************************
17ok: [localhost] => {
18    "msg": "Oh noes, website is down :("
19}
20
21TASK [save result] *************************************************************
22ok: [localhost]
23...

Developing customized plugins

The ansible.eda collection contains some examples to guide development. The development documentation is currently quite thin and clearly in need of improvement.

During a recent Ansible hackathon, I had the idea to extend the Uyuni Ansible collection with proactive functions. Until now, this still used Python code that was developed a long time ago in the katprep project. The corresponding library was previously used by means of relative import as a module utility (module_utils/uyuni.py). However, exactly this does not seem to work with EDA due to a bug.

Annoying, but I wanted to clean up the code anyway and move it out to a dedicated library - so, thanks for the extrinsic motivation. 🫠

To add EDA functionality to a collection, simply create the following folder structure:

1.
2└── extensions
3    └── eda
4        ├── plugins
5        │   ├── event_filter
6        │   └── event_source
7        │       ├── XXX.py
8        └── rulebooks
9            └── XXX.yml

As already suspected, the plugins folder contains the event source plugins, while the rulebooks folder contains the corresponding rulebooks. For filtering information there is a separate plugin type, the corresponding folder name is plugins/event_filter. A good starting point is the official example code on GitHub.

I developed an event source plugin requires_reboot.py within a few hours. This returns whether a specific system needs a reboot - for example after installing a kernel update. Admittedly not a very important use case - but that's not necessarily what hackathons are about 🙂 .

To use the plugin, a rulebook is still missing:

trigger_reboots.yml

 1---
 2- name: Rebooting hosts
 3  hosts: localhost
 4  gather_facts: false
 5  tasks:
 6    - name: Show system that will be rebooted
 7      ansible.builtin.debug:
 8        msg: "Host to be rebooted: {{ ansible_eda.event.host }}"
 9
10    - name: Reboot system
11      stdevel.uyuni.reboot_host:
12        uyuni_host: 192.168.1.10
13        uyuni_user: admin
14        uyuni_password: admin
15        uyuni_verify_ssl:
16        name: "{{ ansible_eda.event.host }}"

In order for this to be executed, an inventory is still missing. The whole thing is then executed as follows:

1$ ansible-rulebook -i inventory.ini --rulebook trigger_reboots.yml

If you now install an update that requires a reboot, EDA recognises this and counteracts it:

 1checking host uyuni-client.xxx.de
 2
 3PLAY [Rebooting hosts] *********************************************************
 4
 5TASK [Show system that will be rebooted] ***************************************
 6ok: [localhost] => {
 7    "msg": "Host to be rebooted: uyuni-client.xxx.de"
 8}
 9
10TASK [Reboot system] ***********************************************************
11changed: [localhost]

Cool, isn't it? 🙂

Outlook

The current development status can be observed in the corresponding pull request - further use cases are to follow in the future. Here I am always happy to receive feedback and constructive criticism. Both the Python library pyuyuni and the EDA plugins are in an early stage of development.

Conceivable options would be, for example:

• react to hosts that no longer respond (e.g. restart Salt-Minion)
• reacting to corresponding system events (e.g. hardening systems specified as insecure by OpenSCAP).

So it remains exciting! 🙂