Ansible: Create sublist from list using "startswith" comparison - loops

I have system (Zabbix) that uses crude group/subgroup definition based on "/" delimiter which are defined in plain list.
For example:
"Grp(1)"
"Grp(1)/Subgrp(A)"
"Grp(1)/Subgrp(B)"
"Grp(2)"
"Grp(2)/Subgrp(X)"
This defines two groups, Grp(1) with two subgroups (A and B) and Grp(2) with one subgroup (X)
If I logically assign user to "Grp(1)" it is expected that user also automatically have right to "Grp(1)/Subgrp(A)" and "Grp(1)/Subgrp(B)"
Example vars file looks like:
---
groups_vars:
- "Grp(1)"
- "Grp(1)/Subgrp(A)"
- "Grp(1)/Subgrp(B)"
- "Grp(2)"
- "Grp(2)/Subgrp(X)"
The vars are used in ansible galaxy module community.zabbix, there is simplified usage for Grp(1):
- name: Ensure user groups are created and right to itself and subgroups are assigned
community.zabbix.zabbix_usergroup:
name: Grp(1)
rights:
- {host_group: ["Grp(1)","Grp(1)/Subgrp(A)","Grp(1)/Subgrp(B)"], permission: "read-write" }
I have tried to achieve "set_fact" transformation of input vars into format more suitable for ansible loop:
---
groups_vars:
-
name: Grp(1)
rights:
host_group:
- Grp(1)
- Grp(1)/Subgrp(A)
- Grp(1)/Subgrp(B)
permission: read-write
-
name: Grp(1)/Subgrp(A)
rights:
host_group:
- Grp(1)/Subgrp(A)
permission: read-write
-
name: Grp(1)/Subgrp(B)
rights:
host_group:
- Grp(1)/Subgrp(B)
permission: read-write
-
name: Grp(2)
rights:
host_group:
- Grp(2)
- Grp(1)/Subgrp(X)
permission: read-write
-
name: Grp(2)/Subgrp(X)
rights:
host_group:
- Grp(2)/Subgrp(X)
permission: read-write
But I failed to define the transformation. The select('match', ) function that I try to use for filtering is regex based but itself can contain regex directives (name "Grp(1)" contains parenthesis that are regex directives) and I cannot fing any "startswith" method for finding subgroups.
My idea was, that for each group from original group_vars defined above I find all items, that begins with the group name (so for "Grp(2)" I will find "Grp(2)" and "Grp(2)/Subgrp(X)", for "Grp(2)/Subgrp(X)" I will find only "Grp(2)/Subgrp(X)" itself)
Please any ideas how to solve the problem?
Maybe my approach is complete wrong, if there is any more elegant solution, please help.

I finally found working approach for the problem.
I created simple Python script that generates data usable for simple plain ansible loop.
The inner loop is emulated by generation of complex structure (array of dictionaries)
There is source YML file:
---
groups_vars:
- "z_JC(015)"
- "z_JC(015)/Pisek(022)"
- "z_HK(055)"
There is trasformed YML file, the inner loop is emulated by array under "rights:"
usergroups_vars:
- group: z_JC(015)
rights:
- host_group: z_JC(015)
permission: read-write
- host_group: z_JC(015)/Pisek(022)
permission: read-write
- group: z_JC(015)/Pisek(022)
rights:
- host_group: z_JC(015)/Pisek(022)
permission: read-write
- group: z_HK(055)
rights:
- host_group: z_HK(055)
permission: read-write
The playbook works simply with trasformed YML file usin simple plain loop:
- name: Ensure z_ prefixed Zabbix UserGroups are present and linked to eponymous HostGroups and subhostgroups
community.zabbix.zabbix_usergroup:
server_url: "{{ static_hostvars.server_url }}"
login_user: "{{ static_hostvars.login_user }}"
login_password: "{{ static_hostvars.login_password }}"
state: "present"
name: "{{ item.group }}"
rights: "{{ item.rights }}"
loop: "{{ usergroups_vars }}"
There is example of the Python transformation script (using pyyaml library):
import yaml
# Press the green button in the gutter to run the script.
if __name__ == '__main__':
# Load and parse host group var yaml file
hostgroups = None
with open('groups_vars.yaml') as f:
hostgroups = yaml.load(f, Loader=yaml.FullLoader)
# Create eponymous usergroups for hostgroups with prefix 'z_'
usergroups = []
for hostgroup in hostgroups.get('groups_vars'):
if hostgroup.startswith('z_'):
usergroups.append(hostgroup)
# Find subgroups ut of list og groups delimited by '/'
# (E.g array ['grp1','grp1/subgrp1'] defined one group 'grp1' and one subgroup 'subgrp1')
usergrpsubgrps = []
for onegrp in usergroups:
# Find subgroups (including the group itself)
subgrps = []
for onesubgroup in usergroups:
if onesubgroup.startswith(onegrp):
subgrps.append({'host_group': onesubgroup, 'permission': 'read-write'})
usergrpsubgrps.append({'group': onegrp, 'rights': subgrps})
out_yaml = yaml.dump({'usergroups_vars' : usergrpsubgrps})
print(out_yaml)
# Write output yaml to the output ansible vars file
out_file = open('usergroups_vars.yaml','w')
out_file.write(out_yaml)
out_file.close()

Related

Can I create dynamic lists within Ansible vars_files?

I have a variables file that includes important info about our databases; the server they are on, the db version, the DB_HOME directory, etc. In the variables file, I would like to dynamically create lists that capture the unique values of those properties, so they can be easily iterated through in a task.
I have equivalent functionality by creating the list on the fly in a task's loop option, but that means repeating that loop syntax (violates DRY principle) and I would like less sophisticated Ansible colleagues to be able to use a pre-defined list.
example of the variables file databases.yml:
databases:
- name: test_db1
server: ora_901
listener: LISTENER_XYZ
version: '11.2.0.4'
oracle_home: '/app/oracle/product/11.2.0.4/db_home'
- name: test_db2
server: ora_902
listener: LISTENER_ABC
version: '11.2.0.4'
oracle_home: '/app/oracle/product/11.2.0.4/db_home'
## This didn't work... was hoping I could build this list dynamically
listeners:
- name: "{{ item }}"
loop: "{{ databases | map(attribute = 'listener') | list | unique }}"
servers:
- name: "{{ item }}"
loop: "{{ databases | map(attribute = 'server') | list | unique }}"
I would then use this loop through either the 'listeners' or 'servers' lists directly with some tasks.
When I tried a task that referenced the listeners variable, it failed. Referencing databases works and all items are returned, so I know it's getting some data from the vars_file...
- vars_files:
- vars/databases.yml
tasks:
- debug:
msg: "{{ databases }}"
- debug:
msg: "{{ listeners }}"
TASK [debug] **************************************************************************
ok: [FQDN] => {
"msg": [
{
"listener": "LISTENER_XYZ",
"name": "test_db1",
"oracle_home": "/app/oracle/product/11.2.0.4/db_home",
"server": "ora_901",
"version": "11.2.0.4"
},
{
"listener": "LISTENER_ABC",
"name": "test_db2",
"oracle_home": "/app/oracle/product/11.2.0.4/dbhome_1",
"server": "ora_902",
"version": "11.2.0.4"
},
fatal: [FQDN]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'item' is undefined\n\nThe error appears to have been in '/home/xxx/test_vars.yml': line 21, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - debug:\n ^ here\n"}
I would really like to keep these dynamic definitions in the same place as the server definitions, and don't see why it wouldn't be possible, I'm just sure I'm using the wrong mechanism.
Check this out from Ansible doc// Just to give you an idea, about "loop_control".
You can nest two looping tasks using include_tasks. However, by default Ansible sets the loop variable item for each loop. This means the inner, nested loop will overwrite the value of item from the outer loop. You can specify the name of the variable for each loop using loop_var with loop_control:
# main.yml
- include_tasks: inner.yml
loop:
- 1
- 2
- 3
loop_control:
loop_var: outer_item
# inner.yml
- debug:
msg: "outer item={{ outer_item }} inner item={{ item }}"
loop:
- a
- b
- c

ansible register with loops

I want to create user and gather their info in local file but with loop register is now working as expected.
I thought it was an indentation problem but no luck.
My playbook
---
- hosts: localhost
tasks:
- name: Clearing Local file
local_action: shell echo "Zone,docode,doname,testuser Output" > user.csv
- hosts: app
tasks:
- name: Creating user Testuser
become: yes
user:
name: "{{ item }}"
state: present
shell: "/bin/bash"
password: "$6$mysecretsalt$qyctTVhRMS1ZSnCuzQNAM8Y7V/yqSEnyRbal0IYXSqSEVKkXF8ZmXBZoRIaN/PvzE/msq8iOJO830OOCG89va/"
update_password: always
groups: santosh
shell: id "{{item}}"
ragister: userout
loop:
- newuser1
- newuser2
- newuser3
- debug:
var=userout
which gives the following error when executed
ERROR! conflicting action statements: shell, user
The error appears to have been in '/home/santosh/ans-home/playbooks/Create_User_and_Gather_output.yml': line 12, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- name: Creating user Testuser
^ here
You cannot call several modules in one task, you need to separate each call in its own task, as reported by the error message.
I understand why you tried it: to take advantage of the same loop for several task. Unfortunately this is not possible. You could move your set of tasks to a separate file and include it in a loop if you really have to loop over a significant amount of tasks. This is not really required in your situation because:
you only have two tasks
you can compact writing of your loop by using a declared var for reuse
and most essentially because you don't need your second task
In fact, the user module will return the uid of the user it created or that is existing in its result. You just have to register the result of calling the user module.
Just try the following for your second play:
- name: Resgister application users
hosts: app
vars:
user_list:
- newuser1
- newuser2
- newuser3
tasks:
- name: Create the users if they don't exist
user:
name: "{{ item }}"
state: present
shell: "/bin/bash"
password: "$6$mysecretsalt$qyctTVhRMS1ZSnCuzQNAM8Y7V/yqSEnyRbal0IYXSqSEVKkXF8ZmXBZoRIaN/PvzE/msq8iOJO830OOCG89va/"
update_password: always
groups: santosh
register: create_users
loop: "{{ user_list }}"
- name: Show ids of users
debug:
msg: "The uid of user {{ item.name }} is: {{ item.uid }}"
loop: "{{ create_users.results }}"
And as a side note: for your first play, do yourself a favor and stop using the old local_action syntax in favor of delegate_to: localhost for a task. It is not even required in your case as your are already targeting your play to localhost only.

Is there a way to loop on two variables when importing an Ansible role?

I'm importing an Ansible role in a play and running its 'install' task.The role is meant to create VMs on an hypervisor like Vbox and works fine.
However, I want to use it to create several VMs at the same time, and I must provide 2 variables for this purpose :
- vm_ip : the ip of the vm to be created
- vm_name : the name of the vm to be created
I have already tried almost everything with loops, with_items and other things. For instance, this code doesn't work :
- name: Create VMs
hosts: localhost
tasks:
- import_role:
name: vm_creation
tasks_from: install
vars:
vm_ip: "{{ item.ips }}"
vm_name: "{{ item.names }}"
loop:
- { ips: '192.168.20.4', names: 'test4' }
- { ips: '192.168.20.5', names: 'test5' }
It is supposed to create both .20.4 and .20.5 VM's but the play crashes telling me this :"The task includes an option with an undefined variable. The error was: 'item' is undefined
You appear to have mis-indented the loop directive. In doing so, you have defined a variable named loop rather than actually creating a loop (this is why item is undefined).
You will also need to use include_role rather than import_role. You can read about the difference between include_role and import_role in the documentation.
- name: Create VMs
hosts: localhost
tasks:
- include_role:
name: vm_creation
tasks_from: install
vars:
vm_ip: "{{ item.ips }}"
vm_name: "{{ item.names }}"
loop:
- { ips: '192.168.20.4', names: 'test4' }
- { ips: '192.168.20.5', names: 'test5' }

Ansible - Versionfile check

I want to be able to read a versionfile if it exists, and check its contents. Then return True if the version changed or the file does not exists, False if versionfile exists and the version matches the content.
Basically this:
# setup test data
- set_fact:
version_expected: "0001"
version_path: "/path/to/version"
version_owner: "root"
version_group: "root"
# this block is used to check for version changes
- name: check version change
block:
- name: check version file
stat:
path: "{{version_path}}"
register: version_file
- set_fact:
version_remote: "{{ lookup('file', version_path) | default('') }}"
when: version_file.stat.exists
- set_fact:
version_changed: not version_file.stat.exists or version_remote != version_expected
# test writing new version
- name: write file
copy:
dest: "{{version_path}}"
content: "{{version_expected}}"
owner: "{{version_owner}}"
group: "{{version_group}}"
when: version_changed
My problem is: This is somewhat ugly and becoming quite redundant in my roles.
Is there a more elegant way to do this?
Is there maybe a module for this? (though I found none)
Or should I just write a module for this?
Best regards,
2d4r
EDIT:
im only meaning the "check version change" block, the surrounding code is for debugging only.
To be more specific, I want to download a server binary, but only if my expectet version differs from the content of the versionfile.
I want to write the new version to file, if (and only if) the download was successfull, but that is not part of my question.
EDIT2:
I got this by now:
# roles/_helper/tasks/version_check.yml
- name: check if file exists
stat:
path: "{{version_path}}"
register: version_file
- name: get remote version
slurp:
src: "{{version_path}}"
register: version_changed
when: version_file.stat.exists
# (False if versionfile exists and version is expected; True else)
- name: set return value
set_fact:
version_changed: "{{ not version_file.stat.exists or ((version_changed.content | b64decode) is version_compare(version_expected, 'ne')) }}"
used like this:
# /roles/example/tasks/main.yml
- include_role:
name: _helper
tasks_from: version_check
vars:
version_path: "{{file_version_path}}"
version_expected: "{{file_version_expected}}"
- name: doing awesome things
when: version_changed
block:
- name: download server
[...]
- name: write version
copy:
dest: "{{file_version_path}}"
content: "{{file_version_expected}}"
It kills the redundancy, but is still not what I want.
Sadly I can not register a return value from a role.
Delete everything except for write file task and remove the condition.
Ansible does this automatically for you.
- name: write file
copy:
dest: "{{version_path}}"
content: "{{version_expected}}"
owner: "{{version_owner}}"
group: "{{version_group}}"
After you changed the question, given the information provided, the only thing I can point to is to use slurp module instead of lookup, as an lookup plugins work locally in the control machine.
Compare versions using your logic or built-in version_compare filter/test.

ansible loop over list and dictionary at the same time

I am writing a playbook that ensure nodes appear in /etc/fstab.
I am using loops to prevent code duplication.
The logic is first to check if the line appears using grep (with perl regex because it is a multi line) and store the results in a register.
Then I want to add only the lines that are not in fstab file. To achieve that I need to loop over list (the register with the grep return codes) and a dictionary (that contains the fstab entries).
I am having errors with the parallel loop. I tried to follow these steps.
One or more undefined variables: 'str object' has no attribute 'item'
tasks/fstab.yaml:
---
- name: Make dirs
sudo: yes
file: path={{ item.value }} state=directory
with_dict:
"{{ fstab.paths }}"
- name: Check whether declared in fstab
sudo: no
command: grep -Pzq '{{ item.value }}' /etc/fstab
register: is_declared
with_dict:
"{{ fstab.regexs }}"
- name: Add the missing entries
sudo: yes
lineinfile: dest=/etc/fstab line="{{ item.1.item.value }}"
when: item.0.rc == 1
with_together:
- "{{ is_declared.results }}"
- "{{ fstab.entries }}"
vars/main.yml:
---
fstab:
paths:
a: "/mnt/a"
b: "/mnt/b"
regexs:
a: '\n# \(a\)\nfoo1'
b: '\n# \(b\)\nfoo2'
entries:
a: "\n# (a)\nfoo1"
b: "\n# (b)\nfoo2"
I am not using template on purpose (I want to add entries to existing files and not to over write them).
UPDATE: I see ansible has module "mount" which deals with fstab. However I am still looking for a solution to this issue because I might be needed it again later on.
I have a couple ideas as to why your original approach was failing, but let's scratch that for a moment. It looks like you're overcomplicating things- why not use a complex list var to tie it all together, and use the regexp arg to the lineinfile module instead of a separate regex task? (though your sample data should work fine even without the regexp param) Something like:
---
- name: Make dirs
sudo: yes
file: path={{ item.path }} state=directory
with_items: fstab
- name: Add the missing entries
sudo: yes
lineinfile: dest=/etc/fstab line={{ item.entry }} regexp={{ item.regex }}
with_items: fstab
fstab:
- path: /mnt/a
regex: '\n# \(a\)\nfoo1'
entry: "\n# (a)\nfoo1"
- path: /mnt/b
regex: '\n# \(b\)\nfoo2'
entry: '\n# (b)\nfoo2'

Resources