Looping through complex dictionary with Ansible - loops

I'm using the Cisco ACI_REST module. I'm building a playbook to shutdown all of my ACI EPGs. The ACI_EPG module is used to create the dict.
- name: Get List of EPGs
cisco.aci.aci_epg:
host: "{{ inventory_hostname }}"
username: "{{ ansible_user }}"
password: "{{ ansible_password }}"
validate_certs: no
state: query
delegate_to: localhost
register: epg_list
It creates a dictionary that looks like this:
"item": {
"key": "current",
"value": [
{
"fvAEPg": {
"attributes": {
"dn": "uni/tn-PC/ap-PROD_AP/epg-VLAN80_EPG",
"shutdown": "no",
},
"children": [
{
"fvRsBd": {
"attributes": {
"annotation": "",
"tDn": "uni/tn-PC/BD-VLAN80_BD",
"tRn": "BD-VLAN80_BD",
"tType": "name",
"tnFvBDName": "VLAN80_BD",
"uid": "0"
},
"children": [
{
"fvSubnetBDDefCont": {
"attributes": {
"name": "",
"nameAlias": "",
"status": ""
}
}
}
]
}
}
]
}
}
I need to loop through the DN values.
"dn": "uni/tn-PC/ap-PROD_AP/epg-VLAN80_EPG"
I'm playing with dict2items but am stuck. How to I reference the nested items in my dictionary? Here is some test code that I'm working with.
- name: Display EPG List
ansible.builtin.debug:
msg: "{{ item.key }} - {{ item.value }}"
loop: "{{ epg_list | dict2items }}"
Thanks,

Q: "Loop through the DN values."
A: Try json_query, e.g.
- debug:
msg: "{{ item }}"
loop: "{{ epg_list.item.value|
json_query('[].*.attributes.dn')|
flatten }}"
gives
msg: uni/tn-PC/ap-PROD_AP/epg-VLAN80_EPG
Given the data
epg_list:
item:
key: current
value:
- fvAEPg:
attributes:
dn: uni/tn-PC/ap-PROD_AP/epg-VLAN80_EPG
shutdown: 'no'
children:
- fvRsBd:
attributes:
annotation: ''
tDn: uni/tn-PC/BD-VLAN80_BD
tRn: BD-VLAN80_BD
tType: name
tnFvBDName: VLAN80_BD
uid: '0'
children:
- fvSubnetBDDefCont:
attributes:
name: ''
nameAlias: ''
status: ''
If the data is
epg_list:
current:
- fvAEPg:
attributes:
dn: uni/tn-PC/ap-PROD_AP/epg-VLAN80_EPG
shutdown: 'no'
children:
- fvRsBd:
attributes:
annotation: ''
tDn: uni/tn-PC/BD-VLAN80_BD
tRn: BD-VLAN80_BD
tType: name
tnFvBDName: VLAN80_BD
uid: '0'
children:
- fvSubnetBDDefCont:
attributes:
name: ''
nameAlias: ''
status: ''
the task below gives the same result
- debug:
msg: "{{ item }}"
loop: "{{ epg_list.current|
json_query('[].*.attributes.dn')|
flatten }}"

Related

Registering output to complex variable and placing conditionals on sub-components

I'm getting a bit lost with next variable types and am hoping for some direction in a specific task please:
The Goal:
Based on a list of username:publickey values. I'd like to:
ensure the user exists on the target system
if the user does exist then:
- ensure the "/home/$user/.ssh/authorized_keys" file exists with the correct permissions through the path.
The Scene:
I have a variable:
ssh_vars:
auth_keys:
bob: "bobs_public_key_string"
anne: "annes_public_key_string"
anon: "anons_public_key_string
I need to iterate over this variable and for each auth_keys item call a tasklist:
- name: loop through the auth_keys and call ssh_dirs.yml for each
ansible.builtin.include_tasks: "ssh_dirs.yaml"
loop: "{{ ssh_vars.auth_keys }}"
However, I only really want to do this when the auth_key(key) is a user which already exists on the host.
I have been playing with getent, within "ssh_dirs.yaml":
- name: "Ensure the user exists on the target system"
ansible.builtin.getent:
database: passwd
key: "{{ item.name }}"
fail_key: false
register: userlookup
which creates what i think is a list of dictionaries:
ok: [ans-client.local] => {
"userlookup": {
"changed": false,
"msg": "All items completed",
"results": [
{
"ansible_facts": {
"getent_passwd": {
"bob": [
"x",
"1003",
"1003",
"",
"/home/bob",
"/usr/bin/bash"
]
}
},
"ansible_loop_var": "item",
"changed": false,
"failed": false,
"invocation": {
"module_args": {
"database": "passwd",
"fail_key": false,
"key": "bob",
"service": null,
"split": null
}
},
"item": {
"key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDvIZuaBhAIGShw21rkvgqyvNePunbVs6OtOBhYJOY2P anne#ans-server",
"name": "bob"
}
},
{
"ansible_facts": {
"getent_passwd": {
"anne": [
"x",
"1000",
"1000",
"anne",
"/home/anne",
"/bin/bash"
]
}
},
"ansible_loop_var": "item",
"changed": false,
"failed": false,
"invocation": {
"module_args": {
"database": "passwd",
"fail_key": false,
"key": "anne",
"service": null,
"split": null
}
},
"item": {
"key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKr/76O3hLJlcyZuy7EJxf7sC1z9BSHMuxGsFGBibJY3 anne#ans-server",
"name": "anne"
}
},
{
"ansible_facts": {
"getent_passwd": {
"anon": null
}
},
"ansible_loop_var": "item",
"changed": false,
"failed": false,
"invocation": {
"module_args": {
"database": "passwd",
"fail_key": false,
"key": "anon",
"service": null,
"split": null
}
},
"item": {
"key": "SOMEKEY",
"name": "anon"
},
"msg": "One or more supplied key could not be found in the database."
}
],
"skipped": false
}
}
But I can't figure out how to isolate this list to ensure the include_tasks: is not called if the user doesn't exist.
- name: loop through the auth_keys and call ssh_dirs.yml for each
ansible.builtin.include_tasks: "ssh_dirs.yaml"
loop: "{{ ssh_vars.auth_keys }}"
when: userlookup.results.???????
How can I figure out how to reference this nested variable, and how best to isolate a non-missing user?
Something like userlookup.results.msg is not defined might work but it's very loose - is there something better I'm missing?
See registering variables with loop
The global idea is
loop over you var to get the existing/unavailable users
loop over the results of that previous task for you next one. The original loop variable is available in the item key of each result and you can filter as you like.
For your particular case, in a nutshell (untested):
- name: Ensure the user exists on the target system
ansible.builtin.getent:
database: passwd
key: "{{ item.name }}"
register: userlookup
ignore_errors: true
loop: "{{ ssh_vars.auth_keys }}"
- name: Call ssh_dirs.yml for each existing users
ansible.builtin.include_tasks: ssh_dirs.yaml
loop: "{{ userlookup.results | select('success') }}"
loop_control:
loop_var: user_checked
vars:
userloop: "{{ user_checked.item }}"
I think I've solved it, although maybe there's a better thing to look for in the getent response than just the msg?
The logic and variable reference which works:
- name: "Ensure the user exists on the target system"
ansible.builtin.getent:
database: passwd
key: "{{ item.name }}"
fail_key: false
register: userlookup
loop: "{{ ssh_vars.auth_keys }}"
- name: Build a list of usernames which don't exist on the remote host (missing_users)
ansible.builtin.set_fact:
missing_users: "{{ missing_users | default([]) + [usercheck.item.name | string] }}"
loop: "{{ userlookup.results }}"
loop_control:
loop_var: usercheck
when: usercheck.msg is defined
- name: loop through the users and ensure the necessary user folders and files are there
ansible.builtin.include_tasks: "ssh_dirs.yaml"
loop: "{{ ssh_vars.auth_keys }}"
loop_control:
loop_var: userloop
when: userloop.name not in missing_users
Although this is still checking the msg: output mindlessly so only a partial solution

how to loop over hostvars of register output

I'm struggling to loop over hostvars of the registered output of all the hosts in dynamic inventory.
Here is the code.
$cat collect.yaml
---
- hosts: "{{ env }}"
become: True
tasks:
- name: Get dockerinfo
docker_host_info:
containers: yes
register: result
- name: Debug dockerInfo
debug:
var: result.containers
- name: dynamic grouping
add_host:
name: "{{ item[0] }}"
groups: "{{ item[1].Image | regex_replace('.*?/(.*?):.*', '\\1') }}"
loops:
- "{{ ansible_play_batch }}"
- "{{ hostvars[item].result.containers }}"
Error i get is item not defined. I would want the hosts refer to their respective result.containers. Not sure on how to use hostvars for the host to refer their respective result.containers.
Here is result.container output.
TASK [Debug dockerInfo]
ok: [vm1.nodekite.com] => {
"result.containers": [
{
"Image": "ca.docker/webproxy:1.0.0",
},
{
"Image": "docker.local/egacustomer:1.0.1",
},
]}
ok: [vm2.nodekite.com ] => {
"result.containers": [
{
"Image": "ca.docker/webproxyui:1.0.0",
},
{
"Image": "cna-docker-local/lega-customer:1.0.1",
},
]}
Here is the what i'm trying to achieve
changed: [vm1.nodekite.com] => {
"add_host": {
"groups": [
"webproxy"
],
"host_name": "vm1.nodekite.com",
},
changed: [vm1.nodekite.com] => {
"add_host": {
"groups": [
"egacustomer"
],
"host_name": "vm1.nodekite.com",
},
changed: [vm2.nodekite.com] => {
"add_host": {
"groups": [
"webproxy" >> this should be webproxyui
],
"host_name": "vm2.nodekite.com",
},
changed: [vm2.nodekite.com] => {
"add_host": {
"groups": [
"egacustomer" >> this should be lega-customer
],
"host_name": "vm2.nodekite.com",
},
Any help would be greatly appreciated.
I would run this task using Images Names
- hosts: "{{ group }}"
gather_facts: false
become: true
become_method: sudo
tasks:
- name: stop or restart docker containers
command: "docker {{ state }} {{ container_name }}"
How about just group_by paired with the loop?
- hosts: "{{ env }}"
become: True
tasks:
- name: Get dockerinfo
docker_host_info:
containers: yes
register: result
- debug:
var: result.containers
- group_by:
key: "container_{{ item.image | regex_replace('.*?/(.*?):.*', '\\1') }}"
loop: "{{ result.containers }}"
- debug:
var: group_names
You don't need to add the prefix. But this would add each host to groups with their container image prefixes. You should be able to use the group later in the play or playbook.

Ansible: How to loop over registered output of dynamic hosts in the next task

I am struggling to loop over registered results.containers for all the hosts in the dynamic inventory.
Here is the code.
$cat collect.yaml
---
- hosts: "{{ env }}"
become: True
tasks:
- name: Get dockerinfo
docker_host_info:
containers: yes
register: result
- name: Debug dockerInfo
debug:
var: result.containers
- name: dynamic grouping
add_host:
name: "{{ item[0] }}"
groups: "{{ item[1].Image | regex_replace('.*?/(.*?):.*', '\\1') }}"
container_name: '{{ item[1].Names[0] | regex_replace("^/", "") }}'
with_nested:
- "{{ ansible_play_batch }}"
- "{{ result.containers }}"
Here is result.containers output.
TASK [Debug dockerInfo]
ok: [vm1.nodekite.com] => {
"result.containers": [
{
"Image": "ca.docker/webproxy:1.0.0",
"Names": [
"/customer1"
],
},
{
"Image": "docker.local/egacustomer:1.0.1",
"Names": [
"/webproxy"
],
},
]}
ok: [vm2.nodekite.com ] => {
"result.containers": [
{
"Image": "ca.docker/webproxy:1.0.0",
"Names": [
"/webproxyui"
],
},
{
"Image": "cna-docker-local/lega-customer:1.0.1",
"Names": [
"/webproxy"
],
},
]}
ok: [vm3.nodekite.com ] => {
"result.containers": [
{
"Image": "ca.docker/webproxy:1.0.0",
"Names": [
"/webproxy"
],
},
{
"Image": "local.docker/saga-customer:1.0.1",
"Names": [
"/customerr
],
},
]}
Right now item[1].Image and item[1].Names[0] is only taken from first host's(vm1.nodekite.com) results.containers output. I would like to loop over for every hosts. So that, I could create dynamic group for all the hosts with their respective containers. With my code, hosts vm1,vm2,vm3 all are referring to vm1.nodekite.com's result.containers but i want the hosts to refer to their respective containers. Any help would be greatly appreciated.
I have update dynamic grouping task ouput for clarification.
changed: [vm1.nodekite.com] => {
"add_host": {
"groups": [
"webproxy"
],
"host_name": "vm1.nodekite.com",
"host_vars": {
"container_name": "customer1" }
},
changed: [vm1.nodekite.com] => {
"add_host": {
"groups": [
"egacustomer"
],
"host_name": "vm1.nodekite.com",
"host_vars": {
"container_name": "webproxy" }
},
changed: [vm2.nodekite.com] => {
"add_host": {
"groups": [
"webproxy" >> this should be webproxy
],
"host_name": "vm2.nodekite.com",
"host_vars": {
"container_name": "customer1" } >> this should be webproxyui
},
changed: [vm2.nodekite.com] => {
"add_host": {
"groups": [
"egacustomer" >> this should be lega-customer
],
"host_name": "vm2.nodekite.com",
"host_vars": {
"container_name": "webproxy" } >> this should be webproxy
},
if you see vm2 is still referring to vm1's result.containers output.
when i try this...i get item not defined error.
- name: adding it to groups using images
add_host:
name: "{{ item[0] }}"
groups: "{{ item[1].Image | regex_replace('.*?/(.*?):.*', '\\1') }}"
container_name: '{{ item[1].Names[0] | regex_replace("^/", "") }}'
loop:
- "{{ ansible_play_batch }}"
- "{{ myresult.containers }}"
vars:
myresult: "{{ hostvars[item].result }}"
run_once: true
Q: "Hosts shall refer to their respective containers."
A: Use hostvars. For example
- name: dynamic grouping
debug:
msg:
- "name: {{ item }}"
- "groups: {{ my_result.containers|
map(attribute='Image')|
map('regex_replace', '.*?/(.*?):.*', '\\1')|
list }}"
- "container_names: {{ my_result.containers|
map(attribute='Names')|
map('regex_replace', '\/', '')|
list }}"
loop: "{{ ansible_play_batch }}"
vars:
my_result: "{{ hostvars[item].result }}"
run_once: true
gives
ok: [vm1.nodekite.com] => (item=vm1.nodekite.com) =>
msg:
- 'name: vm1.nodekite.com'
- 'groups: [''webproxy'', ''egacustomer'']'
- 'container_names: ["[''customer1'']", "[''webproxy'']"]'
ok: [vm1.nodekite.com] => (item=vm2.nodekite.com) =>
msg:
- 'name: vm2.nodekite.com'
- 'groups: [''webproxy'', ''lega-customer'']'
- 'container_names: ["[''webproxyui'']", "[''webproxy'']"]'
ok: [vm1.nodekite.com] => (item=vm3.nodekite.com) =>
msg:
- 'name: vm3.nodekite.com'
- 'groups: [''webproxy'', ''saga-customer'']'
- 'container_names: ["[''webproxy'']", "[''customer'']"]'
(Feel free to fit the code to your needs.)
I was having an issue of getting the item passed into the name below to be a plain string of: item='nginx' and not item='[u'/nginx]'
To get around this, I did the following:
- name: Get docker containers
become: docker
community.docker.docker_container
containers: yes
register: docker_info
- name: Stop running containers
become: docker
community.docker.docker_container
name: "{{ item }}"
state: stopped
loop: "{{ docker_info.containers | sum(attribute='Names', start=[]) | map('regex_replace','\\/','') | list }}"
when: item in apps.split(,)
In this case the apps is a comma deliminated string variable I passed into the ansible playbook to limit which apps to stop.
The sum piece, flattens the Names of all the apps running into a single list.
The regex piece removes the / in the Names parameter

Ansible; using a loop to cycle through variables

I have an inventory in which many host_vars are set. Each host will contain a different number of datasets.
e.g.
host: host1
ip-addr: 192.0.2.12/24
datasets:
set1:
var1: 'east'
var2: 'west'
set2:
var1: 'north'
var2: 'south'
I can create a loop to count the datasets, but I don't seem able to use it to reference [varX]:
- name: "test loop"
debug:
msg:
- "{{ item }}"
- "{{ 'datasets.set' + item + '.var1' }}"
- "{{ datasets.set1.var1 }}"
loop: "{{ query('sequence', 'start=1 end='+((datasets|length)|string)) }}"
This appears to assemble the variable name I'm attempting to reference, however does not return the value associated with it. Manually calling that variable does return the interesting value.
ok: [host1] => (item=1) => {
"msg": [
"1",
"datasets.set1.var1",
"east"
]
}
ok: [host1] => (item=2) => {
"msg": [
"2",
"datasets.set2.var1",
"east"
]
}
Is what I'm doing possible, or should I be approaching it from another angle?
thanks in advance.
The task
- debug:
msg: "set{{ item }}.var1 = {{ datasets['set' ~ item].var1 }}"
loop: "{{ range(1, datasets|length+1)|list }}"
gives
"msg": "set1.var1 = east"
"msg": "set2.var1 = north"

Ansible: iterate over a list of dictionaries - loop vs. with_items

I'm getting different results when using loop vs with_items when trying to iterate over a list of dictionaries.
I've tried using loop|dict2items (the structure isn't a dictionary, & it tells me as much. heh) and loop with the flatten filter.
Here is the list of dictionaries:
"msg": [
{
"id": "id1",
"ip": "ip1",
"name": "name1"
},
{
"id": "id2",
"ip": "ip2",
"name": "name2"
},
{
"id": "id3",
"ip": "ip3",
"name": "name3"
},
{
"id": "id4",
"ip": "ip4",
"name": "name4"
}
]
}
Here is the task in the playbook:
- name: Add privateIp windows_instances to inventory
add_host:
name: "{{ item.ip }}"
aws_name: "{{ item.name }}"
groups: windows_instances
aws_instanceid: "{{ item.id }}"
ansible_user: "{{ windows_user }}"
ansible_password: "{{ windows_password }}"
ansible_port: 5985
ansible_connection: winrm
ansible_winrm_server_cert_validation: ignore
loop:
- "{{ list1 | flatten(levels=1) }}"
When attempting to run the above code, I get the "list object has no attribute" error. I've tried different flatten levels to no avail.
HOWEVER...
If I simply replace the loop above with:
with_items:
- "{{ list1 }}"
Everything works perfectly. I'm missing something in the with_items > loop translation here...
Don't put a - before your list.
And here, you have a list of dicts, so you don't need to flatten neither.
This playbook:
- hosts: localhost
gather_facts: no
vars:
demo_list:
- ip: 1.2.3.4
id: 1
name: demo1
- ip: 2.2.3.4
id: 2
name: demo2
- ip: 3.2.3.4
id: 3
name: demo3
tasks:
- name: the list
debug:
msg: "{{ demo_list }}"
- name: unflattened list
debug:
msg: "{{ item.id }} {{ item.ip }} {{ item.name }}"
loop:
"{{ demo_list }}"
- name: flattened list == unflattened list in this case
debug:
msg: "{{ item.id }} {{ item.ip }} {{ item.name }}"
loop:
"{{ demo_list | flatten(levels=1) }}"
gives this result:
PLAY [localhost] ***************************************************************************************
TASK [the list] ****************************************************************************************
ok: [localhost] => {
"msg": [
{
"id": 1,
"ip": "1.2.3.4",
"name": "demo1"
},
{
"id": 2,
"ip": "2.2.3.4",
"name": "demo2"
},
{
"id": 3,
"ip": "3.2.3.4",
"name": "demo3"
}
]
}
TASK [unflattened list] ********************************************************************************
ok: [localhost] => (item=None) => {
"msg": "1 1.2.3.4 demo1"
}
ok: [localhost] => (item=None) => {
"msg": "2 2.2.3.4 demo2"
}
ok: [localhost] => (item=None) => {
"msg": "3 3.2.3.4 demo3"
}
TASK [flattened list == unflattened list in this case] *************************************************
ok: [localhost] => (item=None) => {
"msg": "1 1.2.3.4 demo1"
}
ok: [localhost] => (item=None) => {
"msg": "2 2.2.3.4 demo2"
}
ok: [localhost] => (item=None) => {
"msg": "3 3.2.3.4 demo3"
}
PLAY RECAP *********************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0

Resources