So, I am working with Ansible and I am retrieving specific information from some devices. In particular, I have a task like this:
<rpc-reply message-id="urn:uuid:1914-b84d7ff">
<lldp-neighbors-information style="brief">
<lldp-neighbor-information>
<lldp-local-port-id>A1</lldp-local-port-id>
<lldp-local-parent-interface-name>-</lldp-local-parent-interface-name>
<lldp-remote-chassis-id-subtype>A2</lldp-remote-chassis-id-subtype>
<lldp-remote-chassis-id>A3</lldp-remote-chassis-id>
<lldp-remote-port-id-subtype>A4</lldp-remote-port-id-subtype>
<lldp-remote-port-id>A5</lldp-remote-port-id>
<lldp-remote-system-name>A6</lldp-remote-system-name>
</lldp-neighbor-information>
<lldp-neighbor-information>
<lldp-local-port-id>B1</lldp-local-port-id>
<lldp-local-parent-interface-name>-</lldp-local-parent-interface-name>
<lldp-remote-chassis-id-subtype>B2</lldp-remote-chassis-id-subtype>
<lldp-remote-chassis-id>B3</lldp-remote-chassis-id>
<lldp-remote-port-id-subtype>B4</lldp-remote-port-id-subtype>
<lldp-remote-port-id>B5</lldp-remote-port-id>
<lldp-remote-system-name>B6</lldp-remote-system-name>
</lldp-neighbor-information>
<lldp-neighbor-information>
<lldp-local-port-id>C1</lldp-local-port-id>
<lldp-local-parent-interface-name>-</lldp-local-parent-interface-name>
<lldp-remote-chassis-id-subtype>C2</lldp-remote-chassis-id-subtype>
<lldp-remote-chassis-id>C3</lldp-remote-chassis-id>
<lldp-remote-port-id-subtype>C4</lldp-remote-port-id-subtype>
<lldp-remote-port-id>C5</lldp-remote-port-id>
</lldp-neighbor-information>
</lldp-neighbors-information>
</rpc-reply>
My goal is to build a dictionary like this :
{'A6': ['A1', 'A2'], 'B6': ['B1', 'B2']}
What I have done is creating a list with all my keys by doing the following tasks:
- name: Retrieve lldp system names
xml:
xmlstring: "{{ item.string | regex_replace('\n', '') }}"
xpath: "{{ item.path }}"
content: text
loop:
- { path: "/rpc-reply/lldp-neighbors-information/lldp-neighbor-information/lldp-remote-system-name", string: "{{xml_reply.xml}}" }
register: sys_names
- name: Save all sys names in a list
set_fact:
sys_names_list: "{{ sys_names.results[0].matches | map('dict2items') | list | json_query('[].value') }}"
Then I could create a second list for the elements [A1, B1] and a 3rd list [A2, B2] respectively. So then I could just combine the 3 lists and create my dictionary.
So there are 3 questions here :
Is there a way to directly build an ansible-dictionary from xml elements or should I keep writing my own module ?
Since the last element C6 would be my 3rd key BUT does NOT exist, I want to skip it. How is that possible in my task above?
How do I combine combine the 3 lists and create my dictionary, having skipped the 3rd element ? Otherwise my lists will not match the correct info ..
You can perform this by combining a filter that converts your xml file to a structured json in order to facilitate data manipulations.
In this example, I use this converter from xml to json.
File filter_plugins/from_xml.py :
#!/usr/bin/python
# From https://github.com/nasgoncalves/ansible-xml-to-json-filter/blob/master/filter_plugins/xml_to_json.py
import json
import xml.etree.ElementTree as ET
from collections import defaultdict
class FilterModule(object):
def etree_to_dict(self, t):
d = {t.tag: {} if t.attrib else None}
children = list(t)
if children:
dd = defaultdict(list)
for dc in map(self.etree_to_dict, children):
for k, v in dc.items():
dd[k].append(v)
d = {t.tag: {k: v[0] if len(v) == 1 else v
for k, v in dd.items()}}
if t.attrib:
d[t.tag].update(('#' + k, v)
for k, v in t.attrib.items())
if t.text:
text = t.text.strip()
if children or t.attrib:
if text:
d[t.tag]['#text'] = text
else:
d[t.tag] = text
return d
def filters(self):
return {
'from_xml': self.from_xml,
'xml_to_json': self.xml_to_json
}
def from_xml(self, data):
root = ET.ElementTree(ET.fromstring(data)).getroot()
return self.etree_to_dict(root)
def xml_to_json(self, data):
return json.dumps(self.from_xml(data))
File playbook.yml, I use a jinja2 template to filter key and combine values in a list, then I convert this list to a dict with filter items2dict :
---
- hosts: localhost
gather_facts: no
connection: local
tasks:
- set_fact:
xml_message: |
<rpc-reply message-id="urn:uuid:1914-b84d7ff">
<lldp-neighbors-information style="brief">
<lldp-neighbor-information>
<lldp-local-port-id>A1</lldp-local-port-id>
<lldp-local-parent-interface-name>-</lldp-local-parent-interface-name>
<lldp-remote-chassis-id-subtype>A2</lldp-remote-chassis-id-subtype>
<lldp-remote-chassis-id>A3</lldp-remote-chassis-id>
<lldp-remote-port-id-subtype>A4</lldp-remote-port-id-subtype>
<lldp-remote-port-id>A5</lldp-remote-port-id>
<lldp-remote-system-name>A6</lldp-remote-system-name>
</lldp-neighbor-information>
<lldp-neighbor-information>
<lldp-local-port-id>B1</lldp-local-port-id>
<lldp-local-parent-interface-name>-</lldp-local-parent-interface-name>
<lldp-remote-chassis-id-subtype>B2</lldp-remote-chassis-id-subtype>
<lldp-remote-chassis-id>B3</lldp-remote-chassis-id>
<lldp-remote-port-id-subtype>B4</lldp-remote-port-id-subtype>
<lldp-remote-port-id>B5</lldp-remote-port-id>
<lldp-remote-system-name>B6</lldp-remote-system-name>
</lldp-neighbor-information>
<lldp-neighbor-information>
<lldp-local-port-id>C1</lldp-local-port-id>
<lldp-local-parent-interface-name>-</lldp-local-parent-interface-name>
<lldp-remote-chassis-id-subtype>C2</lldp-remote-chassis-id-subtype>
<lldp-remote-chassis-id>C3</lldp-remote-chassis-id>
<lldp-remote-port-id-subtype>C4</lldp-remote-port-id-subtype>
<lldp-remote-port-id>C5</lldp-remote-port-id>
</lldp-neighbor-information>
</lldp-neighbors-information>
</rpc-reply>
- set_fact:
my_dict: |
{% set json_message = xml_message | from_xml %}
{% for item in json_message['rpc-reply']['lldp-neighbors-information']['lldp-neighbor-information'] %}
{% if "lldp-remote-system-name" in item %}
- key: {{ item['lldp-remote-system-name'] }}
value: [{{ item['lldp-local-port-id'] }}, {{ item['lldp-remote-chassis-id-subtype'] }}]
{% endif %}
{% endfor %}
- set_fact:
my_dict: "{{ my_dict | from_yaml | items2dict }}"
- debug:
var: my_dict
Executing this playbook with ansible-playbook playbook.yml returns :
PLAY [localhost] **********************************
TASK [set_fact] ***********************************
ok: [localhost]
TASK [set_fact] ***********************************
ok: [localhost]
TASK [set_fact] ***********************************
ok: [localhost]
TASK [debug] **************************************
ok: [localhost] => {
"my_dict": {
"A6": [
"A1",
"A2"
],
"B6": [
"B1",
"B2"
]
}
}
Related
If I have this data structure:
blahblah:
- name: firstdict
touch:
- file: name1
type: file
- file: name2
type: directory
- name: seconddict
touch:
- file: name3
type: file
How can I loop over this to ensure each file exists and is of type type whilst handling the event that the touch value might not even be present?
I have tried:
- name: Blah
file:
path: "{{ item.1.file }}"
state: "{{ item.1.type }}"
with_subelements:
- "{{ blahblah }}"
- touch
It seems to work but fails if the touch key isn't present in the dictionary. Is there a way to provide a default empty list if touch isn't specified?
Use the loop syntax instead of the with_subelements one.
Not only it is kind of recommended by Ansible, nowadays – see the notes in the "Loop" documentation page – it will also force you to use the subelements filter, and with it, discover its skip_missing parameter.
Given the task:
- debug:
msg: "{{ item }}"
loop: "{{ blahblah | subelements('touch', skip_missing=True) }}"
loop_control:
label: "{{ item.0.name }}"
vars:
blahblah:
- name: first dict
touch:
- file: name1
- file: name2
- name: second dict
touch:
- file: name3
- name: without touch
This would yield:
ok: [localhost] => (item=firstdict) =>
msg:
- name: firstdict
touch:
- file: name1
type: file
- file: name2
type: directory
- file: name1
type: file
ok: [localhost] => (item=firstdict) =>
msg:
- name: firstdict
touch:
- file: name1
type: file
- file: name2
type: directory
- file: name2
type: directory
ok: [localhost] => (item=seconddict) =>
msg:
- name: seconddict
touch:
- file: name3
type: file
- file: name3
type: file
I have a data structure that looks like this:
all_vms:
clusterA:
group1:
- vm-01
- vm-02
group2:
- vm-03
- vm-04
clusterB:
group1:
- vm-05
- vm-06
The key names are not known beforehand. There may be shared "group" key names between clusters.
I want to loop through that data structure and run a task with each of the arrays, but I need the names of the keys they're contained within. The looping task would look something like:
- task:
vms: "{{ item.value }}"
group: "{{ item.key }}"
cluster: "{{ item.parentkey }}"
loop: "{{ all_vms | ??? }}"
And that would unroll to:
- task:
vms:
- vm-01
- vm-02
group: group1
cluster: clusterA
- task:
vms:
- vm-03
- vm-04
group: group2
cluster: clusterA
- task:
vms:
- vm-05
- vm-06
group: group3
cluster: clusterB
I cannot change the main cluster/group structure, but I can change the structure of the elements that are currently arrays. I have considered just duplicating the keys as values, like this:
all_vms:
clusterA:
group1:
cluster: "clusterA"
group: "group1"
vms:
- vm-01
- vm-02
group2:
cluster: "clusterA"
group: "group2"
vms:
- vm-03
- vm-04
clusterB:
group1:
cluster: "clusterB"
group: "group1"
vms:
- vm-05
- vm-06
I would rather not do that, because it's terrible, but I can. But I can't even figure out a way to pop each of those things out into an array. (Edit: Actually, I think figured that out right after posting: all_vms | json_query('*.* | []'). I guess I can go with that if there's not a way to use the tidier data structure.)
Or if I could just use a #!#$% nested loop, if ansible would let me:
- block:
- task:
vms: "{{ item.value }}"
group: "{{ item.key }}"
cluster: "{{ cluster.key }}"
loop: "{{ cluster.value | dict2items }}"
loop: "{{ all_vms | dict2items }}"
loop_control:
loop_var: cluster
(Yes, I could do this with include_tasks, but having to have a separate file for a nested loop is just ridiculous.)
Any ideas how to iterate over this data structure without having to resort to a separate file just to do nested looping?
And here is the solution using several combinations of filters directly in Ansible / Jinja.
It combines the first level keys and values with a zip filter, in order to have a know subelements name — 1 — on which we can then use a subelements.
The second level key / value pair is accessible thanks to a dict2items mapped on the first level values.
The task ends up being
- set_fact:
tasks: "{{ tasks | default([]) + [_task] }}"
loop: >-
{{
all_vms.keys()
| zip(all_vms.values() | map('dict2items'))
| subelements([1])
}}
loop_control:
label: "{{ item.0.0 }} — {{ item.1.key }}"
vars:
_task:
task:
vms: "{{ item.1.value }}"
group: "{{ item.1.key }}"
cluster: "{{ item.0.0 }}"
Given the playbook:
- hosts: localhost
gather_facts: no
tasks:
- set_fact:
tasks: "{{ tasks | default([]) + [_task] }}"
loop: >-
{{
all_vms.keys()
| zip(all_vms.values() | map('dict2items'))
| subelements([1])
}}
loop_control:
label: "{{ item.0.0 }} — {{ item.1.key }}"
vars:
_task:
task:
vms: "{{ item.1.value }}"
group: "{{ item.1.key }}"
cluster: "{{ item.0.0 }}"
all_vms:
clusterA:
group1:
- vm-01
- vm-02
group2:
- vm-03
- vm-04
clusterB:
group1:
- vm-05
- vm-06
- debug:
var: tasks
This yields:
TASK [set_fact] ***************************************************************
ok: [localhost] => (item=clusterA — group1)
ok: [localhost] => (item=clusterA — group2)
ok: [localhost] => (item=clusterB — group1)
TASK [debug] ******************************************************************
ok: [localhost] =>
tasks:
- task:
cluster: clusterA
group: group1
vms:
- vm-01
- vm-02
- task:
cluster: clusterA
group: group2
vms:
- vm-03
- vm-04
- task:
cluster: clusterB
group: group1
vms:
- vm-05
- vm-06
Although this is definitely possible using several combinations of filters directly in Ansible/jinja2, my 2 cent: save you nerves and time using a custom filter.
Demo below is for a a filter in the dedicated filters_plugin folder adjacent to your playbook. See the roles and collections documentations for better ways to distribute your filter if need be. Note that this was written real quick to put you on track and although it is fully functional with your above data example, you will need to add some data and errors checks if you intend to use this for a larger audience.
filters_plugins/my_cluster_filters.py
def all_vms_list(vm_dict):
"""Return a list of dicts for all vms with their cluster an group information"""
vms_list = []
for cluster, groups in vm_dict.items():
for group_name, hosts in groups.items():
for host in hosts:
current_vm = {
'cluster': cluster,
'group': group_name,
'host': host
}
vms_list.append(current_vm)
return vms_list
class FilterModule(object):
"""my cluster data filters."""
def filters(self):
"""Return the filter list."""
return {
'all_vms_list': all_vms_list
}
Then the test playbook.yml:
---
- hosts: localhost
gather_facts: false
vars:
all_vms:
clusterA:
group1:
- vm-01
- vm-02
group2:
- vm-03
- vm-04
clusterB:
group1:
- vm-05
- vm-06
tasks:
- debug:
msg: "Host {{ item.host }} belongs to group {{ item.group }} inside cluster {{ item.cluster }}"
loop: "{{ all_vms | all_vms_list }}"
gives:
PLAY [localhost] **************************************************************************************************************************************************************************************************************
TASK [debug] ******************************************************************************************************************************************************************************************************************
ok: [localhost] => (item={'cluster': 'clusterA', 'group': 'group1', 'host': 'vm-01'}) => {
"msg": "Host vm-01 belongs to group group1 inside cluster clusterA"
}
ok: [localhost] => (item={'cluster': 'clusterA', 'group': 'group1', 'host': 'vm-02'}) => {
"msg": "Host vm-02 belongs to group group1 inside cluster clusterA"
}
ok: [localhost] => (item={'cluster': 'clusterA', 'group': 'group2', 'host': 'vm-03'}) => {
"msg": "Host vm-03 belongs to group group2 inside cluster clusterA"
}
ok: [localhost] => (item={'cluster': 'clusterA', 'group': 'group2', 'host': 'vm-04'}) => {
"msg": "Host vm-04 belongs to group group2 inside cluster clusterA"
}
ok: [localhost] => (item={'cluster': 'clusterB', 'group': 'group1', 'host': 'vm-05'}) => {
"msg": "Host vm-05 belongs to group group1 inside cluster clusterB"
}
ok: [localhost] => (item={'cluster': 'clusterB', 'group': 'group1', 'host': 'vm-06'}) => {
"msg": "Host vm-06 belongs to group group1 inside cluster clusterB"
}
PLAY RECAP ********************************************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
I have a scenario where I need to print only 0th array and 1st array of 'N'of array in ansible.
Could someone help me to achieve this
Sample code:
Array;
ID = [1,2,3,4]
Ansible Code:
- hosts: localhost
gather_facts: no
tasks:
- name: Print the first 2 values in the array
debug:
msg: "{{ item }}"
with_items: "{{ ID }}"
Expected Output:
PLAY [localhost] ***********************************************************************************************************************************************************************************************
TASK [Print the first 2 values in the array] *******************************************************************************************************************************************************************
ok: [localhost] => (item=1) => {
"msg": "1"
}
ok: [localhost] => (item=2) => {
"msg": "2"
}
Actual Output:
PLAY [localhost] ***********************************************************************************************************************************************************************************************
TASK [Print the first 2 values in the array] *******************************************************************************************************************************************************************
ok: [localhost] => (item=1) => {
"msg": "1"
}
ok: [localhost] => (item=2) => {
"msg": "2"
}
ok: [localhost] => (item=3) => {
"msg": "3"
}
ok: [localhost] => (item=4) => {
"msg": "4"
}
You could do this with Playbook Loops and with_sequence
- name: Show sequence
debug:
msg: "{{ item }}"
with_sequence:
- "0-1"
Thanks to
Ansible with_items in range
Loop from 0 to 100 with Ansible
To get the value of an array you would use ID[item]. You may have a look into the following loop example and how it works.
---
- hosts: localhost
become: false
gather_facts: false
vars:
ID: [A, B, C, D]
tasks:
- name: Show entry
debug:
msg:
- "{{ item }} {{ ID[item | int] }}"
with_sequence:
- "0-1"
Use slice notation, e.g.
- debug:
var: item
loop: "{{ ID[0:2] }}"
gives (abridged)
item: 1
item: 2
You can concatenate slices if you want to, e.g. to get the 1st, 2nd, and 4th item
- debug:
var: item
loop: "{{ ID[0:2] + ID[3:4] }}"
gives (abridged)
item: 1
item: 2
item: 4
Hi I have a file with list of filenames .I want to read each line and add a variable {{version}} to the end and pass it to another task which will download the artifact .
MY_File
cat file/branches.txt
mhr-
mtr-
tsr
I want to get each line add {{version}} to it
mhr-1.1-SNAPSHOT
mtr-1.1-SNAPSHOT
tsr-1.1-SNAPSHOT
I couldn't get the file names and append them. below is a failed attempt by me .
---
- name: get file names
shell: echo {{item}}{{version}}
register: result
with_lines: cat files/branches.txt
- include_tasks: 2main.yml
with_items: "{{result.results}}"
I think what you need a new variable which contains text like mhr-1.1-SNAPSHOT depending on the source file. You can use set_fact to achieve this.
Example:
vars:
file_versions: []
version: "1.1-SNAPSHOT"
tasks:
- set_fact:
file_versions: "{{ file_versions }} + [ '{{ item }}{{ version }}' ]"
with_lines: cat file/branches.txt
- debug:
msg: "{{ item }}"
with_items: "{{ file_versions }}"
- include_tasks: 2main.yml
with_items: "{{ file_versions }}"
Gives below debug output:
TASK [debug] ************************************************************************************************************************
ok: [localhost] => (item=mhr-1.1-SNAPSHOT) => {
"msg": "mhr-1.1-SNAPSHOT"
}
ok: [localhost] => (item=mtr-1.1-SNAPSHOT) => {
"msg": "mtr-1.1-SNAPSHOT"
}
ok: [localhost] => (item=tsr-1.1-SNAPSHOT) => {
"msg": "tsr-1.1-SNAPSHOT"
}
In a nutshell:
---
- name: append version to names found in a local text file line by line
hosts: localhost
gather_facts: false
vars:
# This is the version we want to append
version: 1.1-SNAPSHOT
# This is the file on local controller containing the names
my_file: /tmp/my_file.txt
# This var will contain a list of each name in file with appended version
file_version: "{{ lookup('file', my_file).splitlines() | map('regex_replace', '^(.*)$', '\\g<1>' + version) | list }}"
tasks:
- name: Show the result
debug:
var: file_version
- name: Loop on var with an include (the passed var is item in this case)
include_tasks: some_other_tasks_working_on_item.yml
loop: "{{ file_version }}"
I have a 'with_item' loop in which variable 'USER' needs to be assigned a value based on condition check.
If '{{ command_result.stdout_lines }}' is "Frontend" then USER variable should get value 'user1' else it should be assigned value 'user2'
Below is something I could achieve after seeking help but the problem is that 'USER' always gets assigned 'user2' value even if the condition is met for 'user1'
My playbook:
- debug:
msg: "User was {{ item.split('\t')[3] }}"
with_items: "{{ command_result.stdout_lines }}"
- set_fact:
USER: "user1"
when: item.split('\t')[3] == "FrontEnd"
with_items: "{{ command_result.stdout_lines }}"
- set_fact:
USER: "user2"
when: item.split('\t')[3] == "BackEnd"
with_items: "{{ command_result.stdout_lines }}"
- debug:
msg: "User has {{ USER }}"
with_items: "{{ command_result.stdout_lines }}"
The First debug prints and confirms that the value of
item.split('\t')[3]
The second debug prints 'USER' but as you can see in the output below it has value 'user2' even when the value is Frontend.
Can you please suggest?
TASK [debug]
************************************************************************************************************************************************ ok: [localhost] => (item=10.12.1.13 10.12.1.13\n-rw-rw-r-- user1
2019-09-13 15:39 /was//testingjsp/testingcom.jsp 1786385840
/was//testingjsp FrontEnd) => {
"msg": "User was FrontEnd" } ok: [localhost] => (item=10.12.1.13 10.12.1.13\n-rw-rw-r-- user2 2019-09-13 15:29 /fin/scripts/testingscr.scr 367595418\n-rw-rw-r-- user2 2019-09-13
15:36 /fin/mrt/testingmrt.mrt 1251350031\n-rw-rw-r-- user2 2019-09-13
15:37 /fin/exe/testingexe.exe 1390265645\n-rw-rw-r-- user2 2019-09-13
15:38 /fin/com/testingcom.com 90193476
/fin/scripts\n/fin/mrt\n/fin/exe\n/fin/com BackEnd) => {
"msg": "User was BackEnd" }
TASK [debug]
************************************************************************************************************************************************ ok: [localhost] => (item=10.12.1.13 10.12.1.13\n-rw-rw-r-- user1
2019-09-13 15:39 /was//testingjsp/testingcom.jsp 1786385840
/was//testingjsp FrontEnd) => {
"msg": "User has user2" } ok: [localhost] => (item=10.12.1.13 10.12.1.13\n-rw-rw-r-- user2 2019-09-13 15:29 /fin/scripts/testingscr.scr 367595418\n-rw-rw-r-- user2 2019-09-13
15:36 /fin/mrt/testingmrt.mrt 1251350031\n-rw-rw-r-- user2 2019-09-13
15:37 /fin/exe/testingexe.exe 1390265645\n-rw-rw-r-- user2 2019-09-13
15:38 /fin/com/testingcom.com 90193476
/fin/scripts\n/fin/mrt\n/fin/exe\n/fin/com BackEnd) => {
"msg": "User has user2" }
Below is the tasks which will get you USER for each item in the command_result.
- set_fact:
USER: "{% if item.split('\t')[2] == 'FrontEnd' %}user1{% elif item.split('\t')[2] == 'BackEnd' %}user2{% else %}{% endif %}"
with_items: "{{ command_result.stdout_lines }}"
register: "facts"
- debug:
msg: "{{ item.ansible_facts.USER }}"
with_items: "{{ facts.results }}"
The first task sets the value of USER fact based on the value extracted by item.split('\t')[3] and uses jinja2 templating to set the value of USER.
If there are n items in your command_result, the first task will be registering the USER fact n times. Hence, I have registered the value in facts named variable.
The values are accessed and printed in the second task.
You can similarly use the values as shown in the second task in subsequent tasks.
Hope it helps.