I have a variable which is a list of dictionaries with a nested dictionary, like the following:
my_users:
- { name: "James Doe", user_id: "xman01", user_password: { clear_text: "password1", hash_md5: "", hash_sha512: "" } }
- { name: "John Dean", user_id: "xman02", user_password: { clear_text: "password2", hash_md5: "", hash_sha512: "" } }
- { name: "Jane Burn", user_id: "xman03", user_password: { clear_text: "password3", hash_md5: "", hash_sha512: "" } }
...
During playbook execution I'd like to loop through each item, generate both md5 and sha512 hashes for the clear_text values, and store them back inside the data structure itself, in other words, something like:
- name: update users data with password hashes
set_fact:
item.user_password.hash_md5 = "{{ item.user_password.clear_text | password_hash('md5') }}"
item.user_password.hash_sha512 = "{{ item.user_password.clear_text | password_hash('sha512') }}"
with_items: "{{ my_users }}"
I know this does not work in ansible because the syntax is wrong and I've tried to adapt many similar examples I've found out there without any luck.
Any suggestion?
I'm on ansible 2.3.x due to CentOS-5/RHEL-5 compatibility constraints.
Related
If I have this data structure:
blahblah:
- name: firstdict
touch:
- file: name1
type: file
- file: name2
type: directory
- name: seconddict
touch:
- file: name3
type: file
How can I loop over this to ensure each file exists and is of type type whilst handling the event that the touch value might not even be present?
I have tried:
- name: Blah
file:
path: "{{ item.1.file }}"
state: "{{ item.1.type }}"
with_subelements:
- "{{ blahblah }}"
- touch
It seems to work but fails if the touch key isn't present in the dictionary. Is there a way to provide a default empty list if touch isn't specified?
Use the loop syntax instead of the with_subelements one.
Not only it is kind of recommended by Ansible, nowadays – see the notes in the "Loop" documentation page – it will also force you to use the subelements filter, and with it, discover its skip_missing parameter.
Given the task:
- debug:
msg: "{{ item }}"
loop: "{{ blahblah | subelements('touch', skip_missing=True) }}"
loop_control:
label: "{{ item.0.name }}"
vars:
blahblah:
- name: first dict
touch:
- file: name1
- file: name2
- name: second dict
touch:
- file: name3
- name: without touch
This would yield:
ok: [localhost] => (item=firstdict) =>
msg:
- name: firstdict
touch:
- file: name1
type: file
- file: name2
type: directory
- file: name1
type: file
ok: [localhost] => (item=firstdict) =>
msg:
- name: firstdict
touch:
- file: name1
type: file
- file: name2
type: directory
- file: name2
type: directory
ok: [localhost] => (item=seconddict) =>
msg:
- name: seconddict
touch:
- file: name3
type: file
- file: name3
type: file
I have a data structure that looks like this:
all_vms:
clusterA:
group1:
- vm-01
- vm-02
group2:
- vm-03
- vm-04
clusterB:
group1:
- vm-05
- vm-06
The key names are not known beforehand. There may be shared "group" key names between clusters.
I want to loop through that data structure and run a task with each of the arrays, but I need the names of the keys they're contained within. The looping task would look something like:
- task:
vms: "{{ item.value }}"
group: "{{ item.key }}"
cluster: "{{ item.parentkey }}"
loop: "{{ all_vms | ??? }}"
And that would unroll to:
- task:
vms:
- vm-01
- vm-02
group: group1
cluster: clusterA
- task:
vms:
- vm-03
- vm-04
group: group2
cluster: clusterA
- task:
vms:
- vm-05
- vm-06
group: group3
cluster: clusterB
I cannot change the main cluster/group structure, but I can change the structure of the elements that are currently arrays. I have considered just duplicating the keys as values, like this:
all_vms:
clusterA:
group1:
cluster: "clusterA"
group: "group1"
vms:
- vm-01
- vm-02
group2:
cluster: "clusterA"
group: "group2"
vms:
- vm-03
- vm-04
clusterB:
group1:
cluster: "clusterB"
group: "group1"
vms:
- vm-05
- vm-06
I would rather not do that, because it's terrible, but I can. But I can't even figure out a way to pop each of those things out into an array. (Edit: Actually, I think figured that out right after posting: all_vms | json_query('*.* | []'). I guess I can go with that if there's not a way to use the tidier data structure.)
Or if I could just use a #!#$% nested loop, if ansible would let me:
- block:
- task:
vms: "{{ item.value }}"
group: "{{ item.key }}"
cluster: "{{ cluster.key }}"
loop: "{{ cluster.value | dict2items }}"
loop: "{{ all_vms | dict2items }}"
loop_control:
loop_var: cluster
(Yes, I could do this with include_tasks, but having to have a separate file for a nested loop is just ridiculous.)
Any ideas how to iterate over this data structure without having to resort to a separate file just to do nested looping?
And here is the solution using several combinations of filters directly in Ansible / Jinja.
It combines the first level keys and values with a zip filter, in order to have a know subelements name — 1 — on which we can then use a subelements.
The second level key / value pair is accessible thanks to a dict2items mapped on the first level values.
The task ends up being
- set_fact:
tasks: "{{ tasks | default([]) + [_task] }}"
loop: >-
{{
all_vms.keys()
| zip(all_vms.values() | map('dict2items'))
| subelements([1])
}}
loop_control:
label: "{{ item.0.0 }} — {{ item.1.key }}"
vars:
_task:
task:
vms: "{{ item.1.value }}"
group: "{{ item.1.key }}"
cluster: "{{ item.0.0 }}"
Given the playbook:
- hosts: localhost
gather_facts: no
tasks:
- set_fact:
tasks: "{{ tasks | default([]) + [_task] }}"
loop: >-
{{
all_vms.keys()
| zip(all_vms.values() | map('dict2items'))
| subelements([1])
}}
loop_control:
label: "{{ item.0.0 }} — {{ item.1.key }}"
vars:
_task:
task:
vms: "{{ item.1.value }}"
group: "{{ item.1.key }}"
cluster: "{{ item.0.0 }}"
all_vms:
clusterA:
group1:
- vm-01
- vm-02
group2:
- vm-03
- vm-04
clusterB:
group1:
- vm-05
- vm-06
- debug:
var: tasks
This yields:
TASK [set_fact] ***************************************************************
ok: [localhost] => (item=clusterA — group1)
ok: [localhost] => (item=clusterA — group2)
ok: [localhost] => (item=clusterB — group1)
TASK [debug] ******************************************************************
ok: [localhost] =>
tasks:
- task:
cluster: clusterA
group: group1
vms:
- vm-01
- vm-02
- task:
cluster: clusterA
group: group2
vms:
- vm-03
- vm-04
- task:
cluster: clusterB
group: group1
vms:
- vm-05
- vm-06
Although this is definitely possible using several combinations of filters directly in Ansible/jinja2, my 2 cent: save you nerves and time using a custom filter.
Demo below is for a a filter in the dedicated filters_plugin folder adjacent to your playbook. See the roles and collections documentations for better ways to distribute your filter if need be. Note that this was written real quick to put you on track and although it is fully functional with your above data example, you will need to add some data and errors checks if you intend to use this for a larger audience.
filters_plugins/my_cluster_filters.py
def all_vms_list(vm_dict):
"""Return a list of dicts for all vms with their cluster an group information"""
vms_list = []
for cluster, groups in vm_dict.items():
for group_name, hosts in groups.items():
for host in hosts:
current_vm = {
'cluster': cluster,
'group': group_name,
'host': host
}
vms_list.append(current_vm)
return vms_list
class FilterModule(object):
"""my cluster data filters."""
def filters(self):
"""Return the filter list."""
return {
'all_vms_list': all_vms_list
}
Then the test playbook.yml:
---
- hosts: localhost
gather_facts: false
vars:
all_vms:
clusterA:
group1:
- vm-01
- vm-02
group2:
- vm-03
- vm-04
clusterB:
group1:
- vm-05
- vm-06
tasks:
- debug:
msg: "Host {{ item.host }} belongs to group {{ item.group }} inside cluster {{ item.cluster }}"
loop: "{{ all_vms | all_vms_list }}"
gives:
PLAY [localhost] **************************************************************************************************************************************************************************************************************
TASK [debug] ******************************************************************************************************************************************************************************************************************
ok: [localhost] => (item={'cluster': 'clusterA', 'group': 'group1', 'host': 'vm-01'}) => {
"msg": "Host vm-01 belongs to group group1 inside cluster clusterA"
}
ok: [localhost] => (item={'cluster': 'clusterA', 'group': 'group1', 'host': 'vm-02'}) => {
"msg": "Host vm-02 belongs to group group1 inside cluster clusterA"
}
ok: [localhost] => (item={'cluster': 'clusterA', 'group': 'group2', 'host': 'vm-03'}) => {
"msg": "Host vm-03 belongs to group group2 inside cluster clusterA"
}
ok: [localhost] => (item={'cluster': 'clusterA', 'group': 'group2', 'host': 'vm-04'}) => {
"msg": "Host vm-04 belongs to group group2 inside cluster clusterA"
}
ok: [localhost] => (item={'cluster': 'clusterB', 'group': 'group1', 'host': 'vm-05'}) => {
"msg": "Host vm-05 belongs to group group1 inside cluster clusterB"
}
ok: [localhost] => (item={'cluster': 'clusterB', 'group': 'group1', 'host': 'vm-06'}) => {
"msg": "Host vm-06 belongs to group group1 inside cluster clusterB"
}
PLAY RECAP ********************************************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Hi I have a file with list of filenames .I want to read each line and add a variable {{version}} to the end and pass it to another task which will download the artifact .
MY_File
cat file/branches.txt
mhr-
mtr-
tsr
I want to get each line add {{version}} to it
mhr-1.1-SNAPSHOT
mtr-1.1-SNAPSHOT
tsr-1.1-SNAPSHOT
I couldn't get the file names and append them. below is a failed attempt by me .
---
- name: get file names
shell: echo {{item}}{{version}}
register: result
with_lines: cat files/branches.txt
- include_tasks: 2main.yml
with_items: "{{result.results}}"
I think what you need a new variable which contains text like mhr-1.1-SNAPSHOT depending on the source file. You can use set_fact to achieve this.
Example:
vars:
file_versions: []
version: "1.1-SNAPSHOT"
tasks:
- set_fact:
file_versions: "{{ file_versions }} + [ '{{ item }}{{ version }}' ]"
with_lines: cat file/branches.txt
- debug:
msg: "{{ item }}"
with_items: "{{ file_versions }}"
- include_tasks: 2main.yml
with_items: "{{ file_versions }}"
Gives below debug output:
TASK [debug] ************************************************************************************************************************
ok: [localhost] => (item=mhr-1.1-SNAPSHOT) => {
"msg": "mhr-1.1-SNAPSHOT"
}
ok: [localhost] => (item=mtr-1.1-SNAPSHOT) => {
"msg": "mtr-1.1-SNAPSHOT"
}
ok: [localhost] => (item=tsr-1.1-SNAPSHOT) => {
"msg": "tsr-1.1-SNAPSHOT"
}
In a nutshell:
---
- name: append version to names found in a local text file line by line
hosts: localhost
gather_facts: false
vars:
# This is the version we want to append
version: 1.1-SNAPSHOT
# This is the file on local controller containing the names
my_file: /tmp/my_file.txt
# This var will contain a list of each name in file with appended version
file_version: "{{ lookup('file', my_file).splitlines() | map('regex_replace', '^(.*)$', '\\g<1>' + version) | list }}"
tasks:
- name: Show the result
debug:
var: file_version
- name: Loop on var with an include (the passed var is item in this case)
include_tasks: some_other_tasks_working_on_item.yml
loop: "{{ file_version }}"
So, I am working with Ansible and I am retrieving specific information from some devices. In particular, I have a task like this:
<rpc-reply message-id="urn:uuid:1914-b84d7ff">
<lldp-neighbors-information style="brief">
<lldp-neighbor-information>
<lldp-local-port-id>A1</lldp-local-port-id>
<lldp-local-parent-interface-name>-</lldp-local-parent-interface-name>
<lldp-remote-chassis-id-subtype>A2</lldp-remote-chassis-id-subtype>
<lldp-remote-chassis-id>A3</lldp-remote-chassis-id>
<lldp-remote-port-id-subtype>A4</lldp-remote-port-id-subtype>
<lldp-remote-port-id>A5</lldp-remote-port-id>
<lldp-remote-system-name>A6</lldp-remote-system-name>
</lldp-neighbor-information>
<lldp-neighbor-information>
<lldp-local-port-id>B1</lldp-local-port-id>
<lldp-local-parent-interface-name>-</lldp-local-parent-interface-name>
<lldp-remote-chassis-id-subtype>B2</lldp-remote-chassis-id-subtype>
<lldp-remote-chassis-id>B3</lldp-remote-chassis-id>
<lldp-remote-port-id-subtype>B4</lldp-remote-port-id-subtype>
<lldp-remote-port-id>B5</lldp-remote-port-id>
<lldp-remote-system-name>B6</lldp-remote-system-name>
</lldp-neighbor-information>
<lldp-neighbor-information>
<lldp-local-port-id>C1</lldp-local-port-id>
<lldp-local-parent-interface-name>-</lldp-local-parent-interface-name>
<lldp-remote-chassis-id-subtype>C2</lldp-remote-chassis-id-subtype>
<lldp-remote-chassis-id>C3</lldp-remote-chassis-id>
<lldp-remote-port-id-subtype>C4</lldp-remote-port-id-subtype>
<lldp-remote-port-id>C5</lldp-remote-port-id>
</lldp-neighbor-information>
</lldp-neighbors-information>
</rpc-reply>
My goal is to build a dictionary like this :
{'A6': ['A1', 'A2'], 'B6': ['B1', 'B2']}
What I have done is creating a list with all my keys by doing the following tasks:
- name: Retrieve lldp system names
xml:
xmlstring: "{{ item.string | regex_replace('\n', '') }}"
xpath: "{{ item.path }}"
content: text
loop:
- { path: "/rpc-reply/lldp-neighbors-information/lldp-neighbor-information/lldp-remote-system-name", string: "{{xml_reply.xml}}" }
register: sys_names
- name: Save all sys names in a list
set_fact:
sys_names_list: "{{ sys_names.results[0].matches | map('dict2items') | list | json_query('[].value') }}"
Then I could create a second list for the elements [A1, B1] and a 3rd list [A2, B2] respectively. So then I could just combine the 3 lists and create my dictionary.
So there are 3 questions here :
Is there a way to directly build an ansible-dictionary from xml elements or should I keep writing my own module ?
Since the last element C6 would be my 3rd key BUT does NOT exist, I want to skip it. How is that possible in my task above?
How do I combine combine the 3 lists and create my dictionary, having skipped the 3rd element ? Otherwise my lists will not match the correct info ..
You can perform this by combining a filter that converts your xml file to a structured json in order to facilitate data manipulations.
In this example, I use this converter from xml to json.
File filter_plugins/from_xml.py :
#!/usr/bin/python
# From https://github.com/nasgoncalves/ansible-xml-to-json-filter/blob/master/filter_plugins/xml_to_json.py
import json
import xml.etree.ElementTree as ET
from collections import defaultdict
class FilterModule(object):
def etree_to_dict(self, t):
d = {t.tag: {} if t.attrib else None}
children = list(t)
if children:
dd = defaultdict(list)
for dc in map(self.etree_to_dict, children):
for k, v in dc.items():
dd[k].append(v)
d = {t.tag: {k: v[0] if len(v) == 1 else v
for k, v in dd.items()}}
if t.attrib:
d[t.tag].update(('#' + k, v)
for k, v in t.attrib.items())
if t.text:
text = t.text.strip()
if children or t.attrib:
if text:
d[t.tag]['#text'] = text
else:
d[t.tag] = text
return d
def filters(self):
return {
'from_xml': self.from_xml,
'xml_to_json': self.xml_to_json
}
def from_xml(self, data):
root = ET.ElementTree(ET.fromstring(data)).getroot()
return self.etree_to_dict(root)
def xml_to_json(self, data):
return json.dumps(self.from_xml(data))
File playbook.yml, I use a jinja2 template to filter key and combine values in a list, then I convert this list to a dict with filter items2dict :
---
- hosts: localhost
gather_facts: no
connection: local
tasks:
- set_fact:
xml_message: |
<rpc-reply message-id="urn:uuid:1914-b84d7ff">
<lldp-neighbors-information style="brief">
<lldp-neighbor-information>
<lldp-local-port-id>A1</lldp-local-port-id>
<lldp-local-parent-interface-name>-</lldp-local-parent-interface-name>
<lldp-remote-chassis-id-subtype>A2</lldp-remote-chassis-id-subtype>
<lldp-remote-chassis-id>A3</lldp-remote-chassis-id>
<lldp-remote-port-id-subtype>A4</lldp-remote-port-id-subtype>
<lldp-remote-port-id>A5</lldp-remote-port-id>
<lldp-remote-system-name>A6</lldp-remote-system-name>
</lldp-neighbor-information>
<lldp-neighbor-information>
<lldp-local-port-id>B1</lldp-local-port-id>
<lldp-local-parent-interface-name>-</lldp-local-parent-interface-name>
<lldp-remote-chassis-id-subtype>B2</lldp-remote-chassis-id-subtype>
<lldp-remote-chassis-id>B3</lldp-remote-chassis-id>
<lldp-remote-port-id-subtype>B4</lldp-remote-port-id-subtype>
<lldp-remote-port-id>B5</lldp-remote-port-id>
<lldp-remote-system-name>B6</lldp-remote-system-name>
</lldp-neighbor-information>
<lldp-neighbor-information>
<lldp-local-port-id>C1</lldp-local-port-id>
<lldp-local-parent-interface-name>-</lldp-local-parent-interface-name>
<lldp-remote-chassis-id-subtype>C2</lldp-remote-chassis-id-subtype>
<lldp-remote-chassis-id>C3</lldp-remote-chassis-id>
<lldp-remote-port-id-subtype>C4</lldp-remote-port-id-subtype>
<lldp-remote-port-id>C5</lldp-remote-port-id>
</lldp-neighbor-information>
</lldp-neighbors-information>
</rpc-reply>
- set_fact:
my_dict: |
{% set json_message = xml_message | from_xml %}
{% for item in json_message['rpc-reply']['lldp-neighbors-information']['lldp-neighbor-information'] %}
{% if "lldp-remote-system-name" in item %}
- key: {{ item['lldp-remote-system-name'] }}
value: [{{ item['lldp-local-port-id'] }}, {{ item['lldp-remote-chassis-id-subtype'] }}]
{% endif %}
{% endfor %}
- set_fact:
my_dict: "{{ my_dict | from_yaml | items2dict }}"
- debug:
var: my_dict
Executing this playbook with ansible-playbook playbook.yml returns :
PLAY [localhost] **********************************
TASK [set_fact] ***********************************
ok: [localhost]
TASK [set_fact] ***********************************
ok: [localhost]
TASK [set_fact] ***********************************
ok: [localhost]
TASK [debug] **************************************
ok: [localhost] => {
"my_dict": {
"A6": [
"A1",
"A2"
],
"B6": [
"B1",
"B2"
]
}
}
I have looked at simular threads, but to no success. What I'm trying to do is update my localstorage through an update function. The functions look as follows:
The code to make the variables to call:
var localProfile = JSON.parse(localStorage.getItem("profile"));
if(localProfile != undefined && localProfile.length>0)
{ this.profiles = localProfile; }
else {
this.profile = [
{ id: "1526", name: "berserk", password: "berserk", age: "31", gender: "male"},
{ id: "1358", name: "Johnathan", password: "test", age: "17", gender: "male"},
{ id: "2539", name: "Britney", password: "test", age: "18", gender: "female"},
{ id: "1486", name: "Kevin", password: "test", age: "7", gender: "male"},
{ id: "7777", name: "jesus", password: "holy", age: "unknown", gender: "male"},
{ id: "6666", name: "satan", password: "hell", age: "unknown", gender: "unknown"}
];
}
The code to update the variable:
this.updateProfile = function(profile) {
profile.updating = false;
console.log(profile);
localStorage.setItem("profile", profile);
}
As I noted in the title I am currently using Angular. I have used the console.log(-line and the response seems to be exactly what it's supposed to be. I have tried using JSON.parse( and JSON.stringify as well as a couple of other combinations. I seem to get either the error above or another error when trying to reload the page. Apperently I either cannot execute the statement, or I end up corrupting the data so reloading the page returns a simular error.
In case the data in variable profile is in doubt:
Array [ Object, Object, Object, Object, Object, Object ]
And when taking a closer look at the data:
age:"17"
gender:"male"
id:"1358"
name:"johnathan"
password:"test"
The other object looks identical with no weird defaults in them. I already took care of the $$hashkey just incase that was the problem.
Any help on how to execute the call correctly is greatly apreciated and if the information is insufficient please do tell.
The problem is your not using JSON.stringify when saving your data. So when you are parsing from localStorage its not json.
Add a factory to be used across your application that handles JSON.parse() and JSON.stringify()
.factory('LocalStorageUtil', function() {
return {
get: get,
set: set,
remove: remove
}
function get(key) {
return JSON.parse(localStorage.getItem(key));
}
function set(key, val) {
localStorage.setItem(key, JSON.stringify(val));
}
function remove(key) {
return localStorage.removeItem(key);
}
})
Here is a JS Bin tiny sample app using this LocalStorageUtil.
http://jsbin.com/fijagiy/2/edit?js,output