creating dictionary with ansible - loops

I want to create a dictionary in ansible from a list; using some variables for the value in the key-value pair of the dictionary, but it seems to be not working.
I've simplified just to the problem and created a sample playbook to reproduce the issue, can someone help me out. Thanks!
here is my playbook
---
- name: create dictionary test
hosts: all
connection: local
gather_facts: False
vars:
ports: [80, 443]
server_base: "org.com"
tasks:
- name: print the ports
debug:
msg: "ports: {{ports}}"
- name: create a dictionary
set_fact:
#server_rules: "{{server_rules|default([]) + [{'server': '{{server_base}}-{{item}}', 'port': item}]}}"
server_rules: "{{server_rules|default([]) + [{'server': '{{server_base}}', 'port': item}]}}"
loop: "{{ports|flatten(1)}}"
- name: output
debug:
msg: "server_rules: {{server_rules}}"
With the above it works, the output as below:
$ansible-playbook -i "localhost," dicttest.yaml
PLAY [create dictionary test] ***************************************************************************************************************************************
TASK [print the ports] **********************************************************************************************************************************************
ok: [localhost] => {
"msg": "ports: [80, 443]"
}
TASK [create a dictionary] ******************************************************************************************************************************************
ok: [localhost] => (item=80)
ok: [localhost] => (item=443)
TASK [output] *******************************************************************************************************************************************************
ok: [localhost] => {
"msg": "server_rules: [{'server': 'org.com', 'port': 80}, {'server': 'org.com', 'port': 443}]"
}
PLAY RECAP **********************************************************************************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
But when I change set fact to (uncomment the one commented line and comment the other one)
server_rules: "{{server_rules|default([]) + [{'server': '{{server_base}}-{{item}}', 'port': item}]}}"
it fails with the following error
$ansible-playbook -i "localhost," dicttest.yaml
PLAY [create dictionary test] ***************************************************************************************************************************************
TASK [print the ports] **********************************************************************************************************************************************
ok: [localhost] => {
"msg": "ports: [80, 443]"
}
TASK [create a dictionary] ******************************************************************************************************************************************
ok: [localhost] => (item=80)
ok: [localhost] => (item=443)
TASK [output] *******************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'item' is undefined\n\nThe error appears to be in '/Users/dev/dicttest.yaml': line 22, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - name: output\n ^ here\n"}
PLAY RECAP **********************************************************************************************************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
Can someone explain how to get this working.

Q: Given the data
ports: [80, 443]
server_base: org.com
create a list of dictionaries
server_rules:
- port: 80
server: org.com-80
- port: 443
server: org.com-443
A: The task below gives the expected result
- name: create a list of dictionaries
set_fact:
server_rules: "{{ server_rules|default([]) +
[{'server': server_base + '-' + item|string,
'port': item}] }}"
loop: "{{ ports }}"
It's not necessary to iterate the list in a task. The declaration of the variables below does the same job
servers: "{{ [server_base]|
product(ports)|
map('join', '-')|
map('community.general.dict_kv', 'server')|
list }}"
server_rules: "{{ servers|
zip(ports|map('community.general.dict_kv', 'port'))|
map('combine')|
list }}"
Notes:
Double braces "{{ }}" can't be nested. The expression below is wrong
"{{ var1 + ['{{server_base}}-'] }}"
Correct
"{{ var1 + [server_base + '-'] }}"
In YAML, the operator plus "+" is used both to concatenate strings and lists. This is because a string in YAML is technically a list of characters. It's recommended to use "~" to concatenate strings
Also correct
"{{ var1 + [server_base ~ '-'] }}"
Use var attribute in debug. The output is more readable with stdout_callback = yaml
- debug:
var: server_rules
The created variable server_rules is a list. The items are dictionaries. Hence, it's a list of dictionaries.
The variable ports is a simple list. There is no need to use the filter flatten.
The combination of "hosts: all" and "connection: local" would make to run all hosts at the localhost
hosts: all
connection: local
Use "hosts: localhost" if you want to run the playbook at the localhost. In this case "connection: local" is the default
hosts: localhost
If you want to run a task at the localhost, but still want the play to read the variables for all hosts, use "delegate_to: localhost" and limit the task to "run_once: true". For example
- hosts: all
tasks:
- copy:
content: "{{ ansible_play_hosts|to_nice_yaml }}"
dest: /tmp/ansible_play_hosts.yml
delegate_to: localhost
run_once: true

got an answer from another person outside of stackoverflow, posting here just to make sure anyone looking at this, gets the solution as well
changing as follows fixes the problem
server_rules: "{{server_rules|default([]) + [{'server': '{{server_base}}-' + item|string, 'port': item}]}}"
The complete working playbook
---
- name: create dictionary test
hosts: all
connection: local
gather_facts: False
vars:
ports: [80, 443]
server_base: "org.com"
tasks:
- name: print the ports
debug:
msg: "ports: {{ports}}"
- name: create a dictionary
set_fact:
server_rules: "{{server_rules|default([]) + [{'server': '{{server_base}}-' + item|string, 'port': item}]}}"
loop: "{{ports|flatten(1)}}"
- name: output
debug:
msg: "server_rules: {{server_rules}}"

Related

Ansible: loop with using collection and role

I´m doing the first steps in Ansible this week and I break on include_tasks for looping ofer a role.
The needed task is to create Letsencrypt certificates for a bunch of domains, thanks to T-Systems-MMS, there is already a collection to do this via APIs of letsencrypt and AutoDNS (see https://github.com/T-Systems-MMS/ansible-collection-acme/blob/master/docs/dns-challenge/autodns.md).
Filling this playbook with my settings, it is working fine for one domain. My try to loop over is (hopefully there was no mistake while anonymising the code):
playbook_getsslcert_main.yml:
---
- hosts: localhost
connection: local
vars:
ansible_python_interpreter: auto
tasks:
- name: Get SSL certificate
include_tasks: playbook_getsslcert_task.yml
loop:
- sub1.domain1.com
- sub2.domain1.com
playbook_getsslcert_task.yml:
---
- name: Doing letsencrypt ACME with AutoDNS
collections:
- t_systems_mms.acme
roles:
- acme
vars:
nbb_emailadress: my.email#example.com
nbb_autodnsuser: login.user#other.com
acme_domain:
certificate_name: "{{ item }}"
zone: "domain1.com"
email_address: "{{ nbb_emailadress }}"
subject_alt_name:
- "{{ item }}"
acme_challenge_provider: autodns
acme_use_live_directory: true
acme_conf_dir: /etc/letsencrypt
acme_account_email: "{{ nbb_emailadress }}"
acme_dns_user: "{{ nbb_autodnsuser }}"
acme_dns_password: "supersecret"
The error I get is
fatal: [localhost]: FAILED! => {"reason": "conflicting action statements: hosts, roles\n\nThe error appears to be in 'playbook_getsslcert_task.yml': line 2, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n---\n- name: Doing letsencrypt ACME with AutoDNS\n ^ here\n"}
My collegues and me are experienced Linux guys, we tested a lot; also we checked the YAML with formatcheckers and so on, did different styles for looping, tried an example tasks.ym just for writing a message, checked file formats (for linefeeds, correct HEX values,...) and so on.
But Ansible doesnt like the playbook.
Thanks for all your suggestions.
Edit:
Ubuntu 18.04 LTS, Python 3.6.9, Ansible 2.9.27
Thanks to #Zeitounator (sorry for overlooing your first link), a suitable and working solution have been found:
---
- hosts: all
connection: local
vars:
ansible_python_interpreter: auto
tasks:
- name: "Doing letsencrypt ACME with AutoDNS for {{ nbb_domain }}"
collections:
- t_systems_mms.acme
include_role:
name: acme
vars:
nbb_emailadress: my.email#example.com
nbb_autodnsuser: login.user#other.com
acme_domain:
certificate_name: "{{ nbb_domain }}"
zone: "domain1.com"
email_address: "{{ nbb_emailadress }}"
subject_alt_name:
- "{{ nbb_domain }}"
acme_challenge_provider: autodns
acme_use_live_directory: true
acme_conf_dir: /etc/letsencrypt
acme_account_email: "{{ nbb_emailadress }}"
acme_dns_user: "{{ nbb_autodnsuser }}"
acme_dns_password: "supersecret"
loop:
- sub1.domain1.com
- sub2.domain1.com
loop_control:
loop_var: nbb_domain

Ansible loop using ansible_hostnames

I'm trying to update a configuration file for a NiFi deployment, the inital deployment configuration needs to include the nodes to allow HTTPS connections to be established between them.
I have an ansible tasks that makes the required structural changes to the configuration files, but I can't seem to get the right details inserted.
- name: Add each host to the authorizers.xml
lineinfile:
path: /opt/nifi/conf/authorizers.xml
line: "<property name=\"Node Identity {{ item }}\">CN={{ item }}, OU=NiFi</property>"
insertafter: <!--accessPolicyProvider Node Identities-->
loop: "{{ query('inventory_hostnames', 'nifi') }}"
This puts the ip addresses for the hosts, and I need to get the ansible_hostname for each node instead.
I've played around with ansible_play_batch and loop: "{{ groups['nifi'] }}" but I'm getting the result, outputting the ip addresses instead of the short hostnames each time.
The short hostnames are not stored in my ansible configuration anywhere, they are (if I understand correctly) determined at run time via the gathering facts process. I'd really like to not have to put the node names into a list variable.
Q: "Get the ansible_hostname for each node"
A: Given the inventory
shell> cat hosts
[nifi]
10.1.0.51
10.1.0.52
The playbook below
- hosts: nifi
tasks:
- debug:
var: ansible_hostname
gives (abridged)
ok: [10.1.0.51] =>
ansible_hostname: test_01
ok: [10.1.0.52] =>
ansible_hostname: test_02
It's possible to iterate the hosts in the group and get ansible_hostname from the hostvars. For example, delegate_to localhost and run_once
- debug:
msg: "{{ hostvars[item].ansible_hostname }}"
loop: "{{ groups.nifi }}"
delegate_to: localhost
run_once: true
gives
ok: [10.1.0.51 -> localhost] => (item=10.1.0.51) =>
msg: test_01
ok: [10.1.0.51 -> localhost] => (item=10.1.0.52) =>
msg: test_02

Constructing a loop in Ansible

I'm writing an Ansible role that sets up the network. For each type of interface (ethernet, bond, bridge and vlan) I made a variable that contains the relevant data.
The idea is that I have to make a loop that runs the number of times that there are elements in a list variable ('bridge_ports') and for each pass a configuration file is created via a template.
Parts of the variable for bridge interfaces look like this:
my_network__bridge_interface:
address: "192.168.1.48",
bootproto: "static",
bridge_ports:
- eth0
- eth1
device: "br-mgmt",
...
To make the pass, I have tried with a with_subelements loop - but this does not go well.
- name: Create the network configuration file for the port on the bridge devices
template:
src: "{{ ansible_os_family }}.bridge_port.j2"
dest: "{{ my_network__ifconf_path }}/ifcfg-{{ item.1 }}"
with_subelements
- "{{ my_network__bridge_interface }}"
- bridge_ports
when: device_conf.type == 'bridge'
register: my_network__bridge_port_result
When I run the code, the error message comes: "could not find 'bridge_ports' key in iterated item '{}'".
I can see that I use with_subelements in the wrong way, but I don't really know what type of loop I would otherwise need.
The issue is with the yml definition. The below yml works:
my_network__bridge_interface:
- address: "192.168.1.48"
bootproto: static
bridge_ports:
- eth0
- eth1
device: br-mgmt
playbook -->
---
- hosts: localhost
tasks:
- include_vars: vars.yml
- debug:
msg: "{{ item.1 }}"
with_subelements:
- "{{ my_network__bridge_interface }}"
- bridge_ports
output -->
TASK [debug] ****************************************************************************************************************************
ok: [localhost] => (item=[{u'device': u'br-mgmt', u'bootproto': u'static', u'address': u'192.168.1.48'}, u'eth0']) => {
"msg": "eth0"
}
ok: [localhost] => (item=[{u'device': u'br-mgmt', u'bootproto': u'static', u'address': u'192.168.1.48'}, u'eth1']) => {
"msg": "eth1"
}

How to read CSV file data in ansible-playbook using with_lines?

I have situation where in one csv file i have 2 columns like below
cat report.csv
Field1,Field2,Field3
name3,3,5
name4,5,6
now i want to use the lines which are in bold.
Each column will be an input to one of the ansible role.
it should go like
roles:
- { role: arti_master, mod_name: "{{ item.name}}" , version: "{{ item.version}}"
with_lines:
- "cat report.csv|cut -d, -f2"
Here is an example:
- name: "Sending email"
hosts: localhost
gather_facts: no
tasks:
- name: "Reading user information"
read_csv:
path: users.csv
register: users
- name: "Sending an e-mail using Gmail SMTP servers"
mail:
host: smtp.gmail.com
port: 587
username: <email>
password: <pass>
to: "{{ user.email }}"
subject: Email Subjet
body: |
Hi {{ user.first_name }},
How are you?
loop: "{{ users.list }}"
loop_control:
loop_var: user
And the CSV:
first_name,last_name,email
Joel,Zamboni,joel.zamboni#example.com
The read_csv returns the data in two formats, as a dict or as a list and in this case I am 'looping' over the list to send emails.
Best,
Joel
I believe you have two (and a half) ways that I can think of:
Do as you said and run the file through cut or python -c "import csv;..." or other "manual" processing, then capture the output in a variable
Anything that looks like JSON when fed into a vars: or set_fact: will become a list or dict, so you'd just want the text to go into a tool looking like CSV and come out of the tool looking like JSON
Use the lookup("csvfile") to actually read the file using an "approved" mechanism
(this is the "half" part:) if the csv is on the remote machine, then use fetch: to pull it to your controlling machine, then run lookup("csvfile") on it
read_csv module was recently added to ansible, and is now available from ansible 2.8. After upgrading ansible, you can read line by line as follows:
- name: read the csv file
read_csv:
path: "{{ report.csv }}"
delimiter: ','
register: report_csv
You can then access it as list by using report_csv.list and it'll hold values as a list of dictionaries:
[{'Field1': 'name3', 'Field2': 3, 'Field3': 5}, {'Field1': 'name4', 'Field2': 5, 'Field3': 6}]
Sharing my ansible code as well this is what worked for me https://stackoverflow.com/a/56910479/1679541
playbook.yaml
---
- name: Read Users
gather_facts: True
hosts: localhost
tasks:
- read_csv:
path: users.csv
register: userlist
- debug:
msg: "{{ user.username }} and password is {{ user.password }}"
loop: "{{ userlist.list }}"
loop_control:
loop_var: user
users.csv
username,password
user0,test123
user1,test123
Ansible Output
PLAY [Read Users] *************************************************
TASK [Gathering Facts] *********************************************************
ok: [127.0.0.1]
TASK [read_csv] ****************************************************************
ok: [127.0.0.1]
TASK [debug] *******************************************************************
ok: [127.0.0.1] => (item={u'username': u'user0', u'password': u'test123'}) => {
"msg": "user0 and password is test123"
}
ok: [127.0.0.1] => (item={u'username': u'user1', u'password': u'test123'}) => {
"msg": "user1 and password is test123"
}
PLAY RECAP *********************************************************************
127.0.0.1 : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Filtering Array of Vars in Ansible

Currently we have a huge file that contains all of our nginx configs for each site we work on. The file has about 150 lines or so of sites like this:
- { nginx_tempalte: 'site.conf.tpl', domain: 'example.com', server: 'ServerA', enabled: true, conf_name: 'example_site' }
Our playbook loops through each var 2 times. Once for getting it into sites-enabled and another for the symlink. This takes about 5 minutes each loop which isn't ideal.
I tried setting up a nested loop that takes in a registered variabled that has all the config names from the sites-available and checks them against the given var from earlier. However this seems like more of the same approach.
I would love some help filtering down these files.
It depends on how are you getting the dict. If it is a variable, you can have:
---
- name: Test
hosts: localhost
gather_facts: False
# with predefined vars
vars:
nginx: { nginx_tempalte: 'site.conf.tpl', domain: 'example.com', server: 'ServerA', enabled: true, conf_name: 'example_site' }
tasks:
- name: Fact
set_fact:
domain: "{{ nginx['domain'] }}"
server: "{{ nginx['server'] }}"
- name: Print Domain
debug:
var: domain
- name: Print Server
debug:
var: server
And you will have both values at the same time:
PLAY [Test] ********************************************************************************************************************
TASK [Fact] ********************************************************************************************************************
ok: [localhost]
TASK [Print Domain] ************************************************************************************************************
ok: [localhost] => {
"domain": "example.com"
}
TASK [Print Server] ************************************************************************************************************
ok: [localhost] => {
"server": "ServerA"
}
PLAY RECAP *********************************************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0

Resources