Currently we have a huge file that contains all of our nginx configs for each site we work on. The file has about 150 lines or so of sites like this:
- { nginx_tempalte: 'site.conf.tpl', domain: 'example.com', server: 'ServerA', enabled: true, conf_name: 'example_site' }
Our playbook loops through each var 2 times. Once for getting it into sites-enabled and another for the symlink. This takes about 5 minutes each loop which isn't ideal.
I tried setting up a nested loop that takes in a registered variabled that has all the config names from the sites-available and checks them against the given var from earlier. However this seems like more of the same approach.
I would love some help filtering down these files.
It depends on how are you getting the dict. If it is a variable, you can have:
---
- name: Test
hosts: localhost
gather_facts: False
# with predefined vars
vars:
nginx: { nginx_tempalte: 'site.conf.tpl', domain: 'example.com', server: 'ServerA', enabled: true, conf_name: 'example_site' }
tasks:
- name: Fact
set_fact:
domain: "{{ nginx['domain'] }}"
server: "{{ nginx['server'] }}"
- name: Print Domain
debug:
var: domain
- name: Print Server
debug:
var: server
And you will have both values at the same time:
PLAY [Test] ********************************************************************************************************************
TASK [Fact] ********************************************************************************************************************
ok: [localhost]
TASK [Print Domain] ************************************************************************************************************
ok: [localhost] => {
"domain": "example.com"
}
TASK [Print Server] ************************************************************************************************************
ok: [localhost] => {
"server": "ServerA"
}
PLAY RECAP *********************************************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0
Related
I want to create a dictionary in ansible from a list; using some variables for the value in the key-value pair of the dictionary, but it seems to be not working.
I've simplified just to the problem and created a sample playbook to reproduce the issue, can someone help me out. Thanks!
here is my playbook
---
- name: create dictionary test
hosts: all
connection: local
gather_facts: False
vars:
ports: [80, 443]
server_base: "org.com"
tasks:
- name: print the ports
debug:
msg: "ports: {{ports}}"
- name: create a dictionary
set_fact:
#server_rules: "{{server_rules|default([]) + [{'server': '{{server_base}}-{{item}}', 'port': item}]}}"
server_rules: "{{server_rules|default([]) + [{'server': '{{server_base}}', 'port': item}]}}"
loop: "{{ports|flatten(1)}}"
- name: output
debug:
msg: "server_rules: {{server_rules}}"
With the above it works, the output as below:
$ansible-playbook -i "localhost," dicttest.yaml
PLAY [create dictionary test] ***************************************************************************************************************************************
TASK [print the ports] **********************************************************************************************************************************************
ok: [localhost] => {
"msg": "ports: [80, 443]"
}
TASK [create a dictionary] ******************************************************************************************************************************************
ok: [localhost] => (item=80)
ok: [localhost] => (item=443)
TASK [output] *******************************************************************************************************************************************************
ok: [localhost] => {
"msg": "server_rules: [{'server': 'org.com', 'port': 80}, {'server': 'org.com', 'port': 443}]"
}
PLAY RECAP **********************************************************************************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
But when I change set fact to (uncomment the one commented line and comment the other one)
server_rules: "{{server_rules|default([]) + [{'server': '{{server_base}}-{{item}}', 'port': item}]}}"
it fails with the following error
$ansible-playbook -i "localhost," dicttest.yaml
PLAY [create dictionary test] ***************************************************************************************************************************************
TASK [print the ports] **********************************************************************************************************************************************
ok: [localhost] => {
"msg": "ports: [80, 443]"
}
TASK [create a dictionary] ******************************************************************************************************************************************
ok: [localhost] => (item=80)
ok: [localhost] => (item=443)
TASK [output] *******************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'item' is undefined\n\nThe error appears to be in '/Users/dev/dicttest.yaml': line 22, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - name: output\n ^ here\n"}
PLAY RECAP **********************************************************************************************************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
Can someone explain how to get this working.
Q: Given the data
ports: [80, 443]
server_base: org.com
create a list of dictionaries
server_rules:
- port: 80
server: org.com-80
- port: 443
server: org.com-443
A: The task below gives the expected result
- name: create a list of dictionaries
set_fact:
server_rules: "{{ server_rules|default([]) +
[{'server': server_base + '-' + item|string,
'port': item}] }}"
loop: "{{ ports }}"
It's not necessary to iterate the list in a task. The declaration of the variables below does the same job
servers: "{{ [server_base]|
product(ports)|
map('join', '-')|
map('community.general.dict_kv', 'server')|
list }}"
server_rules: "{{ servers|
zip(ports|map('community.general.dict_kv', 'port'))|
map('combine')|
list }}"
Notes:
Double braces "{{ }}" can't be nested. The expression below is wrong
"{{ var1 + ['{{server_base}}-'] }}"
Correct
"{{ var1 + [server_base + '-'] }}"
In YAML, the operator plus "+" is used both to concatenate strings and lists. This is because a string in YAML is technically a list of characters. It's recommended to use "~" to concatenate strings
Also correct
"{{ var1 + [server_base ~ '-'] }}"
Use var attribute in debug. The output is more readable with stdout_callback = yaml
- debug:
var: server_rules
The created variable server_rules is a list. The items are dictionaries. Hence, it's a list of dictionaries.
The variable ports is a simple list. There is no need to use the filter flatten.
The combination of "hosts: all" and "connection: local" would make to run all hosts at the localhost
hosts: all
connection: local
Use "hosts: localhost" if you want to run the playbook at the localhost. In this case "connection: local" is the default
hosts: localhost
If you want to run a task at the localhost, but still want the play to read the variables for all hosts, use "delegate_to: localhost" and limit the task to "run_once: true". For example
- hosts: all
tasks:
- copy:
content: "{{ ansible_play_hosts|to_nice_yaml }}"
dest: /tmp/ansible_play_hosts.yml
delegate_to: localhost
run_once: true
got an answer from another person outside of stackoverflow, posting here just to make sure anyone looking at this, gets the solution as well
changing as follows fixes the problem
server_rules: "{{server_rules|default([]) + [{'server': '{{server_base}}-' + item|string, 'port': item}]}}"
The complete working playbook
---
- name: create dictionary test
hosts: all
connection: local
gather_facts: False
vars:
ports: [80, 443]
server_base: "org.com"
tasks:
- name: print the ports
debug:
msg: "ports: {{ports}}"
- name: create a dictionary
set_fact:
server_rules: "{{server_rules|default([]) + [{'server': '{{server_base}}-' + item|string, 'port': item}]}}"
loop: "{{ports|flatten(1)}}"
- name: output
debug:
msg: "server_rules: {{server_rules}}"
Very beginner question, but trying to understand what's going on with loops vs not using loops. (ansible 2.9.9)
Here's my setup
# setup_servers.yml
- hosts: all
gather_facts: false
roles:
- baseline
# staging.yml
---
all:
children:
hosts:
host1:
host2:
host3:
# baseline/tasks/main.yml
---
- name: Create Users Task
user:
name: "{{ item }}"
state: present
loop: "{{ developers }}"
# baseline/vars/main.yml
---
developers:
- bob
- jane
- sally
And I when I run this: ansible-playbook -i staging.yml setup_servers.yml I get the following:
TASK [baseline : Create Users Task] *************************************************************************************************************************************************************************************************
ok: [host1] => (item=bob)
ok: [host2] => (item=bob)
ok: [host3] => (item=bob)
ok: [host1] => (item=jane)
ok: [host2] => (item=jane)
ok: [host3] => (item=jane)
ok: [host1] => (item=sally)
ok: [host2] => (item=sally)
ok: [host3] => (item=sally)
However, if I don't use a loop in the task:
---
- name: Create Users Task
user:
name: "bob"
state: present
user:
name: "jane"
state: present
user:
name: "sally"
state: present
I get this different output:
TASK [baseline : Create Users Task] *************************************************************************************************************************************************************************************************
ok: [host1]
ok: [host2]
ok: [host3]
The way the output is shown it appears that in the looping case above it connects to each host separately for each item in the loop. Whereas with the non-looping task, it shows that it connects to each host just once. Also, the looping case takes quite a bit longer to complete (8x longer - basic check with time prepended to the command for a couple of runs of each).
Is the output not indicative of the actual ssh behavior? Any insight into what's happening here would be helpful. If the looping scenario IS connecting once for each host AND item, is there a way to set this up so it connects to each host once and then loops through creating the users?
After re-reading all the docs on loops and many google searches I have not found an answer.
Thanks.
Your non-looping task is invalid. You have three user calls, but only the last is actually executed. YAML is not a programming language. The YAML processing overwrites earlier items in a list that have the same key.
I'm writing an Ansible role that sets up the network. For each type of interface (ethernet, bond, bridge and vlan) I made a variable that contains the relevant data.
The idea is that I have to make a loop that runs the number of times that there are elements in a list variable ('bridge_ports') and for each pass a configuration file is created via a template.
Parts of the variable for bridge interfaces look like this:
my_network__bridge_interface:
address: "192.168.1.48",
bootproto: "static",
bridge_ports:
- eth0
- eth1
device: "br-mgmt",
...
To make the pass, I have tried with a with_subelements loop - but this does not go well.
- name: Create the network configuration file for the port on the bridge devices
template:
src: "{{ ansible_os_family }}.bridge_port.j2"
dest: "{{ my_network__ifconf_path }}/ifcfg-{{ item.1 }}"
with_subelements
- "{{ my_network__bridge_interface }}"
- bridge_ports
when: device_conf.type == 'bridge'
register: my_network__bridge_port_result
When I run the code, the error message comes: "could not find 'bridge_ports' key in iterated item '{}'".
I can see that I use with_subelements in the wrong way, but I don't really know what type of loop I would otherwise need.
The issue is with the yml definition. The below yml works:
my_network__bridge_interface:
- address: "192.168.1.48"
bootproto: static
bridge_ports:
- eth0
- eth1
device: br-mgmt
playbook -->
---
- hosts: localhost
tasks:
- include_vars: vars.yml
- debug:
msg: "{{ item.1 }}"
with_subelements:
- "{{ my_network__bridge_interface }}"
- bridge_ports
output -->
TASK [debug] ****************************************************************************************************************************
ok: [localhost] => (item=[{u'device': u'br-mgmt', u'bootproto': u'static', u'address': u'192.168.1.48'}, u'eth0']) => {
"msg": "eth0"
}
ok: [localhost] => (item=[{u'device': u'br-mgmt', u'bootproto': u'static', u'address': u'192.168.1.48'}, u'eth1']) => {
"msg": "eth1"
}
I have a playbook, which replace the IP and hostname. The playbook works fine till it reboots the machine, but unable to connect it back as now the IP of that server is changed to something else.
Is there any better way to handle this problem.
---
- hosts: test
remote_user: root
vars:
IP_TO_REPLACE: '192.168.1.15'
IP_TO_REPLACE_WITH: '192.168.1.16'
HOSTNAME_TO_REPLACE: 'devops-15'
HOSTNAME_TO_REPLACE_WITH: 'devops-16'
tasks:
- name: modifying the ifcfg-BR0 file
replace:
path: "{{ item.path }}"
regexp: "{{ item.regexp1 }}"
replace: "{{ item.replace }}"
backup: yes
with_items:
- { path: '/etc/sysconfig/network/ifcfg-br0', regexp1: "{{ IP_TO_REPLACE }}", replace: "{{ IP_TO_REPLACE_WITH }}" }
- { path: '/etc/hosts', regexp1: "{{ IP_TO_REPLACE }}", replace: "{{ IP_TO_REPLACE_WITH }}" }
- { path: '/etc/hosts', regexp1: "{{ HOSTNAME_TO_REPLACE }}", replace: "{{ HOSTNAME_TO_REPLACE_WITH }}" }
- { path: '/etc/hostname', regexp1: "{{ HOSTNAME_TO_REPLACE }}", replace: "{{ HOSTNAME_TO_REPLACE_WITH }}" }
register: task_result
- name: Reboot immediately the server to take latest changes
shell: "sleep 15 && reboot"
async: 1
poll: 0
when: task_result is changed
- name: Wait for the reboot to complete if there was a changed
wait_for:
port: 22
host: '{{ (ansible_ssh_host |default(ansible_host))|default(inventory_hostname) }}'
search_regex: OpenSSH
delay: 10
timeout: 90
when: task_result is changed
- name: Check the uptime of the server
shell: "uptime"
register: uptime_result
- debug: var=uptime_result
Actual Output
TASK [Reboot immediately the server to take latest changes] *****************************************************************************
fatal: [192.168.1.15]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Shared connection to 192.168.1.15 closed.\r\n", "unreachable": true}
Expected Output
It should connect to the new IP 192.168.1.16
My research lead me to the following solution
Inspire by the suggestion of "drenthe73" here:
https://www.reddit.com/r/ansible/comments/n6xfyv/change_current_ip_address_and_reboot_into_the_new/
playbook with 2 plays
---
# play 1
hosts:
- host_ip_initial
# ...
tasks:
- name: "change ip address"
# ...
- name: "reboot target"
ansible.builtin.shell: "reboot"
async: 1
poll: 0
# play2
gather_facts: no
hosts:
- host_ip_final
# ...
tasks:
- name: "Wait for the target to reboot before probing it is up and running"
ansible.builtin.wait_for:
timeout: 300
delegate_to: localhost
- name: "wait for boot up"
ansible.builtin.wait_for_connection:
connect_timeout: 5
sleep: 5
delay: 30
timeout: 600
You need to know the final IP address and creates in your inventory the proper homolog entries.
Ansible various way to restart target:
https://www.redhat.com/sysadmin/automate-reboot-ansible
I am a bit late, but it might help someone else...
I have situation where in one csv file i have 2 columns like below
cat report.csv
Field1,Field2,Field3
name3,3,5
name4,5,6
now i want to use the lines which are in bold.
Each column will be an input to one of the ansible role.
it should go like
roles:
- { role: arti_master, mod_name: "{{ item.name}}" , version: "{{ item.version}}"
with_lines:
- "cat report.csv|cut -d, -f2"
Here is an example:
- name: "Sending email"
hosts: localhost
gather_facts: no
tasks:
- name: "Reading user information"
read_csv:
path: users.csv
register: users
- name: "Sending an e-mail using Gmail SMTP servers"
mail:
host: smtp.gmail.com
port: 587
username: <email>
password: <pass>
to: "{{ user.email }}"
subject: Email Subjet
body: |
Hi {{ user.first_name }},
How are you?
loop: "{{ users.list }}"
loop_control:
loop_var: user
And the CSV:
first_name,last_name,email
Joel,Zamboni,joel.zamboni#example.com
The read_csv returns the data in two formats, as a dict or as a list and in this case I am 'looping' over the list to send emails.
Best,
Joel
I believe you have two (and a half) ways that I can think of:
Do as you said and run the file through cut or python -c "import csv;..." or other "manual" processing, then capture the output in a variable
Anything that looks like JSON when fed into a vars: or set_fact: will become a list or dict, so you'd just want the text to go into a tool looking like CSV and come out of the tool looking like JSON
Use the lookup("csvfile") to actually read the file using an "approved" mechanism
(this is the "half" part:) if the csv is on the remote machine, then use fetch: to pull it to your controlling machine, then run lookup("csvfile") on it
read_csv module was recently added to ansible, and is now available from ansible 2.8. After upgrading ansible, you can read line by line as follows:
- name: read the csv file
read_csv:
path: "{{ report.csv }}"
delimiter: ','
register: report_csv
You can then access it as list by using report_csv.list and it'll hold values as a list of dictionaries:
[{'Field1': 'name3', 'Field2': 3, 'Field3': 5}, {'Field1': 'name4', 'Field2': 5, 'Field3': 6}]
Sharing my ansible code as well this is what worked for me https://stackoverflow.com/a/56910479/1679541
playbook.yaml
---
- name: Read Users
gather_facts: True
hosts: localhost
tasks:
- read_csv:
path: users.csv
register: userlist
- debug:
msg: "{{ user.username }} and password is {{ user.password }}"
loop: "{{ userlist.list }}"
loop_control:
loop_var: user
users.csv
username,password
user0,test123
user1,test123
Ansible Output
PLAY [Read Users] *************************************************
TASK [Gathering Facts] *********************************************************
ok: [127.0.0.1]
TASK [read_csv] ****************************************************************
ok: [127.0.0.1]
TASK [debug] *******************************************************************
ok: [127.0.0.1] => (item={u'username': u'user0', u'password': u'test123'}) => {
"msg": "user0 and password is test123"
}
ok: [127.0.0.1] => (item={u'username': u'user1', u'password': u'test123'}) => {
"msg": "user1 and password is test123"
}
PLAY RECAP *********************************************************************
127.0.0.1 : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0